将设为首页浏览此站
开启辅助访问 天气与日历 收藏本站联系我们切换到窄版

易陆发现论坛

 找回密码
 开始注册
查看: 45|回复: 2
收起左侧

Hadoop学习之路Hadoop集群搭建和简单应用

[复制链接]
发表于 2022-11-7 11:18:00 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有帐号?开始注册

x
正文
% U* O9 x' h1 r9 v) t& F' W/ I$ t$ R  R+ p. @- S
回到顶部; D. c+ x) L  n
分布式集群的通用问题
: M$ U" x9 K* ~当前的HDFS和YARN都是一主多从的分布式架构,主从节点---管理者和工作者  r! |% m* y3 _* p, U% C5 {# S2 z: f
9 h- R% d' ^5 T. ?3 u; s  G
问题:如果主节点或是管理者宕机了。会出现什么问题?5 O) p7 E/ i& L7 S7 M+ l
8 V2 {0 {1 p! l
群龙无首,整个集群不可用。所以在一主多从的架构中都会有一个通用的问题:2 {" B' a0 L, U$ u  B

% L+ b( A& w( g# U2 ^9 T当集群中的主节点宕机之后,整个集群不可用。这个现象叫做:单点故障。SPOF
+ u' p, S* `/ V# \/ t, o) w1 F  {* ~
单点故障讲述的概念有两点/ |7 \& D* x" `4 P1 K
1、如果说宕机的那个节点是从节点,那么整个集群能够继续运行,并且对外提供正常的服务。$ h" o- r1 Y* q+ L9 b
2 E0 J+ k- P+ p% }; f
2、如果说宕机的那个节点是主节点,那么整个集群就处于宕机状态。
1 i0 y# X" f! n% i9 }; g; u7 k5 ], e/ l  C
通用的解决方案:高可用
1 \. k$ X5 B# J. }) f* W3 W2 B& e' }9 z' ]* R1 m- M+ U
概念:当正在对外提供服务器的主从节点宕机,那么备用的主节点立马上位对外提供服务。无缝的瞬时切换。& D6 ]6 j. A& J. ~- _% b! x& i+ O
( Y0 u* V+ ~. a

  l$ v0 z0 W6 w  u# H1 ?7 B' I
% Q2 n. X/ F- Z' S/ B) {回到顶部2 O, K$ y; `& ]- I0 l* [# s7 U
集群的搭建的集中通用模式
% A! ?3 L* M" z& ~' ]5 }1、单机模式
' r' n# t6 A; f: e0 o5 O  表示所有的分布式系统都是单机的。9 @/ ^9 N( a# T! r1 W# G/ ~
: l4 B4 A7 @, v: U" x
2、伪分布式模式(搭建在了只有一个节点的集群中)3 h2 q2 y4 L1 i4 [
  表示集群中的所有角色都分配给了一个节点。$ N: x# o+ L: f  a, f6 T" @' {# B
6 o! k* ]* m3 q: X" W
  表示整个集群被安装在了只有一个节点的集群中的。: T4 R( {+ _2 }2 P, m8 i5 q" h9 z
( t+ E/ N  A; X% H7 z  g
  主要用于做快速使用,去模拟分布式的效果。/ y& B" X: |- Q2 S" ~1 J/ f/ z3 Z

. K+ C; q3 G1 [( e$ n9 C; `3、分布式模式
1 K2 ^% M, j8 f; q! e( d* k- W  表示集群中的节点会被分配成很多种角色,分散在整个集群中。! |1 q! u  X$ \! G/ Z) |

7 X$ @1 m9 B: H/ n9 R/ R  主要用于学习测试等等一些场景中。
: L6 `: \, ^. E! B/ v2 o7 `) u- A: }( l' G+ q% q$ b: }
4、高可用模式/ [( S! d! E2 ^. k' o' z0 [
  表示整个集群中的主节点会有多个
4 v) L  R( s5 Q) t+ c1 U( o! C
$ v* y% `6 h# H8 _) w/ E+ t; n  注意区分:能够对外提供服务的主节点还是只有一个。其他的主节点全部处于一个热备的状态。
4 u1 s8 W/ X8 {4 |5 l* Y2 H) b
, N& Y6 [8 `! A, a5 x( q  正在对外提供服务的主节点:active  有且仅有一个
0 K: F. j  W9 h6 |; E& L
: T8 O0 a, _' i3 k2 o3 s  热备的主节点:standby  可以有多个7 F$ t7 _# a8 m' k: G
$ g- Y5 k0 s. k! a/ u8 h5 X
  工作模式:1、在任意时刻,只有一个主节点是active的,active的主节点对外提供服务2 K& d& L, S0 i) p

; W5 O3 F1 ^) U# T' K       2、在任意时刻,都应至少有一个standby的主节点,等待active的宕机来进行接替* q% f. u" l" n* t$ x) t* O

1 `' g, T+ C/ ~* w5 K7 g% C1 i  架构模式:就是为了解决分布式集群中的通用问题SPOF+ N* \2 Z  {1 N& U& e5 d. P  ?+ B0 R

: R% X& M/ r0 J" z1 v; X* W5 M  不管是分布式架构还是高可用架构,都存在一个问题:主从结构---从节点数量太多了。最直观的的问题:造成主节点的工作压力过载,主节点会宕机,当前的这种现象是一种死循环
- f* D( [. T# R# }$ D! g; T! w9 ^. m" [# o0 y
5、联邦模式) s/ k0 P6 I9 h( Y+ R
  表示当前集群中的主从节点都可以有很多个。5 Y) X! R* [+ `. R
( T6 \/ \+ P( _* o2 c2 n
  1)主节点:可以有很多个的意思是说:同时对外提供服务的主节点有很多个。- c2 \" |( C8 u

" |5 N; L  J; L3 N) E            重点:每一个主节点都是用来管理整个集群中的一部分
8 S1 p! O/ B4 u4 M0 M% p" H& [3 H0 y! n' Y/ J
  2)从节点:一定会有很多个。5 p1 n0 R. J+ S! G. |/ \* ~+ |: b

; \: _& K8 {- w: f- K! U  在联邦模式下还是会有问题:
3 l$ o0 N8 Z" D
- l% s# Y1 s  x' B  虽然这个集群中的一个主节点的压力被分摊到了多个主节点。但是这个多个主节点依然会有一个问题:SOFP  E% Z+ u' v/ p

2 x* R! o% l2 \+ o( \2 @) `回到顶部+ O' L7 A: `: s
安装Hadoop集群中的一些通用问题
( E4 Q% y6 x# w5 {/ f1 ~2 e$ ]4 N: V1、假如安装不成功,并且不知道应该怎么去解决这个安装错误:重装
1 ]/ M7 ^5 s6 C; y  _* n
5 B- h& c9 o$ h/ Q   需要做的处理:处理安装步骤中不同的部分即可。第一次安装和重装时候的不同步骤:, v9 L3 d6 g0 G- F) u+ p

& h# q" }( g  S! Q, h   1)到修改配置文件以前,全部都不用动; M: Z# v; a) g  v
! ~5 ^9 u# y+ v2 w" l( [( `! |
   2)检查配置文件是否都正确
+ y- P0 u& K' d- K9 _9 b
/ I7 V0 w, X6 r( L    先检查一个节点上的配置文件是否都正确,如果都正确,重新分发一次即可$ n$ S- g4 t: @, \) ?% j! c5 Y& t7 F
/ a2 ]: o9 ^; p1 {0 [
   3)在安装分布式集群时,所有节点中的安装的安装目录和安装者,需要检查和确定
" F3 p7 B0 @( o: ]" Y( s) j
, i9 `4 z8 b$ |% m0 T/ D; Q/ e   4)删掉数据目录
' {6 G0 B% q( O- w, y2 L) h' e  _1 _' V* k
    A. 删除主节点的工作目录:namenode的数据目录
# w& c7 J5 u3 [1 L. o2 L, \" @! m& Q+ k5 Z( b
        删除即可,只需要在主节点删除即可
5 N, g$ Z* y6 X! _
# X# Y7 G6 S. S! r7 \    B. 删除从节点的工作目录:datanode的数据目录" d0 b" n& C: m* m+ n- w/ K" }
5 M+ T" @  N- p3 @1 e
        删除即可,把每个从节点上的这个对应数据目录都删掉  }, I" j) [( B1 o0 n0 H+ E0 b1 R
+ R& V' }1 [5 o# c
        如果以上两份数据都被删除了之后。整个集群当中就相当于没有存储任何的历史数据。所以就是一个全新的集群
$ w; t6 z% J4 i( [$ g3 S
$ g9 l% A  H5 J5 Q) ~6 y  5)在确保数据正常和安装包都正常之后,进行重新初始化
) X2 K; N: V( A0 z; y1 G  ]+ _3 f) c% `
     重点强调: hadoop集群的初始化,其实就是初始化HDFS集群, 只能在主节点进行初始化
. J* R' @$ d- `" G0 P1 I+ C1 ~- ^5 O0 `7 Y# n6 m
     如果你只需要搭建YARN集群,那么是可以不用做初始化的。& c2 @9 b- M4 r' W( w2 {
2 u0 P$ h4 X% p) U
  6)启动集群
7 E% {# }0 s9 ?+ u3 k0 Q8 t' ]5 @8 O; e# L* i; ^8 Y
  7)验证集群是否成功+ Y* x. y. p: @& s( C
1 t- z6 i; @* x2 ^8 o5 F
回到顶部3 o  ?* b" Z# T4 r7 S
Linux环境变量加载的顺序7 g+ a  y/ h. f! R& |" m
用户环境变量 :仅仅只是当前用户使用 ~/.bashrc   ~/.bash_profile
) z& C9 X" B2 y; _# t) y$ H1 S系统环境变量 :给当前系统中的所有用户使用 /etc/profile
- p: ?/ |6 A; Q: |
7 W6 V; |) C; ~& @0 n3 Y" y任何普通用户在进行登录的时候:会同时加载几个环境变量的配置文件:  {5 Y( Z/ b, J- ]$ P" R

+ O# q2 e, z* _- k" H+ @按顺序:# S0 L* T% t% o3 O& D
1、/etc/profile2 G. |6 j4 ?+ A! ]8 S- h
2、~/.bash_profile; W) ^- v9 ^+ C8 _" h
3、~/.bashrc) g+ b) L* X3 ], d, |

& a5 n, X  z: o5 ]4 H. Z8 ^5 I
 楼主| 发表于 2022-11-11 16:27:25 | 显示全部楼层
Hadoop是Apache软件基金会旗下的一个开源分布式计算平台。以Hadoop分布式文件系统HDFS(Hadoop Distributed Filesystem)和MapReduce(Google MapReduce的开源实现)为核心的Hadoop为用户提供了系统底层细节透明的分布式基础架构。
对于Hadoop的集群来讲,可以分成两大类角色:MasterSalve。一个HDFS集群是由一个NameNode和若干个DataNode组成的。其中NameNode作为主服务器,管理文件系统的命名空间和客户端对文件系统的访问操作;集群中的DataNode管理存储的数据。MapReduce框架是由一个单独运行在主节点上的JobTracker和运行在每个从节点的TaskTracker共同组成的。主节点负责调度构成一个作业的所有任 务,这些任务分布在不同的从节点上。主节点监控它们的执行情况,并且重新执行之前的失败任务;从节点仅负责由主节点指派的任务。当一个Job被提交时,JobTracker接收到提交作业和配置信息之后,就会将配置信息等分发给从节点,同时调度任务并监控TaskTracker的执行。
从上面的介绍可以看出,HDFS和MapReduce共同组成了Hadoop分布式系统体系结构的核心。HDFS在集群上实现分布式文件系统,MapReduce在集群上实现了分布式计算和任务处理。HDFS在MapReduce任务处理过程中提供了文件操作和存储等支持,MapReduce在HDFS的基础上实现了任务的分发、跟踪、执行等工作,并收集结果,二者相互作用,完成了Hadoop分布式集群的主要任务。
1.2 环境说明
我的环境是在虚拟机中配置的,Hadoop集群中包括4个节点:1个Master,2个Salve,节点之间局域网连接,可以相互ping通,节点IP地址分布如下:
! n' x5 m9 }# @0 I# {2 h; m" Z
虚拟机系统
机器名称
IP地址
Ubuntu 13.04
Master.Hadoop
192.168.1.141
Ubuntu 9.11
Salve1.Hadoop
192.168.1.142
Fedora 17
Salve2.Hadoop
192.168.1.137
+ E3 B* O: y  o

# I) C! Q2 |! |
Master机器主要配置NameNode和JobTracker的角色,负责总管分布式数据和分解任务的执行;3个Salve机器配置DataNode 和TaskTracker的角色,负责分布式数据存储以及任务的执行。其实应该还应该有1个Master机器,用来作为备用,以防止Master服务器宕机,还有一个备用马上启用。后续经验积累一定阶段后补上一台备用Master机器(可通过配置文件修改备用机器数)。
    注意:由于hadoop要求所有机器上hadoop的部署目录结构要求相同(因为在启动时按与主节点相同的目录启动其它任务节点),并且都有一个相同的用户名账户。参考各种文档上说的是所有机器都建立一个hadoop用户,使用这个账户来实现无密码认证。这里为了方便,分别在三台机器上都重新建立一个hadoop用户。
1.3 环境配置
Hadoop集群要按照1.2小节表格所示进行配置,下面介绍如何修改机器名称和配置hosts文件,以方便使用。
注意:我的虚拟机都采用NAT方式连接网络,IP地址是自动分配的,所以这里就使用自动分配的IP地址而未特地修改为某些IP地址。
(1)修改当前机器名称
假定我们发现我们的机器的主机名不是我们想要的。
1)在Ubuntu下修改机器名称
修改文件/etc/hostname里的值即可,修改成功后用hostname命令查看当前主机名是否设置成功。
       另外为了能正确解析主机名,最好也修改/etc/hosts文件里对应的主机名. j, e0 r4 Y3 {/ {. i& R
      
2)在Fedora下修改机器名称
通过对"/etc/sysconfig/network"文件修改其中"HOSTNAME"后面的值,改成我们规定的名称。
命令:vi /etc/sysconfig/network,修改如下:/ S+ J  M: v* H, Y4 z7 d
    * J' H6 J/ e8 n) |
        
    同样为了能正确解析主机名,最好也修改/etc/hosts文件里对应的主机名。
(2)配置hosts文件(必须)
"/etc/hosts"这个文件是用来配置主机将用的DNS服务器信息,是记载LAN内接续的各主机的对应[HostName  IP]用的。当用户在进行网络连接时,首先查找该文件,寻找对应主机名对应的IP地址。
我们要测试两台机器之间知否连通,一般用"ping 机器的IP",如果想用"ping 机器的主机名"发现找不见该名称的机器(这也就是为什么在修改主机名的同时最好修改该文件中对应的主机名),解决的办法就是修改"/etc/hosts"这个文件,通过把LAN内的各主机的IP地址和HostName的一一对应写入这个文件的时候,就可以解决问题。
例如:机器为"Master.Hadoop:192.168.1.141"对机器为"Salve1.Hadoop:192.168.1.142"用命令"ping"记性连接测试。测试结果如下:/ u( s" K* f, D; M9 X
   
从上图中的值,直接对IP地址进行测试,能够ping通,但是对主机名进行测试,发现没有ping通,提示"unknown host——未知主机",这时查看"Master.Hadoop"的"/etc/hosts"文件内容会发现里面没有"192.168.1.142  Slave1.Hadoop"内容,故而本机器是无法对机器的主机名为"Slave1.Hadoop" 解析。
在进行Hadoop集群配置中,需要在"/etc/hosts"文件中添加集群中所有机器的IP与主机名,这样Master与所有的Slave机器之间不仅可以通过IP进行通信,而且还可以通过主机名进行通信。所以在所有的机器上的"/etc/hosts"文件中都要添加如下内容:
192.168.1.141 Master.Hadoop
192.168.1.142 Slave1.Hadoop
192.168.1.137 Slave2.Hadoop
命令:vi /etc/hosts,添加结果如下:, j7 [0 V- i: r1 K. d
现在我们在进行对机器为"Slave1.Hadoop"的主机名进行ping通测试,看是否能测试成功。
从上图中我们已经能用主机名进行ping通了,说明我们刚才添加的内容,在局域网内能进行DNS解析了,那么现在剩下的事儿就是在其余的Slave机器上进行相同的配置。然后进行测试。
1.4 所需软件
(1)JDK软件
    JDK版本:jdk-7u25-linux-i586.tar.gz
(2)Hadoop软件
    Hadoop版本:hadoop-1.1.2.tar.gz
2、SSH无密码验证配置
Hadoop运行过程中需要管理远端Hadoop守护进程,在Hadoop启动以后,NameNode是通过SSH(Secure Shell)来启动和停止各个DataNode上的各种守护进程的。这就必须在节点之间执行指令的时候是不需要输入密码的形式,故我们需要配置SSH运用无密码公钥认证的形式,这样NameNode使用SSH无密码登录并启动DataName进程,同样原理,DataNode上也能使用SSH无密码登录到 NameNode。
注意:如果你的Linux没有安装SSH,请首先安装SSH
Ubuntu下安装ssh:sudo apt-get install openssh-server
Fedora下安装ssh:yum install openssh-server
2.1 SSH基本原理和用法
1)SSH基本原理
    SSH之所以能够保证安全,原因在于它采用了公钥加密。过程如下:
(1)远程主机收到用户的登录请求,把自己的公钥发给用户。
(2)用户使用这个公钥,将登录密码加密后,发送回来。
(3)远程主机用自己的私钥,解密登录密码,如果密码正确,就同意用户登录。
2)SSH基本用法
    假如用户名为java,登录远程主机名为linux,如下命令即可:
    $ ssh java@linux
    SSH的默认端口是22,也就是说,你的登录请求会送进远程主机的22端口。使用p参数,可以修改这个端口,例如修改为88端口,命令如下:
    $ ssh -p 88 java@linux
    注意:如果出现错误提示:ssh: Could not resolve hostname linux: Name or service not known,则是因为linux主机未添加进本主机的Name Service中,故不能识别,需要在/etc/hosts里添加进该主机及对应的IP即可:
    linux     192.168.1.107
2.2 配置Master无密码登录所有Salve
1)SSH无密码原理
Master(NameNode | JobTracker)作为客户端,要实现无密码公钥认证,连接到服务器Salve(DataNode | Tasktracker)上时,需要在Master上生成一个密钥对,包括一个公钥和一个私钥,而后将公钥复制到所有的Slave上。当Master通过SSH连接Salve时,Salve就会生成一个随机数并用Master的公钥对随机数进行加密,并发送给Master。Master收到加密数之后再用私钥解密,并将解密数回传给Slave,Slave确认解密数无误之后就允许Master进行连接了。这就是一个公钥认证过程,其间不需要用户手工输入密码。
2)Master机器上设置无密码登录
a. Master节点利用ssh-keygen命令生成一个无密码密钥对。
在Master节点上执行以下命令:
ssh-keygen –t rsa –P ''
运行后询问其保存路径时直接回车采用默认路径。生成的密钥对:id_rsa(私钥)和id_rsa.pub(公钥),默认存储在"/home/用户名/.ssh"目录下。2 q8 O# Q& [3 \- O: F
      
查看"/home/用户名/"下是否有".ssh"文件夹,且".ssh"文件下是否有两个刚生产的无密码密钥对。
   
b. 接着在Master节点上做如下配置,把id_rsa.pub追加到授权的key里面去。
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
查看下authorized_keys的权限,如果权限不对则利用如下命令设置该文件的权限:
chmod 600 authorized_keys
c. 用root用户登录修改SSH配置文件"/etc/ssh/sshd_config"的下列内容。
检查下面几行前面”#”注释是否取消掉:
RSAAuthentication yes # 启用 RSA 认证
PubkeyAuthentication yes # 启用公钥私钥配对认证方式
AuthorizedKeysFile  %h/.ssh/authorized_keys # 公钥文件路径+ i1 _3 T: ^$ C% ?3 \
   
设置完之后记得重启SSH服务,才能使刚才设置有效。
      
退出root登录,使用普通用户验证是否设置成功。
从上图中得知无密码登录本级已经设置完毕,接下来的事儿是把公钥复制
的Slave机器上。
    注意:有时候在测试时可能会出现错误: Agent admitted failure to sign using the key.解决办法是:ssh-add   ~/.ssh/id_rsa ,如下所示:
   
    d.使用ssh-copy-id命令将公钥传送到远程主机上(这里以Slave1.Hadoop为例)。
e. 测试是否无密码登录其它机器成功。
到此为止,我们经过5步已经实现了从"Master.Hadoop"到"Slave1.Hadoop"SSH无密码登录,下面就是重复上面的步骤把剩余的两台(Slave2.Hadoop和Slave3.Hadoop)Slave服务器进行配置。这样,我们就完成了"配置Master无密码登录所有的Slave服务器"。
接下来配置所有Slave无密码登录Master,其和Master无密码登录所有Slave原理一样,就是把Slave的公钥追加到Master的".ssh"文件夹下的"authorized_keys"中,记得是追加(>>)
注意:期间可能会出现一些问题如下:
(1)如果在ssh连接时出现错误“ssh:connect to host port 22: Connection refused”,如下图所示:8 c& k8 G( g5 I- I5 C
则可能是因为远程登录的那台机器没有安装ssh服务或安装了没有开启ssh服务,下面到Slave3.Hadoop主机进行测试:
, \& r$ \( F3 m4 j$ O
为了一劳永逸,设置系统启动时开启服务:# systemctl enable sshd.service
' Q5 @7 T: i. i' V& I/ T8 q
(2)如果在用命令ssh-copy-id时发现找不到该命令“ssh-copy-id:Command not found”,则可能是ssh服务的版本太低的原因,比如若你的机器是Redhat系统就可能该问题,解决办法是:手动复制本地的pubkey内容到远程服务器,命令如下:
cat ~/.ssh/id_rsa.pub | ssh hadoop@Master.Hadoop 'cat >> ~/.ssh/authorized_keys'
该命令等价于下面两个命令:
①在本地机器上执行:scp ~/.ssh/id_rsa.pub hadoop@Master.Hadoop:/~
②到远程机器上执行:cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
3、Java环境安装
所有的机器上都要安装JDK,现在就先在Master服务器安装,然后其他服务器按照步骤重复进行即可。安装JDK以及配置环境变量,需要以"root"的身份进行。
3.1 安装JDK
首先用root身份登录"Master.Hadoop"后在"/usr"下创建"java"文件夹,再将"jdk-7u25-linux-i586.tar.gz"复制到"/usr/java"文件夹中,然后解压即可。查看"/usr/java"下面会发现多了一个名为"jdk1.7.0_25"文件夹,说明我们的JDK安装结束,删除"jdk-7u25-linux-i586.tar.gz"文件,进入下一个"配置环境变量"环节。
3.2 配置环境变量
(1)编辑"/etc/profile"文件
    编辑"/etc/profile"文件,在后面添加Java的"JAVA_HOME"、"CLASSPATH"以及"PATH"内容如下:
# set java environment
export JAVA_HOME=/usr/java/jdk1.7.0_25/
export JRE_HOME=/usr/java/jdk1.7.0_25/jre
export CLASSPATH=.:CLASSPATH:CLASSPATH:JAVA_HOME/lib:$JRE_HOME/lib
export PATH=PATH:PATH:JAVA_HOME/bin:$JRE_HOME/bin
或者
# set java environment
export JAVA_HOME=/usr/java/jdk1.7.0_25/
export CLASSPATH=.:CLASSPATH:CLASSPATH:JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=PATH:PATH:JAVA_HOME/bin:$JAVA_HOME/jre/bin
以上两种意思一样,那么我们就选择第1种来进行设置。
(2)使配置生效
保存并退出,执行下面命令使其配置立即生效。
source /etc/profile 或 . /etc/profile
3.3 验证安装成功
配置完毕并生效后,用下面命令判断是否成功。
java -version
从上图中得知,我们确定JDK已经安装成功。
3.4 安装剩余机器
这时用普通用户hadoop通过scp命令格式把"/usr/java/"文件复制到其他Slave上面,剩下的事儿就是在其余的Slave服务器上按照上图的步骤配置环境变量和测试是否安装成功,这里以Slave1.Master为例:
scp -r /usr/java seed@Slave1.Master:/usr/
注意:有的机器库函数版本较低,可能安装不了高版本的JDK,比如有些Redhat9,此时不可以选择较低版本的JDK进行安装,因为所有集群中的JDK版本必须相同(经过测试),有两种方法可解决:一是放弃该机器,选用另一台能装该版本的JDK的机子;二是选择低版本的JDK,在所有机器上重新安装。
4、Hadoop集群安装
所有的机器上都要安装hadoop,现在就先在Master服务器安装,然后其他服务器按照步骤重复进行即可。安装和配置hadoop需要以"root"的身份进行。
4.1 安装hadoop
首先用root用户登录"Master.Hadoop"机器,将下载的"hadoop-1.1.2.tar.gz"复制到/usr目录下。然后进入"/usr"目录下,用下面命令把"hadoop-1.1.2.tar.gz"进行解压,并将其重命名为"hadoop",把该文件夹的读权限分配给普通用户hadoop,然后删除"hadoop-1.0.0.tar.gz"安装包。
cd /usr
tar –xzvf hadoop-1.1.2.tar.gz
mv hadoop-1.1.2 hadoop
chown –R hadoop:hadoop hadoop #将文件夹"hadoop"读权限分配给hadoop普通用户
rm -rf hadoop-1.1.2.tar.gz
最后在"/usr/hadoop"下面创建tmp文件夹,并把Hadoop的安装路径添加到"/etc/profile"中,修改"/etc/profile"文件,将以下语句添加到末尾,并使其生效(. /etc/profile):
# set hadoop path
export HADOOP_HOME=/usr/hadoop
export PATH=PATH:PATH:HADOOP_HOME/bin
4.2 配置hadoop
(1)配置hadoop-env.sh
该"hadoop-env.sh"文件位于"/usr/hadoop/conf"目录下。
在文件中修改下面内容:
export JAVA_HOME=/usr/java/jdk1.7.0_25
Hadoop配置文件在conf目录下,之前的版本的配置文件主要是Hadoop-default.xml和Hadoop-site.xml。 由于Hadoop发展迅速,代码量急剧增加,代码开发分为了core,hdfs和map/reduce三部分,配置文件也被分成了三个core- site.xml、hdfs-site.xml、mapred-site.xml。core-site.xml和hdfs-site.xml是站在 HDFS角度上配置文件;core-site.xml和mapred-site.xml是站在MapReduce角度上配置文件。
(2)配置core-site.xml文件
修改Hadoop核心配置文件core-site.xml,这里配置的是HDFS master(即namenode)的地址和端口号。
   
        hadoop.tmp.dir
        /usr/hadoop/tmp
        (备注:请先在 /usr/hadoop 目录下建立 tmp 文件夹)
        A base for other temporary directories.
   
   
        fs.default.name
        hdfs://192.168.1.141:9000
   
备注:如没有配置hadoop.tmp.dir参数,此时系统默认的临时目录为:/tmp/hadoo-hadoop。而这个目录在每次重启后都会被删掉,必须重新执行format才行,否则会出错。
(3)配置hdfs-site.xml文件
修改Hadoop中HDFS的配置,配置的备份方式默认为3。
   
        dfs.replication
        1
        (备注:replication 是数据副本数量,默认为3,salve少于3台就会报错)
   
(4)配置mapred-site.xml文件
修改Hadoop中MapReduce的配置文件,配置的是JobTracker的地址和端口。
   
        mapred.job.tracker
        http://192.168.1.141:9001
   
(5)配置masters文件
有两种方案:
    (1)第一种
    修改localhost为Master.Hadoop
    (2)第二种
    去掉"localhost",加入Master机器的IP:192.168.1.141
为保险起见,启用第二种,因为万一忘记配置"/etc/hosts"局域网的DNS失效,这样就会出现意想不到的错误,但是一旦IP配对,网络畅通,就能通过IP找到相应主机。
(6)配置slaves文件(Master主机特有
    有两种方案:
    (1)第一种
    去掉"localhost",每行添加一个主机名,把剩余的Slave主机名都填上。
    例如:添加形式如下:
Slave1.Hadoop
Slave2.Hadoop
    (2)第二种
    去掉"localhost",加入集群中所有Slave机器的IP,也是每行一个。
    例如:添加形式如下
192.168.1.142
192.168.1.137
原因和添加"masters"文件一样,选择第二种方式。
现在在Master机器上的Hadoop配置就结束了,剩下的就是配置Slave机器上的Hadoop。
最简单的方法是将 Master上配置好的hadoop所在文件夹"/usr/hadoop"复制到所有的Slave的"/usr"目录下(实际上Slave机器上的slavers文件是不必要的, 复制了也没问题)。用下面命令格式进行。(备注:此时用户可以为普通用户也可以为root)   
scp -r /usr/hadoop root@服务器IP:/usr/
例如:从"Master.Hadoop"到"Slave1.Hadoop"复制配置Hadoop的文件。
scp -r /usr/hadoop root@Slave1.Hadoop:/usr/
以root用户进行复制,当然不管是用户root还是普通用户,虽然Master机器上的"/usr/hadoop"文件夹用户hadoop有权限,但是Slave1上的hadoop用户却没有"/usr"权限,所以没有创建文件夹的权限。所以无论是哪个用户进行拷贝,右面都是"root@机器 IP"格式。因为我们只是建立起了普通用户的SSH无密码连接,所以用root进行"scp"时,扔提示让你输入"Slave1.Hadoop" 服务器用户root的密码。
    查看"Slave1.Hadoop"服务器的"/usr"目录下是否已经存在"hadoop"文件夹,确认已经复制成功。查看结果如下:
从上图中知道,hadoop文件夹确实已经复制了,但是我们发现hadoop权限是root,所以我们现在要给"Slave1.Hadoop"服务器上的用户hadoop添加对"/usr/hadoop"读权限。
root用户登录"Slave1.Hadoop",执行下面命令。
chown -R hadoop:hadoop(用户名:用户组) hadoop(文件夹
接着在"Slave1 .Hadoop"上修改"/etc/profile"文件,将以下语句添加到末尾,并使其有效(source /etc/profile):
# set hadoop environment
export HADOOP_HOME=/usr/hadoop
export PATH=PATH:PATH:HADOOP_HOME/bin
如果不知道怎么设置,可以查看前面"Master.Hadoop"机器的"/etc/profile"文件的配置,到此为止在一台Slave机器上的Hadoop配置就结束了。剩下的事儿就是照葫芦画瓢把剩余的几台Slave机器进行部署Hadoop。
4.3 启动及验证
(1)格式化HDFS文件系统
在"Master.Hadoop"上使用普通用户hadoop进行操作。(备注:只需一次,下次启动不再需要格式化,只需 start-all.sh)
hadoop namenode -format
从上图中知道我们已经成功格式化了,但是美中不足就是出现了一个警告,从网上得知这个警告并不影响hadoop执行,但是也有办法解决,详情看后面的"常见问题FAQ"。
(2)启动hadoop
在启动前关闭集群中所有机器的防火墙,不然会出现datanode开后又自动关闭。使用下面命令启动。
start-all.sh
可以通过以下启动日志看出,首先启动namenode 接着启动datanode1,datanode2,…,然后启动secondarynamenode。再启动jobtracker,然后启动tasktracker1,tasktracker2,…。
启动 hadoop成功后,在 Master 中的 tmp 文件夹中生成了 dfs 文件夹,在Slave 中的 tmp 文件夹中均生成了 dfs 文件夹和 mapred 文件夹。
(3)验证hadoop
(1)验证方法一:用"jps"命令
在Master上用 java自带的小工具jps查看进程。
在Slave2上用jps查看进程。
如果在查看Slave机器中发现"DataNode"和"TaskTracker"没有起来时,先查看一下日志的,如果是"namespaceID"不一致问题,采用"常见问题FAQ6.2"进行解决,如果是"No route to host"问题,采用"常见问题FAQ6.3"进行解决。
(2)验证方式二:用"hadoop dfsadmin -report"
用这个命令可以查看Hadoop集群的状态。
4.4 网页查看集群
(1)访问"http://192.168.1.141:50030"
(2)访问"http://192.168.1.142:50070"
5、常见问题FAQ
5.1 关于 Warning: $HADOOP_HOME is deprecated.
hadoop安装完之后敲入hadoop命令时,是提示这个警告:
    Warning: $HADOOP_HOME is deprecated.
经查hadoop-1.1.2/bin/hadoop脚本和"hadoop-config.sh"脚本,发现脚本中对HADOOP_HOME的环境变量设置做了判断,其实根本不需要设置HADOOP_HOME环境变量。
解决方案一:编辑"/etc/profile"文件,去掉HADOOP_HOME的变量设定,重新输入hadoop fs命令,警告消失。
解决方案二:编辑"/etc/profile"文件,添加一个环境变量,之后警告消失:
    export HADOOP_HOME_WARN_SUPPRESS=1
5.2 解决"no datanode to stop"问题
当我停止Hadoop时发现如下信息:
    no datanode to stop
原因:每次namenode format会重新创建一个namenodeId,而tmp/dfs/data下包含了上次format下的id,namenode format清空了namenode下的数据,但是没有清空datanode下的数据,导致启动时失败,有两种解决方案:
第一种解决方案如下:
1)先删除"/usr/hadoop/tmp"
rm -rf /usr/hadoop/tmp
2)创建"/usr/hadoop/tmp"文件夹
mkdir /usr/hadoop/tmp
3)删除"/tmp"下以"hadoop"开头文件
rm -rf /tmp/hadoop*
4)重新格式化hadoop
hadoop namenode -format
5)启动hadoop
start-all.sh
使用第一种方案,有种不好处就是原来集群上的重要数据全没有了。假如说Hadoop集群已经运行了一段时间。建议采用第二种。
第二种方案如下:
1)修改每个Slave的namespaceID使其与Master的namespaceID一致。
   或者
2)修改Master的namespaceID使其与Slave的namespaceID一致。
该"namespaceID"位于"/usr/hadoop/tmp/dfs/name/current/VERSION"文件中,前面蓝色的可能根据实际情况变化,但后面红色一般是不变的。
例如:查看"Master"下的"VERSION"文件
本人建议采用第二种,这样方便快捷,而且还能防止误删。
5.3 Slave服务器中datanode启动后又自动关闭
查看日志发下如下错误。
    ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to ... failed on local exception: java.net.NoRouteToHostException: No route to host
解决方案是:关闭防火墙
5.4 从本地往hdfs文件系统上传文件
出现如下错误:
INFO hdfs.DFSClient: Exception in createBlockOutputStream java.io.IOException: Bad connect ack with firstBadLink
INFO hdfs.DFSClient: Abandoning block blk_-1300529705803292651_37023
WARN hdfs.DFSClient: DataStreamer Exception: java.io.IOException: Unable to create new block.
解决方案是:
1)关闭防火墙
2)禁用selinux
    编辑 "/etc/selinux/config"文件,设置"SELINUX=disabled"
5.5 安全模式导致的错误
出现如下错误:
org.apache.hadoop.dfs.SafeModeException: Cannot delete ..., Name node is in safe mode
在分布式文件系统启动的时候,开始的时候会有安全模式,当分布式文件系统处于安全模式的情况下,文件系统中的内容不允许修改也不允许删除,直到安全模式结束。安全模式主要是为了系统启动的时候检查各个DataNode上数据块的有效性,同时根据策略必要的复制或者删除部分数据块。运行期通过命令也可以进入安全模式。在实践过程中,系统启动的时候去修改和删除文件也会有安全模式不允许修改的出错提示,只需要等待一会儿即可。
解决方案是:关闭安全模式
hadoop dfsadmin -safemode leave
5.6 解决Exceeded MAX_FAILED_UNIQUE_FETCHES
出现错误如下:
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out
程序里面需要打开多个文件,进行分析,系统一般默认数量是1024,(用ulimit -a可以看到)对于正常使用是够了,但是对于程序来讲,就太少了。
解决方案是:修改2个文件。
1)"/etc/security/limits.conf"
    vi /etc/security/limits.conf
加上:
    soft nofile 102400
    hard nofile 409600
2)"/etc/pam.d/login"
    vim /etc/pam.d/login
添加:
    session required /lib/security/pam_limits.so
针对第一个问题我纠正下答案:
这是reduce预处理阶段shuffle时获取已完成的map的输出失败次数超过上限造成的,上限默认为5。引起此问题的方式可能会有很多种,比如网络连接不正常,连接超时,带宽较差以及端口阻塞等。通常框架内网络情况较好是不会出现此错误的。
5.7 解决"Too many fetch-failures"
出现这个问题主要是结点间的连通不够全面。
解决方案是:
1)检查"/etc/hosts"
要求本机ip 对应服务器名
要求要包含所有的服务器ip +服务器名
2)检查".ssh/authorized_keys"
要求包含所有服务器(包括其自身)的public key
5.8 处理速度特别的慢
出现map,但是reduce,而且反复出现"reduce=0%"。
解决方案如下:
结合解决方案5.7,然后修改"conf/hadoop-env.sh"中的"export HADOOP_HEAPSIZE=4000"
5.9 解决hadoop OutOfMemoryError问题
出现这种异常,明显是jvm内存不够得原因。
解决方案如下:要修改所有的datanode的jvm内存大小。
    Java –Xms 1024m -Xmx 4096m
一般jvm的最大内存使用应该为总内存大小的一半,我们使用的8G内存,所以设置为4096m,这一值可能依旧不是最优的值。
8 Y$ u! z; Y( v7 E. X  Y
 楼主| 发表于 2022-11-14 13:52:09 | 显示全部楼层
<?xml version="1.0"?>0 P- x: t; s* ?) ?1 b1 F3 w
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>7 e2 J; B7 N! y  b  T4 u0 f: v
<!--. d# A1 i* k0 z5 f
   Licensed to the Apache Software Foundation (ASF) under one or more
2 o9 N! Q. T/ S3 Q. R- m" }- S   contributor license agreements.  See the NOTICE file distributed with
; \( c) o: @" O% U   this work for additional information regarding copyright ownership.
4 m/ C! o, [4 f. I# H& [( i2 K   The ASF licenses this file to You under the Apache License, Version 2.0
6 X2 D- X( i: [; {: P   (the "License"); you may not use this file except in compliance with
( Y+ T9 j3 x( x% _   the License.  You may obtain a copy of the License at# y) g1 N+ X) D& \/ h7 p
       http://www.apache.org/licenses/LICENSE-2.05 }2 A6 a( W5 Z% I$ [: m+ U
   Unless required by applicable law or agreed to in writing, software4 `0 i/ d# p( K2 R2 J4 Q$ z4 d
   distributed under the License is distributed on an "AS IS" BASIS,
) L4 [. Y+ o) U' l$ V! r' B9 ~# R   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
7 G0 c) W6 v% y+ P, a& Y; U- j# W   See the License for the specific language governing permissions and
0 w( ?* }" x$ J- f* W9 \3 x+ |& k   limitations under the License.
! Q7 @6 Q4 |+ p' q# L2 ]-->  k" ]+ q1 [1 o* V- t6 b0 d
<!-- Do not modify this file directly.  Instead, copy entries that you -->
3 c+ l- p& `, ?$ d2 i  W3 i* v<!-- wish to modify from this file into core-site.xml and change them -->
3 z$ v4 G) ?" G. d6 u* D5 [3 D! |<!-- there.  If core-site.xml does not already exist, create it.      -->
* ~& u" \: Z" E" c8 }<configuration>
, ~  ^7 t$ M' o8 [<!--- global properties -->% X& C( f+ }8 I7 k" f3 k( H
<property>, e/ l" X1 r# B3 P7 }3 N1 e2 c' C
  <name>hadoop.common.configuration.version</name>. q5 ~: A- F- J( s3 [' X2 _) I
  <value>3.0.0</value>4 Y) G4 T( x# d- B- T) M1 P1 w
  <description>version of this configuration file</description>
! \% p$ i: N0 D4 E3 c0 [/ V</property>" A+ |' m( x( W6 U$ M% P3 `, _
<property>4 t7 B( M3 \3 x& q
  <name>hadoop.tmp.dir</name>1 m: K" P5 H2 r
  <value>/tmp/hadoop-${user.name}</value>& I. Y4 ]4 M, n8 D
  <description>A base for other temporary directories.</description>) k; Z" O" {& b  i$ n$ k0 ]
</property>
- {( O5 m9 V6 |7 [7 G<property>
  y3 j( C6 a8 v! H* Y  <name>hadoop.http.filter.initializers</name>; }6 k& O& l, ]& h) ]; q  [" d
  <value>org.apache.hadoop.http.lib.StaticUserWebFilter</value>
- i' _1 K8 Q( J# i. w  <description>A comma separated list of class names. Each class in the list
2 e5 K; p- n# G) R! N  must extend org.apache.hadoop.http.FilterInitializer. The corresponding0 y$ i0 `* w& R* N3 Q) R. G8 |; k
  Filter will be initialized. Then, the Filter will be applied to all user7 K- @4 n5 ^7 |7 B
  facing jsp and servlet web pages.  The ordering of the list defines the  ~  F* |2 q) c$ n2 ~* y
  ordering of the filters.</description># p  \& b7 C8 b( N, T( Q$ E0 a: E4 T9 }
</property>4 v3 E& Z7 O: A- A% h9 b
<!--- security properties -->
! }4 Q, R' U6 [% ~6 Y% S<property>
1 l5 Y& J0 v' h( J( q. G  <name>hadoop.security.authorization</name>! S+ ~2 P4 j# ^- d
  <value>false</value>' D+ ~7 o* e$ K% S# c" c$ n
  <description>Is service-level authorization enabled?</description>  A$ {* C  s5 S$ n$ K) \
</property>- i- m5 y) d! A+ s4 @, T9 n
<property>
, e$ C$ P! W! y( r0 s5 R  <name>hadoop.security.instrumentation.requires.admin</name>4 s& X0 ^% ]5 u0 `0 g7 c
  <value>false</value>
0 \  M& G  ?( Z) Q; V% k% f  <description>
# _- {" u* }% u    Indicates if administrator ACLs are required to access+ F5 [7 W- `8 k  T2 L! r5 O9 B  F
    instrumentation servlets (JMX, METRICS, CONF, STACKS).  M( |$ f$ s5 U' p' S& o+ g0 n
  </description>! r+ b) J* Z8 G( o
</property>
  G  n& a5 {' _) h# t/ ]8 F<property>
2 S% ?7 g. U' }, t  <name>hadoop.security.authentication</name>: Q4 h; X/ D4 m
  <value>simple</value>
) w' _9 x' u4 A  V. R; J  <description>Possible values are simple (no authentication), and kerberos+ K9 m7 f! E. y, f
  </description>. h3 \# ?4 ^* {+ F' P
</property>2 H% k9 _! l, X4 o' g' W' Q, e
<property>3 Q4 z  n: l! h' _" S
  <name>hadoop.security.group.mapping</name>
4 O7 v; A! p/ v# Z7 K  <value>org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback</value>0 k4 u$ \. F% h
  <description>
, k/ H8 v1 j1 M, J    Class for user to group mapping (get groups for a given user) for ACL.0 T+ Q' @* ~$ k, F* A
    The default implementation,* w/ |( X( X# l/ g: @! T1 |
    org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback,$ Z7 S- A7 _8 u; |3 M/ E
    will determine if the Java Native Interface (JNI) is available. If JNI is* ?0 }' Q, o+ ?# M7 ~
    available the implementation will use the API within hadoop to resolve a/ b, Y" g* m! V7 ?! b' t; B5 p
    list of groups for a user. If JNI is not available then the shell
& X6 C; V+ n" }9 _    implementation, ShellBasedUnixGroupsMapping, is used.  This implementation0 _4 V) p/ P5 \$ a" @1 I
    shells out to the Linux/Unix environment with the! ~) @4 M5 m9 ]8 X4 \
    <code>bash -c groups</code> command to resolve a list of groups for a user.
. z, `/ d6 E$ Z) |7 G: c  </description>9 _0 L* p$ c( o7 x5 c% F
</property>
2 c- A( M( ?5 f1 a<property>7 G6 l- [5 j1 x+ F, l. R
  <name>hadoop.security.dns.interface</name># I- P" Z& [  v" P$ i& g8 z, ~1 y5 p2 d
  <description>8 M1 s( ?* U! b3 K
    The name of the Network Interface from which the service should determine
' [4 m( C6 n3 z' @    its host name for Kerberos login. e.g. eth2. In a multi-homed environment,% d9 x) F' r  w; r- S1 ]& F+ F
    the setting can be used to affect the _HOST substitution in the service
( g' i7 o, c; E. \: p  G, Y% t' _    Kerberos principal. If this configuration value is not set, the service
* g2 g- y4 X% N9 ?# f    will use its default hostname as returned by5 }; H% i. Y4 S8 b( n0 l, ^
    InetAddress.getLocalHost().getCanonicalHostName().8 [  h9 g& |; o+ q6 n3 K6 b
    Most clusters will not require this setting.# E- A3 l2 M, M8 q$ _
  </description>" y. N% ~3 Y6 m' r+ J$ @9 M9 U1 p
</property>
" X1 ~0 Q) K: k<property>
6 S, p2 w' k" d, r8 n- R6 k, R  <name>hadoop.security.dns.nameserver</name>9 c' H  z. R: f2 N$ x0 N
  <description>: U9 x" u# V+ C: W' E6 y
    The host name or IP address of the name server (DNS) which a service Node: d. J$ C2 R( V9 M, a# n
    should use to determine its own host name for Kerberos Login. Requires! c5 ]# U3 g. J7 K; W" g# [; a
    hadoop.security.dns.interface." A" t2 n" t+ f/ g/ z
    Most clusters will not require this setting.# O. G. a1 e; S4 t9 M, }) ]9 A4 f
  </description>" G( m- ]0 B( ?% w" |7 p
</property>
# `) ]7 J  y4 ?<property># c# P/ w+ a' |2 i+ C
  <name>hadoop.security.dns.log-slow-lookups.enabled</name>
0 N. ]9 ^) I1 y& \7 Y1 e5 C0 r3 A  <value>false</value>
. `( n  G0 N. y7 D  <description>
* z; p$ ~$ m- G1 ^6 M7 K    Time name lookups (via SecurityUtil) and log them if they exceed the
6 g5 R6 X  Q0 y) ]$ y) w2 B    configured threshold.
( w9 u! M/ K( z  </description>7 ~, O) V* N( g- q
</property>
% X" r+ @( `. [% M<property>
. Q2 d5 a$ ?" X+ D6 t8 ]7 o  <name>hadoop.security.dns.log-slow-lookups.threshold.ms</name>
# t# y: H% _$ V' v( m  <value>1000</value>, G/ t: l! Q( P
  <description>/ _; o1 M- g5 z# g9 P
    If slow lookup logging is enabled, this threshold is used to decide if a
- ]3 S* \$ @5 e3 x( z/ }; x6 I0 c" K    lookup is considered slow enough to be logged.
3 t1 D# t5 V9 q6 F# ]: W& ~4 }$ g- ^  </description>
5 P+ F% N9 y  s4 P" N) {' O1 k</property>
7 K$ g" t9 \/ A6 f, R, t5 E0 b6 v2 [<property>& S1 y0 |! _# H! w* a) N( U
  <name>hadoop.security.groups.cache.secs</name>
; s, S2 G( J% i5 [6 o5 y  <value>300</value>
  u6 i3 F* H+ L. [( C" Q7 U6 _2 r  <description>* q8 y5 r3 l) H
    This is the config controlling the validity of the entries in the cache
2 N" g0 G8 U- W6 ?+ \0 X    containing the user->group mapping. When this duration has expired,
0 j: g$ S' m1 e) C2 n# f2 m  o/ R    then the implementation of the group mapping provider is invoked to get
' j7 ^' ~9 i( D8 ?" t    the groups of the user and then cached back.
+ r& ~% t+ F) H9 b  </description>" X) F! [+ ]( [
</property>, g  D, C. J& U- |$ Z$ ^0 e8 H
<property>
% ~: r3 }2 o" ^3 P: Y+ {# Y4 s# y  <name>hadoop.security.groups.negative-cache.secs</name>
- r% d0 E$ z" [& K' l  <value>30</value>9 Q( M" C, o$ g2 l8 @/ ]
  <description>) P7 S2 u) T; w, ^7 \
    Expiration time for entries in the the negative user-to-group mapping2 W1 [9 \2 z! v6 b' U) D. H
    caching, in seconds. This is useful when invalid users are retrying7 q5 Y  ^- V" e( l; O9 \0 E" q
    frequently. It is suggested to set a small value for this expiration, since# S% _) ~" e( ^7 c* h4 v
    a transient error in group lookup could temporarily lock out a legitimate. p3 ~3 ^) l& w' U2 o. M
    user.9 i0 ^) r% ^% h# q1 [" z
    Set this to zero or negative value to disable negative user-to-group caching.- Z9 i4 ^" A% f. F  @
  </description>  K9 o9 j. I  p3 t  s2 D5 c; o" N
</property>7 M7 b) @! d+ F
<property>2 T6 P9 q: r- x) r; Y7 y0 J
  <name>hadoop.security.groups.cache.warn.after.ms</name>3 A7 t! q9 |! {6 `
  <value>5000</value>
- g, M4 Y" J" E0 h2 B  <description>
; M2 J3 r! x% {5 L% x$ k    If looking up a single user to group takes longer than this amount of7 P+ o: e& l# T" e8 v0 ^
    milliseconds, we will log a warning message.
7 V/ ~2 @" L, U1 G' L; U: m  </description>; a, v. Y- ?" A) d. F- m. F
</property>
" u. s( Z( ], @8 h<property>
( {( @- r. i* \2 U  <name>hadoop.security.groups.cache.background.reload</name>
# L% u& }" @4 k% b9 S- {3 T  <value>false</value>
& m7 i/ G( M! ]( ]  <description>
3 ]. K$ s8 i4 D! p    Whether to reload expired user->group mappings using a background thread- q5 D  T: ^  P5 @# J
    pool. If set to true, a pool of
# |" X: _9 x* n7 N( w0 W    hadoop.security.groups.cache.background.reload.threads is created to
% k2 a; n5 d7 W0 n2 Q    update the cache in the background.- p5 y0 n& J2 K+ Z+ ^. L
  </description>7 n8 g! |0 Z6 v! K; h* G
</property>
! Y' _+ ?/ F' E- ]" t<property>. M( [( x3 X8 T! N
  <name>hadoop.security.groups.cache.background.reload.threads</name>4 w$ Y/ ^" E3 |) e+ P% q5 o, i
  <value>3</value>, R$ K( }0 }/ D+ a: A
  <description>
: B9 m0 ^" G$ e# {! L    Only relevant if hadoop.security.groups.cache.background.reload is true.  G: k9 L" N" C0 Q
    Controls the number of concurrent background user->group cache entry/ O1 z+ F& u( ^3 j' I8 k; F
    refreshes. Pending refresh requests beyond this value are queued and5 Y) c0 F  k# a/ I& x( y+ G! r5 F
    processed when a thread is free.+ p/ I- p8 H& N
  </description>( D! s' j; X7 `- F. I- c; O
</property>, |3 X: g: [& J
<property>+ |9 G# k7 U& X7 J
  <name>hadoop.security.groups.shell.command.timeout</name>
, [1 S) E7 z( d  <value>0s</value>
6 z9 |+ N  z& {- s0 `' U4 H7 q  <description>6 O, l% p$ E0 q- `
    Used by the ShellBasedUnixGroupsMapping class, this property controls how- }$ r, _8 o6 W
    long to wait for the underlying shell command that is run to fetch groups.# S& g. p$ P/ o  P4 {0 Y& N- w7 I
    Expressed in seconds (e.g. 10s, 1m, etc.), if the running command takes
/ y& Y1 Q4 S6 E    longer than the value configured, the command is aborted and the groups
; V- |2 J( j. o4 m    resolver would return a result of no groups found. A value of 0s (default)/ j$ `: r# g/ q! V
    would mean an infinite wait (i.e. wait until the command exits on its own).
# `0 N# B7 X1 P1 d/ s( ?# w  </description>
' ?* K$ f" b: m* }  \9 D' r  ^</property>" i& S2 P! r, H9 Z% ^# n
<property>
' y& E+ a  e* i0 A& ?8 C/ O  <name>hadoop.security.group.mapping.ldap.connection.timeout.ms</name># d! K/ i6 c0 E6 ?% o# y
  <value>60000</value>/ I& M7 c3 M; i+ f! y; a1 Y1 v
  <description># n8 ]) e* f9 a) V+ Z/ B; U
    This property is the connection timeout (in milliseconds) for LDAP* G: L" ^1 K3 l
    operations. If the LDAP provider doesn't establish a connection within the
/ Y) ~( x+ B: R' y/ U- G    specified period, it will abort the connect attempt. Non-positive value' i( v& _; x4 j0 D7 T  H1 d/ @1 [
    means no LDAP connection timeout is specified in which case it waits for the
4 Z) v, F9 _  T! Z& b' q4 S6 O    connection to establish until the underlying network times out.
+ J$ k& b6 {0 ~/ p! H. U! {  </description>
3 [/ ^& P$ Q& G6 O  d8 b</property>- X, C; o% Q5 Q5 d( T" y' }: a
<property>
7 V! _9 J3 X  X; Z8 N) a  <name>hadoop.security.group.mapping.ldap.read.timeout.ms</name>
4 G, |6 {; \, Y4 o  <value>60000</value>
0 K$ Z6 D, C8 y2 C, N% t  <description>
) g/ X% d7 j# n! j    This property is the read timeout (in milliseconds) for LDAP
7 j7 d. ~( X$ q9 G    operations. If the LDAP provider doesn't get a LDAP response within the
3 S: H7 d$ S1 q$ F5 H. V    specified period, it will abort the read attempt. Non-positive value  t: T  M) w6 k. S' |5 I
    means no read timeout is specified in which case it waits for the response" a$ x, |3 U% w- |) {8 B1 w; V8 a
    infinitely.5 _4 _1 Y! p4 v8 B. H
  </description>
( m3 e; Y- w& O& t# H- @: V6 v</property>7 L) Z, a8 L" X0 F1 U" t7 X3 l
<property>
. S' Y' K) i/ g" j3 u' ]; @  <name>hadoop.security.group.mapping.ldap.url</name>
" [: _4 E( g3 h! v' r+ G. M  <value></value>
: [; U2 r! a1 P( V" {& y! l  <description>
4 L  ]5 g0 H, d, L: _    The URL of the LDAP server to use for resolving user groups when using
$ J1 F1 X2 j) Y    the LdapGroupsMapping user to group mapping.
9 Y7 X6 G0 {; V8 C  </description>0 ?! U) k0 P& ?# k
</property>3 |, ]$ s8 ]1 |$ y2 j* F2 @
<property>
% u+ R0 a8 J$ J5 I  <name>hadoop.security.group.mapping.ldap.ssl</name>
5 i$ N7 _( y% j3 `  <value>false</value>
' k. o+ [' a  e; o  <description>
( p; W5 Q$ I' O" t2 P' Z    Whether or not to use SSL when connecting to the LDAP server., b/ K: d* k0 J. L
  </description>
6 Q, n- I- }  b0 E) d</property>
9 p3 E1 _3 G7 a& A$ ?<property>' O5 n9 i4 g7 C
  <name>hadoop.security.group.mapping.ldap.ssl.keystore</name>
6 `; {% h  ~' `& g# X, h, l  <value></value>0 u, c# Y4 k0 N6 r4 o2 A2 _
  <description>( s, x4 O, Z' V: V% m3 \; ?0 ^- q8 |
    File path to the SSL keystore that contains the SSL certificate required
# j0 B' x% a! s! L    by the LDAP server.
+ Q2 y1 R5 {$ w* u& ?1 q3 m3 [1 [  </description>
+ W$ s: `1 f0 F( z; {3 h+ Z* }</property>
- ?! l7 z+ K# Q! h, W<property>( L4 k/ v! }) C9 u  }7 i7 ]
  <name>hadoop.security.group.mapping.ldap.ssl.keystore.password.file</name>: F( |, R3 I" `- S& x6 `' t! N
  <value></value>
$ B, ^- J: u% W) f3 w) ]  <description>
# \# Z" R: X* s" a# Z    The path to a file containing the password of the LDAP SSL keystore. If* a* X  N8 V9 y1 D
    the password is not configured in credential providers and the property
# A/ z* [% B7 [2 {3 j    hadoop.security.group.mapping.ldap.ssl.keystore.password is not set,
& K! O3 ^5 w4 [4 A/ z& v    LDAPGroupsMapping reads password from the file.* N$ |) s( R1 y' ?+ N# Q
    IMPORTANT: This file should be readable only by the Unix user running/ Q% ^6 S' F- O6 a
    the daemons and should be a local file.
2 j$ h' I2 Q1 q  </description>
. F9 N# D3 g2 i, {% l</property>
, l6 F# e5 k* i<property>
9 y/ n* z  Y" `1 W) \1 S  <name>hadoop.security.group.mapping.ldap.ssl.keystore.password</name>
( f8 C$ k4 T7 R. |) A  <value></value>
  L8 T9 `3 R# O/ u- R+ @. N& e  <description>2 f' f7 }, v# b8 G% G
    The password of the LDAP SSL keystore. this property name is used as an
. s5 f9 {6 ~3 ]* t! C  e6 m( |    alias to get the password from credential providers. If the password can
2 s2 U4 c% c4 p7 n6 ]1 D( S    not be found and hadoop.security.credential.clear-text-fallback is true' ?- \+ A; G0 D- t$ V* C
    LDAPGroupsMapping uses the value of this property for password.
6 N+ x. J. A( X' ^3 i3 T# H  </description>/ m: g& H0 V' F; e/ d% I: _
</property>. c& y9 J) Y8 u3 L
<property>
# }# B5 K/ C! Q3 w% F  <name>hadoop.security.group.mapping.ldap.conversion.rule</name>
2 C/ b; H+ ]8 B  <value>none</value>8 ]" \9 ~0 B- T
  <description>* L; f. Q6 o. _0 n
    The rule is applied on the group names received from LDAP when
5 o4 X( q7 u) i    RuleBasedLdapGroupsMapping is configured.
: j. H/ m3 l9 q/ p+ M- a    Supported rules are "to_upper", "to_lower" and "none"." {5 D8 d  s. @/ V) S3 y
    to_upper: This will convert all the group names to uppercase.
. J$ p6 G1 z& L9 b; h4 f- S    to_lower: This will convert all the group names to lowercase.
: p3 ~6 N# v* v8 \# p# n    none: This will retain the source formatting, this is default value.( T6 M5 ]6 ^' Q% j0 n: F
  </description>
' Z  E( J! A4 o; t. E</property>. f( O! K6 `/ [7 M9 ~
<property>7 j9 Z. f6 p0 f& ~
  <name>hadoop.security.credential.clear-text-fallback</name>! T& B: U; f7 B+ M; O
  <value>true</value>
; m; Y. H) u: o2 L/ T0 W  <description>9 \# L1 m' i2 c- a3 |$ R: A; p
    true or false to indicate whether or not to fall back to storing credential- U4 E" U3 X1 _: S6 {6 d
    password as clear text. The default value is true. This property only works2 {1 {/ U+ |! M
    when the password can't not be found from credential providers.
! e$ q' o) |' P  </description>
8 Y. T  \$ R7 z; g# ~) A- B% N</property>: j3 I5 W7 ]3 ~9 C4 C/ [. h- |7 K
<property>
7 _2 G& _- V" f  <name>hadoop.security.credential.provider.path</name>0 b6 D3 A9 i! L% k+ t
  <value></value>! ]9 T: Y! d) |! n. ]
  <description>! Z" |# {. Y0 [# a1 p4 V
    A comma-separated list of URLs that indicates the type and
  D9 C) l  V3 }' J% ~% G9 n    location of a list of providers that should be consulted.4 c+ _9 s2 k$ Y% i  x5 d
  </description>
/ o. u4 d# u4 @</property>
, B% |* w( \1 L& v( Y<property>
) `2 l  [# i. h9 V; g  <name>hadoop.security.credstore.java-keystore-provider.password-file</name>
+ A: d0 j; X/ {+ }- s6 H+ G3 }9 K2 D( E  <value></value>
) G# i. Y- |; {) y2 `* T$ v  <description>* k& [, }2 K6 ^0 M; a3 S+ O
    The path to a file containing the custom password for all keystores
5 [: Z6 n) p1 P1 M! t    that may be configured in the provider path.
/ F1 I5 C+ X: `! X! _9 J  M  </description>( A1 I- d) }- U  X1 M7 m3 q, A  }. M
</property>/ s$ W/ Z( c4 [3 r7 I) C9 d
<property>' t% L  X5 s. n( w
  <name>hadoop.security.group.mapping.ldap.ssl.truststore</name>+ j: R$ m# [* C$ w: f7 h5 C
  <value></value>
+ W- _4 f- G( X3 C  <description>* B2 A) K6 l0 o; W& b! S
    File path to the SSL truststore that contains the root certificate used to) T/ V7 [& H! |; M, k+ A( d$ V. ~7 _6 k
    sign the LDAP server's certificate. Specify this if the LDAP server's
7 C- X8 y8 D3 d9 Z, e) K    certificate is not signed by a well known certificate authority.
" E/ Q  @$ ^% U. U! h- Y* N6 A  </description># q) M* Q, [0 Y% `. f# K
</property>
% a+ p1 V$ z: J7 d6 t<property>
5 ]& |5 X: U* x7 G1 W/ ~. y/ B  <name>hadoop.security.group.mapping.ldap.ssl.truststore.password.file</name>
' A4 `' ]# ?. m  <value></value>" K7 q6 ~  t" L' ]$ q
  <description>
$ e! H1 w/ ]2 e- J/ K    The path to a file containing the password of the LDAP SSL truststore.
& G* O7 z$ t) |+ B& b8 H: S4 Z    IMPORTANT: This file should be readable only by the Unix user running8 w/ D$ X# s- E* u
    the daemons.
; y$ c5 {- M& J& j. ]  </description>, [* F4 |( {1 B' A" `0 @9 M
</property>
) e: y6 _3 p8 v! c<property>
! Q' Z# c* Z* Y( T2 O5 g1 a7 E$ k  <name>hadoop.security.group.mapping.ldap.bind.user</name>5 z) ]) f2 d# Z
  <value></value>
  |$ _: z# L) p! m( U* a  <description>" K) k8 e  U2 O, K9 `
    The distinguished name of the user to bind as when connecting to the LDAP3 v' d: w  L1 N' |) S1 E
    server. This may be left blank if the LDAP server supports anonymous binds.
- L7 b# u$ u: X/ ^- j  </description>0 H* f# _+ ^+ R- k: `
</property>* m/ R/ ~+ n8 J" F
<property>
- G% L6 T5 M: H/ m  <name>hadoop.security.group.mapping.ldap.bind.password.file</name>
% G- a3 ]% e# R9 \0 M# R3 @  <value></value>
# n% Z# }1 M  \' M$ S8 Y4 w  <description>
; p0 C1 o! @; p0 t) B! v- `! J: X$ J    The path to a file containing the password of the bind user. If
; N  A7 @) c& V* I5 a, |" I    the password is not configured in credential providers and the property
) Q1 @, I: Q2 S" B    hadoop.security.group.mapping.ldap.bind.password is not set,2 I: `3 E& D' Z, L6 b# T4 q
    LDAPGroupsMapping reads password from the file.
8 M4 Q# j# t% R    IMPORTANT: This file should be readable only by the Unix user running; v* U5 k" f9 ~$ Z
    the daemons and should be a local file.
" c, L# N$ n! z  </description>& ~# X8 z, d2 s  [8 j
</property>0 x2 h' \+ }1 K
<property>) z9 ], a$ n* n- b. W7 I! M4 A
  <name>hadoop.security.group.mapping.ldap.bind.password</name>; L5 C+ R" w2 S. h# y/ p
  <value></value>
9 A' b/ D) d9 }& U  <description>' ^+ k1 p2 E% |- J( ]/ n
    The password of the bind user. this property name is used as an
4 [' R4 D( L! W    alias to get the password from credential providers. If the password can
" R5 z5 e$ q6 _    not be found and hadoop.security.credential.clear-text-fallback is true
4 q! ]- Z$ D- a7 r" n* `9 ]7 h    LDAPGroupsMapping uses the value of this property for password.2 s, J! R8 l0 `: F# U
  </description>
6 e) k0 c* T9 e& [4 E- G% X$ h</property>
6 M+ d1 d2 N3 {8 }8 \1 R* n* I: Q<property>3 v# \. s, U2 ~/ T; V: J! e1 Q
  <name>hadoop.security.group.mapping.ldap.base</name>
. a1 t5 e, M  d( G8 l  <value></value>
1 w5 q* L/ a7 u  <description>. f' i6 b; B# ^$ C
    The search base for the LDAP connection. This is a distinguished name,5 |7 ^6 w- F5 f2 C  Y- u/ b
    and will typically be the root of the LDAP directory.7 [  Q/ A4 C1 q% |/ [. `" {/ h& i! e4 b
  </description>
* }; Y. Y9 r6 {7 m" n: z</property>
% K6 P: J4 t7 P<property>; ]6 [0 M3 X: t
  <name>hadoop.security.group.mapping.ldap.userbase</name>: B1 u" _& T1 A$ F& P6 m. `% }
  <value></value>
! [7 N3 Y2 A' g, R$ p7 d+ f7 K  <description>6 N+ h6 V' Y) m7 t8 v9 t) \
    The search base for the LDAP connection for user search query. This is a7 r  y6 V$ T, v2 o% @3 t# v
    distinguished name, and its the root of the LDAP directory for users.# N3 q$ |; z3 j5 d5 U
    If not set, hadoop.security.group.mapping.ldap.base is used.
, N- T* v) u" v  </description>; l8 D3 @9 P! s& t+ n3 I
</property>
$ t. ^* f6 j) j+ i<property>
1 }% S, L; c! ?! \  <name>hadoop.security.group.mapping.ldap.groupbase</name>
* F8 g% x3 I# I3 U+ \2 T0 w  <value></value>7 d; B2 S! h. I; J
  <description>
4 ?8 l( V# A6 O$ t7 ~5 u$ `$ _    The search base for the LDAP connection for group search . This is a2 v* J3 C: A3 o" t7 z5 X2 `$ \" M
    distinguished name, and its the root of the LDAP directory for groups.( k4 i# J; f4 f# W4 p
    If not set, hadoop.security.group.mapping.ldap.base is used.
! g; y; v- d& T  </description>
! i# k! [: V( ^9 c6 c</property>
2 ?) x) L; ^& [- I8 i/ O, F<property>
( y! Z4 G  q6 s! Q' E  <name>hadoop.security.group.mapping.ldap.search.filter.user</name>
: a, t/ n% F7 }' I1 [2 b  <value>(&(objectClass=user)(sAMAccountName={0}))</value>4 m' G) g2 P' g9 x+ a+ ?8 ^0 k" ^
  <description>8 A" q4 h6 Z  a. e' G) `' W
    An additional filter to use when searching for LDAP users. The default will% L# A6 ~* }  X$ _: J5 C2 K# [
    usually be appropriate for Active Directory installations. If connecting to
! y4 n% Q3 }. j    an LDAP server with a non-AD schema, this should be replaced with1 Y3 q8 N( a: O- B
    (&(objectClass=inetOrgPerson)(uid={0}). {0} is a special string used to1 O2 [, A. H+ G: M: ?  L
    denote where the username fits into the filter.* Q) n! z0 ]  o' Y( c
    If the LDAP server supports posixGroups, Hadoop can enable the feature by
2 `6 z. |! n% ~    setting the value of this property to "posixAccount" and the value of1 D- W2 v$ S4 Y; \+ d  q4 W
    the hadoop.security.group.mapping.ldap.search.filter.group property to
+ @" H( V/ P7 G    "posixGroup".
3 E) _( [; Y. J0 ^  </description>
1 b0 F: b. y" Q' h) B</property>
: g0 i' b- P/ P# z+ u2 D<property>
! ~8 t0 s7 i* X% g5 I1 p  <name>hadoop.security.group.mapping.ldap.search.filter.group</name>
1 h9 L' a) Z, m% N" j( v5 [  <value>(objectClass=group)</value>( ?- e5 U' w4 h6 |
  <description>* J* b$ Y& v1 c( L
    An additional filter to use when searching for LDAP groups. This should be% j0 E6 R) F4 x4 d' S; k
    changed when resolving groups against a non-Active Directory installation.
% _* D& U" n+ [* M3 q2 b    See the description of hadoop.security.group.mapping.ldap.search.filter.user0 I8 g" r# @& T9 }) t7 U( N' \
    to enable posixGroups support.
! d9 R  }& v$ y  </description>* k/ H$ W  m- b! V) `3 c
</property>
) L8 o  c( I% N# d% ^- e<property>; |! L7 \6 v9 ~9 t7 J
    <name>hadoop.security.group.mapping.ldap.search.attr.memberof</name>
% A/ i7 _+ V* B, G: L+ d    <value></value>
5 f8 u; N/ m8 U5 t    <description>
! `1 q) q1 c/ u/ T  ~+ e' k      The attribute of the user object that identifies its group objects. By8 b, E, g: B+ ]
      default, Hadoop makes two LDAP queries per user if this value is empty. If' ]* d* C" S& {+ ?7 w0 l5 ^
      set, Hadoop will attempt to resolve group names from this attribute,
& s$ W% u/ e2 z4 O8 w      instead of making the second LDAP query to get group objects. The value% u* E% {" l: X. y2 J1 j4 d* P
      should be 'memberOf' for an MS AD installation.
( E! T* _  u5 s2 r, F% v, s    </description>- N* r  m7 J. M$ W4 x
</property>; F5 W# J; s3 a1 C, V: _( a- k' M
<property>
2 [8 E+ Y. i0 r; w  <name>hadoop.security.group.mapping.ldap.search.attr.member</name>
8 X1 f+ g1 W( b  <value>member</value>
' }  ~( u$ }) x. n5 }  <description>: ?- ^4 v0 ^, \6 `& j: H. I
    The attribute of the group object that identifies the users that are
5 _6 {+ {) u8 [    members of the group. The default will usually be appropriate for
' \1 \: S. q6 c' p6 z* ], a  G    any LDAP installation.
/ X& j% q+ h8 q2 I# L4 _  </description>7 g- [4 ~9 S* t
</property>- _! ?: ]; G* O; S3 Q$ s6 s
<property>6 l) q. N* e: @2 l/ p
  <name>hadoop.security.group.mapping.ldap.search.attr.group.name</name>
9 y$ p  U% ^) z- X9 A  <value>cn</value>9 H/ P3 {8 g0 E' W' o4 V
  <description>
( `! R4 j6 B, V" N5 @* ]8 N    The attribute of the group object that identifies the group name. The% Z/ D4 L: o) k: O7 g+ D5 U  ~: ?
    default will usually be appropriate for all LDAP systems.
0 c, z7 z$ f$ d" h$ w4 V  </description>8 u4 @6 J( K' s, g$ x! y3 N
</property>4 x0 [/ H5 N4 v7 g9 L1 x
<property>9 ^8 @7 ]' _* |; t4 p
  <name>hadoop.security.group.mapping.ldap.search.group.hierarchy.levels</name>, r- J  ]) L3 g* m( Q9 s+ `
  <value>0</value>8 q$ ^* `2 M  c" ]( ]
  <description>
( g- ]/ i1 C' U8 ]7 t1 _# |# y    The number of levels to go up the group hierarchy when determining( b' M+ d- q6 C: n1 Q4 r
    which groups a user is part of. 0 Will represent checking just the
  p8 V3 |/ F2 t% g1 q: T. {. |    group that the user belongs to.  Each additional level will raise the" I8 ~" g; ?! d" y/ I+ I- j4 n
    time it takes to execute a query by at most
: W; o; D) a, e2 g: t    hadoop.security.group.mapping.ldap.directory.search.timeout.
- R. ], p( [5 B2 }" n' i. j; I    The default will usually be appropriate for all LDAP systems.; Z7 [( Q: Q. J  Q9 \& l. d
  </description>
" H1 m% I2 }: V4 O# f1 z</property>
! \3 A- I, T6 G<property>
! z( s+ T; G( X: y8 f/ o  <name>hadoop.security.group.mapping.ldap.posix.attr.uid.name</name>
9 [* B' E+ [+ J% r/ N  <value>uidNumber</value>
+ c' c6 H( p  W  @, H  <description>
" `# r  w  [1 E1 F# Y    The attribute of posixAccount to use when groups for membership.
. y6 t8 g' ]: ]5 V9 b  Y. n( c    Mostly useful for schemas wherein groups have memberUids that use an, i% [) W9 J1 d4 o* Y
    attribute other than uidNumber.
5 G% n3 B: `/ s. ^6 t5 z1 E1 ~3 U& S  </description>
# |/ x4 n& F1 t* e4 Y</property>
7 I5 y& X$ Y  A) G5 r6 ]<property>6 _2 K* `* ]1 y* y) D8 Z
  <name>hadoop.security.group.mapping.ldap.posix.attr.gid.name</name>7 i. i+ O2 ]0 a
  <value>gidNumber</value>
4 A+ {! }5 w# B8 {( `5 Y4 G  <description>" B) k9 ~0 x2 x$ @  o4 b; U2 o
    The attribute of posixAccount indicating the group id.
- h, J5 }) q; m! S6 S0 x# Y0 X  </description>
7 Y4 s  u' v3 Q</property>/ S5 Y3 X7 C. |1 b. e" y
<property>
' ^7 D* Y" t& b8 c. _( O  <name>hadoop.security.group.mapping.ldap.directory.search.timeout</name>
; F/ M% U7 B# I1 b! i$ x% M  <value>10000</value>
) b8 r8 u& _; \, K  <description>
. X4 z3 ~) o* X- C3 ?3 @    The attribute applied to the LDAP SearchControl properties to set a
  `% B  s( ^! T4 l0 ~+ {+ [    maximum time limit when searching and awaiting a result.
# o) p- n) x. L! k3 d    Set to 0 if infinite wait period is desired.3 A( o& e' r+ d0 [7 ~5 `! h: {
    Default is 10 seconds. Units in milliseconds.
5 c) U2 y' K% ]  \8 r/ f3 k9 }! y  </description>8 b; _% g! c8 K! s; k* J9 @1 `
</property>
1 U% \0 S3 Q5 d<property>
( k5 T6 }3 ?0 r/ {$ G; d& O  <name>hadoop.security.group.mapping.providers</name>
( \2 G) U& K7 x% b& t# e  <value></value>
& \' r4 l% {* C% R  <description>
" e/ ]& F( Y3 F8 z' N8 T& K- G    Comma separated of names of other providers to provide user to group
+ t" g# D4 Q0 p, F8 ]    mapping. Used by CompositeGroupsMapping.
* j0 o7 j( u+ R+ N$ X' d- S; S  </description>
9 g0 g. Z) q5 a% l4 a' F</property>+ _- u9 `/ ]3 a
<property>2 u% o' c0 G/ m
  <name>hadoop.security.group.mapping.providers.combined</name>
0 q# p6 w/ W& }8 {0 y, d) n  <value>true</value>
8 C- w0 P/ r' L- j9 U: ], W  <description>0 {! ]# V! F8 ]. L6 ~
    true or false to indicate whether groups from the providers are combined or: j! t7 e9 b. D- c$ [
    not. The default value is true. If true, then all the providers will be
% b" E) _1 c& t# O  S( y- ~    tried to get groups and all the groups are combined to return as the final
2 d# P; p( B8 n5 M: R/ V    results. Otherwise, providers are tried one by one in the configured list( [& ]* n5 Z) O$ B! F
    order, and if any groups are retrieved from any provider, then the groups# _+ u0 Q1 l4 P4 `- Z  C
    will be returned without trying the left ones.7 c6 G, w. k) |  Z' l6 C
  </description>
1 L! M: m- O7 K4 g$ D7 |; x</property>
/ V$ U/ f* X1 n. V7 _<property>0 b& T3 E# D" Q1 S# Z0 ?$ U
  <name>hadoop.security.service.user.name.key</name>' c  J& d+ f6 o  f
  <value></value>! @' D# }+ T/ S
  <description>7 Z5 T1 R5 w6 x4 s2 z2 R- ]4 T
    For those cases where the same RPC protocol is implemented by multiple/ _$ Z( \" r4 a8 j/ x. g8 O
    servers, this configuration is required for specifying the principal
* ]  P' y% X8 x    name to use for the service when the client wishes to make an RPC call.3 @1 L) b3 u' \; ^2 ^
  </description>! w+ T6 Q. t( O4 F; I9 ~  t9 Y2 h% |
</property>& I" i% F! G2 j( P; W9 E$ B
  <property>8 ^+ r0 D) r2 ~" w
    <name>fs.azure.user.agent.prefix</name>8 d9 j" R" v! G4 h
    <value>unknown</value>
6 L, U. y: ]  Y( O6 c$ `    <description>
% y! N  ^; x. E/ q/ H. u3 \. M      WASB passes User-Agent header to the Azure back-end. The default value
! t. _! F9 A- f) t      contains WASB version, Java Runtime version, Azure Client library version,: n2 E4 w7 Z$ ~: S' s
      and the value of the configuration option fs.azure.user.agent.prefix.! t3 v; x/ ^- S) n/ p2 I/ b$ Z
    </description>* O9 P( w5 A- X- q7 ^6 a: |
  </property>
6 G4 Z! Y. P2 s& ^, \# c- c+ G<property>
6 |4 ]+ [, m+ |" _! ]) f    <name>hadoop.security.uid.cache.secs</name>9 {3 @9 w: a+ z- y; e" x: w# N. @
    <value>14400</value>
" u5 G0 H6 P! e: T7 ~1 L! z    <description>
- ?# U$ o- _9 F6 U- O        This is the config controlling the validity of the entries in the cache# q" K% E) \% c3 S
        containing the userId to userName and groupId to groupName used by
9 J4 R  H+ o, j% }1 j/ L        NativeIO getFstat().6 h% @' a! V, M3 ~. |7 X9 f  i
    </description>- R1 }1 k  ~+ w  M2 A
</property>( u3 {  }+ r$ U2 {" a* I
<property>
2 h$ G8 n" ?: L7 u) {- O: H8 S: B  <name>hadoop.rpc.protection</name>
0 y/ ]( m; X' z1 P  l9 j  <value>authentication</value># h  M8 l7 X1 {; H
  <description>A comma-separated list of protection values for secured sasl
6 @$ p+ f5 }0 m2 Q      connections. Possible values are authentication, integrity and privacy.* r  F( Z: V. z# N" V) O# i+ G3 `
      authentication means authentication only and no integrity or privacy;
- \: t* |, l5 E$ y      integrity implies authentication and integrity are enabled; and privacy
/ c! I% o: r$ W# w; r" H4 t      implies all of authentication, integrity and privacy are enabled.' C3 W4 M! `  X& u  x  K2 [' [
      hadoop.security.saslproperties.resolver.class can be used to override
# h' ~6 u- i% G+ F  M! V6 h$ c; |; r      the hadoop.rpc.protection for a connection at the server side.
* ]4 c" h! ~  x  e' ?, E  </description>
  O% G6 \( m, Y' z2 C</property>
0 T0 S3 M! M1 b7 n<property>; r( u4 J% Z8 K2 Y+ s% H! ~4 w
  <name>hadoop.security.saslproperties.resolver.class</name>- ]+ w0 m1 o# E  k. r7 y
  <value></value>
/ Y* I/ o  @( o, O5 Q  <description>SaslPropertiesResolver used to resolve the QOP used for a
/ W4 K* h: ]! B* w" p& }3 m( ^. b      connection. If not specified, the full set of values specified in
! @& H: d! O' f" W8 k6 |" Y5 I1 I      hadoop.rpc.protection is used while determining the QOP used for the
4 g5 p, L% B, S      connection. If a class is specified, then the QOP values returned by
/ i6 X+ l; C4 x      the class will be used while determining the QOP used for the connection.) Y% [# p0 k  Q$ ]) p2 I
  </description>; w( m7 c: h$ p1 I4 B
</property>: c7 l5 K3 w( Y0 j3 t
<property>
! w0 c& H% l7 M  <name>hadoop.security.sensitive-config-keys</name>; {9 P: S" |: j9 z1 l+ k8 j" A/ u
  <value>
4 h/ B- w& H4 G7 N      secret$5 D1 p& N! k! H- e
      password$# S3 e# N6 D3 O8 h9 r; ]
      ssl.keystore.pass$
* v& I2 W( n, a; P* Y$ X6 N# E9 |      fs.s3.*[Ss]ecret.?[Kk]ey
1 F# U! z- G8 |4 Y      fs.s3a.*.server-side-encryption.key
. O% k9 N9 H2 k4 l" j" `      fs.azure.account.key.*+ `9 p" T# e% A3 J- w% E* N
      credential$
% W9 k; s" y# T* _- V  ]      oauth.*token$
" ]) h% @5 T/ ]) y- C* x' F      hadoop.security.sensitive-config-keys
4 e8 h" {) q* i0 D( m" R! A  </value>8 e5 T8 i1 S7 H; F: r8 y
  <description>A comma-separated or multi-line list of regular expressions to& {3 E" I1 D7 [* A9 p2 }
      match configuration keys that should be redacted where appropriate, for! K" i! `3 X6 O/ a1 K
      example, when logging modified properties during a reconfiguration,
; S: u) Q; p. j. ~1 x+ ], j      private credentials should not be logged.1 O* N; p5 ^# o$ s9 a% {" s+ {
  </description>
( E' @; t! }9 K& p. U</property>& s' l: U' `5 w3 o
<property>
/ {. `% x3 H! P7 s  <name>hadoop.workaround.non.threadsafe.getpwuid</name>4 S+ f; E5 w& }( b
  <value>true</value>
3 H3 [. l3 e) ~, _) L* r# o- C  <description>Some operating systems or authentication modules are known to9 d# r. W2 e- G6 W4 g
  have broken implementations of getpwuid_r and getpwgid_r, such that these
( |) S; g2 q; C8 t, V  calls are not thread-safe. Symptoms of this problem include JVM crashes
  I: r, w4 U# z+ j- `+ ]  with a stack trace inside these functions. If your system exhibits this
* X; K2 F0 w; _+ g+ L  issue, enable this configuration parameter to include a lock around the7 f0 V( j! h3 G, e* e$ M$ H+ c& k
  calls as a workaround., e) r1 Z1 |: I2 ^% i
  An incomplete list of some systems known to have this issue is available' I. [8 x! n' `7 a/ s& p
  at http://wiki.apache.org/hadoop/KnownBrokenPwuidImplementations: h8 r9 E. K& `5 q
  </description># v  v4 b% o! Q% _' u8 \2 R
</property>
% u7 B  G. w8 |, d+ _" p<property>7 _0 E) {4 \1 ~% s! |1 I; t% S( J
  <name>hadoop.kerberos.kinit.command</name>
9 `; @4 g7 k+ V: X% y  <value>kinit</value>
9 b: i" C! P) J1 k7 ^/ j' Q& k  <description>Used to periodically renew Kerberos credentials when provided% C% ?0 Z4 s' c& h9 Q
  to Hadoop. The default setting assumes that kinit is in the PATH of users
& Y: `& _1 l' D5 y- r! @) k  running the Hadoop client. Change this to the absolute path to kinit if this" u' b9 h) e% _2 L& Z
  is not the case.
9 \, C9 f( X! S* W  `  v  </description>
$ P8 l1 ?+ x% k2 z: H</property>/ c3 f8 n7 {! R4 I3 w4 k
<property>" |4 r: p; d: l6 [
    <name>hadoop.kerberos.min.seconds.before.relogin</name>! N2 _- E+ R# Q7 F. p$ f/ M- ~
    <value>60</value># ?1 @5 t4 p: B3 ]7 z1 M
    <description>The minimum time between relogin attempts for Kerberos, in
2 p/ c5 M! \8 @0 a" b$ \* X' u" p    seconds.
0 Y  F# R. I+ R+ V- E9 v+ C' [$ M! ]    </description>
' F6 [; ^( u- y" C1 ]$ M5 ~</property>) {5 x  T2 x  s" k% A6 P/ y. q6 r
<property>! Q& D. ~; V; K8 r
  <name>hadoop.security.auth_to_local</name>$ E8 c8 E- i- E% b  x9 M9 Y" o- k
  <value></value>
' T" ?; `$ W* q- x  <description>Maps kerberos principals to local user names</description>: ]$ X( J7 ?- c& |4 ?% h: H$ v& P! p
</property>
  g0 K1 `' s8 z$ I# g) N4 R, {6 x2 r* l<property>5 e% \# @. D% ?- ~( ?, K& [
  <name>hadoop.token.files</name>
' n) g6 ~' E2 M  <value></value>
: S! |! {+ _, e/ F: h  <description>List of token cache files that have delegation tokens for hadoop service</description>
) T1 {0 y9 z5 D# Y3 _</property>
% K" h$ h5 A( A<!-- i/o properties -->& N% b* ?4 R3 b- {! N9 b
<property>
( M3 U- x$ I/ }3 I5 Z% l- Q# Y  <name>io.file.buffer.size</name>
; |4 N$ t# y& t% p; l: D9 a  <value>4096</value>
* K! ^( Q* o' e  b  <description>The size of buffer for use in sequence files.) f) ~: i& u6 s- X
  The size of this buffer should probably be a multiple of hardware4 B8 i4 u, F* V  ?; @. e
  page size (4096 on Intel x86), and it determines how much data is
3 [/ |- C( h. [5 p. m  buffered during read and write operations.</description>- `  {3 |8 Q" ?* W5 j3 F
</property>
  w; q* C3 B! |  a6 I0 a! J<property>7 t" N9 v: _8 b
  <name>io.bytes.per.checksum</name>
! P6 m1 I( J& c3 q2 r  <value>512</value>; f: V, y% d% e8 N" s5 Y
  <description>The number of bytes per checksum.  Must not be larger than' f! k) Y! B$ m$ d0 F7 g8 _
  io.file.buffer.size.</description>* n( x: |; S/ o  F$ D
</property>" t2 ]2 R2 R9 _5 u
<property>
$ C" K, M" A: n: ?! w. ?1 e  <name>io.skip.checksum.errors</name>$ v7 |$ x) B4 U' T, V7 x
  <value>false</value>
1 ]" q, [( k- [' h( E  <description>If true, when a checksum error is encountered while7 F: x3 D; U/ E1 j
  reading a sequence file, entries are skipped, instead of throwing an& f9 v9 g. Y: G
  exception.</description>
! j- K( e& P) k</property>, |; N5 Y1 h6 ?  l
<property>$ H" U" f0 m# I" u( {
  <name>io.compression.codecs</name>$ k: _4 Q6 T+ F
  <value></value>
4 }0 c$ t% Z; ^4 M; {5 R: X  <description>A comma-separated list of the compression codec classes that can; h+ o9 Y% k% A$ X) G- o9 n
  be used for compression/decompression. In addition to any classes specified
0 S) G8 q, E% @, S0 w& l1 W2 i  y  with this property (which take precedence), codec classes on the classpath
9 D# U/ B9 h% s$ {  are discovered using a Java ServiceLoader.</description>; u, g* \0 U+ O' u
</property>6 j' V' r, e$ j, W
<property>4 Y# K; f  z1 [
  <name>io.compression.codec.bzip2.library</name>, c+ h: Q; N7 [9 p
  <value>system-native</value>' H4 {* }1 u5 Y" N2 u
  <description>The native-code library to be used for compression and# I* o" g3 r5 e- G  d$ X
  decompression by the bzip2 codec.  This library could be specified/ a; N1 X9 _' e* Z; \' G
  either by by name or the full pathname.  In the former case, the; j4 P! ^) I/ w6 ]" n$ Z; G$ f
  library is located by the dynamic linker, usually searching the/ S% C. i  |. R# o. j; U6 J
  directories specified in the environment variable LD_LIBRARY_PATH.
/ C4 z9 l# u' K+ v1 W  The value of "system-native" indicates that the default system- n& g$ z* N. I2 n* ]6 n( [
  library should be used.  To indicate that the algorithm should
- P* ^0 W2 M% c& z  operate entirely in Java, specify "java-builtin".</description>
9 y: y$ l9 k7 Z8 L% T- d</property>
2 ^. {+ {' c$ @, o! T<property>; P7 d! [' F8 q: x: h
  <name>io.serializations</name>. p( m9 n% t1 @- e3 B
  <value>org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization</value>
+ m/ B+ h1 N& L  k& j  <description>A list of serialization classes that can be used for
$ V4 M4 }1 {+ j4 Y) j6 ^' |: |  obtaining serializers and deserializers.</description>
: W4 W7 k& Y+ {5 _  A! h</property>7 V$ d3 X4 J# ^& C; }/ x) w) t
<property>& L6 l( r; }9 Z. ?6 F: r$ d
  <name>io.seqfile.local.dir</name>
. L# X) H2 Q- X& l  <value>${hadoop.tmp.dir}/io/local</value>& q3 h0 _; l/ s# }
  <description>The local directory where sequence file stores intermediate
* r6 k0 y% Z' m: T  data files during merge.  May be a comma-separated list of* \0 C9 d# Y) f
  directories on different devices in order to spread disk i/o.
/ f2 X7 K9 q+ H+ ?  Directories that do not exist are ignored.$ g  C% ?/ g$ E+ V9 M, _& q5 H
  </description>9 b* [6 S% I/ [$ J
</property>
+ a8 p, T, Q, r; M0 q<property>3 y9 v  H! e0 j1 O
  <name>io.map.index.skip</name>! }" H0 g% y3 R& F( J3 R
  <value>0</value>  R8 [7 K4 J' z/ u5 v
  <description>Number of index entries to skip between each entry.
6 p" K, W- U& z6 h9 _  Zero by default. Setting this to values larger than zero can  v! L/ V# t# ?" i3 r
  facilitate opening large MapFiles using less memory.</description>2 B* \' M5 o) v% H+ z" a# i
</property>
" \$ y% C+ s2 W<property>( g* [' }% j% Z- `5 S) F
  <name>io.map.index.interval</name>
; h8 B1 B) \- I% f! O. v  <value>128</value>
, }. V; i  U: X0 r' x  <description>
3 H1 f, d. k$ @! w, b    MapFile consist of two files - data file (tuples) and index file
( ?1 }$ ?9 K4 M# w! l/ L. u    (keys). For every io.map.index.interval records written in the
" w& r1 R5 V* f: S- D) `    data file, an entry (record-key, data-file-position) is written
" P5 L: l4 i. ^    in the index file. This is to allow for doing binary search later  M% J" t6 y1 O# [' k
    within the index file to look up records by their keys and get their6 N% h$ o0 \+ I, j0 q: G
    closest positions in the data file.! x2 H# A5 R  D: E. g3 {
  </description>
! Z$ O4 M* \9 O' p9 p/ e6 T3 j</property>( o* M4 d& i1 u, k% ~
<property>
4 l! S, ~6 V% e3 {# `+ B7 ^: o  <name>io.erasurecode.codec.rs.rawcoders</name>
5 _4 @( z! g5 t! R& a/ U$ `  <value>rs_native,rs_java</value>0 d5 w% z8 U- `" A
  <description>5 `. }9 r. N3 B) I" ]1 C( d: ~/ G
    Comma separated raw coder implementations for the rs codec. The earlier% P( Z, W0 w, t( g1 ?! w+ @
    factory is prior to followings in case of failure of creating raw coders.
' H3 \& j3 a- a+ @  </description>
5 x! G/ Y( N( S4 M; m</property>
) `( I" h/ K6 q3 k<property>
% c( P: L6 u/ f  <name>io.erasurecode.codec.rs-legacy.rawcoders</name>
. x" a3 B% N' `, l/ k/ u- i% n  <value>rs-legacy_java</value>1 v7 @0 t& s. }1 O! _6 r& P
  <description>- i& `5 `8 }, j9 b- Z& `% _; M
    Comma separated raw coder implementations for the rs-legacy codec. The earlier) P$ I3 k& i4 K/ N
    factory is prior to followings in case of failure of creating raw coders.
& D3 ~8 L5 m- c9 K2 F  </description>7 {2 r3 V4 E" x7 r
</property>
, o8 m7 j5 n  W6 |2 _" _<property>. F2 Q8 C6 e  U8 E" [
  <name>io.erasurecode.codec.xor.rawcoders</name>
4 B8 Z! c' F( i3 W+ k2 ]7 v. {  <value>xor_native,xor_java</value>' V0 w& x, U$ b8 `; [4 {3 y
  <description>: j, A' w+ i2 x( @* |
    Comma separated raw coder implementations for the xor codec. The earlier
/ ^) h" R1 g5 k2 f; }2 d    factory is prior to followings in case of failure of creating raw coders.) j* X4 @' F' s  x, k& @5 Z/ S6 ]8 K
  </description>
5 k: ^) T% |& O5 P" K3 u</property>
! ]. z$ U' e- X0 t' C0 d  <!-- file system properties -->
) q% F$ [; y7 A5 o4 o: M! f" a<property>
. J. t% {8 G4 f( X  p0 p0 ~. Y0 v  <name>fs.defaultFS</name>8 w* G+ [9 \4 C! r9 y2 v- \
  <value>file:///</value>
! b" g7 D  ^9 o$ \# y* m$ N  <description>The name of the default file system.  A URI whose
5 k  k) Z' }: `( \/ Q  scheme and authority determine the FileSystem implementation.  The4 m( Y& N! T/ y4 T4 _% n  B! H3 c4 e( x
  uri's scheme determines the config property (fs.SCHEME.impl) naming
9 x# ]4 f1 |( h+ f, S  the FileSystem implementation class.  The uri's authority is used to
9 q" L  N0 M: N7 ?/ E2 o  determine the host, port, etc. for a filesystem.</description>8 s6 X% y* H. W% C4 l+ g$ y( D  E' M
</property>
: h- _; E( P9 g<property>
; a' I2 ?" F* a  <name>fs.default.name</name>
" h8 B4 E& m/ {  <value>file:///</value>
/ z" t- U7 E0 V3 ~! d2 Y  <description>Deprecated. Use (fs.defaultFS) property3 W" h6 o/ b' y3 ^: S# q+ o
  instead</description>
# w2 M( q8 U' A: \- W1 E/ T</property>" \$ F# C7 n3 C5 z
<property>
8 b3 K, k! K5 D* G  <name>fs.trash.interval</name>
9 Q* d4 p$ H6 h6 J9 c, W6 C  <value>0</value>) Q$ Q3 @- `# V
  <description>Number of minutes after which the checkpoint- n* a0 @! Y9 W' i
  gets deleted.  If zero, the trash feature is disabled.3 D% U+ ?" h) r+ Z1 r; y0 n& H
  This option may be configured both on the server and the( j# M4 ]  m: [$ _- }/ b( b
  client. If trash is disabled server side then the client
( y, }5 a' A# e' {2 G/ E  side configuration is checked. If trash is enabled on the" V* U# C: v! @
  server side then the value configured on the server is
# g* j% e( n) p" K  used and the client configuration value is ignored.
% D+ `4 [) N' ^( w  </description>' i. c. M  H7 c% A
</property>
  }+ B+ ~2 H1 I, _# }2 `6 ?* [<property>
2 }: d8 Y- C: L  <name>fs.trash.checkpoint.interval</name>
- {/ c  I: {! p% G$ q. D. ~  <value>0</value>* \6 m$ T/ x  t1 D- w+ l6 d
  <description>Number of minutes between trash checkpoints.
1 g9 `1 P- G/ A* O- _, t  Should be smaller or equal to fs.trash.interval. If zero,
2 v- k$ D! q! [% E* }  the value is set to the value of fs.trash.interval.
$ _( X9 d7 C6 i9 t+ k- X3 s1 R2 Z# U  Every time the checkpointer runs it creates a new checkpoint
" f+ Y- Q+ A( Y6 i8 ^0 K; o  out of current and removes checkpoints created more than, j8 k  l) a7 M! o) @
  fs.trash.interval minutes ago.
1 x! `: F6 a, R6 o  </description>
( Z, v/ C( V7 i8 ~8 Q</property>
& n  v- _& k' n) R& A0 E. P0 L9 N) {<property>
; s! p9 r% j8 b( X/ q7 a  <name>fs.protected.directories</name>* _! p: w3 D, T6 G  R. p$ b% M
  <value></value>
/ A: K: E5 |6 z  <description>A comma-separated list of directories which cannot
, C6 K; j  k8 I+ e+ v* g0 ]    be deleted even by the superuser unless they are empty. This
6 y# C) b7 ~2 ]2 k5 p1 f5 Z+ N& d    setting can be used to guard important system directories4 ]4 O+ ?! E7 u% U9 @1 _1 q* }5 V
    against accidental deletion due to administrator error.
3 ]9 M1 @( i' X; o- ?+ e  </description>. f$ `7 A( r/ {" p- `+ n# C
</property>0 S0 o+ h4 L+ x" t4 J
<property>
+ k# s( f! g( W: q: e, d  <name>fs.AbstractFileSystem.file.impl</name>0 w; k( h' C$ V! H* `
  <value>org.apache.hadoop.fs.local.LocalFs</value>
& B' w  k8 n. t5 F  <description>The AbstractFileSystem for file: uris.</description>5 \7 I, _2 u& j3 A: W, k- r1 B- P
</property>
$ O  h9 U  n  ]<property>
3 n3 K8 D. {# Y- M! G4 C  <name>fs.AbstractFileSystem.har.impl</name>
* ^1 ~4 `4 x0 D) n; a/ L9 P. l  <value>org.apache.hadoop.fs.HarFs</value>$ \. `" ?1 c5 O( r2 b; b, B+ N
  <description>The AbstractFileSystem for har: uris.</description>  ~% W, W7 @% {& V! ^1 E
</property>
+ F) \2 ?2 F& ^! Y0 C( F<property># y0 t1 T) ?" l0 E* E
  <name>fs.AbstractFileSystem.hdfs.impl</name>5 k7 R/ G8 W% I' f- U
  <value>org.apache.hadoop.fs.Hdfs</value>
* ~$ v/ P' w/ M- ^$ f  <description>The FileSystem for hdfs: uris.</description>
2 l$ P, H% o4 ~% y</property>' g8 S5 f. b/ e4 ]8 G
<property>
. M1 l. n, ~: t9 L  R; p! J. {; w  <name>fs.AbstractFileSystem.viewfs.impl</name>
" t. O) ~# R0 B& R/ G) x  <value>org.apache.hadoop.fs.viewfs.ViewFs</value>
3 W; @' b  e6 o5 `" q9 d7 e  <description>The AbstractFileSystem for view file system for viewfs: uris
7 E7 Z2 h% U  C3 ?  _  (ie client side mount table:).</description># {- D, ]3 @3 J) a4 G# e( r
</property>
( z2 D3 O+ K2 x& a3 j# q<property>  v6 L) K# o' I" X
  <name>fs.viewfs.rename.strategy</name>
. j. D9 F0 I+ O7 _5 B  <value>SAME_MOUNTPOINT</value>
' a! I/ |2 o. f( O3 F7 ~  <description>Allowed rename strategy to rename between multiple mountpoints.& a( _# C/ s/ S  z) m; b+ r: C
    Allowed values are SAME_MOUNTPOINT,SAME_TARGET_URI_ACROSS_MOUNTPOINT and, Q& [4 ]/ I4 l& `/ H, ?
    SAME_FILESYSTEM_ACROSS_MOUNTPOINT.0 }, }  A6 Q7 I+ a
  </description>2 w! q- L8 I! H) i8 _$ d
</property>
4 l' g$ z& O& {- Z6 o6 z<property>
2 h& y$ L$ m3 ]; C6 m/ s# f  <name>fs.AbstractFileSystem.ftp.impl</name>
- f9 P  _$ B1 a" l5 Z  <value>org.apache.hadoop.fs.ftp.FtpFs</value>
3 K% J7 U& P7 X: i% d; W% t  <description>The FileSystem for Ftp: uris.</description>; `& n% Q) x) m9 |: a& V. z" [
</property>
1 z% B) T: w% C# D. {<property>& Q4 M( c, ~+ E
  <name>fs.ftp.impl</name>+ k! _4 A% C" h  n: q# ~9 `+ Y: [
  <value>org.apache.hadoop.fs.ftp.FTPFileSystem</value>8 p" t5 X) l& {" Q6 m# n% ~& W
  <description>The implementation class of the FTP FileSystem</description>
8 _3 J. u. V- j+ {* o</property>: i8 G# }2 I! f/ X/ h& b$ Z+ S6 x
<property>
. D1 D6 Y9 J* b2 V3 |4 w  <name>fs.AbstractFileSystem.webhdfs.impl</name>
, y$ h: m8 Q4 m1 m. v6 N$ i  <value>org.apache.hadoop.fs.WebHdfs</value># k# F; {3 m! D# j
  <description>The FileSystem for webhdfs: uris.</description>
' A; B: a9 c" `2 y! g</property>/ r/ G4 X* M! L+ }5 m/ A  D$ G5 S! Y
<property>
9 T% h( ~- X1 [' H) W  <name>fs.AbstractFileSystem.swebhdfs.impl</name>
+ g, p* f0 Z7 D+ z  <value>org.apache.hadoop.fs.SWebHdfs</value># J0 U7 W5 _1 a8 {
  <description>The FileSystem for swebhdfs: uris.</description>+ o+ z. s& }+ l! L. \" F2 \# E
</property>/ q! z' O2 w- o
<property>
  l; W) `# W4 O( H  <name>fs.ftp.host</name>" R' K  I% B6 ~1 R, Y
  <value>0.0.0.0</value>1 w7 T7 R( w8 r$ P0 M
  <description>FTP filesystem connects to this server</description>
* B$ @$ z% }- c$ P</property>! m5 R- G6 O' Y) ^& o" U
<property>
6 X5 m7 S- F& @5 Q; Y  N. f' L  <name>fs.ftp.host.port</name>$ ^, K- \, W3 r
  <value>21</value>$ A& c+ C% l! W3 F- \. {/ v  I
  <description>' i& {& R6 ~2 D8 w7 ~" j0 k) e
    FTP filesystem connects to fs.ftp.host on this port
6 n+ ^) D: }2 N/ _  </description>( ]+ d& |+ `% M, w3 v9 y
</property>
  t+ n' X) d* B& }" @<property>
: U# \6 f" h" m" c9 y1 N% q  <name>fs.ftp.data.connection.mode</name>
" O6 M1 W9 c8 N; S7 }+ Q  <value>ACTIVE_LOCAL_DATA_CONNECTION_MODE</value>: p0 ^. i! W" v1 d8 i8 a
  <description>Set the FTPClient's data connection mode based on configuration.
8 z8 s  h8 c! c- R  w    Valid values are ACTIVE_LOCAL_DATA_CONNECTION_MODE,$ W9 o4 A- m$ u! P( J' k0 N5 K
    PASSIVE_LOCAL_DATA_CONNECTION_MODE and PASSIVE_REMOTE_DATA_CONNECTION_MODE.
( w# @0 h6 q& D) F; O9 v  </description>" }. O$ L% ^/ @  x5 S9 a# _/ a
</property>- ]- ~3 Q7 ^4 B
<property>
& a, {) z/ `8 s& _# D# M) ~  <name>fs.ftp.transfer.mode</name>
3 j& g9 H& l6 q! U3 x( W( R# V  <value>BLOCK_TRANSFER_MODE</value>
  z3 _0 |% u8 J, \/ m  <description>5 x# F) i4 z$ r! B% Y% o& }  ]
    Set FTP's transfer mode based on configuration. Valid values are" T) m: u5 q8 j3 r5 y5 q+ a; e
    STREAM_TRANSFER_MODE, BLOCK_TRANSFER_MODE and COMPRESSED_TRANSFER_MODE.
, t3 F& `) r3 Z3 o7 R  </description>( i0 u  W: L$ _8 E& _
</property>$ K5 m6 v. @+ u  h8 X  A1 O: P
<property>
5 K2 z! p/ K! d6 D9 x0 Y  <name>fs.df.interval</name>5 S: D% N5 X  p9 [' R; C  t- M
  <value>60000</value>/ F% C# |: k: v$ @
  <description>Disk usage statistics refresh interval in msec.</description>. S. r1 {+ T  s) z0 U1 R) u8 m0 ^% p
</property>- a& J1 W( m, D5 E
<property>
( S& d% |/ Y7 }6 H& M, {. H  <name>fs.du.interval</name>9 s# y) o& x8 |! T4 \5 R* g3 g$ B; Q
  <value>600000</value>, k& J; _: D! g% b6 V
  <description>File space usage statistics refresh interval in msec.</description>1 d8 J) t: ~4 N, K. y
</property>& k5 v% _. g5 O' ]5 e
<property>. Q/ l7 F' N- _0 k' X) ^5 C
  <name>fs.swift.impl</name>
# C1 o: q: Z, w+ z$ Q+ b/ [  <value>org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem</value>
3 W1 ^0 Y8 Q) u. p3 h  <description>The implementation class of the OpenStack Swift Filesystem</description>
4 e) [4 ]$ P: V( @/ p* k1 A5 ~</property>
" \1 e- x, F: m3 c  ~! C/ J<property>( w. ]7 C* f, Z
  <name>fs.automatic.close</name>, b# o2 Z0 h" w6 Q* ^
  <value>true</value>
  n4 E7 F0 p- s' g# \7 J  <description>By default, FileSystem instances are automatically closed at program2 g3 y# H) i+ T8 q) d3 t# P
  exit using a JVM shutdown hook. Setting this property to false disables this4 {1 c9 J1 o5 D4 u% P0 h6 V( j. S+ t
  behavior. This is an advanced option that should only be used by server applications
6 z: o% V; H8 c7 ?* q  requiring a more carefully orchestrated shutdown sequence.
$ M' J* H, u- k  </description>
. k! N0 G% D  R1 P: |& l</property>- b* F$ l6 V2 Z% I+ _
<property>) p8 q) W" P7 R! K" y
  <name>fs.s3a.access.key</name>
! d( c$ G5 K& C: `1 }  \/ R  <description>AWS access key ID used by S3A file system. Omit for IAM role-based or provider-based authentication.</description>
  n" _2 a9 N% ^+ _9 O</property>
4 I8 U  D! ]6 W8 [  P+ B- M+ n<property>
9 [( Q, z4 }- N# y2 c$ |$ X  <name>fs.s3a.secret.key</name>1 g" A2 @) f& t  v; M. n6 Q+ ^6 i
  <description>AWS secret key used by S3A file system. Omit for IAM role-based or provider-based authentication.</description>4 j- s9 L- j- E- x& @
</property>* f9 C4 Z+ N/ s. b9 Q/ u
<property>2 @- R" `2 N3 l# m* Q' `
  <name>fs.s3a.aws.credentials.provider</name>
$ F% E* K& t* i; G: c# Q0 y% l  <description>% u7 W+ d( `4 H) [( H
    Comma-separated class names of credential provider classes which implement7 I- E: [8 A2 j1 P( Y0 A# n
    com.amazonaws.auth.AWSCredentialsProvider.  i* z2 L/ Y* x0 Q% F8 q; e  F
    These are loaded and queried in sequence for a valid set of credentials.
, m% _0 S) h: K. w9 H8 c# Z    Each listed class must implement one of the following means of
: ~# d& |9 o$ d" p8 D    construction, which are attempted in order:( m/ Y# o. T0 E  o4 Y
    1. a public constructor accepting java.net.URI and' V, n+ B( P, a* U& G+ F, {
        org.apache.hadoop.conf.Configuration,# {( N( j- q4 q6 l2 T5 N
    2. a public static method named getInstance that accepts no
+ y' E/ C# @$ I       arguments and returns an instance of' p, _7 G" z( {. N0 @
       com.amazonaws.auth.AWSCredentialsProvider, or. f) L' _  |# M3 j4 v. e" i* g
    3. a public default constructor.* L4 ]: w, Y; I, U# N/ x7 Z2 H
    Specifying org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider allows3 C' Q7 `8 J' ~* N) h- [
    anonymous access to a publicly accessible S3 bucket without any credentials.
/ u" D- n9 O( z  J! R! c- P    Please note that allowing anonymous access to an S3 bucket compromises
: p0 N7 v$ w" J, X: ]6 F6 s# Z( `    security and therefore is unsuitable for most use cases. It can be useful
$ g1 V  z. u0 G, m    for accessing public data sets without requiring AWS credentials.
2 l% s6 \9 o$ D* u: ~8 O& E) L/ G    If unspecified, then the default list of credential provider classes,: R: I$ p# h7 j, U3 J( S9 h1 c! e
    queried in sequence, is:
$ u  s; U( y5 W    1. org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider: supports static
& w; ^. `. l+ V6 X1 W        configuration of AWS access key ID and secret access key.  See also
+ K8 C: V1 Y$ l) x2 \: O        fs.s3a.access.key and fs.s3a.secret.key.) m0 s; Y/ ?/ J1 B. r) R
    2. com.amazonaws.auth.EnvironmentVariableCredentialsProvider: supports
3 i; G% E; P5 V" ]        configuration of AWS access key ID and secret access key in
2 }7 J; ?9 w0 R/ {        environment variables named AWS_ACCESS_KEY_ID and
2 |. T* j# n8 j4 ^1 N        AWS_SECRET_ACCESS_KEY, as documented in the AWS SDK.4 u+ r- X; R7 x" f
    3. com.amazonaws.auth.InstanceProfileCredentialsProvider: supports use% [) F3 j4 w- d; O
        of instance profile credentials if running in an EC2 VM./ y: r3 q! q, c+ _/ A
  </description>  J8 T. ]2 o! q) c8 @' E4 Y  Z
</property>
& w3 v$ ^' E. q! o& m9 L0 S' e<property>8 }/ x; P, M! j& ?) V2 M, }' K
  <name>fs.s3a.session.token</name>/ I' x. v8 v3 j) ~# B' v5 q
  <description>Session token, when using org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
% [: M' Z4 n* u    as one of the providers.: l4 }- f: Q1 ?  R8 u
  </description>8 ~0 }% d- ~8 V
</property>8 P1 H3 h9 P7 z: s- m1 w
<property>' M4 _* s  j- h
  <name>fs.s3a.security.credential.provider.path</name>' n/ v# Y5 h% @" Q3 q3 Q8 Z
  <value />
  z' e6 c, i: Y5 x1 A. D$ c  <description>5 q7 L7 J* T0 m
    Optional comma separated list of credential providers, a list
' Q2 _. k' X4 c! f& B    which is prepended to that set in hadoop.security.credential.provider.path8 ^# b9 D  o9 ^; v6 f
  </description>
& [8 g  D9 e( \- U! Y  R% Q) x* L! J</property>! l% G( R5 D, V6 }# F
<property>0 h+ Y8 e  w: T3 D3 U8 x
  <name>fs.s3a.assumed.role.arn</name>: p1 o1 ^9 u& y) B+ U
  <value />& H* @0 j! S( |4 F6 e; y: V
  <description>
" T" g8 n3 j  B) \; \% p    AWS ARN for the role to be assumed.
, S3 l: a2 |# ?$ M# g    Required if the fs.s3a.aws.credentials.provider contains
# [+ S9 C/ l  \# ^8 m# g! |    org.apache.hadoop.fs.s3a.AssumedRoleCredentialProvider% n' I8 ~3 W0 M
  </description>
& V4 {6 s" o3 A; e8 p2 w* X, O, [</property>: D- R$ L1 K3 N8 S1 Y7 A7 A3 x
<property>' T, p1 T8 S2 N3 U! e
  <name>fs.s3a.assumed.role.session.name</name>
2 N; S6 a- ^" }5 h. B. y. g+ i/ \  <value />! y; Y# I0 h/ w  o% R4 H) K+ w, ~
  <description>
' B" v" B$ z  P" ]# m  P$ e    Session name for the assumed role, must be valid characters according to
) r. N, g4 Q: z6 q% q, Q3 Q    the AWS APIs./ y- a* a6 f/ a: [
    Only used if AssumedRoleCredentialProvider is the AWS credential provider.
7 h, V$ A; o$ t' l    If not set, one is generated from the current Hadoop/Kerberos username." {0 W1 C0 W+ o: [& C. `- U
  </description>
; E  e" @4 @9 T3 q$ I) S</property>
! @1 E9 P: }' v: I; G0 [<property>8 m. u/ Y3 s# w- p1 e# k
  <name>fs.s3a.assumed.role.policy</name>' q8 ^- J$ j8 ?  U
  <value/># d' O' }! Y$ R8 i& w, ^0 d8 s
  <description>* V1 \- {6 }4 s- u( t& X; C( A: R
    JSON policy to apply to the role.2 Q2 v7 d4 G* p
    Only used if AssumedRoleCredentialProvider is the AWS credential provider.6 D6 _" F# M' P; r6 \, |4 l7 g, p
  </description>
8 B8 |1 G) i+ j3 q</property>( i( S, o" L/ z$ G9 e  L+ `
<property>
$ O' l0 H+ N/ i2 J  <name>fs.s3a.assumed.role.session.duration</name>
  h5 J- }# P+ Q2 {1 P* H+ I2 P2 H" Q/ U  <value>30m</value>0 k/ x6 L% Q3 B& f2 ~
  <description>1 p$ ?% E- q0 [* v: {6 ]& M
    Duration of assumed roles before a refresh is attempted.# J8 \' S( y( C; c
    Only used if AssumedRoleCredentialProvider is the AWS credential provider.* v1 ?9 O4 }% p2 O7 t7 n( I& d
    Range: 15m to 1h
5 N5 Q7 g" `$ @% z, Q* U  </description>
5 V7 R  D, ^9 P  L1 q# O2 f</property>
4 G# x7 u8 j& W( [5 u8 l: A$ B<property>
7 V( T3 c3 \# m5 v1 w! d4 G  <name>fs.s3a.assumed.role.sts.endpoint</name>
6 ~5 {. V5 }/ M2 A2 i  <value/>$ g6 z' Z4 x4 u9 k
  <description>
6 l# V% n4 l6 S$ P: O    AWS Simple Token Service Endpoint. If unset, uses the default endpoint.3 W) G2 I8 y( @4 }
    Only used if AssumedRoleCredentialProvider is the AWS credential provider.3 O/ I2 ]6 R% ^5 \0 l1 H5 @3 S
  </description>' M" a3 \% ?( N% A( k
</property>
4 |' r8 P: \# r2 t1 V<property>
$ n& R2 [; J/ Z# U9 z# A9 \  <name>fs.s3a.assumed.role.credentials.provider</name>
" a# F5 h5 M& h1 b& Z6 k! v& w  <value>org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider</value>
$ O; U$ S1 T* S, c1 D1 G  <description>
( h4 X: D( S1 Z    List of credential providers to authenticate with the STS endpoint and- a( E3 N# g" d
    retrieve short-lived role credentials.3 p& G) ^& i: j: U3 \
    Only used if AssumedRoleCredentialProvider is the AWS credential provider.  U' r0 W4 a1 c, h
    If unset, uses "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"./ W. L- P' B- ^/ H0 d
  </description>- \! N. h' F; {& k
</property>- r& S' v/ q/ B
<property>2 \% C0 i' J; D8 m- K2 e2 h$ g
  <name>fs.s3a.connection.maximum</name>
1 u, ~: }2 l0 C  H  <value>15</value>2 T4 A3 s) P; X
  <description>Controls the maximum number of simultaneous connections to S3.</description>: x- X6 i( _' z5 d9 k
</property>
5 A- w$ Z, S5 P  k' t5 j4 C8 [<property>% Y% U4 n3 O& K% g
  <name>fs.s3a.connection.ssl.enabled</name>. u  k6 X# b8 C
  <value>true</value>
7 G, y0 ?- D* j8 n  <description>Enables or disables SSL connections to S3.</description>$ d. ]+ J% g. Y" c/ Y, r* S3 C
</property>
% {) ?( Q, v( n<property>6 b6 m) P. d5 w3 [* J! H
  <name>fs.s3a.endpoint</name>
! |& c' L2 |( Z6 n  N  <description>AWS S3 endpoint to connect to. An up-to-date list is
; R) w3 ]6 r  e( z! \. H3 f    provided in the AWS Documentation: regions and endpoints. Without this6 k- V; h- C" C; A
    property, the standard region (s3.amazonaws.com) is assumed.7 h; u7 U8 A" _  C( E' b& L
  </description>
! G1 N+ q# u8 x8 {% k</property>
" z/ \5 w2 @6 [  ~# E- X<property>
5 i9 O# `( ]/ d( m9 ~  <name>fs.s3a.path.style.access</name>
6 |9 N( h* ?& w  <value>false</value>1 i9 |% Y+ f; K8 y! C/ _0 i
  <description>Enable S3 path style access ie disabling the default virtual hosting behaviour.
, k" L0 g* ^8 t" U7 @" N6 @    Useful for S3A-compliant storage providers as it removes the need to set up DNS for virtual hosting.8 C* t4 u  V; K3 g& v$ T
  </description>" t$ N7 y8 N0 M, M% X
</property>
+ O0 |( Q* i2 ~( S' k7 a/ D, H<property>4 `( L4 J% s; H0 M
  <name>fs.s3a.proxy.host</name>3 ?* `: K, z- U! m4 A0 p
  <description>Hostname of the (optional) proxy server for S3 connections.</description>. {* O  m, \; w5 S, q
</property>% b- O& ^! x3 q5 S
<property>
) S+ }- b! I. X  y  <name>fs.s3a.proxy.port</name>1 a  |# Q; i' d
  <description>Proxy server port. If this property is not set- P. s& {/ A( ]5 h4 T
    but fs.s3a.proxy.host is, port 80 or 443 is assumed (consistent with
, W/ \3 m+ f/ J5 X3 w7 a- Z- H    the value of fs.s3a.connection.ssl.enabled).</description>! z0 g9 {4 k: x2 y- ~& I' y
</property>3 H* {$ Z# I2 w* ^5 X6 m
<property>
7 Y% M- x/ F! ?) y, u  <name>fs.s3a.proxy.username</name>; }& G: x4 ?! w: P, b
  <description>Username for authenticating with proxy server.</description>
6 u: Q  j% d! M4 y. t/ Y. K1 \</property>$ s. Z, D& I; c: A- L! i
<property>) v/ M* s: [3 }/ k8 {5 C6 y3 |$ \
  <name>fs.s3a.proxy.password</name>2 j2 S/ Q. m0 [5 w: \- E2 a# n
  <description>Password for authenticating with proxy server.</description>
1 ]8 K& c% q4 p! T  j2 S7 J, u  w</property>5 T7 S$ r7 g7 _$ M3 B  U
<property>4 D9 M! ?3 {, R9 ^4 P
  <name>fs.s3a.proxy.domain</name>
7 m8 M  K- C9 `# a) F2 ], T  <description>Domain for authenticating with proxy server.</description>  i; S+ L" J6 @0 `
</property>
) T" U5 T( a% L<property># D2 O+ ~$ H2 p" u9 B, `
  <name>fs.s3a.proxy.workstation</name>+ y- j, l& q. `6 N' L1 J
  <description>Workstation for authenticating with proxy server.</description>5 g8 Z# ^: n, y  D% U2 M2 b
</property>$ ^7 A8 `* b' f; J/ i8 L+ ?+ p
<property>8 G4 P/ S0 [% W! \' ?% v
  <name>fs.s3a.attempts.maximum</name>7 U3 ]4 M* |9 \6 ?8 u; v+ ~
  <value>20</value>
: A1 [& C+ k  D2 t. r6 }- ?" _  <description>How many times we should retry commands on transient errors.</description>
5 U0 ^) \- y0 {. Z</property># [+ F0 B7 G( W+ {: q
<property>
; T- C8 {& v& O' z  <name>fs.s3a.connection.establish.timeout</name>$ k( I4 ?! q6 p& ~: w, C
  <value>5000</value>
( S( W/ Z  m# Q( I: k  <description>Socket connection setup timeout in milliseconds.</description>' g+ f! E1 s( H: E* Z5 Q9 f
</property>" b0 N' I+ y& P2 B
<property>5 ?) D0 l: t& O: h7 u0 W7 a
  <name>fs.s3a.connection.timeout</name>
7 ?- X7 N' t# T& I  <value>200000</value>( `+ l1 [& I% ]
  <description>Socket connection timeout in milliseconds.</description>; A6 I& ~0 O1 i
</property>
2 B' U  W* U4 E  i5 z9 t" N4 d<property>/ L5 w; C8 U! q( k% t3 k, q6 a7 I
  <name>fs.s3a.socket.send.buffer</name>
! w6 n5 ]. j( s/ {1 x$ @  <value>8192</value>
- T" K3 Q6 E3 s' S( O; M9 p) ]$ B  <description>Socket send buffer hint to amazon connector. Represented in bytes.</description>
4 r+ X; O+ P$ d+ T% p( ~) A7 C2 z</property>7 S: i1 `) V! \9 m8 ^, o6 D: e
<property>
1 O9 v& X$ r8 D  <name>fs.s3a.socket.recv.buffer</name>
7 j2 A$ P' H/ o& J  <value>8192</value>
- c$ I' V* Y+ t1 Z  <description>Socket receive buffer hint to amazon connector. Represented in bytes.</description>: [0 e! F9 ^5 e; I
</property>
& {4 j4 V' e; m, A, }<property>8 e/ u  J8 @3 C( m# @! ]' P
  <name>fs.s3a.paging.maximum</name>  }. V  W) S4 ]! j  Y8 c& v- w: \
  <value>5000</value>
. S$ K0 `" X& y& y' Z$ G- z  <description>How many keys to request from S3 when doing6 _7 j. W$ L' D, o+ H
     directory listings at a time.</description>
* Q) y8 v5 T  w/ s# c! {</property>8 k' z2 {" u. g: q& G
<property>
4 w3 ?8 M" e7 S! v  <name>fs.s3a.threads.max</name>0 D  z; v3 E# m+ j9 ?9 @
  <value>10</value>) q0 }0 m' N! B2 U3 _
  <description>The total number of threads available in the filesystem for data
4 ^1 @$ Z. q& D1 a$ _  b1 c4 e    uploads *or any other queued filesystem operation*.</description>6 q' g7 B! _" m  M( x1 O
</property>
1 m: f' K9 e9 [4 h<property>
7 \  k" g4 v5 E+ @. D- c* j" h* c  <name>fs.s3a.threads.keepalivetime</name>) j. ^5 r6 w  \9 [: z
  <value>60</value>
, {( I% h0 d  Q$ A# ?9 }  <description>Number of seconds a thread can be idle before being
8 B/ u; T8 D2 W# F4 _    terminated.</description>2 b7 G% u1 u" e
</property>: P8 p$ C. L; T2 B; u' ~
<property>
1 h3 ~, N4 r/ g  S) r  <name>fs.s3a.max.total.tasks</name>
" c  k2 u9 U/ Z* Q  <value>5</value>/ `- n0 U; \% `' Y" S% x* C
  <description>The number of operations which can be queued for execution</description>
" l; |) m0 a& z4 C+ L</property>7 s' B- f1 _% P
<property>) X! i, G9 u$ u1 Y, h
  <name>fs.s3a.multipart.size</name>5 ]0 f5 j- x- J' F) T
  <value>100M</value>5 h, v, D8 R- O. {" L
  <description>How big (in bytes) to split upload or copy operations up into.
& u: v, a' i- H  `1 w    A suffix from the set {K,M,G,T,P} may be used to scale the numeric value.
" g9 K! p' d! l1 q- |, H- B; v# ?  </description>1 ~: U4 }4 |+ A, L
</property>
) M  ^4 w( F, ?( N: z- f<property>
7 a% G) A. `3 D* n' ?6 s! v  <name>fs.s3a.multipart.threshold</name>
  b& c( s3 S& \% o  <value>2147483647</value>
6 H4 D- \. t* U: R$ n- L3 p- @, @  <description>How big (in bytes) to split upload or copy operations up into.* d( d6 t' O. k  r$ i# b. T
    This also controls the partition size in renamed files, as rename() involves- C5 l3 Q, p3 t; Y" N' h- j
    copying the source file(s).! `7 z; Z6 z' b" F
    A suffix from the set {K,M,G,T,P} may be used to scale the numeric value.
  W7 n1 S$ }8 x' P  E# ^/ D0 d7 u/ s  </description>
6 U% [( G. Y9 e</property>: U2 ?4 L8 J0 R7 H: w% ]
<property>
2 f- V( e+ v: G! _( X! E9 L  <name>fs.s3a.multiobjectdelete.enable</name>
" C4 d6 F) g; e8 w: ?: f/ Q( |1 @  <value>true</value>
* Q3 S7 }) f  Q0 `" C: x  G5 W  <description>When enabled, multiple single-object delete requests are replaced by  w! }' b4 K; c3 P( r+ l0 K' ], K
    a single 'delete multiple objects'-request, reducing the number of requests.
' x3 m8 A: n) F1 X1 F) j  I+ S7 D    Beware: legacy S3-compatible object stores might not support this request.
2 ^+ f4 h) ^- n  </description>2 r+ o1 v9 {0 t+ e( N% {
</property>
, `. q3 O7 K' I8 Z6 u( b3 P; I<property>
' T6 v" f: X' Y0 D9 X! }  <name>fs.s3a.acl.default</name>4 G7 Z8 P$ \( J5 Z' O3 t/ Y8 c8 r1 X* `& N
  <description>Set a canned ACL for newly created and copied objects. Value may be Private,
7 p0 i; c! A& r/ U/ C' K1 F" {      PublicRead, PublicReadWrite, AuthenticatedRead, LogDeliveryWrite, BucketOwnerRead,
1 ]; `, S* }( f( R" @      or BucketOwnerFullControl.</description>
3 u. P* P/ z& _8 K: v' e8 [3 G8 I6 p8 K: s</property>
( Q0 v) {, c6 Q  k) N- L* V<property>
! ?' M, H8 B" x$ f9 R  K* m  <name>fs.s3a.multipart.purge</name>
8 m4 {9 C9 ~  D7 }  <value>false</value>6 g( \, s, N# m8 `$ Q& l0 G
  <description>True if you want to purge existing multipart uploads that may not have been7 G; ?- t- d# k" ^
    completed/aborted correctly. The corresponding purge age is defined in
2 {( v- U8 m9 \; Y2 ~    fs.s3a.multipart.purge.age.& `  f) t. e+ x1 [* ^, j7 v3 i
    If set, when the filesystem is instantiated then all outstanding uploads) o+ \( I8 |3 @* l; Y2 w8 ]
    older than the purge age will be terminated -across the entire bucket.
/ ]3 @" G  v- A! X    This will impact multipart uploads by other applications and users. so should. y, D* n- R/ |' u8 M
    be used sparingly, with an age value chosen to stop failed uploads, without
8 |5 M8 Z+ [# ^, @$ u" _3 z    breaking ongoing operations.
( |% f% |1 i/ P5 j4 ^' R  </description>
" Q: b/ t' U7 T+ B, F. Z- H: B</property>( Z3 ~" C6 m4 q
<property>
+ P  P! d3 s4 m2 r0 c. d& v  <name>fs.s3a.multipart.purge.age</name>
9 U! U9 H" C$ n6 b  <value>86400</value>* z( F" J0 i, [( P/ z. `
  <description>Minimum age in seconds of multipart uploads to purge0 @5 @% V0 a- ]8 k) J6 @
    on startup if "fs.s3a.multipart.purge" is true
2 y( I# W3 q3 U2 m  </description>8 q$ h7 X, A- E! S) F* A' w
</property>
4 ]7 M* G4 a- u+ s<property>2 R" P. Q1 I1 p) }
  <name>fs.s3a.server-side-encryption-algorithm</name>
2 ~- A( H3 N$ D/ n9 |  <description>Specify a server-side encryption algorithm for s3a: file system.# j' }1 ^" \2 ~4 o4 F
    Unset by default.  It supports the following values: 'AES256' (for SSE-S3),
- M2 a" x( E, H; b( g    'SSE-KMS' and 'SSE-C'.
2 ~( N' I. N+ _/ K2 t  </description>
  X' c) |0 S" W, D, n& H</property>
5 F6 o6 K( a! O. F% X' L3 p$ U* ]  E4 k<property>
" l3 t5 O2 W8 C& v  <name>fs.s3a.server-side-encryption.key</name>. d" D/ R* Y& b8 a( E
  <description>Specific encryption key to use if fs.s3a.server-side-encryption-algorithm& Y  H9 A6 {) h
    has been set to 'SSE-KMS' or 'SSE-C'. In the case of SSE-C, the value of this property
' G5 z# u6 s& S    should be the Base64 encoded key. If you are using SSE-KMS and leave this property empty,; V. V+ z* u' \* k$ j, I
    you'll be using your default's S3 KMS key, otherwise you should set this property to* R; J; Q$ k% B0 W  D; D
    the specific KMS key id.1 t# ~0 G6 o$ p8 C/ j
  </description>
- C4 F6 J2 e6 D  V</property>. G, G3 s2 k  K( i
<property>
5 {7 }. y( G9 N4 t2 k* O  <name>fs.s3a.signing-algorithm</name>/ H$ X8 t' V3 P$ W
  <description>Override the default signing algorithm so legacy/ l# V  p$ D2 t" Y2 e6 T! B
    implementations can still be used</description>
9 D, L, P% D( T* x0 L$ Y( W5 y8 o</property>
9 t) \, K% |: u<property>" G9 T, p0 x* r+ W, D0 d7 _
  <name>fs.s3a.block.size</name>' g6 j' ?) N: p  |& E& S, k1 F
  <value>32M</value>/ }3 T- A- I8 v& p& C8 j
  <description>Block size to use when reading files using s3a: file system.' N  Y; d, a0 |& g9 i9 z
    A suffix from the set {K,M,G,T,P} may be used to scale the numeric value.
( c8 V+ L4 T) s8 T1 A) c  `& r+ n: K  </description>0 L; q9 i8 I# M3 E( T5 Q
</property>& Z: `# \! S& I$ M9 Z7 \, M; q9 w  U
<property>
/ y& F/ b# [5 g8 e4 S' H+ O  <name>fs.s3a.buffer.dir</name>0 V2 _6 ~" ?% A" n# g9 p
  <value>${hadoop.tmp.dir}/s3a</value>; ?. C/ e  O( N6 q" H5 N
  <description>Comma separated list of directories that will be used to buffer file9 A7 T; b# T  O9 B6 v
    uploads to.</description>
( Q7 J& x8 R7 ?, w2 f( Q</property>
3 C" q, @/ g. L( n<property># s6 v6 _4 ^! J$ h$ P
  <name>fs.s3a.fast.upload.buffer</name>5 M' V8 u9 h) c( {: q) y
  <value>disk</value>; A) j" L  P! Q7 }6 P; s
  <description>: Q( E8 z3 Y3 d  x# |% _
    The buffering mechanism to for data being written., {4 b2 e2 S3 ?" ]; r
    Values: disk, array, bytebuffer.
. b* X! b& p1 P- B8 U3 I7 Z    "disk" will use the directories listed in fs.s3a.buffer.dir as
& O, S% L; ^0 L0 W- j    the location(s) to save data prior to being uploaded.+ P- x* A; i5 C. Y
    "array" uses arrays in the JVM heap
2 c+ g6 f' Y' u8 _6 R    "bytebuffer" uses off-heap memory within the JVM.
5 E% m- Y4 A" }6 l    Both "array" and "bytebuffer" will consume memory in a single stream up to the number" u; T& |# O8 {' Y) w
    of blocks set by:
* U! g' f" \; {        fs.s3a.multipart.size * fs.s3a.fast.upload.active.blocks.$ ]$ \; W. |1 `! k3 s. L
    If using either of these mechanisms, keep this value low
6 w, k1 c0 ?! O+ e9 X% f    The total number of threads performing work across all threads is set by
) z7 k. w& b# O* Z3 R% f. [    fs.s3a.threads.max, with fs.s3a.max.total.tasks values setting the number of queued
2 H1 f, L6 F$ @* A9 @    work items.
3 P# `3 _2 E3 p  ^7 t2 Z/ f, Z  </description>
  W6 N# V/ h, f: H# d</property>
8 @( j0 J0 O3 e. i$ b<property>; z% e% p+ m4 Y! Z6 i$ h
  <name>fs.s3a.fast.upload.active.blocks</name>
6 h* `9 A% n/ b/ t  <value>4</value>2 N( a( q1 O9 F/ V7 I3 ]# C
  <description>
$ f3 y& @6 I/ k, T    Maximum Number of blocks a single output stream can have
$ ]; d: I2 O: K8 w5 q    active (uploading, or queued to the central FileSystem& ?9 }. [& B3 C7 d: z1 ?" J
    instance's pool of queued operations.& H# s9 w0 I$ v' L- f5 X
    This stops a single stream overloading the shared thread pool.
7 p. N3 h; g1 p" A2 t/ h; V1 l8 J; M  </description>6 x- O# }# x8 F6 _9 \- k# v* p
</property>2 A$ h9 G! A! h  w, Z1 J( F
<property>7 A7 q7 j1 H2 A& J0 b, O
  <name>fs.s3a.readahead.range</name>" m8 r- q' d! I% p- t4 O% `! w5 S
  <value>64K</value>+ ?, W9 v( J% Q; `
  <description>Bytes to read ahead during a seek() before closing and
, q) y8 ]" @+ J6 O  re-opening the S3 HTTP connection. This option will be overridden if& [7 j5 T; X0 m$ W' ^
  any call to setReadahead() is made to an open stream.
8 e* F. ~- j$ S# {  A suffix from the set {K,M,G,T,P} may be used to scale the numeric value.
* |' A( V5 K" [  </description>6 X$ R- k6 V  n, ~: @' S
</property>
  Q# l* ^. N$ Q4 L0 H- `5 z" o' S. N<property>8 M; X! [% `: Y- C
  <name>fs.s3a.user.agent.prefix</name>
1 y! ^0 K3 g" l, f( _  <value></value>
" O  R5 C1 N9 n  <description>. \# ~4 |0 J( s
    Sets a custom value that will be prepended to the User-Agent header sent in' w* l5 S6 e0 I- ^
    HTTP requests to the S3 back-end by S3AFileSystem.  The User-Agent header
9 {0 c' ~( R2 R7 s3 m9 g6 u" J' R7 A    always includes the Hadoop version number followed by a string generated by/ [' K6 b6 Q7 m: u( J
    the AWS SDK.  An example is "User-Agent: Hadoop 2.8.0, aws-sdk-java/1.10.6".7 B" W$ t8 g4 O: ], ?: m, Q
    If this optional property is set, then its value is prepended to create a
6 Q1 M4 v2 V4 A9 h; h    customized User-Agent.  For example, if this configuration property was set
- D- }' G6 z7 S7 T) K& F0 Q6 g, v+ l    to "MyApp", then an example of the resulting User-Agent would be
# r: T9 x, m  p! }. q% j    "User-Agent: MyApp, Hadoop 2.8.0, aws-sdk-java/1.10.6".
' z; Z/ g+ d* q3 b# {  </description>
9 c; m( `" M  n6 z. ^7 r</property>
9 G! j; m5 y2 D) E- J9 c: A<property>
# _9 T* ]* J& ~5 H1 k: t    <name>fs.s3a.metadatastore.authoritative</name>3 C/ T9 b+ X1 A+ l0 u( j5 B( R/ r
    <value>false</value>
, X' [5 h; W: T4 ?% E) P9 N    <description>
7 i) j: t+ F" b        When true, allow MetadataStore implementations to act as source of0 b' m9 D2 U0 r; M) h
        truth for getting file status and directory listings.  Even if this
8 W% s/ z/ a4 t6 B& E2 N        is set to true, MetadataStore implementations may choose not to/ k1 r6 D+ e$ I; E
        return authoritative results.  If the configured MetadataStore does
3 |7 N$ _3 ^" b8 p        not support being authoritative, this setting will have no effect.
. }+ \, u( r$ D/ N, Y% g    </description>
! N% E$ x: x) R</property>. S! ]$ F8 n5 c
<property>' [( _/ U( e( C1 n  ~
    <name>fs.s3a.metadatastore.impl</name>
; j- L5 d- c  g7 k6 E    <value>org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore</value>
/ j+ x4 ~3 T/ E0 R4 @3 U+ R    <description>
7 _( u, U& u7 e9 E! o( Y  Z! S3 [! [        Fully-qualified name of the class that implements the MetadataStore' A; D( e! f6 R2 \) z2 h+ V
        to be used by s3a.  The default class, NullMetadataStore, has no5 ]4 L4 S/ n+ C& u# I5 l
        effect: s3a will continue to treat the backing S3 service as the one, r. E6 _  C7 N- m
        and only source of truth for file and directory metadata.- b; _( g4 H8 @# l1 m
    </description>% h* y( m" A6 K5 t% I
</property>6 F( J1 Q/ v0 `8 u/ A
<property>: A5 G, |3 H! z) I+ i8 r: x: ~
    <name>fs.s3a.s3guard.cli.prune.age</name>
6 J5 ~  n; S" _    <value>86400000</value>
- W4 G( ^* n2 p2 {+ y/ R6 m    <description>( `$ m2 W/ L  C* m
        Default age (in milliseconds) after which to prune metadata from the
6 R% k/ U2 Z; Y. J6 a, u  E8 F5 ]        metadatastore when the prune command is run.  Can be overridden on the
& V* g% J% A* q- t& `! J, J        command-line.
6 j. L1 G9 K' z3 T$ I3 g    </description>4 y9 C* L8 A: O# n
</property>
# v- r7 z9 K; N/ D3 I<property>: a0 m7 {+ B9 L' p8 E1 I
  <name>fs.s3a.impl</name>
4 C1 B% P6 O& f0 E/ ?9 w  <value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
  F% M: z, F0 t0 K# e  <description>The implementation class of the S3A Filesystem</description>
0 p9 Z7 V6 Z. p! M</property>
0 @7 P4 W! i6 _<property>- e2 u; r: Q5 Q( V- S$ E% O
  <name>fs.s3a.s3guard.ddb.region</name>/ l" X% I8 Y0 v2 Q9 |- e9 G+ q/ ?
  <value></value>$ A, G/ ?2 n6 f+ O
  <description>
% p- Q& K  B5 v0 E  W8 E    AWS DynamoDB region to connect to. An up-to-date list is
; O; A# d- T4 P    provided in the AWS Documentation: regions and endpoints. Without this
7 [4 ~+ C; D# U& [0 `( C! p1 B* q    property, the S3Guard will operate table in the associated S3 bucket region.
- \, T/ `  D( ~# K% B  </description>, V$ [6 c% J! N7 _0 [; Z# Z
</property>
1 s: K1 w, n$ K5 z% K( ^' F( Z<property>
0 r* q, q& R& {; i4 Z  <name>fs.s3a.s3guard.ddb.table</name>
3 U* k: E  ]0 E5 @- I  T$ i4 J. \  <value></value>
! I6 `  |8 V0 {; o  <description>8 \" e0 b/ Q& e, c2 w  A
    The DynamoDB table name to operate. Without this property, the respective
- T0 A( X$ V& y1 o) t' H    S3 bucket name will be used.7 x7 c0 e2 }9 m- \6 M6 ^0 O
  </description>3 x/ U' |, B$ o1 k
</property>
5 y. `: x. J% F0 N2 u% A7 Z<property>( Y& l# o# v6 g
  <name>fs.s3a.s3guard.ddb.table.create</name>. u+ _! M3 d" r6 V+ v
  <value>false</value>
+ {2 t9 E( J8 h% L5 j5 A  <description>
* Y- {7 q; w( ~6 y    If true, the S3A client will create the table if it does not already exist.
$ x9 v* M3 _+ \5 W& o+ y  </description>' \, }" O  J* P& V1 X
</property>
% t# o+ H! k8 R<property>
$ k' U3 Z! R2 p0 V, M2 Z$ g  <name>fs.s3a.s3guard.ddb.table.capacity.read</name>  ~, i# u! j) B
  <value>500</value>& c4 g! z8 o% M/ H: k7 z4 Y/ m
  <description>
& d" @$ P  F+ \; _3 q    Provisioned throughput requirements for read operations in terms of capacity
! S2 l/ ~' _9 {4 g    units for the DynamoDB table.  This config value will only be used when
+ K; D1 z+ m( X& I    creating a new DynamoDB table, though later you can manually provision by
6 z' _( V* l4 Q0 y    increasing or decreasing read capacity as needed for existing tables.
. i8 Q6 ]' l# s    See DynamoDB documents for more information.; J3 J' N5 @* c6 b& l' `
  </description>. ^: U5 P0 n: \5 S5 }  K' h  [
</property>! f0 z; e/ I/ B7 i1 }0 X
<property>
8 C, V! {8 m/ \2 t6 D+ V' h/ Y( J4 m  <name>fs.s3a.s3guard.ddb.table.capacity.write</name>% u1 `- _- u! d/ @
  <value>100</value>
1 u" @7 Q% J2 s- C' O  P7 f. y  <description>) `4 R( Q6 O2 J  d$ h  k( F
    Provisioned throughput requirements for write operations in terms of
- v% _& D! y$ D0 B3 R0 z& U' j& b    capacity units for the DynamoDB table.  Refer to related config& ~* B  @; G/ N& a" c
    fs.s3a.s3guard.ddb.table.capacity.read before usage.  t5 E0 l9 K  n; Z9 r# {
  </description>3 G' w7 {" o3 K+ V- ]# Q
</property>3 r# g( L% ^( ]6 _( k5 d4 Y$ A2 {
<property>1 N( {; N$ K+ P) Y/ w5 \; B
  <name>fs.s3a.s3guard.ddb.max.retries</name>5 F* `; V& D, z& d; a
  <value>9</value>) M; y% ?9 n4 F/ q8 h
    <description>
: t3 i1 C  R" v8 f* d      Max retries on batched DynamoDB operations before giving up and
& A- t, q/ j$ A" M/ Y      throwing an IOException.  Each retry is delayed with an exponential% ^' g0 O/ d; K0 A3 [4 t' U+ v
      backoff timer which starts at 100 milliseconds and approximately
0 }9 l: c" C1 {) B      doubles each time.  The minimum wait before throwing an exception is) L& {5 v! T' ~$ ^2 g1 J
      sum(100, 200, 400, 800, .. 100*2^N-1 ) == 100 * ((2^N)-1)& [, t1 r9 I7 ]6 j+ N
      So N = 9 yields at least 51.1 seconds (51,100) milliseconds of blocking
% X6 A) Y2 O6 Q% P0 M0 N      before throwing an IOException.
/ K2 |' u; j% a" n2 Q    </description>
1 X9 s2 J+ ^2 _+ I, Z  x</property>
1 k, L$ y1 }5 {* g4 r  L" Y( w3 t<property>
2 R& y% `' {. W2 T' x0 Z7 j$ ?  <name>fs.s3a.s3guard.ddb.background.sleep</name>
2 n+ C+ c! c- M. `1 h$ X# ?  <value>25</value>
6 f( K8 g! j1 u2 q  d# T! |  <description>' D0 o2 e" v& k" H5 p' l" y
    Length (in milliseconds) of pause between each batch of deletes when
, s8 n$ {4 _; |$ t    pruning metadata.  Prevents prune operations (which can typically be low, W+ T; L7 S7 E. c! X
    priority background operations) from overly interfering with other I/O
- B8 d3 C. E0 C. c( E$ I    operations.
: B6 D  n# X, k+ @  </description>
) e1 I3 f9 @- H6 k4 l</property>4 y  p% q: [' V0 h5 X8 F' m. y0 z
<property>: m: ?" n$ Y7 l' s7 Q3 q
  <name>fs.s3a.retry.limit</name>
! u2 L2 K. e  X, _) h* v  <value>${fs.s3a.attempts.maximum}</value>. T7 e# t! v* q, _) @
  <description>
& o0 G, \, {: s9 f    Number of times to retry any repeatable S3 client request on failure,
  P. S! q3 B8 ?$ u1 `9 Z7 S$ J    excluding throttling requests.( U' ?! W' b. C" F# h: x
  </description>
* E/ l" V8 u, L: ]7 U$ c</property>
) J4 T& L: z/ R7 s<property>) {3 ^  O# F# o$ @8 y3 W
  <name>fs.s3a.retry.interval</name>
6 \( o0 D3 g( _6 A2 M  <value>500ms</value>. b* L1 b, @% [# {  c
  <description>2 Y( n5 J! R  y0 z
    Interval between attempts to retry operations for any reason other# f3 e2 n% b2 q2 J; d) |! R# v# B& d5 e- Z
    than S3 throttle errors.
  M' B# b2 r5 r5 B  </description>
& J  C" }, Y9 y( [, `# o# E</property>( d, J# l0 F6 H1 @; X. e3 _3 F0 J
<property>
& M! J. g, }# W" @/ H  <name>fs.s3a.retry.throttle.limit</name>
0 q) q4 ^" K, H) s) H  <value>${fs.s3a.attempts.maximum}</value>4 R3 `5 ~8 g1 K9 s# X* y$ Y
  <description>+ d6 j7 ]7 i/ C9 z; K" \2 h
    Number of times to retry any throttled request.8 O  }+ E" G% w) ^
  </description>
' T  l% W, D& ~. Q3 a</property>2 v; V" S# P9 I
<property>
) {9 }) A/ z# B! \5 K" I  <name>fs.s3a.retry.throttle.interval</name>1 ]' a( f# P, @( K+ Y( ~7 {; F2 y
  <value>1000ms</value>3 C1 H' ?. m' ]* K- g1 ?/ ^2 Q
  <description>3 s7 Q& X* h# K" f' K
    Interval between retry attempts on throttled requests.
5 x2 M1 K+ r/ S+ A6 C  </description>
+ o4 \8 J/ [* w5 E/ q</property>
: @! X4 k) ?" O' z% C7 j5 J  O<property>4 S, N5 |! _4 f. C4 V
  <name>fs.s3a.committer.name</name>
+ Q& F5 f0 [4 Q2 K  <value>file</value>
3 h9 \7 q6 Z+ P: D: `' n2 U  <description>- d5 _5 O1 l) ^( _9 B
    Committer to create for output to S3A, one of:  C- Q; t; |. T  B3 G( \+ P# \
    "file", "directory", "partitioned", "magic".
& X9 h/ ^+ V0 @0 O  </description>) J$ w$ }( ?4 {5 S/ ~6 O4 y
</property>
% K/ r* d, d2 x7 e4 m+ h4 l3 ]<property>
) K# g8 b' _; s& Y2 @  <name>fs.s3a.committer.magic.enabled</name>
5 ~* o, p7 p1 R( ?) P' q  <value>false</value>
5 @% S6 x" \( y  <description>
0 s5 M  H$ ?  m- A4 G    Enable support in the filesystem for the S3 "Magic" committer.
: H2 f+ z: c; e4 |3 D    When working with AWS S3, S3Guard must be enabled for the destination
( x3 E' S0 _; {+ m    bucket, as consistent metadata listings are required.. U; ~  D( p. I' u' H  ^% A/ i
  </description>( y* h, {4 E3 I) k  m- N$ A
</property>9 g3 \1 o) v% a! E1 ~+ S7 G! W
<property>; V) t% y3 Q# z- z% h0 D  }
  <name>fs.s3a.committer.threads</name>
' C- r6 j$ l* g) v  <value>8</value>
9 r$ W6 i3 ?9 b& x. {6 r; R) K  <description>1 N) J& M* l& g, D% v4 @6 C
    Number of threads in committers for parallel operations on files
" f+ p7 G- M5 J" z1 g7 V6 x" t" q' p- f    (upload, commit, abort, delete...)
  ]' u7 K5 _/ X$ G4 w5 _6 Z3 r  </description>
5 V" L7 {# b$ K: L2 [</property>
1 F1 Q( m' f* C. e7 w( }. {4 k<property>
* f) |2 k9 t2 v2 Q( G  <name>fs.s3a.committer.staging.tmp.path</name>5 m  j* f2 H6 ^. J
  <value>tmp/staging</value>7 S8 k  o2 H, V4 H2 m; Y; W
  <description>, b  A7 f% ]+ b
    Path in the cluster filesystem for temporary data.$ t) Z. `& a1 x8 Q
    This is for HDFS, not the local filesystem.. t& P4 l8 V8 M1 j
    It is only for the summary data of each file, not the actual( q( O3 W( n+ n& d" g7 v0 e
    data being committed.; z4 Q- ~( O1 R
    Using an unqualified path guarantees that the full path will be! J0 }; [4 K* d" y' w3 h% B" n
    generated relative to the home directory of the user creating the job,
9 K$ z2 u/ w) q" h+ ~  _3 p    hence private (assuming home directory permissions are secure).
* W# o7 u$ y9 H4 L2 i  </description>& H* U, _0 H4 I- B! k
</property>
, a7 N3 P  X7 x( W; J<property>
& K% ~6 g# n0 [  <name>fs.s3a.committer.staging.unique-filenames</name>( X2 T# L3 S0 P
  <value>true</value>+ l! |) d; Z8 q5 o
  <description>; P# \0 ^( |' {. r6 ^
    Option for final files to have a unique name through job attempt info,! B5 P$ g7 N$ v/ }
    or the value of fs.s3a.committer.staging.uuid
7 T! M; F* o! p! {6 D% ]; O    When writing data with the "append" conflict option, this guarantees. d# ^4 H/ n& |0 x
    that new data will not overwrite any existing data.
  s- x4 b8 Z" W# j  ]; y+ K: X0 S/ H  </description>- v1 d0 Z4 s6 F' H2 v( ?# n: S
</property>
& _) Q. u5 X; i<property>
  g7 M6 o! ~) }/ h$ k' |  <name>fs.s3a.committer.staging.conflict-mode</name>
! ^9 M0 B; Y8 F3 q7 q  <value>fail</value>
' i4 _5 }/ m* G0 u# s  ~4 m# l  <description>
0 {' D, L0 Q0 |5 X: F9 p    Staging committer conflict resolution policy.% u, ?5 S) S! i" [" Q9 K4 q
    Supported: "fail", "append", "replace".
# ?, [( F( i8 F1 G& @* N+ ~/ X  </description>
: n& o6 K& ~0 {7 _. K</property>
' v$ s& a* K6 k2 ^8 J2 e<property>
3 A8 O! ?3 z  V0 U6 W  Q. K( b  <name>fs.s3a.committer.staging.abort.pending.uploads</name>' F# y6 i7 F2 e# g+ Q
  <value>true</value>
) n1 N4 Q7 k* V; x  <description>. e# d& m. O+ ^5 s0 K( Y
    Should the staging committers abort all pending uploads to the destination
& k  z. v' i3 l# V6 U    directory?$ |; r6 M8 @6 r$ S
    Changing this if more than one partitioned committer is
/ ?4 s# ]' ~: W    writing to the same destination tree simultaneously; otherwise
9 h  m7 ^: |5 U4 ?    the first job to complete will cancel all outstanding uploads from the' Z: Q/ w: Z: }! V. X, t
    others. However, it may lead to leaked outstanding uploads from failed/ t/ G& O7 V4 q4 v
    tasks. If disabled, configure the bucket lifecycle to remove uploads! D: |# m3 C! r1 N
    after a time period, and/or set up a workflow to explicitly delete
" `& U; @) R" ?+ d    entries. Otherwise there is a risk that uncommitted uploads may run up
: K( j9 Q4 ^% D1 m5 P( B" y    bills.
4 z  E7 Q5 b! Z5 Z  </description>3 \5 S6 t' T! ~* N4 j! d( ~
</property>2 ?8 e9 q( g! F) I- t
<property>8 D( \9 N- S4 u1 D! R/ s: a' Y
  <name>fs.AbstractFileSystem.s3a.impl</name>
# m5 d/ T. }( e: B# U3 j  <value>org.apache.hadoop.fs.s3a.S3A</value>
' ?+ J  ~/ P) l. Z% O  <description>The implementation class of the S3A AbstractFileSystem.</description>
( [" T/ K* A5 R; w</property>; b" P8 k) w  k$ ]: i" E
<property>
8 n* ~3 V' S9 |3 g  <name>fs.s3a.list.version</name>% M# Z* T5 x% O  m! `
  <value>2</value>
% H% e/ T' W8 Y' l: e  <description>+ C1 h+ L- {8 ^" i$ D
    Select which version of the S3 SDK's List Objects API to use.  Currently& ]; p( K2 x. ]3 v! _/ P5 Q/ n2 o
    support 2 (default) and 1 (older API).( w; b- }. v6 z9 x) h; ~
  </description>, q# A3 z" }9 i& O# [+ M1 q  J
</property>4 Z3 e9 E. C2 K/ l
<property># y. |4 m+ J; S' d
  <name>fs.s3a.etag.checksum.enabled</name>
6 n% y) }1 C% r  r  <value>false</value>
& y- ?+ b' a8 `  <description>6 J+ s3 E$ J) o6 o2 K
    Should calls to getFileChecksum() return the etag value of the remote
' H" I6 q  D/ D, A7 |8 _- E    object.# E$ T0 g( z2 w: r$ E2 m
    WARNING: if enabled, distcp operations between HDFS and S3 will fail unless
1 J9 R& P4 u" u; U: h- |    -skipcrccheck is set.
4 d, f9 w0 s( o1 n2 D  </description>) _5 t; e4 l% W" k. n
</property>' f% G5 k5 Y$ _3 {; |
<!-- Azure file system properties -->
3 P# U$ v+ N* U; V; x$ I, \<property>$ d3 F) n; Z# ~4 y: ^
  <name>fs.wasb.impl</name>
2 K7 B; \( G% }  <value>org.apache.hadoop.fs.azure.NativeAzureFileSystem</value>/ R4 T+ u' b2 L2 O* L8 o9 i
  <description>The implementation class of the Native Azure Filesystem</description>
+ Q4 l( k% q# R6 D5 {3 p0 [  @</property>8 x0 [  `* o+ `  l7 Q
<property>; n' |. w) C& l
  <name>fs.wasbs.impl</name>6 n) j! p  P2 X) T8 H* R' x$ X
  <value>org.apache.hadoop.fs.azure.NativeAzureFileSystem$Secure</value>
( H) ^" h4 @/ B( n# ]  <description>The implementation class of the Secure Native Azure Filesystem</description>6 i+ |! [7 F/ n2 z+ N( d8 f1 p. }
</property>
# a7 e. P; e( F, e/ h2 h9 t1 X<property>2 p5 R: F6 u2 P4 u
  <name>fs.azure.secure.mode</name>
# G) [4 G% w$ z; J% S; J  <value>false</value>
% |7 v4 l- \) {6 _* ?  <description>
2 Z1 R0 M1 l+ G/ [7 n1 t) V    Config flag to identify the mode in which fs.azure.NativeAzureFileSystem needs/ L  d: H- V4 f
    to run under. Setting it "true" would make fs.azure.NativeAzureFileSystem use
# p0 }4 q! ^! R- g4 I    SAS keys to communicate with Azure storage.' T( Q- {. i8 z4 S; g
  </description>1 D+ U3 {6 t3 `
</property>, J8 g1 I, o/ O% ]; O" X1 R' n
<property>
  Y* ^6 w7 F# Z! ]& E  <name>fs.azure.local.sas.key.mode</name>5 _1 K$ g) _4 s5 A3 `9 ~
  <value>false</value>
; a" D9 K! g& v/ b, t/ e1 L  <description>4 g' {' d" ?( d# e6 F& u
    Works in conjuction with fs.azure.secure.mode. Setting this config to true3 K8 z8 W/ w3 W
    results in fs.azure.NativeAzureFileSystem using the local SAS key generation
& \6 D/ Z$ j& i; }* V0 F    where the SAS keys are generating in the same process as fs.azure.NativeAzureFileSystem.: q. l- f% F6 ?
    If fs.azure.secure.mode flag is set to false, this flag has no effect.
9 U4 r  q/ f( ^) m8 i0 F  </description>6 ?9 J7 G) J! j) ]0 S
</property>
& ^% Z( T+ C/ f4 R- X) N1 v<property>5 [' o1 x6 J1 o3 D3 x/ C
  <name>fs.azure.sas.expiry.period</name>
6 h% C7 ~; ~) [$ a  f: X  <value>90d</value>
% `1 H( ^$ J" E# k4 n, S' a) e* a  <description>/ q2 f0 a& d" S  X, S0 `) o
    The default value to be used for expiration period for SAS keys generated.  B3 ]: h) B8 |% B
    Can use the following suffix (case insensitive):8 d4 v$ E5 y' T4 z1 W  V
    ms(millis), s(sec), m(min), h(hour), d(day)
6 r& Q  {& I, X; L2 ^4 z    to specify the time (such as 2s, 2m, 1h, etc.).
+ k, Y+ H& o0 _  </description>1 x4 v+ i, C6 Y$ ?  I  R' ?
</property>
" w1 j! W- V/ h3 ~<property>
2 M/ L" s( k; _0 q! Q* C) ?  <name>fs.azure.authorization</name>
! f( f* _- B, s0 \8 k  <value>false</value>' V0 A( N- t, Y
  <description>5 Z; S: K/ N1 {% x# w& G5 c, ~
    Config flag to enable authorization support in WASB. Setting it to "true" enables. {" }, e2 r. r2 Y3 d( H! J& C6 A
    authorization support to WASB. Currently WASB authorization requires a remote service
: h- c5 j* G4 x( G( ]% e$ z    to provide authorization that needs to be specified via fs.azure.authorization.remote.service.url' i5 G8 q' R; w" D0 p+ c
    configuration
9 G) F  ^. W' h* I$ \3 W  </description>
2 L) e  v; p& A6 l5 w; a$ W- ^</property>- C* g0 ^+ y7 o' F1 _/ ]
<property>
1 m( P2 ?9 E6 t+ F  <name>fs.azure.authorization.caching.enable</name>! W: G8 {; l" t* d2 I7 g2 z: T
  <value>true</value>
2 g9 G6 x9 d( U  <description>  d" U  H& u0 d% h
    Config flag to enable caching of authorization results and saskeys in WASB.
& w; ^/ g3 ~/ j$ p; e    This flag is relevant only when fs.azure.authorization is enabled.
) ?; `9 w* h3 [6 u" s( g9 G  </description>2 q: C8 \; n, X0 J- C/ U! W
</property>
5 t0 i1 s, T5 y5 n/ a, Z7 O<property>( a1 F! ?& C) B/ A. ]4 X
  <name>fs.azure.saskey.usecontainersaskeyforallaccess</name>/ R0 z9 {* T1 R8 G5 M" y
  <value>true</value>
' a9 [; Z, ?& B0 G" W. v. v  <description>1 p; Z' J3 G. i0 f
    Use container saskey for access to all blobs within the container.
2 F8 i9 X. m; `7 v. A4 }    Blob-specific saskeys are not used when this setting is enabled.
0 k' W. O3 s/ u8 m+ j    This setting provides better performance compared to blob-specific saskeys.4 S* w3 h  x+ i' M
  </description>
1 a# p; {6 F  t8 ?8 {: {</property>6 Z/ ~  A! z1 p* _/ e
<property>
, I6 h6 w% n! o- o* j/ R  <name>io.seqfile.compress.blocksize</name>
6 d, b4 q, _/ n" `4 I  <value>1000000</value>7 @& g) y( N% y+ v) k: X9 Y
  <description>The minimum block size for compression in block compressed' }' K0 R7 o6 Z1 q1 M# h* h
          SequenceFiles.
7 z$ L2 j. b3 h/ |- h% n4 j  </description>
8 s6 F2 p+ l8 n; C, Z2 {( K' p! T" w</property>
6 g( r) d2 |/ u  _6 b- B) K: z <property>
4 A" q9 V* s8 s1 ?; C1 Z6 Q4 z  <name>io.mapfile.bloom.size</name>: m- H$ S# P) b
  <value>1048576</value>. h' X# a6 s  j# W( n
  <description>The size of BloomFilter-s used in BloomMapFile. Each time this many
3 D( m5 W  A: H* F4 H  keys is appended the next BloomFilter will be created (inside a DynamicBloomFilter).9 |' b% |- D/ q' ^" _, p9 ^
  Larger values minimize the number of filters, which slightly increases the performance,; V7 _. t* q2 `  R
  but may waste too much space if the total number of keys is usually much smaller# I' n1 P" G8 g0 w: e4 U1 v
  than this number.+ O- c# ^6 ]: m+ E2 q$ J- {; \
  </description>7 U# Q! m6 Y* z9 _' S8 f
</property>) Y, R3 C) c8 f
<property>
& m' h) q7 d/ a; l$ D# X. n  c" G  <name>io.mapfile.bloom.error.rate</name>, [4 v; t: i" d8 D
  <value>0.005</value>, d$ v) X9 u+ J' ^1 e+ t" {
  <description>The rate of false positives in BloomFilter-s used in BloomMapFile.( }& a6 J' i: ]1 g. R& {: Q
  As this value decreases, the size of BloomFilter-s increases exponentially. This* u" c) q6 _1 d) O' b
  value is the probability of encountering false positives (default is 0.5%).. T$ U* b$ ]+ \  _# c
  </description>
/ l# `; O2 H: S6 O; m% v4 w</property>
9 {' U+ v  K- r1 V. D) r+ |8 s<property>
5 d. b8 O3 M* N/ P9 G% }' J) N  <name>hadoop.util.hash.type</name>1 @+ X+ p  A( u# Z9 a. e$ c- X+ J5 W5 i7 W
  <value>murmur</value>  |; ], k1 ]1 ?+ D2 A* L
  <description>The default implementation of Hash. Currently this can take one of the7 Y' w: i4 V' R% }/ g
  two values: 'murmur' to select MurmurHash and 'jenkins' to select JenkinsHash.
% Z# W5 [0 a  i6 Y  </description>
' I  ^  B* T  d5 _( P8 m</property>
0 Z) a5 X. c3 |3 y% z, @<!-- ipc properties -->% Q( d3 Z3 O* O" C  @4 ~- t4 M* o
<property>$ ?/ O3 ^8 p" a3 w3 j: g
  <name>ipc.client.idlethreshold</name>/ O) O' |3 _# t2 M( _) p8 }+ v
  <value>4000</value>
4 [3 M& S8 Y6 F% ^8 k  <description>Defines the threshold number of connections after which) W% C9 }/ k# g! g
               connections will be inspected for idleness.( ~. z( U7 N6 @8 S8 v; r6 M
  </description>9 _$ b& g7 `, `1 E6 y
</property>
  E. Y2 n6 q2 [; E/ ]1 P' |$ L<property>
+ G; _* r9 P1 `) W& x- E  I  <name>ipc.client.kill.max</name>
. e( Y& K; v- E- k, I  <value>10</value>
/ R) L# J8 K. X  <description>Defines the maximum number of clients to disconnect in one go.
3 V1 f* l! |' f4 v' I  </description>
& Q' J2 h% u2 X# R' r4 Q( H</property>
$ @: |! t, T) t; Z0 t  g<property>
* y5 F7 x; m6 C2 [  <name>ipc.client.connection.maxidletime</name>) D% a( S! ]/ Q4 A
  <value>10000</value>! E7 o* M. q7 M8 ?1 e+ [7 [) K/ ]
  <description>The maximum time in msec after which a client will bring down the
" n$ P+ ^% L5 ]& @$ U               connection to the server., F; l2 Q" T3 c7 d' n
  </description>
7 ]* j, {- |" N& I" ]9 [</property>
, b- ~& E6 b% L, K4 ^<property>
2 R6 Y  V6 O, T2 b& k7 z: b  <name>ipc.client.connect.max.retries</name>
! E' C& }; \7 \- i6 V( j  <value>10</value>
8 D# h. S/ C$ u9 a2 w  a  <description>Indicates the number of retries a client will make to establish
( G9 Y$ N% s3 I6 d               a server connection.& _! I- z4 [- n1 H: `! Y
  </description>: t9 K5 y1 S% ?9 W$ U
</property>
( ^- O4 Q1 _$ }+ k/ I* A<property>
5 l* E8 N! l: R3 _  <name>ipc.client.connect.retry.interval</name>
$ D0 i4 a/ s) c0 g7 N7 ^  <value>1000</value>
0 |2 H7 o% u  s4 }4 |  <description>Indicates the number of milliseconds a client will wait for
9 a! c% e$ g9 y0 l    before retrying to establish a server connection.# X% n0 a+ u* K+ \) z% r
  </description>* q7 M* P  s& L5 o' {
</property>
0 l, ?5 }* u, q9 p# z<property>
  p% a4 U  d* D6 M3 D  <name>ipc.client.connect.timeout</name>
: V' M' F7 c9 p( ~0 a7 @4 ?. [  <value>20000</value>
3 q% s, P, V0 H) \5 X3 G1 Q. D  <description>Indicates the number of milliseconds a client will wait for the
* H: x3 a" r6 C1 w5 C# Q               socket to establish a server connection./ L% f6 u* O) ~; D
  </description>' Q- x% [2 {/ [4 @; T
</property>! a! f/ o$ A2 ~3 r- S
<property>7 {+ N9 m7 f) Q& W/ U( `' o1 |" p) c
  <name>ipc.client.connect.max.retries.on.timeouts</name>
" y, G% L  a9 R, |3 t  <value>45</value>: I1 Z7 e2 S3 x/ y5 U1 _
  <description>Indicates the number of retries a client will make on socket timeout
, X, \1 X; C6 n& T5 _: R1 ~               to establish a server connection., P' e/ Y( K, O9 x8 f3 }( H9 O" v
  </description>1 W0 s% [) d; @9 d7 V- v
</property>; V+ X& C$ P! `$ J' c; Z. f
<property>
# K3 \; |3 n0 ?. @. s1 w) h) I  <name>ipc.client.tcpnodelay</name>
' ?+ V+ \- ^5 W  <value>true</value>
; z- }. P3 S* D  <description>Use TCP_NODELAY flag to bypass Nagle's algorithm transmission delays.
9 _' @1 N1 s6 w1 Z/ J- o/ ~' ^) O  </description>' j$ t6 y2 c9 T$ }/ M! h( g
</property>
: Z  k4 j* a6 r$ L- ~0 C$ q2 B<property>1 W% z0 P# j& {: J
  <name>ipc.client.low-latency</name>
% ]5 H. \( Y- K& x: R$ S$ z$ E  <value>false</value>
& f) [/ O6 r: [8 Y$ q! F( c1 T0 c9 n  <description>Use low-latency QoS markers for IPC connections.9 k+ A# {! h* w  Z+ R
  </description>, ]; o* |' n& X6 V) t# s" Z, o5 t4 }
</property>+ ?0 u# M* L8 F" |& F5 H
<property>/ I7 U5 {$ c. C$ Y
  <name>ipc.client.ping</name>5 s6 `8 n% X8 u8 Q8 t+ G3 x
  <value>true</value>
. d# z8 K; E5 }6 h: m; @9 S4 W/ r  <description>Send a ping to the server when timeout on reading the response,
8 i) {; M# G. N; u- G  if set to true. If no failure is detected, the client retries until at least
( C' w0 s. r1 ~$ e# I7 F( ~  a byte is read or the time given by ipc.client.rpc-timeout.ms is passed., D6 ?$ U+ G% g3 M
  </description>; s  @4 L6 s2 u+ t- N' M
</property>
; b9 }. v' ~. Y: G$ ~$ J<property>
( P) U! r2 {$ @9 ]  <name>ipc.ping.interval</name>
/ e; m  @% @' T" y  {) m  <value>60000</value>
& Y6 b; z( f3 I. Z; L  <description>Timeout on waiting response from server, in milliseconds.1 l  j5 x0 h9 U3 G- c$ u! e
  The client will send ping when the interval is passed without receiving bytes,
: E9 ?! A3 Z7 q/ y& Q9 y5 r& l  if ipc.client.ping is set to true.
( x  L1 d) `6 g  </description>
( W( {' y6 o' u) T0 r9 i, q! w</property>" w$ f' V; \) y. F8 u2 P
<property>
5 u: _/ x" i- a# O7 g/ O  <name>ipc.client.rpc-timeout.ms</name>* h+ ?- r, X+ x4 ~  p/ g
  <value>0</value>) a, j/ U% t* o- n. h1 Q
  <description>Timeout on waiting response from server, in milliseconds.% P. M! P- t& g$ ]
  If ipc.client.ping is set to true and this rpc-timeout is greater than2 [( X0 ^9 R2 \; n% I
  the value of ipc.ping.interval, the effective value of the rpc-timeout is
" a" K* ?2 d/ K% L* x" ^4 }" r  rounded up to multiple of ipc.ping.interval./ V. k+ v' o/ x4 q
  </description>
/ V2 |/ G2 b6 @# u0 [* u- [</property>
. S+ o/ L- E' f# j- e<property>
+ J3 g8 I+ L; r: X  <name>ipc.server.listen.queue.size</name>, |. C& h$ J( N) ]/ q
  <value>128</value>
0 ~; X- L" J$ e% l2 B( L( u+ J  <description>Indicates the length of the listen queue for servers accepting
) |8 {1 `6 N/ O$ o: z               client connections.0 N2 H4 O2 X. P2 T# ~6 e% y4 H( `
  </description>
; S" @) j6 a3 b" b" b4 s</property>( }3 V- r  L$ y( U0 T9 n; u% M
<property>
) B( v/ R+ G" G# b9 H    <name>ipc.server.log.slow.rpc</name>" }5 Z+ ~3 g  A4 h
    <value>false</value>
6 v3 F( @% ]  j& g' |9 Y; y    <description>This setting is useful to troubleshoot performance issues for8 g8 p( ]8 I" p4 H; [6 v4 [
     various services. If this value is set to true then we log requests that
5 c. z; c) w% L3 f- D+ k9 M     fall into 99th percentile as well as increment RpcSlowCalls counter.
3 r2 f, g0 p8 B( }' Y5 G    </description>% S- l" |, \' x5 @% b; i
</property>
( j& x8 a3 H# o. _- P0 d) }7 c  Q<property>. b5 G- r8 c" C1 R. i
  <name>ipc.maximum.data.length</name>) F4 i9 r" h3 Q2 Z. h
  <value>67108864</value>. r' Y' `# o8 z/ L/ o
  <description>This indicates the maximum IPC message length (bytes) that can be% F+ [: l- V; v  \
    accepted by the server. Messages larger than this value are rejected by the
! |! p2 X$ K# N2 v2 j- d6 [! P6 X    immediately to avoid possible OOMs. This setting should rarely need to be/ y( o* E% P$ i5 q) ?' c: }2 q$ C
    changed.
. J6 t- V" g8 x. ~# ?  </description>; h# b7 H3 z" w! \  q( S# Y
</property>
0 ]* _2 _- x7 m' }$ y<property>/ U8 d1 W% C- N4 x7 N' F! \
  <name>ipc.maximum.response.length</name>
+ b; p& n6 W/ S+ `$ F" a  <value>134217728</value>
; L9 {% u- O4 Q. X3 h3 M# d6 g) T# A  <description>This indicates the maximum IPC message length (bytes) that can be: s5 P6 S" [! S
    accepted by the client. Messages larger than this value are rejected
: j% C1 A5 ]0 Y# H+ j    immediately to avoid possible OOMs. This setting should rarely need to be- ]( X( Z$ i( ]3 g! d; t$ _
    changed.  Set to 0 to disable.
! L9 W* n/ j9 E0 [- `9 O5 Y  </description>
8 X5 q1 ~* v1 n: l0 T0 K</property>
6 b0 s1 j7 b! s& p$ v) Q6 Y6 F7 ]<!-- Proxy Configuration -->
# k! ~' I8 F2 X<property>
0 T2 D9 I, o. c* D" w; K% d' F  <name>hadoop.security.impersonation.provider.class</name>( i3 H5 E5 k2 V
  <value></value>, ?1 @) I4 ]. s8 `" B
  <description>A class which implements ImpersonationProvider interface, used to9 J+ x' F5 w! A. B, T
       authorize whether one user can impersonate a specific user.9 O/ B: e$ J. z/ Q5 |
       If not specified, the DefaultImpersonationProvider will be used.: X3 E, M4 S" f; N  a6 @" U6 t% v
       If a class is specified, then that class will be used to determine/ b$ ^% J8 u* d5 c# m
       the impersonation capability.  s* P) q2 V3 U. \( R9 t( L
  </description>
/ A/ L9 k2 \2 u3 T5 C0 L9 t</property># T+ l- ^, T( H% G- f/ z& D
<property>
% i6 X# Y/ w% y3 J+ r: @  <name>hadoop.rpc.socket.factory.class.default</name>
; G! d0 j" h4 K' @* O  <value>org.apache.hadoop.net.StandardSocketFactory</value>) Y5 |1 ^# c# E; _1 |" _
  <description> Default SocketFactory to use. This parameter is expected to be
8 f. ~1 @3 T/ {; k( Z    formatted as "package.FactoryClassName".
  G4 N# _2 [  D( ~1 y  </description>9 g+ {; m; z7 B) F
</property>
3 f( j" F, N! |7 h6 \% z: O<property>
4 l% l: N1 _  A: y  <name>hadoop.rpc.socket.factory.class.ClientProtocol</name>
/ d, ^4 A) q6 n) ^' Y  <value></value>
' K! G8 f: S6 R' E  <description> SocketFactory to use to connect to a DFS. If null or empty, use' {7 W9 _4 Q7 q, h7 f+ B4 ~
    hadoop.rpc.socket.class.default. This socket factory is also used by, R$ s2 n$ n2 r5 o- e; k, m5 H6 O
    DFSClient to create sockets to DataNodes.
- u! P* q( p* p7 n  n  </description>
" ]% r' T' V+ G. w5 p$ Y- M- K</property>; t5 _- g* ^2 \! X
<property>
% t9 }, {6 k; j, u  <name>hadoop.socks.server</name>
3 H! Q4 w1 l( ^" V# D2 b" g  <value></value>5 W' I2 }- D" b8 N. ~' C
  <description> Address (host:port) of the SOCKS server to be used by the/ ~7 @7 {. s1 r( n
    SocksSocketFactory.
' ]- y/ c2 s. Y( x5 ~$ |4 m5 k) I" q  </description>
1 M! L$ Z6 e; }</property>6 G6 \( l9 X% Q' U
<!-- Topology Configuration -->/ C; c, V, W* U1 y4 a
<property>7 R# E+ u. M1 m* M: K4 u. v
  <name>net.topology.node.switch.mapping.impl</name>5 f0 s+ O. L! k0 e
  <value>org.apache.hadoop.net.ScriptBasedMapping</value>
" j; W$ j- F& }5 d6 B  <description> The default implementation of the DNSToSwitchMapping. It
4 i! X  q% k5 b: ?8 N7 v4 t: c    invokes a script specified in net.topology.script.file.name to resolve; _, n9 j8 z$ Y5 L. y5 w1 E* p
    node names. If the value for net.topology.script.file.name is not set, the1 _  a) m- t$ D/ ?1 b5 {0 z% ~2 \
    default value of DEFAULT_RACK is returned for all node names.
4 S6 h  r+ B% f* e- k- J  G$ r  </description>; h. H5 Y8 ]! g! G
</property>; t  F3 A% ^. u0 _& i
<property>% O/ i, C) S; C, T
  <name>net.topology.impl</name>
; \- D" |9 |6 ?2 {/ V- Z  <value>org.apache.hadoop.net.NetworkTopology</value>" j; b3 T; C) L5 {  Q8 X
  <description> The default implementation of NetworkTopology which is classic three layer one.0 ^  h* b4 \: S+ \
  </description>
" t8 t) J! }$ {/ B1 Y: u</property>& G* c+ H- t7 D
<property>
% ?* g  [% L# }% Z6 Z( o4 Q; c# Q  <name>net.topology.script.file.name</name>
* b+ ]' d4 S! P% S; i  <value></value>/ X. X, h0 `0 i( h/ s( p3 n3 q
  <description> The script name that should be invoked to resolve DNS names to
. q7 ~: A% y1 z  K    NetworkTopology names. Example: the script would take host.foo.bar as an
& v( n: Q/ u% L+ l8 r! @    argument, and return /rack1 as the output.. E; M" c7 ?( X) K, n% i3 R
  </description>
/ a  Q" j9 A4 W' q0 U; ]" W</property>
1 A$ ~4 J# V7 U5 }, x<property>
! v! g, O6 i; n* c* z  <name>net.topology.script.number.args</name>6 i9 a; i0 \0 `! z" }# w8 U* F
  <value>100</value>5 [: n, S6 Q+ w- |4 ~0 p- x0 ~5 {. |
  <description> The max number of args that the script configured with0 ~% P% _1 i' Y  w
    net.topology.script.file.name should be run with. Each arg is an
% {! t! E: ~4 {/ t0 z7 T! m    IP address.+ f( e& L% G& Z; Q  @( n
  </description>
2 u, u9 J% y3 M) e0 B! Z6 }. E</property>6 O- n- ^( q4 g3 l* s
<property>
. b: ~9 H( L9 Y9 H  <name>net.topology.table.file.name</name>
& I/ X* q7 B$ V: c+ Y  <value></value>. X# @& s  K9 N. j4 E: }
  <description> The file name for a topology file, which is used when the) z" X, g8 r4 W. [) @
    net.topology.node.switch.mapping.impl property is set to
& u9 c3 k4 o- q# `+ q    org.apache.hadoop.net.TableMapping. The file format is a two column text
# x) ~# ~8 g& C* \4 R1 q' ^    file, with columns separated by whitespace. The first column is a DNS or
5 z1 _1 C! @# a( \$ F6 R    IP address and the second column specifies the rack where the address maps.
2 @1 b/ O* S9 L    If no entry corresponding to a host in the cluster is found, then
* W5 R2 ^2 W2 g    /default-rack is assumed.
( u2 X* t0 N- e8 v4 B. v. r# ?) D  </description>
- X8 Y' X/ s+ t+ }</property>
, o( w* C0 {+ z2 U<!-- Local file system -->+ l# e# U4 P8 _1 A
<property>
: s- V2 b5 z. Y& f! S, Z5 L- H  <name>file.stream-buffer-size</name>: z% x2 K% I& ^! E; I
  <value>4096</value>
% J0 @1 R) e& W: O# i8 r" o5 F  <description>The size of buffer to stream files.
6 P% ^; u0 Q+ }; c. [5 |  The size of this buffer should probably be a multiple of hardware% y; r! X+ Y2 Y! ^$ X5 Q
  page size (4096 on Intel x86), and it determines how much data is9 s0 R& `! c3 R
  buffered during read and write operations.</description>
* ^) Q7 f. `/ r% S/ P8 V</property>8 l% s5 @( O& R3 Z4 \+ s. w* w
<property>
: Y+ _4 Z# I3 l. r- i  [* y* P4 `9 g  <name>file.bytes-per-checksum</name>" S. w. ?* k; L. D9 |( G9 P2 N
  <value>512</value>3 @: N# v1 O7 N5 e, }
  <description>The number of bytes per checksum.  Must not be larger than; j; C( U- i+ n+ k) Q7 c
  file.stream-buffer-size</description>
* y8 @# P" m! M2 [# G</property>9 z; s. s7 S6 \( u- ]+ h7 H
<property>
% y3 z9 i; |+ ?. u" v. a  <name>file.client-write-packet-size</name>. h+ v1 r1 V) a- b8 O8 K! M
  <value>65536</value>
+ ]- c  H7 G; j  ~% d" ~  <description>Packet size for clients to write</description>3 u" W! p) U; R+ j/ D
</property>3 m% v3 w  P) p6 r9 X: W" p0 y
<property>
5 m2 Y4 w! l: a5 g" H  <name>file.blocksize</name>
( P" Y. @4 D8 b) i1 Y  <value>67108864</value>
" Q. F# z4 ~* o3 X( f  <description>Block size</description>
9 n9 [4 t9 I: R) X" ~</property>& `7 u+ G' ~) D* j( x) \
<property>- e- J9 _5 O) r5 W$ S
  <name>file.replication</name>& w5 }+ Y! V6 D: Y* P# w# Z( ~+ k# W! @
  <value>1</value>6 t8 e8 K/ A" G2 }8 I3 Q$ C7 w4 p
  <description>Replication factor</description>
' I0 n6 c7 }. T( u</property>6 ?8 D+ e$ g1 r; o1 S( S
<!-- FTP file system -->
. T1 B- x' P; ]2 E$ C* Z<property>
  c9 X  E, T5 z) @) |0 `: y$ B0 u  <name>ftp.stream-buffer-size</name>
! u, t+ c; W/ [! X. W% w5 I' w  <value>4096</value>
( Y: Y5 q8 L' F( K% s( g  <description>The size of buffer to stream files.: f0 {8 M; l/ }  \: V
  The size of this buffer should probably be a multiple of hardware; j9 c% P$ g3 b4 T
  page size (4096 on Intel x86), and it determines how much data is( h; e, {; T% w5 I5 H
  buffered during read and write operations.</description>
5 c* u7 h+ f9 `/ S+ [2 V</property>$ W: I, I3 Y6 Z. E* t5 o: ], _2 ~9 u
<property>3 O( i0 y/ K, Z) l8 Z% T' Y
  <name>ftp.bytes-per-checksum</name>  g8 y- {5 l' m8 ^+ y/ U
  <value>512</value>" w& ^" @+ j# C( U: h9 e
  <description>The number of bytes per checksum.  Must not be larger than
" e. k( Y. m9 @  ftp.stream-buffer-size</description>$ j! O/ U9 N- S! ~; P0 P7 u
</property>) Z& X! V( E- W! [, G2 ~
<property>
* F1 Z* s" ]* P  i5 Y: x  <name>ftp.client-write-packet-size</name>
: m9 c, ^- L" I# S" y8 W. n6 l& Z  <value>65536</value>
3 Q% C4 n6 F) V1 q. S  <description>Packet size for clients to write</description>
: B. H  m5 o, [6 \) \9 }) E</property>4 i0 O2 u; ^3 q* h* j: p
<property>, H8 c3 K8 Q$ M9 ?
  <name>ftp.blocksize</name>
4 {/ n. \7 l  `) A# k; m0 W  <value>67108864</value>
/ L: q) f& j, w  S4 p* z  <description>Block size</description>
* t' Y8 h# X" G8 ~' M  g6 d</property>& |8 b6 `) C; g  v+ R
<property>
0 M( z, [& i; q& n' r. m: a  <name>ftp.replication</name>
8 E) c6 `5 K& i7 T1 C9 S% c) @+ K  <value>3</value>
# m4 i, A2 A' s7 D" n) k% ?6 a  <description>Replication factor</description>
( r4 _' I; P& {; }' O1 i$ n0 {+ B</property>
. g2 h$ w, e; o: `. K' o" b$ S4 {<!-- Tfile -->
- t& S; U4 `4 m& _( e9 a<property>
) q: L" x2 S4 H; U8 y' t6 `  <name>tfile.io.chunk.size</name>. i; Z. a6 O2 b5 w
  <value>1048576</value>3 C) t$ n/ \8 X! X/ j& D" v
  <description>
5 P) d% s) _6 I" W    Value chunk size in bytes. Default  to9 g  o) S! ]8 {! g. m# Z
    1MB. Values of the length less than the chunk size is8 v8 y" C- G& _1 K+ o8 r* t: H$ [
    guaranteed to have known value length in read time (See also
# L" W* p6 C0 q2 b. Q    TFile.Reader.Scanner.Entry.isValueLengthKnown()).
; C: M7 Z: s. K. k  </description>1 G, ~4 O7 a# l3 L% {
</property>' Z$ y: [5 V$ W+ I2 N. z. Q4 z5 M
<property>
' G6 M) T4 Z- S9 @& P, d, i1 X  <name>tfile.fs.output.buffer.size</name>
: j# t* J! b) S/ X5 W0 t" \  <value>262144</value>
: F- ~# Y7 d1 {  <description>
' G! v3 z$ I" D7 a8 A7 X; [    Buffer size used for FSDataOutputStream in bytes.' O& C& Q9 I+ v* E3 @
  </description>
9 Z( ~2 G: b5 f* N6 D</property>
+ ~& O+ l6 ?! s) L' Q" O, D" [4 b<property>
1 ^, F2 @$ {* M( S' m  <name>tfile.fs.input.buffer.size</name>1 |; i) {# r; H1 V: d  h
  <value>262144</value>* x' b2 b- ~7 [7 R$ c6 r' s
  <description>
# ^6 p9 s  ]1 S% e8 w    Buffer size used for FSDataInputStream in bytes.$ k( e: I1 |2 n# M; u
  </description>* p: B& h5 a7 O
</property>
: @2 B: c! L9 T( p) g<!-- HTTP web-consoles Authentication -->
* c* n/ S" s3 G& b2 \, D<property>
7 r) p6 M) S, A! z, H/ x  <name>hadoop.http.authentication.type</name>  J6 p0 C# E: e# B3 k  _
  <value>simple</value>
% I8 E! O+ n1 p  `) D" ]4 a  <description>
% E9 p3 X) g# G3 _2 `    Defines authentication used for Oozie HTTP endpoint.1 O$ P) S4 v' j
    Supported values are: simple | kerberos | #AUTHENTICATION_HANDLER_CLASSNAME#
- }2 O* x! h% ^$ y) U8 @  </description>5 E* X$ A6 u) s" F. J7 U
</property>4 j* e' o3 z% p$ l' R; q
<property>
6 F, f( f0 V5 m' @. G% G6 {2 F* M  <name>hadoop.http.authentication.token.validity</name>
; J) p* M+ V0 d  W5 _  <value>36000</value>
- b, h9 n+ m" U0 O* l2 k/ I( U+ H/ f  <description>
# ?" v3 p- x! `    Indicates how long (in seconds) an authentication token is valid before it has
8 ^; M0 }, r- q  Z    to be renewed.
; H. C5 H/ @! }( u+ V" L  </description>
; |$ ^% c) Q2 I" U9 k</property># o& T5 Y/ K4 B( @
<property>, y  V! q& }" w, f/ n0 m
  <name>hadoop.http.authentication.signature.secret.file</name>! ]+ B0 l( w7 _, x
  <value>${user.home}/hadoop-http-auth-signature-secret</value>
7 u0 \% K5 b# ^4 L+ k+ N  x# j7 \  <description># S9 G! h$ }% g" E) w+ s
    The signature secret for signing the authentication tokens.
( L  [  B! S0 \. R: H# b    The same secret should be used for JT/NN/DN/TT configurations.
( @9 @, ~% X+ O1 U- h: m  </description>* t  b# G  ^0 w/ T2 g3 i% ]
</property>
) s( y7 `: y  \& Z: M+ A<property>
# T) ~( i; I4 |/ [, {: f  <name>hadoop.http.authentication.cookie.domain</name>
  p6 m/ v+ K5 n$ `8 p  <value></value>: {1 g" ^9 r2 ^/ k
  <description>
5 C' o9 \  A3 H( K5 s/ D5 r    The domain to use for the HTTP cookie that stores the authentication token.
" ^( f* h4 _( w0 a    In order to authentiation to work correctly across all Hadoop nodes web-consoles( Y) t) V5 J- }9 j  m6 B4 u* ~$ o; `
    the domain must be correctly set.
# [4 v6 x5 @( `    IMPORTANT: when using IP addresses, browsers ignore cookies with domain settings.4 }# T" Y1 l# Q
    For this setting to work properly all nodes in the cluster must be configured
7 E8 T* O7 N1 Q! e5 F    to generate URLs with hostname.domain names on it.% D$ q' X: _9 {4 S! R) x0 _6 v
  </description>. ^1 z6 u, G! y" E5 n
</property>
9 M! }- a; I- L  F( M" I<property>; C/ A0 [5 s9 l  U; N& d
  <name>hadoop.http.authentication.simple.anonymous.allowed</name>; i( X0 N7 O: L* z- z, K9 |
  <value>true</value>$ A9 L3 I: V2 V& N5 B1 L
  <description>1 _  r7 U; W0 T: A7 {0 J
    Indicates if anonymous requests are allowed when using 'simple' authentication.$ w- \; D, ^8 W
  </description>1 t+ h) l) X0 x* e4 b. ]
</property>" m, e2 Y% P9 \5 B
<property>
" ?6 |4 q. u. H% \) a6 p7 e. |! Z+ a% c* `  <name>hadoop.http.authentication.kerberos.principal</name>3 ]1 b- ^0 `3 K- p/ P+ c' I) y
  <value>HTTP/_HOST@LOCALHOST</value>& E3 o* g1 B8 T2 k- e
  <description>
6 T" P# E( p  n: M# V- u  x# l    Indicates the Kerberos principal to be used for HTTP endpoint.5 j" d# U- l( C, l* ?
    The principal MUST start with 'HTTP/' as per Kerberos HTTP SPNEGO specification.* X0 ~  x% w9 q6 U5 y, `1 S
  </description>
( s' P8 N# m, m4 i. l</property>2 J- `1 s/ M5 e% c. q
<property>
# F: {9 W7 N1 l* T8 A- y% G6 ~  <name>hadoop.http.authentication.kerberos.keytab</name>
9 z  k8 k3 i  V: U7 i  <value>${user.home}/hadoop.keytab</value>3 P' E: j( ~9 t8 Q3 ^3 H" \
  <description>
! b3 E( |* I. q+ f0 X    Location of the keytab file with the credentials for the principal.0 a% i( H; \0 m2 X5 M2 l
    Referring to the same keytab file Oozie uses for its Kerberos credentials for Hadoop.
. A8 k) U6 h/ b  </description>
" F" O6 ?+ T% T9 h5 y; O</property>
9 c" S. G; d2 e<!-- HTTP CORS support -->
# a; X+ a4 n# a1 T<property>
. n& d; V& P) U- N  <name>hadoop.http.cross-origin.enabled</name>7 ]/ H7 _# x: s4 K! s# N# _
  <value>false</value>
+ `$ k; c6 D; z8 m  <description>Enable/disable the cross-origin (CORS) filter.</description>* k" P% I$ O+ e' G* }
</property>
* n, n" U5 l3 D! W' y5 S<property>
; q, b8 {' R9 m  <name>hadoop.http.cross-origin.allowed-origins</name>
6 K3 o) q, z; o$ N; T  <value>*</value>4 [9 n7 w  Z$ p2 t: Z  t- ]
  <description>Comma separated list of origins that are allowed for web services! q2 u/ t- \- w( c7 }
    needing cross-origin (CORS) support. If a value in the list contains an+ p: b9 e: Y  n2 t
    asterix (*), a regex pattern, escaping any dots ('.' -> '\.') and replacing& S/ N; {% L! P/ k7 v- U
    the asterix such that it captures any characters ('*' -> '.*'), is generated.3 }  [) c$ J: k% d
    Values prefixed with 'regex:' are interpreted directly as regular expressions,, o/ ~* G% J8 l9 p, l  \
    e.g. use the expression 'regex:https?:\/\/foo\.bar:([0-9]+)?' to allow any
. o2 |/ Q+ O2 O$ S+ x    origin using the 'http' or 'https' protocol in the domain 'foo.bar' on any  m* r" \2 B- v6 K: f3 r: ?( n
    port. The use of simple wildcards ('*') is discouraged, and only available for9 x8 C8 L1 h. a3 A$ J6 G
    backward compatibility.</description>
3 k* k. U* P4 j; i2 @/ o</property>
- B& g& G% H$ i1 T<property>, r& P7 C) E* J. U# x: T
  <name>hadoop.http.cross-origin.allowed-methods</name>1 J7 S  z6 H& S
  <value>GET,POST,HEAD</value>
1 q0 f/ x8 M8 n. v' W- B- Y  <description>Comma separated list of methods that are allowed for web* D. o2 W# e, [/ B9 z8 ~- Y
    services needing cross-origin (CORS) support.</description>
4 c) X) G4 t* f4 g</property>
7 D- x0 _& K& ~1 W- C<property>
. f; F/ |& _9 ^. Q, m+ b  <name>hadoop.http.cross-origin.allowed-headers</name>
" Y; g- j0 V4 j" F! l9 I# ~  <value>X-Requested-With,Content-Type,Accept,Origin</value>+ A7 }6 ~1 ]3 B- E+ w. |, e0 Z
  <description>Comma separated list of headers that are allowed for web
+ e( `+ w  O4 K2 W' P3 `$ r    services needing cross-origin (CORS) support.</description>. L$ C" [# z% c
</property>9 w1 f" y, U- B; F& X
<property>
  K. n/ d" S7 f: {  <name>hadoop.http.cross-origin.max-age</name>. S) @  q& M7 Q8 _; o: \& m
  <value>1800</value>% L  m; w% n% z* I
  <description>The number of seconds a pre-flighted request can be cached% J/ B0 z% y6 O* R+ g5 b
    for web services needing cross-origin (CORS) support.</description>! r9 K* b% ~4 r* G, f( d* O& Q8 `/ }
</property>
5 L3 e, ^' N5 W/ U/ }" D7 k' P<property># c! @" t% ?1 Z6 Q) i, ?+ h
  <name>dfs.ha.fencing.methods</name>! L6 P  F+ j; M  O
  <value></value>
! q; X8 `6 Z2 G  <description>3 X3 j/ D3 D' V) K
    List of fencing methods to use for service fencing. May contain2 `/ G% f8 m+ `9 R2 U) L$ V, b
    builtin methods (eg shell and sshfence) or user-defined method., E# c/ d; v, A) z0 ?
  </description>
( l9 V, e  v4 j; K2 U7 N9 K</property>
( `7 V0 W: w3 p+ t<property>
9 f' R, E. a, S  n5 R; D  <name>dfs.ha.fencing.ssh.connect-timeout</name>  l; V; g- W5 t7 Y, _1 y
  <value>30000</value>. z9 r  R$ ]" Y$ W: I5 H0 x
  <description>+ I5 w" J4 B8 p. c" k( P
    SSH connection timeout, in milliseconds, to use with the builtin3 R* g+ q5 g& w% ~; U( G
    sshfence fencer.& z& s1 `+ Y* L1 s9 |
  </description>% z: N; O7 `7 ?. `. z" ~( a
</property>
' R- L- n2 _9 m<property>( y- X3 k+ H  e# e3 R% z- {6 H# K# J
  <name>dfs.ha.fencing.ssh.private-key-files</name>
7 \9 M& T! }( h) W. ?  <value></value>
0 J$ M. J. V0 ^1 [3 V' ^" O  <description>
" U4 [+ H' @+ l9 O% @/ _    The SSH private key files to use with the builtin sshfence fencer.
* B; A( [, q! p  </description>
8 z' q; B3 O3 k) M</property>
" I8 |' ]* w# W- J% @<property>
- s; C+ G5 V# [  <name>ha.zookeeper.quorum</name>
' r) j+ Q+ U* m; F8 ^# B  <description>9 A. L  Y8 i" e9 x' [& h
    A list of ZooKeeper server addresses, separated by commas, that are
" `# ~  M7 @5 r8 x' \; z5 }    to be used by the ZKFailoverController in automatic failover.2 C4 T2 `& z% l3 d6 O' l  x
  </description>
1 f+ u& s$ h8 n, G</property>
4 a3 C( Y) c1 {<property>
0 q, X, G1 S( N* \  <name>ha.zookeeper.session-timeout.ms</name>
& r& b$ I: s+ o5 I+ S  <value>10000</value>
6 G6 n5 q. {; |- [* O  <description>: l6 v: X2 I( t9 p
    The session timeout to use when the ZKFC connects to ZooKeeper.
4 S) P$ M$ B- I" M    Setting this value to a lower value implies that server crashes4 l- ^2 N) b  ~$ r, \
    will be detected more quickly, but risks triggering failover too
; m/ }# ?5 o+ b) a: a& G2 q: b9 a    aggressively in the case of a transient error or network blip.
# {/ z* g, i' A/ @3 E/ E9 J* ~  </description>/ o7 f( `1 A1 Q8 V/ E
</property>
1 O2 l! P4 ~7 L<property>
# l6 v) [3 K, r4 o& |  <name>ha.zookeeper.parent-znode</name>
, `' p: i" ~4 X2 Z  <value>/hadoop-ha</value>
. U# t2 R8 g6 a7 R  <description>
1 `# ?7 x$ V" W, y, h    The ZooKeeper znode under which the ZK failover controller stores
$ C! Z  ^' C3 i4 f, Y* s    its information. Note that the nameservice ID is automatically
9 L! Y. A8 s. i& m3 x9 L6 [) U4 \0 M    appended to this znode, so it is not normally necessary to& [; F# ^( l& O
    configure this, even in a federated environment.. e: }, ^9 a! a  f
  </description>5 J9 G% `3 R4 X" j
</property>
( U  @6 N% F/ v<property>
2 H7 ~& z! n( F  <name>ha.zookeeper.acl</name>
4 R5 u8 e( o7 L! G' @  <value>world:anyone:rwcda</value>8 [2 w" z$ o, r) n$ v
  <description>4 X5 y* l4 ^/ B& D
    A comma-separated list of ZooKeeper ACLs to apply to the znodes
' |5 M4 ?+ j( J. h/ \3 Y6 E    used by automatic failover. These ACLs are specified in the same. a/ W1 e% K' P+ E
    format as used by the ZooKeeper CLI.
0 ?5 ?+ K3 z; ?) f: v) P5 z" _    If the ACL itself contains secrets, you may instead specify a" u  w* A8 O* w/ y: `3 K1 ~
    path to a file, prefixed with the '@' symbol, and the value of
7 l  Y0 |1 I3 ~* N2 }    this configuration will be loaded from within.
1 e5 s; J9 d1 Y. F4 A" T8 h5 P' T  </description>5 g/ D; |, Z2 S0 a! u0 j! x8 P
</property>
' n4 l0 s1 O; I; b+ J<property>& P7 Z+ h. v. e* m5 g
  <name>ha.zookeeper.auth</name>' M, ]0 }0 O) n0 S6 Y, h% i6 h
  <value></value>
% A) N6 ?5 F! k/ T5 D7 w  <description>
' O8 J( Z6 y" N2 O" |    A comma-separated list of ZooKeeper authentications to add when5 f- f( G9 N* j& f: H
    connecting to ZooKeeper. These are specified in the same format
$ w  L) Q! {$ p. n! x4 k    as used by the "addauth" command in the ZK CLI. It is
8 P( N9 }$ `) ~! t; a# ~    important that the authentications specified here are sufficient9 y- L/ t1 i% i# l* T; f
    to access znodes with the ACL specified in ha.zookeeper.acl.
5 P0 Z% f- C# l$ e! u    If the auths contain secrets, you may instead specify a- ^# w0 }4 @7 `/ |5 T8 |
    path to a file, prefixed with the '@' symbol, and the value of
! m' y, u7 ^" \' o8 D* P) r9 l8 E    this configuration will be loaded from within.
( S- c4 K* ?  i  </description>
) _0 F. D3 |# U8 `  L# Q$ ]. J' G</property>  \- |. e8 m, L
<!-- Static Web User Filter properties. -->
8 Z* i5 ?. Z# O: |<property>1 [4 E  ^0 A9 N0 }5 s
  <name>hadoop.http.staticuser.user</name>
5 G2 ~( R% N$ |( q: D  l2 U8 K. G  <value>dr.who</value># ]+ ]% U6 x! V3 m) g, x7 V9 k
  <description>  F  x8 j. Q5 [) ?
    The user name to filter as, on static web filters6 |! {5 h3 s% n# K, c, K+ B* w
    while rendering content. An example use is the HDFS  ~' A, {( p7 j
    web UI (user to be used for browsing files).5 i/ _% H/ e+ a: |# B5 H4 d4 q
  </description>5 g, r# S- r( z) b6 r7 n+ D
</property>
4 K, l3 x, a# A, J4 b<!-- SSLFactory configuration -->
1 M" y' B- w' J( m" p! }4 h, l<property>- I9 |: ]" y* g( s
  <name>hadoop.ssl.keystores.factory.class</name>
( e. Z7 {( K' H8 g! J  m  <value>org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory</value>
* p8 r, I' T& i" S- H  h  <description>1 t, O1 p& B' ?& ]
    The keystores factory to use for retrieving certificates.% h  d: z; h# x; w( n
  </description>3 O; m4 u  }& ]' N" ~3 D3 b: X
</property>, p$ \  `) }: a: H) j% T/ ]
<property>
; N8 R3 E! ^- H+ \0 Z  M# t  <name>hadoop.ssl.require.client.cert</name>) l- ^1 q9 L; z4 w; k  s
  <value>false</value>. f3 ^! q. G$ {, `$ q# C
  <description>Whether client certificates are required</description>
9 T! @- q7 P8 L8 E7 B</property>
; x  {5 P1 P" U& S* L8 T<property>- `- H  W' D$ i
  <name>hadoop.ssl.hostname.verifier</name>  l" n0 I7 q( f* E) u# \
  <value>DEFAULT</value>
: G9 T4 I# s4 B% v7 p6 r5 l  <description>
9 e' n5 O6 O" M. O    The hostname verifier to provide for HttpsURLConnections.( p8 Y; q  B, o2 w; _& l) u5 a" p6 \
    Valid values are: DEFAULT, STRICT, STRICT_IE6, DEFAULT_AND_LOCALHOST and
- N3 w& o6 m5 N9 A8 I    ALLOW_ALL
, x9 B; K: s& u" ?) j/ T  </description>  c# G, O4 j; G* j2 X4 b) ^
</property>
8 X; I! M' \6 O4 [" K% c3 }4 z<property>0 h' q) i% F0 W' |7 ~
  <name>hadoop.ssl.server.conf</name>
' o" }. l- H( w4 E: J2 t  <value>ssl-server.xml</value>" V% T+ O1 x5 G( c, F1 a! W! B' Q
  <description>7 Z+ S$ Z# T3 k( Z' s
    Resource file from which ssl server keystore information will be extracted.. D# E) \" X0 t& c" O7 r
    This file is looked up in the classpath, typically it should be in Hadoop" Q, K9 @( c+ @' m+ W0 h9 g+ T
    conf/ directory.
( f$ R$ F5 H' t5 {  </description>
9 w; @5 ~, z5 F# d* o</property>8 i) g7 Z6 X5 y1 o8 t% i% j% q0 y
<property>
2 i/ p3 ~% |. v2 N  <name>hadoop.ssl.client.conf</name>
9 k( S! c: x. t) I  <value>ssl-client.xml</value>
( g3 S& a) @, @* [5 M  <description>
4 S% u2 k  y! D2 k; B! _+ ~7 |& B    Resource file from which ssl client keystore information will be extracted0 X& m7 d5 x2 a" g4 m
    This file is looked up in the classpath, typically it should be in Hadoop1 x9 X/ x& X8 ^+ ^4 O1 k
    conf/ directory.
+ G1 }; h' @2 ?6 K  Y  </description>
8 B  S. ]7 A( R2 s2 m</property>
, l9 _) j( J! y" T6 f) F+ k  Q9 K! [<property>
- W2 ~, ^' S9 [4 b. T# U2 g2 l' G  <name>hadoop.ssl.enabled</name>
' }- V+ \2 ?) I' E8 l% L3 Y3 A7 b  <value>false</value>% n4 h9 [" q! d8 l, H
  <description>. l+ t1 H- ?5 F/ b
    Deprecated. Use dfs.http.policy and yarn.http.policy instead.
/ u3 u$ J# _) e* V, _* w% A  </description>. f: u' G0 X$ H! c+ {: {# Q" h
</property>  h+ v% p2 t, x. [) s. `' c' N
<property>: `) f' H* |: k& A8 p
  <name>hadoop.ssl.enabled.protocols</name>
3 v9 y$ z7 J3 m" k$ `& c  <value>TLSv1,SSLv2Hello,TLSv1.1,TLSv1.2</value>/ Q' E/ {8 [1 q/ S1 \: m$ v+ J
  <description>
0 w7 U9 Y2 Z8 x* \7 o! P; V    The supported SSL protocols.
: k9 k4 O- @# o+ }  </description>
% C. x! k4 B5 W9 H: h$ z5 d</property>& H7 r* |& C2 Y* h3 I9 K
<property>
5 ?5 H' a0 c! G. _6 m  <name>hadoop.jetty.logs.serve.aliases</name>( J, L; S8 b6 k( @* e3 P( A; u
  <value>true</value>
; t. o4 z: u) ]8 R; o0 |. o; K  <description>
2 s9 O, }7 \+ [+ ~    Enable/Disable aliases serving from jetty( i7 U1 v6 B8 W) j1 q! e2 g
  </description>
: ?5 P: }# O3 M7 c9 r</property>5 K2 y3 K  H: @+ K, P9 p/ L
<property>
: r$ S3 j' q4 @  <name>fs.permissions.umask-mode</name>
% G( c2 Z1 j$ x' u# \  <value>022</value>6 E1 I( u- r# B! ~) S  {
  <description>1 m- K7 s/ D4 {
    The umask used when creating files and directories.
8 ^5 {7 `  u- o    Can be in octal or in symbolic. Examples are:& i5 n# n# r# ?' r/ w$ o
    "022" (octal for u=rwx,g=r-x,o=r-x in symbolic),3 I" I2 [. A9 i" P
    or "u=rwx,g=rwx,o=" (symbolic for 007 in octal).
' R2 e5 {3 i% p: l' a+ t" Z  </description>
3 ]. _1 B5 K5 |) F</property>2 D$ l, L9 R) ^; ?
<!-- ha properties -->
/ R( o. L/ T1 Q8 t6 Q3 b<property>
* L5 k4 X; K: k  <name>ha.health-monitor.connect-retry-interval.ms</name>
: d9 h. E% Q4 ]) I. B7 N" G  <value>1000</value>
' n1 r3 k, ]/ J  o$ G1 |7 Y8 E- ]  <description>7 M* a- c, {& L) V  z: u
    How often to retry connecting to the service.7 y+ H( h% J7 Z+ p9 u# h% o
  </description>. Q! }/ L: a: Z5 M* O$ Z- v
</property># _2 i5 I: Y- [+ M3 {8 v* D' T" v
<property>
' B( O. B# o# p3 ^  <name>ha.health-monitor.check-interval.ms</name>' w  T5 {- `* O# z* M+ G& t/ j) M4 z
  <value>1000</value>
% M% Q, J+ V& F* X' Z4 W9 Z  <description>5 G' T6 u. O& ?+ @0 T  I
    How often to check the service.4 ^0 |  @  H3 Z4 v! N6 C; A
  </description>$ [$ s0 w! k7 Z% [  B5 ^) {
</property>( G2 C: C# F- r+ z
<property>
9 T& v% n8 w9 e" A/ Y  <name>ha.health-monitor.sleep-after-disconnect.ms</name>8 k& f- c! D% _" ?  e; \
  <value>1000</value>2 Y) }' R, ^; B2 v% y
  <description>
, T% F7 A. c- L% O8 [    How long to sleep after an unexpected RPC error.
4 z4 b4 ?/ Y3 @( ]6 l0 Q, e  </description>
( \# m/ t6 l" n8 d' Y7 Q</property>5 \7 a$ }+ R; w
<property>
; Z# r  z% u/ N  N% N  <name>ha.health-monitor.rpc-timeout.ms</name>: @" y  G0 A; e4 G! o
  <value>45000</value>! ]8 j* U$ a" z1 N+ y
  <description>& o" z1 Y7 t* N5 w& R
    Timeout for the actual monitorHealth() calls.
6 R. O6 o* y2 j0 [( I4 r$ Z  </description>, I$ q4 l2 p' v- E
</property>
6 V' c' K9 I% ^/ _! _* w<property>
( T# _8 G. \5 _8 }# V$ U  <name>ha.failover-controller.new-active.rpc-timeout.ms</name>/ z6 }& I: v2 v% ^% R. R
  <value>60000</value>
( S; @  s' f3 |& I. d2 Y; x" O  <description>
3 f" O. ~+ n/ D6 q    Timeout that the FC waits for the new active to become active$ h5 Q( }; t8 T1 I6 K# w- c, _3 G4 @
  </description>, k1 N4 J" }& S/ e7 B0 }! \4 H' K
</property>
" R  V  y% u8 C<property>* b% a$ z8 n' p) w  P
  <name>ha.failover-controller.graceful-fence.rpc-timeout.ms</name>, Y! }/ X/ ]( H3 ^$ ]' i' Q& ?7 L
  <value>5000</value>
0 {5 S( Q8 M+ }' z+ r  <description>
+ G4 k9 {6 z! `4 E- F    Timeout that the FC waits for the old active to go to standby" o. q5 F5 m& ?7 Q( S5 f% S
  </description>+ L7 H) Y' E. Z/ I% l* A. H; M* n
</property>
3 N; |/ c. [& X5 `1 z<property>* L! Z8 t/ U2 z4 g! C+ e, K% C
  <name>ha.failover-controller.graceful-fence.connection.retries</name>
- C# E# Z8 T. G3 \  <value>1</value>
6 W' w- h- L8 E5 W4 L0 Z3 c! k  <description>
8 R/ q- D/ V0 F4 ^& k4 G8 B    FC connection retries for graceful fencing
8 {( r8 `4 m* V" v8 @# T  </description>; X9 R( `6 I, |4 r5 d
</property>' [1 h' @( V# d2 M0 x
<property>8 |2 n0 `- P/ X) _4 s
  <name>ha.failover-controller.cli-check.rpc-timeout.ms</name>
( n- T) t8 r  f* A' |  <value>20000</value>* y" K# J) s$ Q
  <description>+ m" t$ L7 y4 [! U
    Timeout that the CLI (manual) FC waits for monitorHealth, getServiceState
# @+ s3 P5 r  W0 V! V4 c' Y  </description>; T9 ]- u) c. s/ p! l' L
</property>) {- D4 s1 }2 Z) H/ E2 B
<property>
3 K( J/ o5 A6 C* g% \+ @  <name>ipc.client.fallback-to-simple-auth-allowed</name>+ f& r+ {9 T  l- y; Y6 D" Y
  <value>false</value>. n  Y  T$ M# w/ \* j( E
  <description>; p: O0 Q8 |( _4 x; T3 h
    When a client is configured to attempt a secure connection, but attempts to6 P0 Z4 |$ |6 r4 y+ G2 @
    connect to an insecure server, that server may instruct the client to
* @% k' p+ K' [5 U' @8 l, M    switch to SASL SIMPLE (unsecure) authentication. This setting controls
) p: g! m  v& ^, {1 Q) S6 W( `    whether or not the client will accept this instruction from the server.
3 t! }4 u' G$ O% Q1 i( c4 |6 [    When false (the default), the client will not allow the fallback to SIMPLE
( V5 R. u4 ~+ g! j3 C    authentication, and will abort the connection.
2 D3 |0 A) \+ m) L3 c. k6 x  </description>
( Q4 r+ i( f. W0 U& ]/ C6 ^</property>
6 e! Q( P, i9 h% v, w7 R<property>
6 [6 N" H( c  M1 g! m  <name>fs.client.resolve.remote.symlinks</name>
4 j, G1 x/ n6 N# |. b  <value>true</value>
/ R1 m& [) H6 M( }# x  <description>: `3 |; }6 A: a& u. J6 I9 s
      Whether to resolve symlinks when accessing a remote Hadoop filesystem.2 s; ^/ J% m; J( a! n4 B! l
      Setting this to false causes an exception to be thrown upon encountering
1 m9 p  k; J2 K- k8 z( C" L( {      a symlink. This setting does not apply to local filesystems, which
2 ?( t+ J2 [8 o. h( a& u% P      automatically resolve local symlinks.5 t! N. g5 U2 Q* |
  </description>0 @! C) G; x$ V
</property>, `% ?' Z+ i0 e0 w1 U2 {
<property>
$ H2 A8 \/ `) b8 [# D  <name>nfs.exports.allowed.hosts</name>9 T  |- s+ E1 s9 s( m2 E" E# l+ a- d
  <value>* rw</value>
/ ~+ i0 ^# }3 a- b, }, C) {  <description>8 h! q# k- ?5 u; Q
    By default, the export can be mounted by any client. The value string3 Q" C! {- g& j4 f4 [' B
    contains machine name and access privilege, separated by whitespace- _7 t, [+ z% R" d3 ~' q! q
    characters. The machine name format can be a single host, a Java regular1 O" s, ~6 f# n5 h% M8 P# `" e3 `
    expression, or an IPv4 address. The access privilege uses rw or ro to
" \) U5 P* X* F( W- j. \    specify read/write or read-only access of the machines to exports. If the
2 S7 ], M% W" O. D    access privilege is not provided, the default is read-only. Entries are separated by ";".$ g) Q, r8 Z6 U
    For example: "192.168.0.0/22 rw ; host.*\.example\.com ; host1.test.org ro;".  K/ e, P/ q! a' j
    Only the NFS gateway needs to restart after this property is updated.
9 `/ ^3 r6 K# R. a' w  </description>" U/ e. i7 {' g6 \. L0 P
</property>8 i6 w$ V* I" f/ w3 G, k
<property>
( g$ R+ J8 e6 I# i4 R+ I  <name>hadoop.user.group.static.mapping.overrides</name>
/ y( c6 h4 Y3 ~; _$ ?; E  <value>dr.who=;</value>
+ x6 F1 P. a0 K( S. |  <description>+ m( g% D: b4 Q/ f% z2 r  I
    Static mapping of user to groups. This will override the groups if, f8 s' x8 l- y* z
    available in the system for the specified user. In other words, groups( l6 b# T1 N9 E
    look-up will not happen for these users, instead groups mapped in this  Y: h2 A1 k) k* e
    configuration will be used.
4 m/ u  A9 M+ ^' Y& }    Mapping should be in this format.
/ G3 V% |. ~; o9 N+ y' s2 I$ ^    user1=group1,group2;user2=;user3=group2;# ?! i. }" {8 m$ t
    Default, "dr.who=;" will consider "dr.who" as user without groups.
/ l( W* d( o) @# F  </description>
' M$ R! R6 H: c6 m</property>
& @' G8 ?$ `/ m. y0 n, a3 g! U<property>$ U; {0 v4 l8 `) X, i: j
  <name>rpc.metrics.quantile.enable</name>, y( w8 l! J- s+ O- H; B: O! r
  <value>false</value>
  ~, I( J5 J# m/ S3 u9 m+ a/ V  S" I  <description>
4 U$ j" {$ D  K4 @, U6 P    Setting this property to true and rpc.metrics.percentiles.intervals; {; }2 ^0 T1 J% D  Q0 }, A: j
    to a comma-separated list of the granularity in seconds, the
  _: ^& F, u1 _; b, o    50/75/90/95/99th percentile latency for rpc queue/processing time in
2 w* x8 q9 E- c# V    milliseconds are added to rpc metrics.! e0 v8 B5 g( _; p! T# p
  </description>0 x/ b4 V( ~9 ^! s) b
</property>
) f; Y! z; N9 [6 _% b# I; N- ~<property>, x  L: g8 n! f5 r( a: R
  <name>rpc.metrics.percentiles.intervals</name>
6 E* G7 @( X( @. B% I- t  <value></value>  B9 X6 ^8 ^7 O! U- z1 o
  <description>
1 u& t. g0 W/ |- P) B6 Z" v    A comma-separated list of the granularity in seconds for the metrics which
: G. V( _* S% F. j    describe the 50/75/90/95/99th percentile latency for rpc queue/processing
/ q& c" C8 {- e/ T% _    time. The metrics are outputted if rpc.metrics.quantile.enable is set to
/ F( l. V$ g* y6 o4 p) H8 a    true.
! v" u0 K9 F0 O( I- Z' j8 s7 p. r# c  </description>' Q1 w  u, f  B. v
</property>
$ V* c- B6 I0 z( f: Q- N<property>
/ D. Y1 d! }. K( H3 p4 E3 g  <name>hadoop.security.crypto.codec.classes.EXAMPLECIPHERSUITE</name>
0 n  g% X/ L- K# ]4 }1 j$ m. F  <value></value>/ J5 a# G% K& w9 v! _
  <description>
7 k! I# @. C( @: m" m    The prefix for a given crypto codec, contains a comma-separated
, B/ V( S) I' [; Q. Z    list of implementation classes for a given crypto codec (eg EXAMPLECIPHERSUITE).
& @7 p% ~- f4 ]7 ]7 j    The first implementation will be used if available, others are fallbacks.6 c: D/ _2 _" q5 d/ }
  </description>5 O9 C- @0 h. ~% Q. g4 U7 g- c
</property>
) n" T/ M( M6 o. ~9 [8 k<property># E( D$ `5 |+ H6 b
  <name>hadoop.security.crypto.codec.classes.aes.ctr.nopadding</name>
! H( |5 Q  E& D" }+ _" h0 D/ _1 Z  <value>org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec, org.apache.hadoop.crypto.JceAesCtrCryptoCodec</value>/ a) n4 \) z* F7 J
  <description>
9 C. v; h* c- p0 r9 i    Comma-separated list of crypto codec implementations for AES/CTR/NoPadding.( y9 o9 k6 G/ K( J9 I* r7 H
    The first implementation will be used if available, others are fallbacks.
+ c" ~( I/ N# b% b  </description>! I) [, c4 A) @  F/ H, X0 @
</property># l# ?# o9 `. k
<property>
  i- e+ D4 x; i! n5 U3 a  <name>hadoop.security.crypto.cipher.suite</name>
8 {9 {5 t8 O3 ?: n+ X7 X4 F  <value>AES/CTR/NoPadding</value># w% r$ \4 ?) x. X
  <description>
  Y, Z8 K5 E+ o! j    Cipher suite for crypto codec.6 L% w5 @$ Y. P) P( d. S/ K
  </description>3 q4 T# v/ f; s3 V$ |) K% |% R
</property>
! B, E) A& v  ^! C0 G% E5 C1 d<property>
- z& b, {2 J- c3 \9 _0 u) ?  <name>hadoop.security.crypto.jce.provider</name>
' p  b% Z  Y5 C+ i* `/ O& B  <value></value>5 j% ~/ z3 {/ I( a( [7 g
  <description>
, a/ L! F$ c- i: w    The JCE provider name used in CryptoCodec.& i( `' D( ~- q% [% h7 ^) W# F
  </description>2 _( }6 |& w. Y& C" [, S1 }+ k' h
</property>& M8 d4 {; a3 B1 N! k
<property>- U* X0 C7 \7 p7 m  ?1 l7 S+ k
  <name>hadoop.security.crypto.jceks.key.serialfilter</name>
4 @, C9 i+ G) v/ R3 Y  <description>
& g0 K; ]) M( C2 \2 A" G9 T/ I6 Z    Enhanced KeyStore Mechanisms in JDK 8u171 introduced jceks.key.serialFilter.
2 A5 s7 H$ D9 i& J/ M! _    If jceks.key.serialFilter is configured, the JCEKS KeyStore uses it during
: R+ y1 K2 ^8 N$ v# G    the deserialization of the encrypted Key object stored inside a! o) y9 }7 b' `1 V5 w% |
    SecretKeyEntry." ^( X( Q. o' N" f
    If jceks.key.serialFilter is not configured it will cause an error when
5 t4 h, B! O( s; _    recovering keystore file in KeyProviderFactory when recovering key from
8 j3 p$ e6 \- [" [' p! ^    keystore file using JDK 8u171 or newer. The filter pattern uses the same& E" z+ R7 a/ r0 t1 {
    format as jdk.serialFilter.. `: h! P" O: o( A
    The value of this property will be used as the following:" y6 V  c* o1 l0 U" _
    1. The value of jceks.key.serialFilter system property takes precedence" E: q6 G& |; I2 E& Q
    over the value of this property.
8 L$ C( C/ Z6 m, |1 o: t% q    2. In the absence of jceks.key.serialFilter system property the value of$ D. {$ Z- N0 b8 l
    this property will be set as the value of jceks.key.serialFilter.2 h- x6 m, [$ K1 t( P
    3. If the value of this property and jceks.key.serialFilter system
) y+ K1 `& ]3 L9 r  m0 Z4 G& K* B    property has not been set, org.apache.hadoop.crypto.key.KeyProvider" D$ T  Z  b% M1 G: ?; ~0 c
    sets a default value for jceks.key.serialFilter.
% n5 p( ^! {4 v, q  </description>& i+ f1 M( L  e1 A
</property>
8 n  D  ~% c1 O' `! m<property>
1 S% ~7 J; p: _" \2 J/ Z  <name>hadoop.security.crypto.buffer.size</name>
' D7 g  k5 J. U  <value>8192</value>
' E8 x7 v1 \1 [; z  <description>0 C& l5 y2 O$ j: I0 W3 P( q
    The buffer size used by CryptoInputStream and CryptoOutputStream.
9 }- y( c" C6 J9 ~% r3 o2 [  </description>
" o0 K7 D1 v4 ?+ d# m</property>+ F* D& m$ C* s( y8 i
<property>
. c$ z5 \5 N$ t9 I, D5 C  <name>hadoop.security.java.secure.random.algorithm</name>3 m0 K; Y+ p3 V
  <value>SHA1PRNG</value>
& s" u' F. h7 H" C  W  <description>/ s- S. l0 c& {2 E: i# A
    The java secure random algorithm.
2 ]' ?. U2 V3 c& `* K6 L  </description>; t  W2 l  Q2 `& ~5 Y+ i
</property>
$ r1 ^" T1 x# B. ~<property>
6 j. Q& Z9 ~% s8 |2 [4 n  <name>hadoop.security.secure.random.impl</name>' H, P) T, P' K$ w
  <value></value>
. g5 _; a/ N2 F8 V  <description>
$ _4 U0 D, D, }3 ?    Implementation of secure random.
8 K* \' r3 p* ?  h2 b; |  </description>3 s8 G: {/ m8 u- M" p+ Q8 N  d
</property>
5 g9 o0 U: \( W5 d8 X3 {<property>+ _1 \, C9 N: d7 T1 O2 n' _; ~) n% M
  <name>hadoop.security.random.device.file.path</name>
! S* Y1 r! S, D' Z% n  <value>/dev/urandom</value>2 H* G0 s9 S" o+ W9 b
  <description>/ X/ K& r% ?# N6 a6 ]
    OS security random device file path.
" o  w; a& l. y/ ?: M% I6 ]# C( j  </description>
/ G+ _9 y! I: \, L* \, {</property>5 O' H: d6 I: @4 @* P. |& ~9 P% y  U
<property>
5 b/ @- k6 X8 Z. S  <name>hadoop.security.key.provider.path</name>' g* J6 s8 u5 N, z  X: B
  <description>; p. a  F) D5 x; A% D6 u$ H$ a
    The KeyProvider to use when managing zone keys, and interacting with+ h- F. l% P% W: n( Q' L5 t
    encryption keys when reading and writing to an encryption zone.6 H  K+ ]+ w. P6 r6 W- J
    For hdfs clients, the provider path will be same as namenode's4 c; z, X9 D0 H4 {
    provider path.
" `% y6 O1 B; G: [( {: Y  </description>
9 p! D7 z; q- y9 C% |, a  a' }+ L; h</property>  z3 a/ D" H" m3 q3 t( _7 i
<property>
. F1 L1 X$ N8 q1 G" R  <name>hadoop.security.key.default.bitlength</name>: o: V2 H; t1 Q+ I9 h
  <value>128</value>
6 t; G. l- Q5 R/ c* c9 F) k  <description>
; h: Y7 `4 V* u, \- V( P( |    The length (bits) of keys we want the KeyProvider to produce. Key length/ Y( g. u. f: ^9 N( @
    defines the upper-bound on an algorithm's security, ideally, it would/ D9 E1 D) L- n8 c) c
    coincide with the lower-bound on an algorithm's security.8 X, w: G& d/ v9 o4 L; _
  </description>
8 U8 p6 W/ N5 t3 ^</property>
3 Q2 N! D3 r* A7 l% j. l1 f) P  L<property>1 \5 s% B- M5 r2 t
  <name>hadoop.security.key.default.cipher</name>
5 X# a  p3 f* U" p7 j0 h  <value>AES/CTR/NoPadding</value>! k* H  K  D; E, I2 O& E
  <description>
. t1 F' X: Z5 X( L- b/ y! [    This indicates the algorithm that be used by KeyProvider for generating
) `4 ~, K% Y2 _5 |  P. p8 O    key, and will be converted to CipherSuite when creating encryption zone.
9 H5 N7 W  l% C- Y, \  </description>
+ H) d' T6 v- J6 T) A</property>/ Q: h; T& {) k" ?0 V
<property>3 s# a4 P+ X, g- D3 f9 F0 ^
  <name>fs.har.impl.disable.cache</name>/ h- i8 d$ K  q% J
  <value>true</value>' D& J, Y& i4 h4 l5 g# J3 |% [( [4 h7 ~
  <description>Don't cache 'har' filesystem instances.</description>0 n3 i( Y2 k1 Y% b) G
</property>
, `: ]' `( k  |) O$ K4 a( L<!--- KMSClientProvider configurations -->. V7 a8 S2 G9 R. P
<property>9 I4 y4 a6 D: G
  <name>hadoop.security.kms.client.authentication.retry-count</name>, |  x- w4 t- \' ~
  <value>1</value>
, \3 w, \4 e: x! d8 A( Y( J  <description>. E- C2 W0 w, M
    Number of time to retry connecting to KMS on authentication failure& a8 f: _: z  }9 @7 ?; r* M
  </description>9 W% T+ }" n: x
</property>3 _# f6 i/ R. A# Y
<property>5 e; P4 {$ M$ b
  <name>hadoop.security.kms.client.encrypted.key.cache.size</name>8 l2 @+ H; [6 u6 [2 B* |6 Q
  <value>500</value>
# z0 ?' i: V- S. l  <description>
7 t, N* W9 l1 c. F+ I    Size of the EncryptedKeyVersion cache Queue for each key
6 x; r, D' p: r' z$ `& K0 v* m  </description>
* A4 g( L: ~6 ]% [) w. k3 f</property>$ W9 U+ R2 C2 {  E
<property>
9 K) h; F; O2 K7 d; I. [9 i  <name>hadoop.security.kms.client.encrypted.key.cache.low-watermark</name>
7 U6 j9 o5 L2 r7 E  <value>0.3f</value>7 j+ P5 u2 D4 G1 {  N% \# K
  <description>
8 w, a% [- f8 R: f$ {    If size of the EncryptedKeyVersion cache Queue falls below the7 E" c3 O& v( ]8 {5 V
    low watermark, this cache queue will be scheduled for a refill
9 b7 J' n9 z+ i  </description>4 b1 e# o; o( O) b6 X4 e# M
</property>
6 w1 j7 d* d) j' K4 S' k9 {<property>
5 V  T+ c2 J3 W6 d5 L# ?+ h  <name>hadoop.security.kms.client.encrypted.key.cache.num.refill.threads</name>1 R0 T% h) g5 I1 i3 }
  <value>2</value>; T* z3 Y3 Q4 Y+ H1 e3 M7 K5 ^
  <description>
7 k6 v8 n6 O0 I( Z. Y5 W7 [    Number of threads to use for refilling depleted EncryptedKeyVersion
& c$ x2 [3 s7 a! K9 G$ g7 J0 ?    cache Queues9 z- |  _9 U/ E# w
  </description>5 h. R$ p1 `) W) T1 D
</property>6 m! S! D: g% Z* }8 u7 t4 ]
<property>7 O. j- r0 Z' G5 o  `$ p8 w
  <name>hadoop.security.kms.client.encrypted.key.cache.expiry</name>( O, T$ B' J& k2 i! R! J1 |
  <value>43200000</value>
7 O8 C: U% [  o4 s  <description>
1 G( A3 o% ?% c; _+ A+ W; o    Cache expiry time for a Key, after which the cache Queue for this
3 Q* T; c7 }0 f    key will be dropped. Default = 12hrs6 P+ Y3 `: ~2 Z
  </description>7 \) }4 Z& H; [+ U5 _3 e
</property>
! z- H) s/ r: z- E" e/ ~<property>
3 a8 o  p( L8 z  z, n  <name>hadoop.security.kms.client.timeout</name>
# x3 c( I, X& f  <value>60</value>
% n2 z' Y$ j0 \  <description>
$ ^6 e, K: j  k% {: N( Y! M3 @    Sets value for KMS client connection timeout, and the read timeout# k3 i' T" u/ p3 i- L; Q
    to KMS servers." W! }; B( t# y" G8 `* h
  </description>' N( w5 `1 i5 n  \) Y9 U! u
</property>
+ F& O/ ?2 B/ _3 r<property>
% ~* S( ?9 G3 h4 z  <name>hadoop.security.kms.client.failover.sleep.base.millis</name>
" V) G8 D, }, c0 g  <value>100</value># O2 S+ ^( R( B* I
  <description>$ G# e' E3 @1 a2 p: a
    Expert only. The time to wait, in milliseconds, between failover" i2 O: ]4 \3 i0 |6 e  T
    attempts increases exponentially as a function of the number of. F/ ~: n( I$ T9 N; w) A  ?
    attempts made so far, with a random factor of +/- 50%. This option
' p( h3 x. e% ^    specifies the base value used in the failover calculation. The! Q# y6 M5 D' Q  @7 w) E: D% ?
    first failover will retry immediately. The 2nd failover attempt
' W! I1 m) \8 ?' @5 D' Z* `    will delay at least hadoop.security.client.failover.sleep.base.millis9 L( h4 ?/ v& l0 I
    milliseconds. And so on.0 g1 z; n- U- ?5 J8 J
  </description>
& C8 P4 {$ S! D! n1 Z( k  }</property>
' X9 @- a, u" [- v! C& w<property>
, w/ Y6 E4 k' I2 x  <name>hadoop.security.kms.client.failover.sleep.max.millis</name>
' k$ @- Y( E( W' O& V) Z  <value>2000</value>  q1 A" w0 \( I
  <description>
$ @; O7 k$ x( I4 ~. P, D    Expert only. The time to wait, in milliseconds, between failover
; @2 u% W/ o% V* m+ v5 y    attempts increases exponentially as a function of the number of/ A* T, ?- t# B* V0 k# c: p
    attempts made so far, with a random factor of +/- 50%. This option
2 c) K' F4 M+ K    specifies the maximum value to wait between failovers.
: f6 i1 o7 R! ?5 F: F    Specifically, the time between two failover attempts will not; [- H9 k2 f% Q% O$ v' \. O! @
    exceed +/- 50% of hadoop.security.client.failover.sleep.max.millis7 m- m! x: S$ F: j  k4 j
    milliseconds.
% e# v( r7 r, X1 ^+ n; U+ m  </description>( E$ b) z! D& `: J+ E
</property>) T" q. L( O. I. e+ b  z9 T- U  Z9 j
<property>
% u8 }" Z! I- q8 ~  L; e" s9 C  <name>ipc.server.max.connections</name>/ |0 D- Z" Z- M* H: V+ Z' u
  <value>0</value>
" q, R6 F3 p. U  <description>The maximum number of concurrent connections a server is allowed
+ y4 ^4 O" |1 w1 q" M    to accept. If this limit is exceeded, incoming connections will first fill8 b& z7 L' n! f  A
    the listen queue and then may go to an OS-specific listen overflow queue.
# O4 H! T; W" B* K! g& b4 z: [- R9 E    The client may fail or timeout, but the server can avoid running out of file/ w5 F) ?( B+ ~& q' ]4 p
    descriptors using this feature. 0 means no limit.
3 I. m: y7 j. ~" j9 L# z( ]( y  </description>
& ~& p6 @# K8 o+ R* U</property>3 o! {$ a" P5 y; C. a
  <!-- YARN registry -->. V. d6 x- N3 R( L8 L+ v, V/ o
  <property>" p% ]+ w& Y( K: ]0 h3 q9 \
    <name>hadoop.registry.rm.enabled</name>3 L8 G9 @' A: R+ j  v  i
    <value>false</value>, O/ S3 Q7 d2 i! ~
    <description>+ C# p; N" I' r" F* p8 N! z* V
      Is the registry enabled in the YARN Resource Manager?2 A; _* W' m. I
      If true, the YARN RM will, as needed.* f: J/ r8 _' r
      create the user and system paths, and purge
5 y: R* H( s. _4 u4 m5 j3 H+ l      service records when containers, application attempts
1 W3 q& r& ~* X8 q+ T; @6 t- b      and applications complete.
2 }/ d5 ?0 p- @$ K; k, ?      If false, the paths must be created by other means,
- \  y4 Q, p9 H- @, |5 Z      and no automatic cleanup of service records will take place.3 M- S$ w& j2 `( ]  b$ D
    </description>
5 x0 b, A8 Y. C& D; O- f7 l$ \1 u  </property>- y3 }& f$ p3 y. V
  <property>
- B/ z3 ?% R$ t) X# `3 ^    <name>hadoop.registry.zk.root</name>1 u( ^. A, S. P7 N  ?9 u% c
    <value>/registry</value>
' f3 e( b* D( P7 Y# L! L' f( {  v+ |    <description>
1 G$ ~( B- l! H+ x, g      The root zookeeper node for the registry' Z( [8 O: n9 k% L8 p/ t
    </description>" k& l( g5 D5 Y( b# X" X
  </property>2 }" [, t5 f% B$ j7 c
  <property>, w$ e+ q( M( q6 d/ }5 @' Z8 l) e
    <name>hadoop.registry.zk.session.timeout.ms</name>1 j3 \' {/ E. ^' s. H
    <value>60000</value>
) s" }2 `, z4 a, X    <description>
& E& N' s8 I" O5 y3 ^' T; C      Zookeeper session timeout in milliseconds
# R6 B/ r0 k2 G: @6 V    </description>
. [( |2 c& }+ n& b0 |9 D/ _# n  </property>  e5 p% @  E* R9 m, F! n
  <property>, Z9 y4 O7 s" Y
    <name>hadoop.registry.zk.connection.timeout.ms</name>- i' X) K/ @9 }, g+ r; n
    <value>15000</value>
  |8 C! h3 {0 a  p    <description>/ Y+ j* C: d- u2 ^" @. S* P
      Zookeeper connection timeout in milliseconds
0 W0 I3 }# |+ w3 l( ^    </description>
$ K' s' g' x/ g  R  i  </property>
* s$ e4 E( s3 Q* T0 X% @" {  <property>8 z4 B8 P  n1 D0 ?
    <name>hadoop.registry.zk.retry.times</name>5 E, N& ?+ ?3 M2 }! |2 q. v  _
    <value>5</value>
3 ?- p1 A" C2 R6 T5 \  ~    <description>' k/ F( ^) v  l$ ?  W
      Zookeeper connection retry count before failing
' @$ z8 ?5 q9 K4 _2 B. |    </description>" Q1 U+ ?6 P( i! [5 Y  a4 ]
  </property>2 N% D& u/ D2 a
  <property>* p# l2 O) l: r1 l( }# v
    <name>hadoop.registry.zk.retry.interval.ms</name>" t. }/ g1 E2 ?
    <value>1000</value>  c/ O6 r, d2 \& X7 V
    <description>
9 _. q$ Y7 D. r- f    </description>
0 \5 A: k' A7 i' y. Q( j  </property>- ^& v4 A  K5 b" L
  <property>0 N: ]2 ?7 v# a$ _
    <name>hadoop.registry.zk.retry.ceiling.ms</name>2 @4 P  ^8 h$ p9 Y- a
    <value>60000</value>
) x; ~. B- S: K7 I    <description>
/ y  d- f0 P2 `      Zookeeper retry limit in milliseconds, during
# f& h$ d% f8 g      exponential backoff.3 L( \1 N5 [1 \" i6 `
      This places a limit even
4 _3 }) G# Q: S9 J) a( t9 b4 K      if the retry times and interval limit, combined, Q6 R1 ^; Z0 o. a
      with the backoff policy, result in a long retry0 |% l2 K: r- i& O5 C5 I
      period
: ?4 c" e5 H/ B; {7 H    </description>8 T! O5 [+ W& M2 A( Q: G
  </property>5 t# F! N% x! N. @* \1 b* N( T+ `* l
  <property>! \* W9 k4 g  E
    <name>hadoop.registry.zk.quorum</name>) o7 K: H* J) T  V' o& N. |
    <value>localhost:2181</value>0 S/ f" A3 Z% }7 }
    <description>8 g# N; T6 y" f- j/ e  Z
      List of hostname:port pairs defining the
) z! F, A- ?& l2 n+ _      zookeeper quorum binding for the registry1 {# u3 O9 h- |* r, I7 c1 B
    </description>. T4 l& A0 A$ O% B6 |( o( l
  </property>
  x3 O- s& z- j" V1 l, _* r  <property>
9 t( f% ?+ i2 O/ ?9 x    <name>hadoop.registry.secure</name>
: v7 h. i6 k$ o    <value>false</value>
; D; V4 ?( s- R$ n4 d1 D    <description>, Y5 f8 S; i! x- W7 W8 v
      Key to set if the registry is secure. Turning it on& L; c$ h% h2 t4 r4 C. C+ P, g( O
      changes the permissions policy from "open access"5 y2 S! q7 T# _) v5 {$ Q' i% K
      to restrictions on kerberos with the option of3 V; s0 `6 T' W8 S# {: X
      a user adding one or more auth key pairs down their- E; S3 \  g3 w! h* \6 B, Y( x
      own tree.
' ~  D; H$ Y( L) K    </description>
$ `1 {5 x5 w3 P3 Z2 F6 Z3 U  </property>
- E9 m# F/ s9 y6 {  <property>
5 Z5 X  E$ A7 f2 I5 [# [    <name>hadoop.registry.system.acls</name>
2 O1 L' R+ q1 B2 \" b# d    <value>sasl:yarn@, sasl:mapred@, sasl:hdfs@</value>; M: ~( D/ l/ l
    <description>
; d- c" _: X2 Z7 \4 l% ^      A comma separated list of Zookeeper ACL identifiers with
  h5 q9 }& c4 E      system access to the registry in a secure cluster.
7 A" N- @( y; X      These are given full access to all entries.: y7 \8 M! G3 N) C4 r
      If there is an "@" at the end of a SASL entry it1 `4 E5 K3 d, m( N( J
      instructs the registry client to append the default kerberos domain.
- U2 k( G! p+ S6 z    </description>
$ n( Z) ^  z$ m9 D  </property>
, f& n1 e  U2 l# X+ q  <property>
9 }7 P" d6 [5 G- t    <name>hadoop.registry.kerberos.realm</name>
* ]8 ^- F1 J: Q& T, f$ e    <value></value>
4 k  I! h) t; J: L# T    <description>
. Z: c0 n; g7 b5 `( e      The kerberos realm: used to set the realm of
( A0 u% g5 G. X/ J* e' ]" R+ K. O      system principals which do not declare their realm,( C6 Q) v2 s6 T' t* k; k1 K6 ~( A
      and any other accounts that need the value.
5 L* W, J" x+ [6 \. ^- P8 y      If empty, the default realm of the running process
6 w9 `* m' `' o8 u9 h      is used.
0 F, L/ v# z  ?0 ]' e' A. M% T      If neither are known and the realm is needed, then the registry
; L& ~7 z7 g* V/ C4 V      service/client will fail.
6 [- y7 [. T4 \- u* g* G    </description>, p+ f) N0 y0 f+ K. ^& u2 V
  </property>
3 b5 z9 @" U. K! t: ?  <property>4 q5 V& F8 P0 `1 R5 y
    <name>hadoop.registry.jaas.context</name>2 ]6 I6 D+ ]: T6 o
    <value>Client</value>
5 O  f* Z; r( r    <description>
' F: ]! I' P7 H      Key to define the JAAS context. Used in secure
% M! e" v. V- i) O& T      mode4 a- m  B( l. n" O
    </description>( |5 p% n. @8 @. A2 _; [" J: M
  </property>
2 _3 p& N4 T6 \1 Z2 W8 ]  <property>2 B& C* o; \) \6 i% l
    <name>hadoop.shell.missing.defaultFs.warning</name>
; w6 e; r/ m% ^9 F    <value>false</value>
% P! r+ g/ B+ c4 u, Q    <description>
* B) w( [) v* m( T1 o2 l3 y2 d      Enable hdfs shell commands to display warnings if (fs.defaultFS) property
  D5 p: B  o0 m% V% l      is not set.
  N# P7 F8 W3 {6 x    </description>
& l$ B$ I+ e' f, x7 k6 D  </property>
, ^3 k! B- M4 {  <property>1 z9 \3 X. z7 P( x1 C( L
    <name>hadoop.shell.safely.delete.limit.num.files</name>4 U: l0 q7 {3 h  ^8 U- P6 x
    <value>100</value>! [+ Z6 y' L/ R8 K+ p) u1 `7 S
    <description>Used by -safely option of hadoop fs shell -rm command to avoid, y9 E  h+ r3 |: d7 v
      accidental deletion of large directories. When enabled, the -rm command
8 D5 D3 d1 M' p7 R      requires confirmation if the number of files to be deleted is greater than5 H  m/ S7 h" a1 y5 e. c
      this limit.  The default limit is 100 files. The warning is disabled if$ i1 _5 E) }+ a1 z1 ~/ T0 y
      the limit is 0 or the -safely is not specified in -rm command.
) R4 k- ?0 E- }" X! }    </description>9 {. f% m, u  M7 W6 s8 o
  </property>; d' z2 N$ a) J- T8 t2 r' G. A# j8 ?! N
  <property>% t% z& Y! u6 L) V  K1 m1 n0 \: ~
    <name>fs.client.htrace.sampler.classes</name>  l: A& P9 ~8 k$ X
    <value></value>( z0 L( j6 y2 u! i
    <description>The class names of the HTrace Samplers to use for Hadoop4 o8 ~! n* A7 x4 \9 \! b
      filesystem clients.& W5 [  G+ \2 Y. H/ X! T' I
    </description>4 B$ l% }! R, P+ o% x
  </property>4 G9 @$ }6 t2 ]
  <property>& y9 @0 g& T% L+ K
    <name>hadoop.htrace.span.receiver.classes</name>
' _5 `! [* x2 c9 n/ g* N7 y8 X( G    <value></value>
# \* |& l. K% @, a; u5 Y) J8 Q. @    <description>The class names of the Span Receivers to use for Hadoop.
) W. _  Y% V4 t3 k$ E* T    </description># e, U) j, N( t) R7 a( M" S
  </property>3 Z  h8 F: M7 B. h* v( M
  <property>
0 Q2 j2 y/ K4 A# |  _    <name>hadoop.http.logs.enabled</name>& f2 k  ~0 G: N# ~' k* n
    <value>true</value>
. w: m' C+ `: G4 d    <description>
6 z- J3 k+ ^! U6 R  K. F$ t% x4 D      Enable the "/logs" endpoint on all Hadoop daemons, which serves local' K$ l0 o- L- n! {6 y
      logs, but may be considered a security risk due to it listing the contents
& J7 P8 `. C' W+ i' U      of a directory.
- r; k$ l: e3 U0 M7 n( W    </description>, X7 z. r, c4 V* j
  </property>$ N* `. U8 W9 @! O# _
  <property>
: d( x# e  [& L: W6 |) G    <name>fs.client.resolve.topology.enabled</name>1 w* W3 r& [9 F+ X$ Z) _9 a. m
    <value>false</value>
, y+ d9 r3 a$ e% ]4 W4 g4 _( j    <description>Whether the client machine will use the class specified by: Y" a& E9 T# B3 ?2 C
      property net.topology.node.switch.mapping.impl to compute the network
+ ~3 t$ g( S2 `: ]9 U( X1 G      distance between itself and remote machines of the FileSystem. Additional. ?( F" @- V& \$ z
      properties might need to be configured depending on the class specified
$ g6 ?4 y1 N" V! z      in net.topology.node.switch.mapping.impl. For example, if' N9 i$ w" q1 y1 |- G7 B
      org.apache.hadoop.net.ScriptBasedMapping is used, a valid script file5 c; h" j. F# G, k: e
      needs to be specified in net.topology.script.file.name.* _6 E/ n1 }6 U
    </description>- C# T- P7 v$ h- Y% u3 ]
  </property>
& V! D8 a# O. s& b5 G- b8 u% `, W  <!-- Azure Data Lake File System Configurations -->8 D/ E4 d  C. Z: w, }& v
  <property>9 \) J% ]3 N+ p; Q/ p9 t- |/ r6 h9 b
    <name>fs.adl.impl</name>
: q, m: m6 S+ o3 {    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
* y8 G! D' E* h. t  </property># r' F$ I- p+ Z7 O) @
  <property>) t, J; K% S1 `; C
    <name>fs.AbstractFileSystem.adl.impl</name>: F! g9 c# P9 }3 D$ j5 G+ q& [" p
    <value>org.apache.hadoop.fs.adl.Adl</value># d4 d  P$ s8 P: c# l: s
  </property>8 [: @; w  u5 b5 B0 z5 t3 r
  <property>' u) |' s% H. J' p4 B6 V0 A% |0 I
    <name>adl.feature.ownerandgroup.enableupn</name>1 G  O* w$ g% s, k) W; d& c  a' {
    <value>false</value>9 L9 g+ Y7 N% X+ u  }0 a, C
    <description>! y) p9 h- |1 u6 A2 y% e4 q
      When true : User and Group in FileStatus/AclStatus response is
, T+ b! r* S$ p/ ]7 i% \      represented as user friendly name as per Azure AD profile.% q" O" L+ U0 z% u( n. y( Y7 `0 k2 ]
      When false (default) : User and Group in FileStatus/AclStatus6 t* S: f- B3 P$ w4 `7 J3 m
      response is represented by the unique identifier from Azure AD- `  v( r0 }$ C" @# _5 r/ ^' K, K
      profile (Object ID as GUID).
% _4 m% M; t9 [& @, E  {      For optimal performance, false is recommended.1 S7 ]4 Q. }: ^8 o+ u, G* c# r7 b
    </description>7 ]# g4 ?6 I% w& D# h, M
  </property>
8 l. c/ F3 K. I  Y3 U  <property>; _5 d/ b4 {& Z6 n9 m
    <name>fs.adl.oauth2.access.token.provider.type</name>8 C; c4 }1 r7 Z& n1 w2 }% {
    <value>ClientCredential</value>. h$ k* l' b- N' r# _9 g
    <description>
  d' N: d! S& H$ a5 f; ^      Defines Azure Active Directory OAuth2 access token provider type.( X* A4 t+ ^) t6 J
      Supported types are ClientCredential, RefreshToken, MSI, DeviceCode,6 \* g2 Q0 k( e3 {8 @
      and Custom.  L' M2 k. t7 O6 i% Z8 x
      The ClientCredential type requires property fs.adl.oauth2.client.id,
* y, m$ x5 ?9 a4 ^5 C8 q      fs.adl.oauth2.credential, and fs.adl.oauth2.refresh.url.
  e$ m6 K* }; ~& j7 b, q; M+ ?* V      The RefreshToken type requires property fs.adl.oauth2.client.id and7 u6 W! M: w- |3 D
      fs.adl.oauth2.refresh.token.0 A9 k) }; j% ?- O  z- R
      The MSI type reads optional property fs.adl.oauth2.msi.port, if specified.
! s" d4 M* x, j1 q/ |      The DeviceCode type requires property& W5 z0 y9 O8 M, f8 L! y
      fs.adl.oauth2.devicecode.clientapp.id.: x2 t/ v0 b6 R7 z
      The Custom type requires property fs.adl.oauth2.access.token.provider.
4 J) C/ ~' p: U5 `) L3 D    </description>
4 N% h4 Q; \( P7 D  A" t! V: a  </property>9 K% f& O" j! C- x/ F
  <property>
$ C! R' A' U8 [# Y% X/ r) F  e    <name>fs.adl.oauth2.client.id</name>
; K" Z( W2 Q/ _    <value></value>" r+ v0 V) G3 D' ?: T
    <description>The OAuth2 client id.</description>. `) B, }; [% N6 p
  </property>
& ~, l' f5 C: b, }  Z+ @4 U  <property>
; |& |# r8 o$ H: P    <name>fs.adl.oauth2.credential</name>% z/ L9 ?8 H  E% s1 s3 C9 a, H
    <value></value>& Q( l" c- ~5 `( {9 o* u
    <description>The OAuth2 access key.</description>
# M6 w7 h  T, H2 M  </property>( |! S) B# l& U! J
  <property>+ X. V9 W* t3 e
    <name>fs.adl.oauth2.refresh.url</name>+ X0 M9 o. V4 O! y" U2 J! @
    <value></value>
' @" C0 L8 X0 e9 `% K2 E    <description>The OAuth2 token endpoint.</description>
2 u1 J5 {+ W9 `+ O9 a& K. l( `  </property>" [2 ]0 v9 h' `5 L# P% v8 ?
  <property>
6 S: g* U: B' a& Y7 A& \/ J    <name>fs.adl.oauth2.refresh.token</name>; f# @2 r! f% e
    <value></value>, |' I( R) Y5 o
    <description>The OAuth2 refresh token.</description>
1 m9 X- H: _- z  </property>8 B* Z# e7 o" z$ n8 r' I! I2 {
  <property>
0 T' X9 c9 F: R! }; s' `    <name>fs.adl.oauth2.access.token.provider</name>
7 O! K4 h4 s+ }7 M7 ^" p& Y    <value></value>3 W, y. |+ M7 o
    <description>
- V* d) e, d5 n; P      The class name of the OAuth2 access token provider.
3 v4 d- m& P- G" F, ~    </description>
8 K  |1 I0 d6 q  d8 W  </property>+ `. ]. O: r; k, H& }
  <property>
& k4 Z( Q! h# S; q    <name>fs.adl.oauth2.msi.port</name>9 f  r$ v. O6 ^6 B1 w2 S
    <value></value>! H4 x7 t. X! R5 j% z
    <description>
) J: e3 `( N# H' E! n5 ~0 S      The localhost port for the MSI token service. This is the port specified8 @2 ?7 [( @' P: N  S- H
      when creating the Azure VM. The default, if this setting is not specified,
9 L9 u4 k5 b: F0 `9 _2 t2 A$ M1 z      is 50342.+ q) y9 P' w2 L" W& M
      Used by MSI token provider.
' X! F* }$ F/ j4 Y! ]0 I7 i2 F* I    </description>
2 N& r9 Y# `& Z) j0 v  </property>) }4 {. L3 @; c8 e/ {4 C
  <property>
5 g# v; K0 k; G9 U7 t    <name>fs.adl.oauth2.devicecode.clientapp.id</name>
9 {9 \& H; ]; E% m% i% v    <value></value>7 _8 _1 F7 u. c9 t. q" r/ C( f4 h8 P
    <description>
# l0 x: J" K% `6 ?" |! g/ S      The app id of the AAD native app in whose context the auth request# d8 I4 c9 I. a( V4 r) h
      should be made.
0 o" g  P* ~1 l4 ?      Used by DeviceCode token provider.
- v6 _2 H, B3 `# u    </description>
' y/ y- n# s6 }1 E6 }' _0 D  </property>
3 `1 ^. g+ N% i4 }  <!-- Azure Data Lake File System Configurations Ends Here-->
, V4 q9 p6 R/ ~& n  <property>
3 R( J$ l0 Q0 O' T    <name>hadoop.caller.context.enabled</name>
1 K! ?+ e7 m9 [! z6 ]    <value>false</value>
9 ~. J" D7 O& g- n. A$ ~    <description>When the feature is enabled, additional fields are written into
$ X% I  n4 v1 \& `; q      name-node audit log records for auditing coarse granularity operations.9 {3 w" Z1 ~2 h. e/ ~
    </description>
) D* Q* e) r9 x( Y/ v  </property>
% r7 A% ]4 j# b* @  <property>
- k$ A2 d& {7 I9 S+ |! d# v# F, U  X/ S    <name>hadoop.caller.context.max.size</name># j1 J" x# B5 P3 @6 o( G6 i0 }
    <value>128</value>8 l" v. w- \' r1 c: V) m, ~2 R
    <description>The maximum bytes a caller context string can have. If the
/ X- ]& m% n# P$ C$ I; K7 e  K* G4 Z      passed caller context is longer than this maximum bytes, client will
( M% Q" K9 y+ T      truncate it before sending to server. Note that the server may have a% u3 p9 P4 a' z! |- C, m
      different maximum size, and will truncate the caller context to the
5 `6 Y' k; ?9 {1 R: ]7 o& H      maximum size it allows.
5 R; K) b6 m: ?3 R! |. Q    </description>) ]1 i* |  b$ `+ }, e
  </property>
% E  \8 K! r) B$ Z* r; N. t  <property>3 K+ ?; P# C$ Q$ V
    <name>hadoop.caller.context.signature.max.size</name>
1 j; j2 |/ M  Z; B$ U1 w    <value>40</value>
* ?8 |. `- m# h; ]0 i/ C; G, t    <description>7 h" J4 Q# H# M( H; l" ], i
      The caller's signature (optional) is for offline validation. If the5 n' A$ _1 A& H* x: P
      signature exceeds the maximum allowed bytes in server, the caller context
1 {! e3 i- k1 z; F* O8 O0 i      will be abandoned, in which case the caller context will not be recorded
$ i- g$ z- h9 d$ L' V, g      in audit logs.
7 A4 Y* ?4 Q1 X    </description>) U1 l  a7 h" v( n8 N
  </property>
% a, U+ E9 e; j" m<!-- SequenceFile's Sorter properties -->
' x* F/ K5 E# q9 I( k  <property>3 c5 ~9 w; q+ P5 a# C
    <name>seq.io.sort.mb</name>+ w6 [' a# o# ?$ @! Z9 p' P' ]
    <value>100</value>
2 A! v& n& V) ^0 @! ?5 y- g$ L    <description># c' t2 {4 K7 t) ^8 `
      The total amount of buffer memory to use while sorting files,2 y% D4 ~8 X2 S7 S+ R' _
      while using SequenceFile.Sorter, in megabytes. By default,
0 R' @4 |5 j- E& a, Y5 a      gives each merge stream 1MB, which should minimize seeks.
* ~5 Q) z' Z3 k. _, ]7 ]* ^) h% G    </description>
3 I% c  U9 W3 \$ I" p  </property>
) [. E+ e$ ]' \4 U6 L( w9 |& u) f  <property>* Y5 K, Z( V1 d5 \0 i
    <name>seq.io.sort.factor</name>- @6 D1 r3 H* n
    <value>100</value>
7 p  g5 P. _! F$ s' B/ {    <description>7 w6 F8 Y1 Y( [& l
      The number of streams to merge at once while sorting6 j& m& m5 o" p" G1 O' B0 a
      files using SequenceFile.Sorter., l' N. {% S7 [5 \3 p  Z; r
      This determines the number of open file handles.
$ v% C+ Y# U: \4 O. ]0 P4 t# I2 N    </description>+ ~4 U+ T0 l0 C1 [5 [
  </property>
' Z0 q, Z' Z' c- |0 z* N, _  <property>
2 A4 r1 L  _$ v6 t- B5 D4 Y    <name>hadoop.zk.address</name>
1 i! z6 }. ^; A% l8 ^* m5 D" e    <!--value>127.0.0.1:2181</value-->
: p6 Y2 U2 g. N. z' g! @    <description>Host:Port of the ZooKeeper server to be used.
2 q6 e( P$ d$ {0 ?5 G    </description>( K$ K; s4 ^2 A5 s
  </property>1 }- M9 X& h" v7 g! \
  <property>
% S: g- d* ?4 v  i    <name>hadoop.zk.num-retries</name>
" U% G% `4 C. r+ H! B0 s    <value>1000</value>+ t; n) X& q' g+ t$ }
    <description>Number of tries to connect to ZooKeeper.</description>
$ J2 d) t" |2 Z$ m( m  T* Y  </property>
' _2 p5 _  R) f6 l4 I- u$ H5 q  <property>
1 f: v% `+ H# z# @    <name>hadoop.zk.retry-interval-ms</name>, S. ^6 H# \6 K2 d9 o# \; G$ _7 N, V
    <value>1000</value>0 ]3 u8 b' `6 L2 H) i
    <description>Retry interval in milliseconds when connecting to ZooKeeper.
7 U5 V9 U# L  r5 N' m4 A# c    </description>
" ?7 b3 U( n' w% E; j) f- y  </property>% ]: R2 S! {* _: f4 y$ |" s- Y
  <property>3 @$ N% r6 T2 |: ^
    <name>hadoop.zk.timeout-ms</name>
/ d* ?) d  O" R/ x8 ~    <value>10000</value>  F7 z2 H& R# G0 R; t) j
    <description>ZooKeeper session timeout in milliseconds. Session expiration# V6 f5 G* e) a
    is managed by the ZooKeeper cluster itself, not by the client. This value is5 Q( B. s7 n! }5 I) W0 ~7 ^, B2 X
    used by the cluster to determine when the client's session expires.  }; y3 J) U' ?8 @9 @) ?
    Expirations happens when the cluster does not hear from the client within7 x4 W* y8 F4 I0 b; s, T- x2 [
    the specified session timeout period (i.e. no heartbeat).</description>
! E* U" i8 e5 F, H  </property>
/ X- }4 A3 i* @# B$ |" s  <property>. [% X4 T) u! I2 [0 h
    <name>hadoop.zk.acl</name>
: g5 e* E8 N0 v) O4 O- V( V    <value>world:anyone:rwcda</value>* c& V: ?0 c: E* x) X
    <description>ACL's to be used for ZooKeeper znodes.</description>% a3 e$ A# h1 ?$ o4 X" E$ n7 y
  </property>( b1 Z: \# S% y0 E/ i! u6 @
  <property>
9 M* s$ [& R$ G1 E    <name>hadoop.zk.auth</name>" D2 X. ^* `/ e$ E( Y: y
    <description>
) i# V0 t; e& @+ F% d! h        Specify the auths to be used for the ACL's specified in hadoop.zk.acl./ V' _. v3 x' [/ r' U2 s* r
        This takes a comma-separated list of authentication mechanisms, each of the
/ N( J* p- I( L+ W2 t/ |        form 'scheme:auth' (the same syntax used for the 'addAuth' command in
9 g6 p9 |, z5 ?; g        the ZK CLI).1 _( Q1 u$ Q" q- r" A5 |
    </description>
+ w/ s, n$ Y% l1 Z  </property>
7 e" e6 E+ p7 `0 O7 [$ k" A+ d  <property>
5 Y+ [: \) P1 @5 @    <name>hadoop.system.tags</name>4 i+ H" K; a5 H3 h4 a- {
    <value>YARN,HDFS,NAMENODE,DATANODE,REQUIRED,SECURITY,KERBEROS,PERFORMANCE,CLIENT/ r, r8 d; c, ^: d9 L- K& @" f
      ,SERVER,DEBUG,DEPRICATED,COMMON,OPTIONAL</value>
" p! ?4 L: F$ s  ^" i    <description>
6 d1 o5 ~8 l+ V# u      System tags to group related properties together.6 I2 Y3 O; }+ r' T' t  y7 e' z
    </description>2 I/ T9 g/ w$ }% R5 v- C/ y
  </property>
; D5 X& p$ Z' w* A0 B" C6 u  <property>
5 ^2 L- j) h. _/ P    <name>ipc.client.bind.wildcard.addr</name>
( @1 n* u/ }' R! w& }% q. B2 n    <value>false</value>; e0 ?6 C- ^6 Z  |/ t8 e: k
    <description>When set to true Clients will bind socket to wildcard3 v5 `' m  K8 U
      address. (i.e 0.0.0.0); x( S# I! U1 A7 T  P% Z+ i
    </description>
3 I- d, g0 L/ v( ]: @  </property>
; ^4 W9 s, h6 l2 z, l</configuration>2 b: s5 }, h& E, Q. C
2.hdfs-default.xml
' @& ?! ^, S5 u. ^1 g+ z$ l- z# c& w0 |; d8 a; m* s1 _% G
<?xml version="1.0"?>
, P) e, d. Z3 U  d: V<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
% p1 P4 K& u/ M<!--
( u4 d3 r! r# x6 V$ T% q- x   Licensed to the Apache Software Foundation (ASF) under one or more& G5 p2 s+ E" Q
   contributor license agreements.  See the NOTICE file distributed with
8 ^  @& {! O% ]5 r6 T" M   this work for additional information regarding copyright ownership.
1 o$ M2 q6 Q/ B5 ^( G   The ASF licenses this file to You under the Apache License, Version 2.0
& z+ M) A3 H/ T: q( G: T4 K8 T+ y. M   (the "License"); you may not use this file except in compliance with6 u/ ]* R) J" ?# ]" i$ C
   the License.  You may obtain a copy of the License at
; o( F( _0 e4 T6 E" r       http://www.apache.org/licenses/LICENSE-2.0
( Q# x3 f0 D) ~& u5 P8 O   Unless required by applicable law or agreed to in writing, software
$ w% `/ ?$ Z" [7 w   distributed under the License is distributed on an "AS IS" BASIS,
1 m, h! i- a; I8 C) d   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
" v2 ~. g1 B' i% ^# L7 A$ t   See the License for the specific language governing permissions and; F& f6 T4 s- q4 {( k) n" J- O
   limitations under the License.0 d8 i: m; w5 |( A4 _7 J, U% L
-->
6 A3 P0 A! i6 _" G1 `<!-- Do not modify this file directly.  Instead, copy entries that you -->
" R9 }# m" r# R<!-- wish to modify from this file into hdfs-site.xml and change them -->( d- m2 S  T$ i8 W. U* T; h; r  J
<!-- there.  If hdfs-site.xml does not already exist, create it.      -->
0 r1 S% V8 B9 \- ~+ M1 \+ x! T<configuration>
2 c6 x' M3 K* B/ B<property>) _) X1 ^' `. Y+ g
  <name>hadoop.hdfs.configuration.version</name>
# [" K/ j# e3 H  ]& T* l9 d  <value>1</value>
! x6 o0 Q7 ?% O8 ]4 W  <description>version of this configuration file</description>* L' G7 y. \; ]# G9 K
</property>" [& b$ H6 x" s. x- X  z
<property>) V. c$ z5 z* W0 a
  <name>dfs.namenode.rpc-address</name>4 n* D- H6 S5 F6 l
  <value></value>; k: |) K. ~! u0 H
  <description>- v! Z2 {: ~+ e: _5 p' L
    RPC address that handles all clients requests. In the case of HA/Federation where multiple namenodes exist," P8 o  j2 _+ x, b  y* V( w. }( J& \
    the name service id is added to the name e.g. dfs.namenode.rpc-address.ns1
( r! G! Q; x- ^4 x. @    dfs.namenode.rpc-address.EXAMPLENAMESERVICE. g- `4 l0 C; l
    The value of this property will take the form of nn-host1:rpc-port. The NameNode's default RPC port is 8020.
2 X6 n9 @: t' ^5 z' m  </description>  m* ]% ]) _& d( `) K$ F6 u
</property>- v. n3 W( `1 U0 L: n- g
<property>: c: u) ]7 T3 E, I( ]
  <name>dfs.namenode.rpc-bind-host</name>
* f8 w" y" T, N, G  <value></value>) U: N1 l7 }' }+ O* |  t$ j
  <description>( d8 v! X4 |: |6 e  z3 c
    The actual address the RPC server will bind to. If this optional address is* v4 ^5 S! i) g5 C& P1 B( ~
    set, it overrides only the hostname portion of dfs.namenode.rpc-address.
9 [% y5 F. A" a7 P5 l    It can also be specified per name node or name service for HA/Federation.7 Q9 q8 i" c# x  m9 ~+ ?: |
    This is useful for making the name node listen on all interfaces by
( i! v! ?0 ^  S  `$ ~, p# }, {    setting it to 0.0.0.0.
& f2 [$ T) Z1 t  j/ V  </description>
+ Q" B8 l" A1 W1 ]  a. Z, T</property>3 c0 V7 ]8 D# x
<property>
  c6 d4 K6 |5 _! K, s. F' _, u  <name>dfs.namenode.servicerpc-address</name>, b) o0 ~  F- ^7 E+ t/ f
  <value></value>- s; ]- J/ }; ~* [% [
  <description>
: y4 }, M( B6 ^* s, w    RPC address for HDFS Services communication. BackupNode, Datanodes and all other services should be
$ L8 T5 k& X& m* Y/ o    connecting to this address if it is configured. In the case of HA/Federation where multiple namenodes exist,
5 O/ U7 k9 {; s+ k4 |! F7 x1 o    the name service id is added to the name e.g. dfs.namenode.servicerpc-address.ns1& O' E- i& F5 J2 U2 V
    dfs.namenode.rpc-address.EXAMPLENAMESERVICE$ @* v5 q# [9 T5 |4 u% U
    The value of this property will take the form of nn-host1:rpc-port.
$ B+ x  e' {, r, |  ~! i    If the value of this property is unset the value of dfs.namenode.rpc-address will be used as the default.
" z& W& u' Q( z3 p7 E* M  </description>8 A9 u$ Y1 q7 ?
</property>
) x- ^  d2 g' E1 T2 C8 V7 E" E<property>+ T$ o! c, f6 d; l+ T2 a; e
  <name>dfs.namenode.servicerpc-bind-host</name>" ]$ a# ^( Q) v8 S' D3 B
  <value></value>
5 O, |- p" l  u# _) _  <description>
! z" m( Q: U- R* N7 l! r    The actual address the service RPC server will bind to. If this optional address is
8 a  R1 W$ W4 _: A. K7 a+ G    set, it overrides only the hostname portion of dfs.namenode.servicerpc-address.
: d0 b' f4 b% `- C5 q3 `    It can also be specified per name node or name service for HA/Federation.
0 V) Z2 J( S& N& l; R- O# U    This is useful for making the name node listen on all interfaces by5 ~0 {0 |1 Z: r+ l( U0 f, z% u% H
    setting it to 0.0.0.0.
% u' v6 F/ z: j- p5 |& O8 Q( D* s  </description>
: }& ?. x2 j. ^" Q. f</property>; K1 v  v" }+ Q5 U
<property>! m4 ]! f/ ^! H, `
  <name>dfs.namenode.lifeline.rpc-address</name>3 @3 O% {+ @: T8 e9 g' g
  <value></value>! H; }9 G9 [% P3 L( R" X
  <description>' q9 L( n( z. G( ^& F3 y
    NameNode RPC lifeline address.  This is an optional separate RPC address
. R4 U$ n% T! K1 W# C0 N6 G/ H    that can be used to isolate health checks and liveness to protect against
# U* A7 K6 G8 e8 \: |3 Z; o* U    resource exhaustion in the main RPC handler pool.  In the case of  M* m9 Z" Q6 j, T
    HA/Federation where multiple NameNodes exist, the name service ID is added* F/ S% Z3 y6 {/ `3 x
    to the name e.g. dfs.namenode.lifeline.rpc-address.ns1.  The value of this& m  e# F0 I2 E( t; ?5 Z; ]% f+ c
    property will take the form of nn-host1:rpc-port.  If this property is not
2 @' Y: g( l% n! F! m( e' f$ C    defined, then the NameNode will not start a lifeline RPC server.  By
% |4 N, X7 L: Q- s! W6 \3 K    default, the property is not defined.
. Q9 H& I. s6 M5 F# x  `& u  </description>
% X8 {$ _4 g: Q1 t; v</property>
) A: D' t: |. i4 H. i3 p0 |<property>
5 f6 ^% n$ c$ u" B* f  <name>dfs.namenode.lifeline.rpc-bind-host</name>+ f/ D2 U& b5 ?* n3 V
  <value></value>9 N# d6 e' g; x! i+ K, i
  <description>1 o7 _( R2 H" o/ o
    The actual address the lifeline RPC server will bind to.  If this optional- u: L% r1 N) x; z4 g7 Z- K' M: A
    address is set, it overrides only the hostname portion of
4 n- ~5 V1 m2 d& o+ O, j    dfs.namenode.lifeline.rpc-address.  It can also be specified per name node1 \6 A; l1 n5 r6 \
    or name service for HA/Federation.  This is useful for making the name node; N9 t8 F$ f5 j' T9 g1 F9 F
    listen on all interfaces by setting it to 0.0.0.0." |$ t. U' i+ C( Y- W
  </description>
$ U8 A0 n$ T" D9 S8 ~' b</property>& U4 Q3 L1 T+ _3 H+ Q4 {4 Q, M
<property>
: a/ j2 q( b. b) ^  <name>dfs.namenode.secondary.http-address</name>$ f# x9 T$ ]' D5 w, S
  <value>0.0.0.0:9868</value>" j+ Q% s1 L: v  k- j
  <description>
$ z( ^( E( G( [9 Z    The secondary namenode http server address and port.
$ Y, o; g4 Q( B  </description>
  u) `* V5 A4 e</property>
- X- m) p+ s( {, K& k<property>
( g7 `' G2 H: s4 \! o  <name>dfs.namenode.secondary.https-address</name>
. i' X! h( s( k2 K6 |% u4 v% W  <value>0.0.0.0:9869</value>% e8 W4 z) {/ p( D8 a9 K  H
  <description>: L' U% Y' l8 _  I; K) l
    The secondary namenode HTTPS server address and port.
) i8 ~' n( _$ N' j$ w" w$ y7 Z3 w  </description>
7 w5 L7 [4 u! o9 m2 I. Y0 O* \</property>
- |  F' I& B3 @3 y) C7 G<property>8 X9 _+ ^0 I$ b1 {5 C; d
  <name>dfs.datanode.address</name>
& S) n# ^8 J, \' A  <value>0.0.0.0:9866</value>
9 H8 w9 J* O: G# C0 J  <description>
  n# v* \& J6 ~    The datanode server address and port for data transfer.
' s- w+ e+ m0 A( a. o( g: j  U- k& |5 @  </description>
0 X3 i. s; P" e# y  x/ j7 w0 z</property>
0 d: D* Y! F) U3 y8 o# c<property>
, m+ \' j0 T" y! _  <name>dfs.datanode.http.address</name>
" I6 D, F, Y5 N  <value>0.0.0.0:9864</value>
6 o, m, _* i- F" X" c  <description>; W# T: w1 @! I! O! m+ |
    The datanode http server address and port.
( \5 G% l" s5 s: i3 w' O  </description>
$ i0 i, {* w0 _0 S</property>3 J) c; _* d; v& M
<property>
5 ?9 _7 i; i9 E8 z( w  <name>dfs.datanode.ipc.address</name>
7 T7 i4 d# w+ K1 _6 m. @  k. [  <value>0.0.0.0:9867</value>
6 o% t" K4 @( }8 u; o0 e  <description>
# g9 U; K* X' Q    The datanode ipc server address and port.& l2 g( o3 R# ^2 _/ _+ ^; G
  </description>
6 b/ i4 F2 ~# L</property>( S7 M# m, B" q, T( e, f. `3 B
<property>2 ~' B0 Y) U7 L3 j2 w
  <name>dfs.datanode.http.internal-proxy.port</name>( `5 h5 E; i5 a- l- [5 k9 }$ @
  <value>0</value>
, ?' d2 e# Q, J! d  <description>% T# \1 x6 ~3 d8 Z4 L
    The datanode's internal web proxy port.
+ j! w; \; x& }+ O. L8 q7 n    By default it selects a random port available in runtime., y- j. B9 B4 J( g
  </description>
6 z( x6 _' K6 |+ V3 d, Y! ?6 G& i( i</property>: X/ j8 J6 d7 G$ f) F0 L0 ]
<property>2 o1 `$ P& [/ C# g
  <name>dfs.datanode.handler.count</name>5 ^& T& G. v, ?9 V
  <value>10</value>' ~" F! d; B6 z
  <description>The number of server threads for the datanode.</description>  z* L! y' M0 Q/ h
</property>8 A; i. ?2 Z, ]6 s
<property>$ u* k4 `. N7 r, E# x, N
  <name>dfs.namenode.http-address</name>
6 U1 K- ]* ~, B) H8 q$ P1 x  <value>0.0.0.0:9870</value>
" g, R5 [) x! X/ u* c- z, p; I  <description>
* ?+ J9 k' b9 b7 G! \    The address and the base port where the dfs namenode web ui will listen on.
$ {  z  L9 @5 c) c* d2 i  </description>
9 y4 y& b: h6 ~3 W2 }</property>
+ Z4 a  l& t2 q. i7 v<property>, k7 m' D5 `4 e7 L) `- P
  <name>dfs.namenode.http-bind-host</name>
1 k2 ^2 ?8 }4 _( T6 S! k  <value></value>2 {8 I  u. O: M: X; L* W0 s5 Z+ h; }
  <description>
+ U7 l$ D& B8 d7 N% d    The actual address the HTTP server will bind to. If this optional address8 L* c( L' p4 {( z. U5 N/ l: a
    is set, it overrides only the hostname portion of dfs.namenode.http-address.. @, `9 x. I2 k& a
    It can also be specified per name node or name service for HA/Federation.3 J4 J7 o  D& C7 G$ t8 ^
    This is useful for making the name node HTTP server listen on all
7 x1 g' n: x/ t# n) Q( M  w, `    interfaces by setting it to 0.0.0.0.
8 e/ [! f! s6 I; c/ }) C  </description>
9 h7 A2 ^. T5 Y5 F</property>
( d; h+ y% d/ i" t7 R) [2 Z<property>; N' \3 [1 r( y
  <name>dfs.namenode.heartbeat.recheck-interval</name>
- A& X1 Q! y) J3 T  <value>300000</value>: q: \  g  i' i7 g, M$ z* l+ X
  <description>
* H$ M' Z% J1 H4 p8 C; m$ B" |- a    This time decides the interval to check for expired datanodes.
* |8 l& N+ r: c; A! ^    With this value and dfs.heartbeat.interval, the interval of4 }7 k8 f* s6 X+ o6 o8 a
    deciding the datanode is stale or not is also calculated.0 W3 \9 T3 ?* [; y+ b+ g
    The unit of this configuration is millisecond.
. _: D/ I2 B+ `% p1 u$ M  b  </description>1 D) ^( L; C  s. r
</property>4 M- b, w& ?" o( d4 Q
<property>0 C6 Z  ~0 N' r9 F0 A, m$ v4 e
  <name>dfs.http.policy</name>
0 l( y& U' i# A# g* {% q+ y  <value>HTTP_ONLY</value>
: L0 l+ A9 y) x/ M! J  <description>Decide if HTTPS(SSL) is supported on HDFS+ q4 n, q' D. S5 a7 }7 X
    This configures the HTTP endpoint for HDFS daemons:! V. k6 c/ ]8 w2 s1 g9 F( G
      The following values are supported:( ]  w: k0 _, h7 f; ?# t
      - HTTP_ONLY : Service is provided only on http
5 h9 u) U% F" L2 n. x      - HTTPS_ONLY : Service is provided only on https" `+ r, d2 ^% q" i! i( ]
      - HTTP_AND_HTTPS : Service is provided both on http and https0 [5 w& t6 t' I: }  I. r. @/ r
  </description>) @: P, O' R# P
</property>
/ o, g( w3 S1 K, D. w<property>. L9 b$ W" D/ O+ W3 `2 s8 W! V& F
  <name>dfs.client.https.need-auth</name>$ I0 g+ g5 _+ \6 ^! B  Y0 R
  <value>false</value>
# k% F% j" N8 E2 V  <description>Whether SSL client certificate authentication is required
* I" G7 y8 q, ^) [$ w  </description>
. u. d6 ]' D7 v</property>
" m7 I4 P4 b5 W: t8 A<property>% t/ E4 S6 A: a
  <name>dfs.client.cached.conn.retry</name>
: }+ N; n! v7 I  M  <value>3</value>
+ u1 }# H: w, _- d, S+ ?  <description>The number of times the HDFS client will pull a socket from the) Y$ Y! O# ?- l% d2 t2 }
   cache.  Once this number is exceeded, the client will try to create a new: @  X, ~; s+ d/ l5 N0 U
   socket.
% k( r0 f: h+ M& V: G  </description>
; d: y. ~( Z4 j  G</property>. {3 V" n5 J/ Q, y
<property>
5 v2 c" ^' l  U/ a  n" Z+ j  <name>dfs.https.server.keystore.resource</name>
: A- X) v2 s) P  <value>ssl-server.xml</value>% s3 K9 l' S+ r' h1 b+ B/ s
  <description>Resource file from which ssl server keystore
4 F! c; X, q$ J* l( v  information will be extracted! x$ i# o! p  q/ D  M! ^* D) h
  </description>* }: a- v+ A$ a
</property>9 p2 E/ x) E! C; k# C# Q1 w# b
<property>$ f8 Q  m# r/ l$ J2 ]; u2 S
  <name>dfs.client.https.keystore.resource</name>) m& B: D* F9 t7 Y. c
  <value>ssl-client.xml</value>0 g# H! M$ S# o% X; Q
  <description>Resource file from which ssl client keystore; h/ G" V; r1 Y- C4 c* {$ w
  information will be extracted
4 r& E7 A& L* o" f4 u5 T0 D  </description># c" {( r# s% V5 b
</property>
! p6 K; P" ~3 g/ G2 f<property>7 R/ B6 p9 `1 G6 _: g4 i
  <name>dfs.datanode.https.address</name>
: r) d/ R3 ?9 B' G2 t( p  <value>0.0.0.0:9865</value>
0 t2 f( v, a9 {! u& @  <description>The datanode secure http server address and port.</description>
4 d2 c$ u% ?, ]  e0 O! q</property>+ k9 ~# Z1 a. V) Y$ X8 L6 i
<property>
5 z' }, B/ l- c0 @+ Z  <name>dfs.namenode.https-address</name>+ M# m! A& G% f& h
  <value>0.0.0.0:9871</value>) K4 q4 S8 m9 u0 s! z1 I
  <description>The namenode secure http server address and port.</description>
; o) `5 l9 f# D# `) M7 U</property>
( @$ k, P+ o% q0 v* v, w<property>
. Z) g/ ^& E' r+ u  <name>dfs.namenode.https-bind-host</name>
1 [/ @# _, U3 g# @- \( s  <value></value>( M% {: B5 [! r
  <description>3 X1 S/ P8 I1 M2 W% I3 K8 [8 k$ w
    The actual address the HTTPS server will bind to. If this optional address
! H0 A. r% O6 @& e# d9 q    is set, it overrides only the hostname portion of dfs.namenode.https-address.! Z' i) M# ~% B1 M0 |9 Z
    It can also be specified per name node or name service for HA/Federation.
- h# V. c$ ?) Q$ Z2 T' ]' `    This is useful for making the name node HTTPS server listen on all
* O5 y) ^: z( ~; p    interfaces by setting it to 0.0.0.0.
4 e$ i2 c# Q( l" Q& B  </description>
) o, C, l; f( M' I</property>  D; w1 Z* \8 c. L5 t  ~
<property>
( c' [, V, N% K* o7 ^   <name>dfs.datanode.dns.interface</name>1 F: ^! L4 r; U5 a4 d' N7 d
   <value>default</value>
2 s3 e% V! ~! W   <description>
0 B1 [. v1 v! m# C$ y: h4 P     The name of the Network Interface from which a data node should  l+ `. ^2 T1 a: ]4 R8 W
     report its IP address. e.g. eth2. This setting may be required for some, W( O( Q, Q) H' _" Y; X+ e$ K
     multi-homed nodes where the DataNodes are assigned multiple hostnames: t0 Q: i: q: `9 k) ~  C2 j
     and it is desirable for the DataNodes to use a non-default hostname.
! _# ?3 P- c0 w2 H! m3 M     Prefer using hadoop.security.dns.interface over
9 W3 V8 u  F( b6 g  I. R4 D     dfs.datanode.dns.interface.
) _6 X. D+ E# ^/ K   </description>
9 P9 p9 k9 F; ] </property>
' L/ @, S) _3 H& H3 @5 @8 W! `<property>7 o; f7 z4 R% ?
  <name>dfs.datanode.dns.nameserver</name>
  J9 Z  V% H8 N  b9 y* S  <value>default</value>
, x+ k7 d# D$ L6 y  <description>
; D" g: }1 h4 w6 q( j. U    The host name or IP address of the name server (DNS) which a DataNode' K7 {+ A( ~. c" b8 }, U
    should use to determine its own host name." _9 K- X; X( e5 a5 a& C& p4 i# a
    Prefer using hadoop.security.dns.nameserver over
1 [2 ~( G& S5 y    dfs.datanode.dns.nameserver.3 N+ e$ n3 N# G( S6 }4 ~2 G  d% t
  </description>
. ?* {' P% s6 o3 z/ R </property>( e4 U3 {- z6 G9 E) Z- |
<property>
+ M3 N3 [6 |8 N2 [. f2 B  <name>dfs.namenode.backup.address</name>
/ @9 i# v% }& Q* R6 w  <value>0.0.0.0:50100</value>4 r: e8 |; c1 b' T
  <description>" d7 f- D) N; |
    The backup node server address and port." Q2 p) u1 g7 |8 _0 ^& P
    If the port is 0 then the server will start on a free port.8 N, v) p8 F8 T5 p2 j
  </description>% I" h- N" k0 F2 Q4 L+ R
</property>" Y2 ^1 o. o% _7 J0 c6 Z
<property>
% A  ~! }3 O3 J. O% ~' l  <name>dfs.namenode.backup.http-address</name>
$ i- W9 }! k1 n% ?, h9 @; ~  <value>0.0.0.0:50105</value>: i0 {+ H% Y. l  J- }5 a" h
  <description>+ _+ v# r7 c7 ~7 h: F- c
    The backup node http server address and port.3 p% S0 J( C1 W6 Z
    If the port is 0 then the server will start on a free port.
. T: O  n1 Q( z7 A8 D  </description>
* d' l' L7 n1 |7 V6 _& {</property>
# J$ ?' @+ R* t& G/ f4 ?- k<property>5 `0 \2 w, X1 P  Y3 T! ]0 V' W
  <name>dfs.namenode.redundancy.considerLoad</name>0 F% d2 \' X' i! t
  <value>true</value>
% q/ e# k/ w& {3 n* y  <description>Decide if chooseTarget considers the target's load or not
4 C% P' f0 l) K  </description>
" C1 Z5 ^1 G: b</property>. {7 V+ n3 p8 V  I% U: K
  <property>! q6 a+ P4 R* A, a( z+ b/ @
    <name>dfs.namenode.redundancy.considerLoad.factor</name># |0 S7 I9 I; q" m0 S6 R+ v  j
    <value>2.0</value>
; S0 }* x" F, t, @    <description>The factor by which a node's load can exceed the average
4 ^. N' R$ \- \; F, I: U0 J+ i      before being rejected for writes, only if considerLoad is true./ k' M2 R- M. A7 t
    </description>
) O  Q! }/ q. V& T$ ^  </property>
& b; ]2 z6 ]) C* P. ~" c<property>
8 V; A  ~) Z* W( O  <name>dfs.default.chunk.view.size</name>
* ~& A9 s4 m- h' l$ \  <value>32768</value>4 Q" j0 D0 B& [
  <description>The number of bytes to view for a file on the browser.. _1 r. T. F. h& X, X2 c9 I
  </description>
* f4 d8 v/ K% }: `</property>
+ f8 Z7 a. e$ e3 E<property>" V" a; H3 a1 i: s" y
  <name>dfs.datanode.du.reserved.calculator</name>
+ f% F# i2 M" c) h5 x& f. J$ T6 c  <value>org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReservedSpaceCalculator$ReservedSpaceCalculatorAbsolute</value>5 L2 c1 P$ L2 Q* Q6 k: R
  <description>Determines the class of ReservedSpaceCalculator to be used for
5 g% e6 G* g/ I3 A8 U6 h7 ]! e    calculating disk space reservedfor non-HDFS data. The default calculator is$ i) C) e- b& K. `0 A1 O$ S) m) H
    ReservedSpaceCalculatorAbsolute which will use dfs.datanode.du.reserved$ o$ c# F! }  K2 t  N' C, t  S: g
    for a static reserved number of bytes. ReservedSpaceCalculatorPercentage% A9 R' M- G% U9 L9 v6 [
    will use dfs.datanode.du.reserved.pct to calculate the reserved number
, k" p5 ^8 ?$ G! h, D1 Y    of bytes based on the size of the storage. ReservedSpaceCalculatorConservative and; o. V9 w5 S# f/ a1 \/ E7 r
    ReservedSpaceCalculatorAggressive will use their combination, Conservative will use
- |6 B% d* O4 C$ a    maximum, Aggressive minimum. For more details see ReservedSpaceCalculator.
8 x9 p$ G4 G( [7 r! u! L  </description>3 [, n9 J' x9 G' \
</property>+ O- x5 x# X  K& N6 |+ t
<property>
7 ?, \$ ]# ?* }0 g. r  <name>dfs.datanode.du.reserved</name># E6 Z7 J" N; W( A
  <value>0</value>. ^7 @* P  [9 {9 a7 [0 z
  <description>Reserved space in bytes per volume. Always leave this much space free for non dfs use.
; _7 \) r" J6 B; f( a, }1 W4 _      Specific storage type based reservation is also supported. The property can be followed with5 ?4 D9 U0 k$ v0 @8 g
      corresponding storage types ([ssd]/[disk]/[archive]/[ram_disk]) for cluster with heterogeneous storage.) u3 |3 X4 _( e& L/ d
      For example, reserved space for RAM_DISK storage can be configured using property* C0 f9 q+ Q3 O
      'dfs.datanode.du.reserved.ram_disk'. If specific storage type reservation is not configured
: k  m! O$ }3 A7 v4 M      then dfs.datanode.du.reserved will be used./ p% N7 ~4 V1 b+ y& c
  </description>
% a- ?# P* L3 ?; q* ^! J: o</property>" q6 o0 j5 F+ t
<property>
7 t( H5 ^4 K1 w; D  <name>dfs.datanode.du.reserved.pct</name>/ Z% V. Z+ U, A/ Y" [. K
  <value>0</value>
7 _6 K$ y; l: u. C  <description>Reserved space in percentage. Read dfs.datanode.du.reserved.calculator to see5 Y& g% h# q$ L! i
    when this takes effect. The actual number of bytes reserved will be calculated by using the* T+ k. K6 E$ {1 T' j( W# m
    total capacity of the data directory in question. Specific storage type based reservation' h+ ^9 n/ W" w1 X8 m' y
    is also supported. The property can be followed with corresponding storage types; A4 Y  h% \3 p
    ([ssd]/[disk]/[archive]/[ram_disk]) for cluster with heterogeneous storage.
+ _0 \+ F. z% R2 f6 L( y/ Q: \( r    For example, reserved percentage space for RAM_DISK storage can be configured using property
5 G& p. F( H/ G$ ^6 L, [# Q2 ?2 W) y$ q    'dfs.datanode.du.reserved.pct.ram_disk'. If specific storage type reservation is not configured
" @" C' D( s, n    then dfs.datanode.du.reserved.pct will be used.
' E5 A/ `1 l: P: W" s$ @! c  </description>2 ]& a, p) `* U# J5 s" X# n+ C
</property>$ C) @* a) k9 P
<property>- y/ W9 c1 G5 ~4 M" Q! ^2 T' ?9 |
  <name>dfs.namenode.name.dir</name>( S+ V, c* a% ?, V* g7 B8 P
  <value>file://${hadoop.tmp.dir}/dfs/name</value>
3 T5 E/ }) k) j/ Q* A6 N) N& [  <description>Determines where on the local filesystem the DFS name node
" {% l6 M: I1 P8 K      should store the name table(fsimage).  If this is a comma-delimited list
! o: V5 j; I- D3 l8 g" w# s  a      of directories then the name table is replicated in all of the4 c$ X- Z- U5 m+ A
      directories, for redundancy. </description>, D1 ^) a$ z1 k+ H
</property>
1 n' E5 z# Y; W0 h! ^( g<property>2 D7 Z8 ]( ]  N8 r/ t( i8 c
  <name>dfs.namenode.name.dir.restore</name>. F7 Q$ l2 [% |- i- u& v
  <value>false</value>
$ X- f$ o2 d9 A9 m: |5 k& d  <description>Set to true to enable NameNode to attempt recovering a
/ k. ^% B* }& Y, P8 ~6 v1 y/ e      previously failed dfs.namenode.name.dir. When enabled, a recovery of any  }$ D: l1 h9 ?' @* p  Y4 \1 b
      failed directory is attempted during checkpoint.</description>7 e1 M) t: p9 C; e/ k( ~# R# w
</property>: Y9 i- s! G: d8 K
<property>8 d" a- F, G* D6 B
  <name>dfs.namenode.fs-limits.max-component-length</name>6 s, f9 I4 F( F. ]6 l( p7 P) a
  <value>255</value>5 S, V+ x7 i- R4 S6 R( s7 f; P% t
  <description>Defines the maximum number of bytes in UTF-8 encoding in each" b! p3 \2 h' O2 i  r
      component of a path.  A value of 0 will disable the check.</description>% A0 T% T" B$ Q: g% r* I$ F8 ?
</property>9 ?2 x2 W  _4 `  w, H
<property>
7 [/ n5 s# c3 c  <name>dfs.namenode.fs-limits.max-directory-items</name>: b; Z: u" K& ~% t1 G7 Q% W9 b6 n
  <value>1048576</value>6 r- B' c2 T! O- g
  <description>Defines the maximum number of items that a directory may" [% H& h. F- X* h
      contain. Cannot set the property to a value less than 1 or more than: l) T" Q1 ?8 ^% z; ]0 f
      6400000.</description>( \! k- X' G8 R
</property>
4 k% t/ Z$ A* V/ p( y7 x<property>+ a: f% F: x  t3 ^+ I1 ]5 m  L( C
  <name>dfs.namenode.fs-limits.min-block-size</name>
( V* s6 z6 ]+ z6 [! z5 @$ T) A  <value>1048576</value>
- u/ {5 X% @0 J' H0 s% n  <description>Minimum block size in bytes, enforced by the Namenode at create
  Q$ y' b- B. S( V4 ]% E# E: ?      time. This prevents the accidental creation of files with tiny block
' c8 m4 w+ U- x: R' e# h      sizes (and thus many blocks), which can degrade
$ z4 {6 v! K5 L0 s& z  z8 f9 l      performance.</description>% k: f0 C" b3 I$ w' r" I/ c% ?
</property>
" H6 T7 B. ^$ R- \<property>. `0 F' G! A  Y) S+ B3 U$ y4 J
    <name>dfs.namenode.fs-limits.max-blocks-per-file</name>
: k* M8 E) U: l& F, c2 B8 F    <value>10000</value>) r3 f1 R# s* ~' }( E4 e4 j
    <description>Maximum number of blocks per file, enforced by the Namenode on3 B( M, G4 J0 T( u& X6 i
        write. This prevents the creation of extremely large files which can4 ]! \( {8 o5 t- A3 G5 N1 m2 Z
        degrade performance.</description>
. x9 N; B) u" ^' s</property>
4 e% N3 }' v: |4 t: g<property>
% c7 ], y  M1 ^$ P9 C3 o  <name>dfs.namenode.edits.dir</name>
$ Z( F; A' N# V  <value>${dfs.namenode.name.dir}</value>0 e' T" I3 ?: ^" ]3 \
  <description>Determines where on the local filesystem the DFS name node
% [" Z0 ^+ b1 w) K3 p: x2 f$ ^      should store the transaction (edits) file. If this is a comma-delimited list% V! s6 U* v3 ]7 D7 [
      of directories then the transaction file is replicated in all of the & K+ @; J" q$ y( h) `
      directories, for redundancy. Default value is same as dfs.namenode.name.dir. f7 K% f0 ~( a% ~
  </description>
3 t" Y. X& B2 G( Y) K1 v/ F- p# j: Q$ \</property>
) n6 i2 C+ K( D+ s5 |<property>
& u  F  D& B9 S6 r& E3 P  <name>dfs.namenode.edits.dir.required</name>
2 Y& Q& T9 L& t7 c( D+ e9 ^  <value></value>
( j7 R: P/ k1 ?! k. q. N, a0 o  <description>This should be a subset of dfs.namenode.edits.dir,! E+ q5 r6 l$ L* p
      to ensure that the transaction (edits) file1 C! I; B! u3 j1 h/ N, d1 t
      in these places is always up-to-date.
6 Y: N4 I% C0 x7 i6 f, C, @4 j# P  </description>
2 h$ o& t9 O) _</property>
! N7 Z1 ~( ~. o# S9 [4 @! @<property>
& X, `6 U1 K  s6 v  <name>dfs.namenode.shared.edits.dir</name>
; c) m4 E! O2 }7 F+ x! ]) D% F  <value></value>$ ]* C' u$ p3 a; u9 |( _7 N1 k7 x
  <description>A directory on shared storage between the multiple namenodes% _9 b  {9 c4 v3 f$ L" t
  in an HA cluster. This directory will be written by the active and read
; m3 ]# Z. W  t' Q  by the standby in order to keep the namespaces synchronized. This directory
! f1 z4 U' r# Z3 v3 h) g5 ]% Z  does not need to be listed in dfs.namenode.edits.dir above. It should be) R& }. Y* n/ U. u/ i
  left empty in a non-HA cluster." _) J8 x9 s- ]0 q2 `" c) O3 l
  </description>/ U( z6 Z* x/ ?5 l! d) w) Y/ e
</property>6 G% T2 f7 y0 i; x8 f2 S; i6 u
<property>  s" ~/ D; O0 B3 ~' H
  <name>dfs.namenode.edits.journal-plugin.qjournal</name>
5 n& u0 P' c4 k# C1 }  <value>org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager</value>
% u; ]0 b- g" V3 I% h</property>
0 B. k, R# u- H- Y<property>
  u& N" G' q4 T* f  <name>dfs.permissions.enabled</name>
# Q4 X0 K) f* `. u7 z  <value>true</value>
" c& K5 i5 J" U- h+ r  <description>
5 W/ P3 H4 H$ c, U    If "true", enable permission checking in HDFS.
: ~3 D/ J- p8 A4 N  g- N    If "false", permission checking is turned off,0 U3 I. f1 _" W  \1 z8 h
    but all other behavior is unchanged.
( ~9 Y% {5 L4 P; z    Switching from one parameter value to the other does not change the mode,
2 \, K% y$ n9 f& F1 y# ]    owner or group of files or directories.
+ X. m. U+ f3 `' I+ J, J  </description>
6 i/ C. y* C: P7 E" _</property>7 Y" j0 A, ?* R7 a3 ~
<property>5 [; V9 E( i+ Q: c1 x* f2 @
  <name>dfs.permissions.superusergroup</name>
, x! t/ }) r8 G4 I: Q4 O  <value>supergroup</value>% \/ s! @. E# N3 y- m$ R8 |
  <description>The name of the group of super-users.6 Q9 `; i" \! p( F  w( r2 b+ F& k
    The value should be a single group name.. w+ r1 L4 V3 B, |3 N- y
  </description>
4 S0 ^+ T5 H& N# U3 k) y7 `</property>
2 M* R; Y1 j5 P, T0 y1 S7 R: y<property>
6 {  |) a; Y2 k! Q3 v   <name>dfs.cluster.administrators</name>
" @' T( m1 F0 _- S   <value></value>
8 y0 ]+ o/ V6 C7 i6 V# @2 N   <description>ACL for the admins, this configuration is used to control
% E& m, u/ R  N; T/ e1 O     who can access the default servlets in the namenode, etc. The value
% v" X, L4 `( Q& @2 A     should be a comma separated list of users and groups. The user list
# ~5 |3 @2 x  T5 m& o. J     comes first and is separated by a space followed by the group list,
# v/ m3 C+ Z& l# X     e.g. "user1,user2 group1,group2". Both users and groups are optional,
5 o' M/ L' Q" F% s0 _     so "user1", " group1", "", "user1 group1", "user1,user2 group1,group2"
9 V. a7 W. r1 \. t     are all valid (note the leading space in " group1"). '*' grants access: q. W" ^" H& O4 o- w& D
     to all users and groups, e.g. '*', '* ' and ' *' are all valid.
, N0 _, q" I7 m( I: w# ]5 h   </description>
4 T; p9 I6 z' ]5 M! H7 u</property>8 P1 Q$ g* {4 D+ n  ]8 a7 u
<property>& t6 _5 b6 U) u7 q  ?0 {
  <name>dfs.namenode.acls.enabled</name>/ c$ h4 f4 B$ U( `0 n" U; l
  <value>false</value>. |6 o& b) A1 _) o4 U5 a" D+ X
  <description>
9 T4 Q+ T' `9 i" l$ Q+ q    Set to true to enable support for HDFS ACLs (Access Control Lists).  By
# A2 S( @; w8 R( F    default, ACLs are disabled.  When ACLs are disabled, the NameNode rejects, u# F. T8 x1 Q/ h2 t0 W3 }+ m: Z  ?
    all RPCs related to setting or getting ACLs.
- b7 j5 s5 \/ c2 n; _% R: M& P1 o. E" o) v  </description>, O$ m* R0 i2 o6 J" k0 ~
</property>, D- G: N: S* K$ u3 ^
  <property>& `2 o/ }- \+ s  h3 v; K# D* X
    <name>dfs.namenode.posix.acl.inheritance.enabled</name>
$ [5 W" N' X! v) Q7 f    <value>true</value>
3 F; m1 a# ^; F) O& y/ c    <description>
; }$ f8 u9 g6 Y3 u* z      Set to true to enable POSIX style ACL inheritance. When it is enabled
% P  u( y& G, Q, L) b+ B' [      and the create request comes from a compatible client, the NameNode- N  T$ W9 [3 d  b/ E: q7 k7 ]' ~
      will apply default ACLs from the parent directory to the create mode- i  T' P: R" P8 Z9 P
      and ignore the client umask. If no default ACL found, it will apply the
: W. R" M/ I; m. a8 J      client umask.
, N  V. @, J9 D: W* C; ^; p    </description>
4 `" ?4 {  I) I, Y1 T5 f  </property>
2 {2 o/ H  [+ F( d8 B/ q0 v, J9 g  <property>0 r0 @  ~9 O3 h+ v1 `
  <name>dfs.namenode.lazypersist.file.scrub.interval.sec</name>! a% r8 ]% x! w. l% K/ N& x$ T8 U
  <value>300</value>
1 M" j/ l1 J7 d$ y0 t  <description>
( Z. W: _% }. ?0 x) c4 E    The NameNode periodically scans the namespace for LazyPersist files with( D. ]/ T! a! w* y( k. W
    missing blocks and unlinks them from the namespace. This configuration key: G0 ^" t" f$ q
    controls the interval between successive scans. If this value is set to 0,% [! z- E8 t! J! _) Q
    the file scrubber is disabled.. R* S" f$ k1 L7 o
  </description>% J8 a1 U% X% z% N# r; Q
</property>
" `; _+ r9 P8 ^, @3 x; g<property>
( E  a- n! z; P# B  <name>dfs.block.access.token.enable</name>/ s. e2 R5 F& R" a0 X
  <value>false</value>
# V. _7 t0 v+ o  <description># K" ]& O. g% H+ d1 k, M
    If "true", access tokens are used as capabilities for accessing datanodes.3 d3 P3 S& r: A( N) h
    If "false", no access tokens are checked on accessing datanodes.' y* x/ ]  j' [6 S
  </description>
' Q8 i" u; X4 X* E$ b& ~</property>6 `; W3 L1 k$ s
<property>% p/ v! A4 `4 }+ y5 f$ i
  <name>dfs.block.access.key.update.interval</name>
6 D! e  A9 K! }  i+ z  <value>600</value>9 P) @2 {! ?  Z6 K
  <description>
1 y2 h9 d3 [$ ?- T0 l    Interval in minutes at which namenode updates its access keys.* H( x* K1 `( M6 n
  </description>
2 h/ d2 ^7 U) q) t! v* Z</property>
$ [4 x( v7 z+ Y# N<property>8 v- f+ g7 Q9 N, M/ A
  <name>dfs.block.access.token.lifetime</name>3 e+ p* i/ A7 b  {- o
  <value>600</value>
" u: A7 `: M1 e' z" U% b  <description>The lifetime of access tokens in minutes.</description>' t8 A: n& P$ [* {
</property>
8 d  A& j5 e# I+ H2 [8 X1 p<property>
5 Y/ j" C4 W1 z% H  <name>dfs.block.access.token.protobuf.enable</name>
& M' ^; l6 G6 m+ B  <value>false</value>
* d- A* B  l# P: |1 q  L. n# G  <description>' c% V1 Q5 w! A3 U6 k/ {0 F
    If "true", block tokens are written using Protocol Buffers.
( F6 |  v$ J3 A* z' Q3 A0 j( O    If "false", block tokens are written using Legacy format.
! ]* @' ?0 ]1 m6 g- c  </description>
& d- n; v9 H$ G2 h3 _' z</property>
9 S2 H, D" \" `+ Y<property>: I# o1 u2 o$ x: Y4 z0 G
  <name>dfs.datanode.data.dir</name>; k' ?+ ~! ~7 ?/ X, k1 C
  <value>file://${hadoop.tmp.dir}/dfs/data</value>
# N: a! g3 i8 b  <description>Determines where on the local filesystem an DFS data node: l1 m, V7 S  Q( i6 I
  should store its blocks.  If this is a comma-delimited$ o% E/ t$ a( h7 W- `/ |: H' A
  list of directories, then data will be stored in all named& b/ `  \/ f3 [0 F# o6 }
  directories, typically on different devices. The directories should be tagged5 e' ?  f" }* q, [! g  K
  with corresponding storage types ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for HDFS
2 V' n6 C; x: u9 J8 ^) }  storage policies. The default storage type will be DISK if the directory does
& G: A- P1 T' o! }$ D9 u: G3 @# d  not have a storage type tagged explicitly. Directories that do not exist will
6 @! ]( n8 h% f- n3 }0 n  be created if local filesystem permission allows.
) L/ D6 }" L7 E! C& [. q# ~  </description>
! S9 z9 O% X0 k" w' }" m  d</property>
' S0 w. o# R) O: p# E7 U% |1 K<property>. o+ b- N5 w( K
  <name>dfs.datanode.data.dir.perm</name>+ V1 t$ R1 b4 x6 \
  <value>700</value>
9 Q6 Y9 U$ k( U4 L% ^" H/ U$ V  <description>Permissions for the directories on on the local filesystem where
/ I) l) ?4 K. e5 T% v3 x7 h  the DFS data node store its blocks. The permissions can either be octal or
$ t, |0 |$ ^; a7 e: L  symbolic.</description>/ \6 T0 E; R: l' M
</property>2 j! ?5 C7 J- ?0 o5 S
<property>
) `& l/ s* S* M% v  <name>dfs.replication</name>/ V* M, Q4 I' p" Q
  <value>3</value>, Q; X2 B& u) @& c- ?3 h
  <description>Default block replication.
# P/ F  r  c* Y% F) ^9 `+ t8 M  The actual number of replications can be specified when the file is created.$ s: T7 R3 u# m! q& I4 V
  The default is used if replication is not specified in create time.
# m2 }$ |! C; K* K  n4 `  </description>3 q& j$ v: `0 l7 Y# F  s: O5 H
</property>
% v0 K7 Z9 ~5 i0 i3 V" t0 X<property>( X( ^1 Z# d( o9 w6 T+ s
  <name>dfs.replication.max</name>
0 i/ N; j! v/ c# a8 K& G  <value>512</value>6 k6 `* o$ e: c& F; V0 G" M1 z
  <description>Maximal block replication.
5 k; J/ P5 B$ z  </description>+ V8 `7 W' X" m5 E3 ?+ Y
</property>2 y' \/ U" @( M
<property>! |* R, ^% l( o1 S" t" K8 @
  <name>dfs.namenode.replication.min</name>
% ^6 s8 u# |, O, l: {* p0 y  <value>1</value>2 {) Z* U, P3 S  r8 L8 f
  <description>Minimal block replication.
4 P- y4 C. \& N' c- f1 S8 y  </description>
/ J; z4 Y* B' N5 G" U8 \</property># n2 D$ o5 q" h5 v( V3 Z5 C8 |9 b
<property>2 o1 V  Q$ M1 w. C
  <name>dfs.namenode.maintenance.replication.min</name>
1 [, Q/ P  q/ I& C" ^. I  S4 g  <value>1</value>
7 O' ~" o4 |% [  <description>Minimal live block replication in existence of maintenance mode., x7 v7 I( N; z& ^; g' K; M4 v( G
  </description>4 f; j7 a* S: M5 G" E
</property>
( F% j1 Z, j8 O. a<property>: S, d6 a; M! R! P
  <name>dfs.namenode.safemode.replication.min</name>& o8 [! S0 H7 d; B: e
  <value></value>( n8 K0 o: I- P" F0 j
  <description>
2 W) a8 N5 I1 b' ~& ^      a separate minimum replication factor for calculating safe block count.8 X& F8 |. q: t' \  A0 V
      This is an expert level setting.
$ a& i0 y2 ?0 N8 ]$ Q1 @      Setting this lower than the dfs.namenode.replication.min
' Q+ Z2 P& X; ^4 G9 O+ n( Y5 m- H      is not recommend and/or dangerous for production setups.
- o# y  t: l7 T2 ~$ n2 ]8 f! A% [      When it's not set it takes value from dfs.namenode.replication.min& b6 y2 V6 {  M9 a& B4 M
  </description>
" g5 V$ B) s' g  n+ {</property>) _. ^1 o6 b9 s3 n# c9 X) p! G
<property>
4 e- S( X3 F3 ]  <name>dfs.blocksize</name>
/ Z6 K. f3 P0 |* K" f  <value>134217728</value>, c1 n0 A' K2 ~( @4 R
  <description>
; J- B  n, a/ T, B4 u: {      The default block size for new files, in bytes.
" q* q5 J+ O% W/ L5 [* l      You can use the following suffix (case insensitive):
8 ?- v! E) N& Q      k(kilo), m(mega), g(giga), t(tera), p(peta), e(exa) to specify the size (such as 128k, 512m, 1g, etc.),
; [( g4 n: T, l      Or provide complete size in bytes (such as 134217728 for 128 MB).
$ B% v5 s! [7 u( N2 g  </description>
) }6 p2 q: j& K' d; @</property>
# h& s( j: g! I4 K: @" U<property>
  k: f) @3 l- J8 N9 _! h  <name>dfs.client.block.write.retries</name>
: u& I8 J' f+ j$ [5 ~* Z  <value>3</value>2 n, j3 v( o2 A$ Q
  <description>The number of retries for writing blocks to the data nodes, 9 l' c# c" e' j" Y$ ]4 r, k1 f
  before we signal failure to the application.
4 P& c) v# N0 n7 B5 e& {8 K  </description>% t' [' {) ~" }/ |
</property>
/ Z, g% R9 M* {<property>
, W' \+ z. I, Y+ l- t/ N0 S. h  <name>dfs.client.block.write.replace-datanode-on-failure.enable</name>
( c! X: F) u; W6 D( t) B( K: R; c  <value>true</value>! Q6 m/ E2 _9 f% ]1 Q
  <description>8 T$ c$ ^0 ]/ B. y
    If there is a datanode/network failure in the write pipeline,. s; A* Q) `! M% D! X, _
    DFSClient will try to remove the failed datanode from the pipeline& _+ L5 U. B' b7 ~  r5 e" o
    and then continue writing with the remaining datanodes. As a result,
  ?5 V, h. K% i" ?4 t6 @* i    the number of datanodes in the pipeline is decreased.  The feature is
7 y/ Y8 g+ ]- }4 U: S0 I8 [. C    to add new datanodes to the pipeline.0 J) J/ p" f( m  n
    This is a site-wide property to enable/disable the feature.
: {" m7 z7 t7 L    When the cluster size is extremely small, e.g. 3 nodes or less, cluster
3 s0 J7 q+ l' u5 q& [+ E0 ~, l    administrators may want to set the policy to NEVER in the default1 g! h% O+ n: m
    configuration file or disable this feature.  Otherwise, users may- i: R6 f1 X( p; Z( f% R
    experience an unusually high rate of pipeline failures since it is
5 S5 ^% k, h( E" @    impossible to find new datanodes for replacement.
& [5 T2 D1 P- Z% p    See also dfs.client.block.write.replace-datanode-on-failure.policy: `/ @, f, S: ~3 q6 N  t
  </description>" y" c5 x2 A- z
</property>4 |& q8 I7 |+ Q  P: l
<property>$ i& O1 O) j$ P* Q8 b
  <name>dfs.client.block.write.replace-datanode-on-failure.policy</name>2 g4 x2 ?3 e6 ~% t4 F2 S/ I0 Z
  <value>DEFAULT</value>5 I/ V4 Z' b1 b4 U
  <description>0 f& @/ e) H: g, E$ j- t
    This property is used only if the value of# f0 S( R0 ~; P. y# a0 C4 _' H! s6 ~
    dfs.client.block.write.replace-datanode-on-failure.enable is true.
/ `& q1 e* @( F! j4 o& `- u    ALWAYS: always add a new datanode when an existing datanode is removed.9 T4 g# g) p# N) Z
    NEVER: never add a new datanode.
7 L$ y9 ~1 {$ k% ?% _( F    DEFAULT: 4 _0 O' d. ^" z: ^3 k1 o' W
      Let r be the replication number.! t6 R7 B4 Q" p- u
      Let n be the number of existing datanodes.  I  [) ]9 E8 A  Q3 Y% \  X
      Add a new datanode only if r is greater than or equal to 3 and either
5 O; P4 E" x' B1 `: n6 v      (1) floor(r/2) is greater than or equal to n; or
) F  i- j8 M6 a! D1 ~; j      (2) r is greater than n and the block is hflushed/appended.! J+ {( k% U, f& i! l
  </description>
' }+ `% H8 k+ [" W) ?  a9 m$ |: k</property>. n. M- |( I/ ]6 {$ r/ F, r+ o5 `
<property>$ i' L8 b, o$ I2 f, }
  <name>dfs.client.block.write.replace-datanode-on-failure.best-effort</name>6 M1 f+ {! {; u9 \7 K* B1 i
  <value>false</value>8 w3 ^- e6 q3 W) u( t
  <description>
: V4 }; [# }2 P9 f' @0 U, H- S& N    This property is used only if the value of" d% A1 g  ^, F+ C1 ~1 ?& R4 D
    dfs.client.block.write.replace-datanode-on-failure.enable is true.2 C) i: |0 L: G) X6 p! V9 f/ g
    Best effort means that the client will try to replace a failed datanode
0 k1 n4 r9 ?% I  _  n' q    in write pipeline (provided that the policy is satisfied), however, it + N' Z; D* S* c" ]1 _, o8 Z
    continues the write operation in case that the datanode replacement also
- B% S+ |; ]1 ?    fails.* K, z0 a- A: g5 y! d
    Suppose the datanode replacement fails.
. `( L3 Y" Q- o# v5 U3 x    false: An exception should be thrown so that the write will fail.
# p# D- c0 `0 ?; \# R    true : The write should be resumed with the remaining datandoes.
& H3 P4 G/ N9 a) p    Note that setting this property to true allows writing to a pipeline
: `3 W8 X- |$ @: c% |, [  {$ t! |. }    with a smaller number of datanodes.  As a result, it increases the8 }7 `: x" r; e& q- s7 b
    probability of data loss.9 U/ b5 F# o+ F, }, c
  </description>
4 e' X3 q$ q6 [. g2 h, e4 k, Z</property>8 O- {, X: @5 E: J  @
  <property>( @* v9 @: H, ~2 q& H8 p
    <name>dfs.client.block.write.replace-datanode-on-failure.min-replication</name>
2 G3 S( g3 `9 k' H: N$ I    <value>0</value>* v$ a0 f/ W7 d- ^/ ~- o: t# x
    <description>
% v$ c( W" @( ^& d. q$ m* c      The minimum number of replications that are needed to not to fail6 Y$ u, E# l  ]' E
      the write pipeline if new datanodes can not be found to replace. \2 R4 W1 q+ ]' O4 v
      failed datanodes (could be due to network failure) in the write pipeline.3 X4 |0 W  @7 h" ~, V5 a8 \: m5 ~
      If the number of the remaining datanodes in the write pipeline is greater
6 Z& f- B2 t1 I) o5 ]# N      than or equal to this property value, continue writing to the remaining nodes.
  y. f3 c  \0 }  ]% d! j      Otherwise throw exception.
' z* Y: Z' q+ h4 e      If this is set to 0, an exception will be thrown, when a replacement, C6 q1 B2 q. |1 `
      can not be found.
" ?) x2 U8 M! }( G  x- Q      See also dfs.client.block.write.replace-datanode-on-failure.policy5 K0 E( w- Q* U7 u6 v& _
    </description>* r+ N8 A% i: [: F
  </property>
7 p6 {" C* l0 X7 N- m<property>) U  U7 O( M: c) L
  <name>dfs.blockreport.intervalMsec</name>
# I- Y1 {1 Z, k/ E1 t  <value>21600000</value>
. b9 o5 F, Z  Z5 ?  <description>Determines block reporting interval in milliseconds.</description>7 q9 j. E) L- B- d
</property>' J: `' L- r7 R6 r4 c; Z$ c* ?
<property>+ i1 ]* K+ r5 m
  <name>dfs.blockreport.initialDelay</name>
# y% K7 X1 e: }( b3 O  <value>0s</value>) Q; b: i$ J% y- X4 e6 J7 u
  <description>+ @* s% x& o% C; @/ _9 ~
    Delay for first block report in seconds. Support multiple time unit; g+ a  s6 L  U% Z; t' F1 u, h
    suffix(case insensitive), as described in dfs.heartbeat.interval.* v- z' {  V( u: |
  </description>: I* R4 q- n. o5 A" x
</property>
  G/ W0 F& b: r4 i<property>5 m) o7 S: p% g& M
    <name>dfs.blockreport.split.threshold</name>
8 P7 j% g- E; a7 N7 a/ B    <value>1000000</value>
. [" d1 m5 }0 P) ], s0 R# ?    <description>If the number of blocks on the DataNode is below this
* c- q$ E3 n: t! j% Z5 _    threshold then it will send block reports for all Storage Directories3 K$ i- U, L' T' y0 R+ a5 A8 q; {
    in a single message.! g1 a+ p/ `. M4 E1 ^; ~
    If the number of blocks exceeds this threshold then the DataNode will
8 G: R- p' @- k& d: m    send block reports for each Storage Directory in separate messages.
: C" [, Q4 k6 R" N0 o1 u. x; e* y    Set to zero to always split.
9 x2 a' Y. o4 A" d  \4 {    </description>/ f5 F3 }/ [! o2 d
</property>8 S/ R9 I% W5 `! q- e& A: B
<property>
! e- e( e4 P/ g1 F8 o- j* a' E3 ^  <name>dfs.namenode.max.full.block.report.leases</name>
# k8 V9 R) j1 d$ I9 t; P  <value>6</value>3 `- G+ V/ n5 L* T  U
  <description>The maximum number of leases for full block reports that the' f" m. N- _3 E/ u+ A7 Q
    NameNode will issue at any given time.  This prevents the NameNode from6 w; x" P, I, A- v+ C: \: j% w; i
    being flooded with full block reports that use up all the RPC handler
& |2 e6 }7 F( J! |+ i    threads.  This number should never be more than the number of RPC handler: [) ~1 \& z; O
    threads or less than 1.8 T1 C4 _/ h1 b
  </description>7 m& R+ }+ I7 P3 u9 M
</property>( Y  _7 J" E) \! }7 E* m4 g
<property>% @& ~6 ^' j9 I% Q
  <name>dfs.namenode.full.block.report.lease.length.ms</name>: Q) z2 f6 Y% Z) V
  <value>300000</value>
7 {9 G* k( ^# C) R! `0 e( b  <description>. q3 }$ q) B+ @/ T3 [& K
    The number of milliseconds that the NameNode will wait before invalidating
& K% Q$ \# H. @    a full block report lease.  This prevents a crashed DataNode from+ ?. V1 u8 f6 \9 G
    permanently using up a full block report lease.
! |+ R/ L! ^; t5 x5 a" k  </description>+ e& Y, n, l7 q" e" E
</property>
( @. d- ?, O: h<property>
' m7 m+ [. w1 h) z9 S  <name>dfs.datanode.directoryscan.interval</name>8 c: B( r. V# X2 I
  <value>21600s</value>$ h/ d; g! G: x# z
  <description>Interval in seconds for Datanode to scan data directories and
- u( j0 R6 ]5 U; D$ b2 E, S9 Y( {  reconcile the difference between blocks in memory and on the disk.
# M# m1 e5 t4 @; Z6 R" S  Support multiple time unit suffix(case insensitive), as described
. {% \0 ^5 c9 C4 s  in dfs.heartbeat.interval.6 u  n% P" x: o+ T7 s( {
  </description>
1 K# b+ ~2 ~5 H+ t& i</property>
% f; P  Q: T8 j: t& t4 M4 w. Q<property># |" q( w$ Z$ S- y4 S$ A
  <name>dfs.datanode.directoryscan.threads</name>; b2 W$ l6 Q# P. l& w: t7 k' ^
  <value>1</value>: b6 Y! \+ Y8 _9 W. ?
  <description>How many threads should the threadpool used to compile reports, x% L# i4 }0 [1 X$ C
  for volumes in parallel have.
. X* t& e% K- M$ \8 T  </description>  K1 M8 p: h! M" ]7 z7 A
</property>
% ^, b" I4 d; O3 O7 d<property>  l: k9 M: i, O: j
  <name>dfs.datanode.directoryscan.throttle.limit.ms.per.sec</name>" |( @9 p0 L( k6 g( P. T6 |5 U  ^
  <value>1000</value>
+ J/ e4 S, M. B  <description>The report compilation threads are limited to only running for3 i4 \" ^. }9 z3 T# e) f# o% L
  a given number of milliseconds per second, as configured by the
  H  a7 g0 [8 W5 w9 x6 _  property. The limit is taken per thread, not in aggregate, e.g. setting
* U5 s9 s7 b& G  a limit of 100ms for 4 compiler threads will result in each thread being
9 Q0 G  @) _7 X4 s  ^- ^  limited to 100ms, not 25ms.5 |. ?  ?8 j1 _% r2 M, T
  Note that the throttle does not interrupt the report compiler threads, so the0 _5 h: U0 \" j+ l
  actual running time of the threads per second will typically be somewhat& [! H! G$ Y- [4 m( [5 E! a
  higher than the throttle limit, usually by no more than 20%.
+ n* b( X) r/ Z2 f+ {+ @  Setting this limit to 1000 disables compiler thread throttling. Only
6 _) ?& B; S# l4 k  values between 1 and 1000 are valid. Setting an invalid value will result
2 B9 y, U6 B* k/ R! G5 v- i" B  in the throttle being disabled and an error message being logged. 1000 is
. L: @# A* k: l" |  the default setting.
1 ?) X% C. `, H7 I- @7 _/ L! ~7 Q  </description>
- P6 ~0 j8 p( q. F* l# A</property>
9 _) q  u& W7 a# g0 R<property>' l$ Z3 E! D3 E, P$ c& C- r7 D- G& j8 Q) X
  <name>dfs.heartbeat.interval</name>4 ~5 ?$ \  \- H* s
  <value>3s</value>
4 C4 E$ c1 j! V$ Q2 Z* K  <description>5 m5 s. ?9 y& R7 {  ]2 c
    Determines datanode heartbeat interval in seconds.
* q; Y7 [2 Q! h* \    Can use the following suffix (case insensitive):
& C7 Y, r( c3 p2 b7 T    ms(millis), s(sec), m(min), h(hour), d(day)
  g9 r& p3 J* Y+ z: |' {    to specify the time (such as 2s, 2m, 1h, etc.).
' e% Y- y0 d; q% e    Or provide complete number in seconds (such as 30 for 30 seconds).
( |$ l# f. q) l. t  </description>
6 v/ g- n; o) P; n</property>
; |1 M: F, ~% k( ^1 }<property>) _/ Z3 @6 Y6 R
  <name>dfs.datanode.lifeline.interval.seconds</name>6 @; `  ~1 L* M) W7 `: e
  <value></value>
3 T3 E" b' \. @& b9 C& ~! z% K. B  <description>
/ P+ S, h" p4 {% {  Z    Sets the interval in seconds between sending DataNode Lifeline Protocol+ e: P  T  X4 v6 G2 G. K. T
    messages from the DataNode to the NameNode.  The value must be greater than8 d* Z+ Y6 a0 z" b& |  y" [
    the value of dfs.heartbeat.interval.  If this property is not defined, then+ [1 j$ w1 `( r8 n$ U( N. j% q
    the default behavior is to calculate the interval as 3x the value of
; k0 ^+ f0 @5 o9 c- Y* G, m; q1 B    dfs.heartbeat.interval.  Note that normal heartbeat processing may cause the, ?/ X7 v- g4 O3 ^
    DataNode to postpone sending lifeline messages if they are not required.. Q9 U4 u4 t' p9 e
    Under normal operations with speedy heartbeat processing, it is possible2 B& c! r( w( V$ q9 {
    that no lifeline messages will need to be sent at all.  This property has no& H+ _& u0 d8 S! y* v
    effect if dfs.namenode.lifeline.rpc-address is not defined.
+ S& ~: j: O4 I2 d' b  </description>
" b4 a8 O5 d2 r4 P$ j& Z</property>
6 X! e" L9 o  K<property>9 R0 x1 @2 C$ D! a, M) @
  <name>dfs.namenode.handler.count</name>1 i1 O+ c: Y: K
  <value>10</value>3 ~$ D/ i: R% L
  <description>The number of Namenode RPC server threads that listen to( y: v8 J3 G0 G1 r
  requests from clients.- S+ w$ p! k2 R5 }: M% Q4 v9 ^
  If dfs.namenode.servicerpc-address is not configured then2 u6 l; O/ J* }" L3 D# [( ]0 ^; Q/ l
  Namenode RPC server threads listen to requests from all nodes.1 x) k) Y: R8 V, K/ i& c
  </description>
/ }+ k4 l3 h9 [2 e</property>
1 w& U1 f  X" F9 O% h<property>
9 H6 |* j6 E$ q3 q3 d! S+ g- z  <name>dfs.namenode.service.handler.count</name>, p' M. N% b  N7 w/ \
  <value>10</value>3 Y) W  x' L4 @: u8 k8 R
  <description>The number of Namenode RPC server threads that listen to# D9 l9 ?3 g0 ~
  requests from DataNodes and from all other non-client nodes.
+ O: O& _# P# P- q  dfs.namenode.service.handler.count will be valid only if, Y9 q$ t& w: U* p! v* ?, z
  dfs.namenode.servicerpc-address is configured.$ K) R0 s7 c" d- g4 c4 L
  </description>; @- E9 }& f5 w; w3 @3 K
</property>
. [, Z  P0 F0 n6 W2 k<property>
( a( a# [5 T! L/ S" o  ^  <name>dfs.namenode.lifeline.handler.ratio</name>
9 V$ D: p1 B  [- ~  F7 v7 D4 L9 {5 ~  <value>0.10</value>
5 V+ U) N% ]& V! K) u' Q' @  <description>
. B* c0 j* ^* x- d/ W    A ratio applied to the value of dfs.namenode.handler.count, which then+ I- k' O; T! Y. e& [, @5 F
    provides the number of RPC server threads the NameNode runs for handling the
8 v4 F# |+ o3 m6 U6 o$ O    lifeline RPC server.  For example, if dfs.namenode.handler.count is 100, and' k6 l% ^* D: ~! I
    dfs.namenode.lifeline.handler.factor is 0.10, then the NameNode starts. k( g! ?% e  W2 J/ f$ o2 r& K
    100 * 0.10 = 10 threads for handling the lifeline RPC server.  It is common; e( t. o. n/ A" f' X
    to tune the value of dfs.namenode.handler.count as a function of the number0 R# y* P/ H' D4 \& ?! ]' f
    of DataNodes in a cluster.  Using this property allows for the lifeline RPC0 I5 {! k7 `* v: C7 |- L, U2 g% S
    server handler threads to be tuned automatically without needing to touch a6 R( O  H8 l; w4 g' M
    separate property.  Lifeline message processing is lightweight, so it is
6 P+ G; Y  Y$ O: ~! Y4 y    expected to require many fewer threads than the main NameNode RPC server.4 N6 d/ W5 b) a4 D* K7 [3 x* J
    This property is not used if dfs.namenode.lifeline.handler.count is defined,
( U6 U( s! v0 v+ E# ^: V: z    which sets an absolute thread count.  This property has no effect if( |! J8 o1 Z1 z2 M8 j. w
    dfs.namenode.lifeline.rpc-address is not defined.+ G, E5 A: x1 j7 ^$ {4 ]
  </description>
9 O* [, ~2 e( E* N' F! j</property>
& B: c4 B0 D9 L% r<property>! F7 d9 J: g5 s6 r
  <name>dfs.namenode.lifeline.handler.count</name>
6 _, M, C. o. @' c$ x  <value></value>' o8 B: ?5 u  r: |' w9 ?% e
  <description>
8 N7 @  _2 l/ N/ B) n+ J& F    Sets an absolute number of RPC server threads the NameNode runs for handling
* ]) J9 R' }- D( o( q    the DataNode Lifeline Protocol and HA health check requests from ZKFC.  If& J& \$ B) j6 R! w5 U; J
    this property is defined, then it overrides the behavior of
( D! c8 \5 e$ L. h    dfs.namenode.lifeline.handler.ratio.  By default, it is not defined.  This
* L, B' |3 S# G+ n    property has no effect if dfs.namenode.lifeline.rpc-address is not defined.' K: U- f, p( m" E
  </description>
$ m3 P& ?2 V  M$ h</property>+ j0 |5 H5 C- Z
<property>
$ H+ E- |7 S* \; B/ q' p; B  <name>dfs.namenode.safemode.threshold-pct</name>
1 Y* \: [% E6 j' \5 z0 L  <value>0.999f</value>
) R- V$ V, r: a" [( I2 w: p* g  <description>8 M+ _% `) \: [8 {5 m3 R
    Specifies the percentage of blocks that should satisfy
; t. ]; c& u: H) O* y    the minimal replication requirement defined by dfs.namenode.replication.min.
2 R+ l# Y$ S# q9 c; _: P( q    Values less than or equal to 0 mean not to wait for any particular0 I% P6 t, k; O3 `
    percentage of blocks before exiting safemode.; z2 n9 I" X- c; I7 b9 w& t+ f
    Values greater than 1 will make safe mode permanent.( H) l. j- F+ j/ E3 v( p. ]! b* ~
  </description>) s% Z$ Q- ?% Y! V; c6 r8 ]
</property>
+ v8 N5 z/ S5 ]: H3 C( f  `4 y<property>3 F% t) d$ o- C; E) k6 `
  <name>dfs.namenode.safemode.min.datanodes</name>
( n4 _+ G4 F. _$ L0 Z& J  <value>0</value>
# U& o* b. r& `% ~2 T  <description>
4 k: W: R& c) E4 O8 `0 o/ [+ S; e$ ^    Specifies the number of datanodes that must be considered alive. r3 e4 ~7 Y* r
    before the name node exits safemode.
3 A! K# T; `. d1 n: j    Values less than or equal to 0 mean not to take the number of live
- g" C5 l) ?7 L, V0 n    datanodes into account when deciding whether to remain in safe mode, v6 g( s# C7 P7 u
    during startup.
( p5 [( ~6 q, |  p# x- r    Values greater than the number of datanodes in the cluster3 C' S# b5 }: K3 {
    will make safe mode permanent.
, y) D$ j  X4 y# o' y  </description>7 R# t* l! t$ |
</property>
# x6 X5 {; U. m3 @3 ~# [5 D<property>( }8 E: C* {8 X5 s, T. T
  <name>dfs.namenode.safemode.extension</name>) B' M0 s- a7 _& t5 G
  <value>30000</value>
# I& u; l( p3 @+ H  <description>$ x$ K/ e& u! T0 V* m! i$ G
    Determines extension of safe mode in milliseconds after the threshold level
. ?. `' U' z) {    is reached.  Support multiple time unit suffix (case insensitive), as0 n" {$ b5 ]6 r- r3 O6 i) _
    described in dfs.heartbeat.interval.& Y+ ?2 O( p: Z
  </description>
4 }) @( J8 l& p! u/ a# y</property>
# ~7 J8 n' |$ ^+ X, d: K<property>$ U- B9 \* l# N8 b6 W  a) c
  <name>dfs.namenode.resource.check.interval</name>6 Q3 G* g3 Q5 Y$ I6 l" S! K0 |
  <value>5000</value>1 ?& J3 O& _/ ~+ c
  <description>
0 Z' s0 r4 A; `& I6 j8 K    The interval in milliseconds at which the NameNode resource checker runs.& B* W' U: z5 I
    The checker calculates the number of the NameNode storage volumes whose( M9 Q0 E& n' |/ H3 a" [! b
    available spaces are more than dfs.namenode.resource.du.reserved, and
" V& t& _. z( |( H7 J9 `8 U    enters safemode if the number becomes lower than the minimum value1 K5 I) U0 a' P$ W) q- F
    specified by dfs.namenode.resource.checked.volumes.minimum.4 h: D5 U- D2 C+ u
  </description>/ U. ^5 j- j/ B8 u
</property>  A% E, o% Z6 |8 H* z
<property>
' Z8 h9 A5 l0 Z7 C( S4 c7 K  <name>dfs.namenode.resource.du.reserved</name>
3 k9 N5 Q- e: H  <value>104857600</value>
8 F4 }, ~4 r& B$ j+ N) P4 d: f( r1 M  <description>" ~) l& S4 c4 B2 o+ w
    The amount of space to reserve/require for a NameNode storage directory
* C; ~5 t/ `! N- d7 p* s9 `: D    in bytes. The default is 100MB.0 r& _+ f% D7 j; ]
  </description>& ]' {0 }7 I" H/ \+ b
</property>2 ?$ ?5 i. o$ t3 V2 ~3 n
<property>
* E4 H% p' q6 E- i  <name>dfs.namenode.resource.checked.volumes</name>
; o- v' H0 ]; I& P3 X) f+ K  <value></value>$ H7 j9 C+ H" n2 Y
  <description>
; j$ T% s3 w6 Z/ e    A list of local directories for the NameNode resource checker to check in% E& e; t/ Y$ ]- C& Y. g+ C9 V5 a
    addition to the local edits directories.
0 D& k2 D1 t! [8 R" O  </description>" k! t1 Q( h( b! R
</property>
9 n9 F1 y% {8 Z# U$ F% _) l& [6 F2 L<property>
( j) N2 x% H2 V% e5 u  <name>dfs.namenode.resource.checked.volumes.minimum</name>
! [( V! W) s! v" R3 H' y& Z) w) }  <value>1</value>
/ L: I* W) G4 k# W3 G3 E  <description>
6 ^( b* ]3 Z) `( \  T7 m1 o) r    The minimum number of redundant NameNode storage volumes required.
7 D. `! f) U* O+ e. f  </description>
2 q$ U. A& f* E  g</property>: R: k. E, l8 P# r6 K
<property>
9 ^* |! X9 s: W- @. b% Z0 a. L  <name>dfs.datanode.balance.bandwidthPerSec</name>
$ u) t+ W4 l$ q* r8 K  <value>10m</value>" N# p0 L! `. F. D! d
  <description>5 o; S4 ~% q  R) F- t+ W
        Specifies the maximum amount of bandwidth that each datanode, Z7 a9 P9 S& x+ x: a
        can utilize for the balancing purpose in term of
' l  {; m$ V* q% G- s; Z        the number of bytes per second. You can use the following
* b# l- |- [8 z# t" @        suffix (case insensitive):  W) d$ a* p; K% y
        k(kilo), m(mega), g(giga), t(tera), p(peta), e(exa)to specify the size
! O' U$ [: F6 k4 f3 X& T% {5 G! i        (such as 128k, 512m, 1g, etc.).- t- T9 m/ |9 r" a$ Z
        Or provide complete size in bytes (such as 134217728 for 128 MB).
$ q2 x& V6 m0 {* B3 e1 }! E( \  </description>
! q' v/ ^% z" q6 T5 }& |</property>8 N- n! S3 b9 J7 v' h
<property>
3 t& Q1 q! X+ G# H  <name>dfs.hosts</name>1 l0 D6 R; Y' ~! J
  <value></value>
; Y+ w3 o/ n* M7 I- ?/ F) z4 t  <description>Names a file that contains a list of hosts that are( l, V1 u  N1 _9 K3 B4 _/ I, L
  permitted to connect to the namenode. The full pathname of the file+ {, s! `- ?- A% z5 d* j# m0 V
  must be specified.  If the value is empty, all hosts are; {9 l- w6 R# S; U' @" R- {" j
  permitted.</description>
! K4 T1 _8 i' K( \# W</property>  ]$ V- z' b4 h. F
<property>2 e* d" C' f1 `" @$ Q, W8 l
  <name>dfs.hosts.exclude</name>
9 r8 B* _! _, p' O+ F$ H+ M* f  <value></value>
& K0 e- C" F+ @8 R0 S  <description>Names a file that contains a list of hosts that are
% Y5 o& M. G$ D3 G  not permitted to connect to the namenode.  The full pathname of the
& Q) y# n8 q$ ?0 o- P  file must be specified.  If the value is empty, no hosts are& O3 F7 ^. G$ ~0 D
  excluded.</description>/ V0 X* c/ b$ j. j* j2 p" C
</property> 0 |: _0 W* D- K! A6 s. ^) @1 V: b
<property>; U# G6 N. l1 R9 R
  <name>dfs.namenode.max.objects</name>, H" G% g/ Q' i! F
  <value>0</value>
, G/ F- v6 e" p% V; {- D  <description>The maximum number of files, directories and blocks8 {# R4 J/ ]5 V& W5 C1 C; F' x+ E
  dfs supports. A value of zero indicates no limit to the number
( g# t& m6 X6 P  Y  of objects that dfs supports.
: v* r! z, \8 D9 B# e5 C' m  </description>; Y5 ]; `0 ?7 {$ t  x/ I1 ^5 ?& A- C
</property>0 U4 Y+ G$ T: ]8 U
<property>
9 `" J1 |- s2 C4 H/ g7 n' [  <name>dfs.namenode.datanode.registration.ip-hostname-check</name>) B9 Z9 i9 ]6 U2 [/ v5 ^8 O% r
  <value>true</value>8 R/ U) U  D% D& C
  <description>; f) `8 T# M, T. K8 w
    If true (the default), then the namenode requires that a connecting
6 ]3 C. q  M9 B* `    datanode's address must be resolved to a hostname.  If necessary, a reverse/ W* t* c7 p- U
    DNS lookup is performed.  All attempts to register a datanode from an
2 ^( r3 G: W* D9 m9 n( T    unresolvable address are rejected.
3 w4 z0 K* k# t4 _1 C5 @/ D    It is recommended that this setting be left on to prevent accidental
) Z& c$ Z. s: u: w    registration of datanodes listed by hostname in the excludes file during a$ Q( e$ G% S* O: H6 d9 z
    DNS outage.  Only set this to false in environments where there is no/ M* |+ U0 s5 v2 S7 b' X/ v" m
    infrastructure to support reverse DNS lookup.
* I# y8 r# ?8 a0 [+ i8 ~+ _( \  </description>3 b) g3 B2 C' m9 S$ _
</property>
* o6 P/ k5 D" i<property>
% T# Z' ~# R0 p# e  <name>dfs.namenode.decommission.interval</name>
% Q  t, q7 Z5 Q2 Y. X0 A  <value>30s</value>
1 p7 Q, k, B" v/ P3 R3 J  <description>Namenode periodicity in seconds to check if: r" ]% r- r! Q! T! R7 J+ F
    decommission or maintenance is complete. Support multiple time unit) Q1 u3 x- l, l/ D
    suffix(case insensitive), as described in dfs.heartbeat.interval./ q- s' Z8 @- G" ?6 e- R! A
  </description>- t( ?( H- _) g2 Y0 p9 S
</property>
  V# E* h9 T- m; A7 C" g<property>
# }/ o. ^4 S" {4 \1 @8 P  <name>dfs.namenode.decommission.blocks.per.interval</name>  p& y. T7 S, U3 p! o
  <value>500000</value>+ I$ c# V: D8 ]1 h) Q0 p* m+ y8 Y1 I
  <description>The approximate number of blocks to process per decommission' F* _3 `; t/ @4 U* e# p
    or maintenance interval, as defined in dfs.namenode.decommission.interval./ p2 ]+ P5 f$ Q/ U
  </description>
4 U" L5 m2 Z# a9 q/ U; A! `</property>
3 b* l- s  P; {<property>
" d8 j* f0 A4 q  k% Q' t+ l) ~  <name>dfs.namenode.decommission.max.concurrent.tracked.nodes</name>
% ?0 \: Y4 {' o7 Z: K8 f  J/ X  <value>100</value>
0 w% e$ o. P7 w- [  <description>1 e2 T' g5 T" A- i
    The maximum number of decommission-in-progress or
% t& E2 r" d- T4 `" ^, O    entering-maintenance datanodes nodes that will be tracked at one time by/ T  l3 c2 r8 o
    the namenode. Tracking these datanode consumes additional NN memory
+ @  T+ V6 |  b! U/ M    proportional to the number of blocks on the datnode. Having a conservative, I# U* B& ~* g# q- p* V/ U9 u
    limit reduces the potential impact of decommissioning or maintenance of
; Q" ]9 A/ k8 C! \4 K    a large number of nodes at once.& q- x% D) r2 G, y- w! T
    A value of 0 means no limit will be enforced.
4 i, |4 }+ o  b8 E3 ?6 X+ n  </description>
$ I; J- B5 K/ e1 G</property>
% I- y  g/ P; n5 O1 a6 `  e& {<property>! x7 K1 e0 g  X- ~3 v8 F+ G
  <name>dfs.namenode.redundancy.interval.seconds</name>) q: ?! Q# _5 v. {: q2 d$ N
  <value>3s</value>
$ K. N9 |( C8 {7 ?0 R. P9 A$ a  <description>The periodicity in seconds with which the namenode computes 7 F+ ]" u6 x7 u* Y1 S
  low redundancy work for datanodes. Support multiple time unit suffix(case insensitive),  N9 Y" d$ Y# z) p6 i$ y, W0 \$ w
  as described in dfs.heartbeat.interval.
5 U3 K& S! J1 t1 Q0 b" n  </description>6 ], R. U5 V  l$ q. d
</property>
5 T: l0 r% N  A- \3 J1 u9 c! ]<property>  @: U, M' K6 i4 {+ S- I: x
  <name>dfs.namenode.accesstime.precision</name>0 e0 h. Z  l) s( y9 S
  <value>3600000</value>2 y! B+ }& ~# E- }
  <description>The access time for HDFS file is precise upto this value. 2 g( d  S8 c/ j5 f. q- A* {
               The default value is 1 hour. Setting a value of 0 disables
, D) z! J3 y- v- R; o               access times for HDFS.
$ {8 n. O1 n! s) k( Y( d& y; j  </description>
6 `" A; A; i# z9 h, Z/ y/ P</property>7 Y9 z0 \  k/ z: z9 Q" X) N5 r
<property>7 v1 K* Y8 T7 U
  <name>dfs.datanode.plugins</name>
: c* P7 I& k! ]7 Z5 J9 K  <value></value>1 |9 B3 x) `) n$ C# @
  <description>Comma-separated list of datanode plug-ins to be activated.
- N+ h4 R$ {- v/ Q  </description>8 ?: Q; W* c2 a: c2 _7 z9 V5 @& \
</property>0 L  k( T6 i+ ]/ T
<property>
/ w! z' @8 ~2 Z( f  <name>dfs.namenode.plugins</name>
5 n, t. x& G$ M4 ~2 M( E  <value></value># Y$ j/ F2 _) T; ~3 ^" V
  <description>Comma-separated list of namenode plug-ins to be activated.
7 P8 h9 F2 t' P3 k$ t# I  </description>- `9 n: A, x; O( S7 f5 u/ o  P5 i- i
</property>
. H# M5 u, Q0 K, J4 T- b<property>5 b! m" A! i& o) x& N7 ?
  <name>dfs.namenode.block-placement-policy.default.prefer-local-node</name>
/ Q. r2 b& [! o- C+ o0 A7 j  <value>true</value>
0 T3 O. k3 j2 I$ r% l  <description>Controls how the default block placement policy places
8 z$ g- M  K, x( P  the first replica of a block. When true, it will prefer the node where" m# w" c7 X9 y! h6 V4 d
  the client is running.  When false, it will prefer a node in the same rack9 Z( y: v7 w+ b$ o$ J
  as the client. Setting to false avoids situations where entire copies of
1 A! P' \6 n( Y1 w6 ?# K" v6 U  large files end up on a single node, thus creating hotspots.
( H  \) S: ^. D  </description>5 p4 ~1 U9 _. {
</property>9 ]) t8 e# h& {
<property>4 i/ i+ ~( L9 n/ n. r
  <name>dfs.stream-buffer-size</name>+ _/ \! E, e2 a
  <value>4096</value>' z: d# Y, h9 u& w9 G3 X7 ?9 B
  <description>The size of buffer to stream files.- I* J3 D/ X  i- r# |
  The size of this buffer should probably be a multiple of hardware
. {* j2 c0 k8 v* x# K  page size (4096 on Intel x86), and it determines how much data is8 h9 T. [7 w+ y0 s0 {+ [! Z
  buffered during read and write operations.</description>
# R8 u/ ?& q* `2 k4 J* ~</property>
  k: x" E9 `1 l5 R; o* {/ w. }<property>
, r- B4 ?' F, U7 y1 A) Y$ `  <name>dfs.bytes-per-checksum</name>8 Z' K* ~8 p- j5 G( q, X9 c6 }
  <value>512</value>( Z$ ~' w+ x; }8 _
  <description>The number of bytes per checksum.  Must not be larger than, o- @- \) i6 l' o8 r7 q1 |' |. }
  dfs.stream-buffer-size</description>
! U4 E( T. M# ?7 `) b7 K</property>
9 r* G3 Y) N2 e6 W4 ~( N<property>  O; u3 e8 u% @$ o1 |! l' O
  <name>dfs.client-write-packet-size</name>: t  M# R+ @0 P, X1 v
  <value>65536</value>) R8 ~+ V0 x& U! G5 s7 {1 V
  <description>Packet size for clients to write</description>! V8 F8 t( m2 Z3 W" y+ @
</property>% i# F+ M( a7 x
<property>
" X9 A+ v- h4 ^; l. }. [  <name>dfs.client.write.exclude.nodes.cache.expiry.interval.millis</name>3 Q! r- I' Y/ u
  <value>600000</value>
" y) _% Z1 H; {: [  <description>The maximum period to keep a DN in the excluded nodes list* n# x  T- b6 j6 P
  at a client. After this period, in milliseconds, the previously excluded node(s) will! x5 v8 E( k8 a9 U9 E3 o
  be removed automatically from the cache and will be considered good for block allocations
/ [, ]) f! M1 j7 d* V" K! L& @  again. Useful to lower or raise in situations where you keep a file open for very long5 G+ E2 I  X- t2 h) c2 \6 A
  periods (such as a Write-Ahead-Log (WAL) file) to make the writer tolerant to cluster maintenance8 Q9 A0 o9 d% {
  restarts. Defaults to 10 minutes.</description>
  S/ {  X9 m6 _5 q# P8 v6 M9 c</property>
" k  `1 p: F% @0 ]- A0 }0 W7 X<property>% \$ I, q: [5 R1 G4 v  ?
  <name>dfs.namenode.checkpoint.dir</name>8 g% H2 }% |4 L) u1 [/ Y$ G$ `
  <value>file://${hadoop.tmp.dir}/dfs/namesecondary</value>
% f$ H' _% u6 C, j" ~  <description>Determines where on the local filesystem the DFS secondary4 f5 ^' z% S, H
      name node should store the temporary images to merge.
1 U2 D7 B$ \) \9 ~9 n% r: F( }: O9 f* f      If this is a comma-delimited list of directories then the image is
) B: s1 J8 V" [      replicated in all of the directories for redundancy.
2 }% ?, I: a0 d) c1 v$ O: `- g) B2 f  </description>9 T) N, T  t; ]! n
</property>
5 l2 B2 x1 _, C& m7 e8 R<property>, c6 s* r8 ]+ W! a) d. K# b
  <name>dfs.namenode.checkpoint.edits.dir</name>
. f+ I% a' G( f* E2 x  r2 u  <value>${dfs.namenode.checkpoint.dir}</value>
+ A; _+ r) w& ^2 I  <description>Determines where on the local filesystem the DFS secondary! k  f  _0 X) o) e8 s9 w6 y
      name node should store the temporary edits to merge.2 E+ d2 W: b' {8 j$ L* `7 G
      If this is a comma-delimited list of directories then the edits is
, [( x, l# t% }' T  y      replicated in all of the directories for redundancy.+ K0 c3 N2 |/ H( U" H
      Default value is same as dfs.namenode.checkpoint.dir" B/ d  X) n/ {8 B5 L2 b  e
  </description>: W, m) V7 E7 m* z9 f" U5 I
</property>
; M, x$ Q3 |: |8 Y% `. e3 A<property>' t3 Q% d# j/ y
  <name>dfs.namenode.checkpoint.period</name>( V. ]$ g7 h% W4 _( D3 R
  <value>3600s</value>3 H& I* i* h3 M0 F; U3 X
  <description>5 L( E* S, R( |. k* M
    The number of seconds between two periodic checkpoints.
& D- d8 h* S6 |& j& C    Support multiple time unit suffix(case insensitive), as described! ^& D2 f5 L6 N2 U+ ~0 t
    in dfs.heartbeat.interval." ]3 i9 U: e3 a& s: X
  </description>7 O% C; ]8 ?" ]7 y" r8 e
</property>
1 L( g6 d  p" V/ f+ ~<property>" H4 j3 K: U% F8 P3 O* Z; t8 O
  <name>dfs.namenode.checkpoint.txns</name>- ~# o' J# ]' E1 R
  <value>1000000</value>
6 }  y1 r, ?5 G1 L' j  H" Y  <description>The Secondary NameNode or CheckpointNode will create a checkpoint; ^' V5 x( d) W# ^0 R
  of the namespace every 'dfs.namenode.checkpoint.txns' transactions, regardless$ J3 N* y' z3 d& s% f) n1 x0 [
  of whether 'dfs.namenode.checkpoint.period' has expired.
: G4 i' C/ z5 w4 @  k0 L, o  </description>; j9 g0 M# B9 I5 K7 v
</property>
4 q' W$ u+ C' ~7 [<property>, P' j5 B; i- ^& F
  <name>dfs.namenode.checkpoint.check.period</name>+ ~. G. r! g$ O" r: K/ g: m
  <value>60s</value>/ C* `( S0 `9 \, |2 n
  <description>The SecondaryNameNode and CheckpointNode will poll the NameNode7 E( b7 w& H5 W
  every 'dfs.namenode.checkpoint.check.period' seconds to query the number# |) Y5 P; K+ J. |
  of uncheckpointed transactions. Support multiple time unit suffix(case insensitive),+ i! A+ E3 n0 f2 m# V8 U) G% [* ?
  as described in dfs.heartbeat.interval.
! \( Q! Y  r4 O3 ?  </description>9 y0 {9 z5 N+ K3 g
</property>/ H' b0 P/ e3 q  p9 Q6 E6 R6 o
<property>% |* a# n! U+ F
  <name>dfs.namenode.checkpoint.max-retries</name>% u: j2 N1 q* o+ X- F) F7 Q, F8 P# J
  <value>3</value>
+ T: t7 _" _0 I' I* K  <description>The SecondaryNameNode retries failed checkpointing. If the
" \# A. S( ^" r& X  m5 h  failure occurs while loading fsimage or replaying edits, the number of% K4 J- o. V/ z5 d! R4 v1 \; M9 r( o
  retries is limited by this variable. 8 t" p! s5 \& C$ Z4 }, h; F
  </description>! ^% _! h2 H; t9 Z: g
</property>2 Q7 ]1 ~/ A  o* D  Z" d
<property>: G* w5 M5 J" l% c, Q4 L5 Q, a
  <name>dfs.namenode.checkpoint.check.quiet-multiplier</name>
' z. G  G% G, G# b6 {) r  <value>1.5</value>! Q9 O8 r: i5 y* [
  <description>7 f4 i0 u# m' r) }  @
    Used to calculate the amount of time between retries when in the 'quiet' period9 X  x& h: ~5 j. X+ F+ p
    for creating checkpoints (active namenode already has an up-to-date image from another6 i) f' x5 y8 b! d* P- a
    checkpointer), so we wait a multiplier of the dfs.namenode.checkpoint.check.period before
, x3 K$ N( Y! I    retrying the checkpoint because another node likely is already managing the checkpoints,: V. k8 N" n: l+ e( G. x
    allowing us to save bandwidth to transfer checkpoints that don't need to be used.
3 j1 [% Q9 W5 G1 X  </description>
" S# K* X6 B. P! C</property>5 C0 m0 o# o9 D1 v" v- ^, I2 d
<property>. a5 _) G- W; m7 L. {( q
  <name>dfs.namenode.num.checkpoints.retained</name>/ G8 A" e1 v/ d$ E/ \2 z
  <value>2</value>% @- r9 p8 m$ ?, Q5 r0 [' M
  <description>The number of image checkpoint files (fsimage_*) that will be retained by
! U' q  ^; _: w2 x. a( m  the NameNode and Secondary NameNode in their storage directories. All edit4 a& P' f. x+ V
  logs (stored on edits_* files) necessary to recover an up-to-date namespace from the oldest retained
( ]( c! _8 G6 q. g0 ~8 n  checkpoint will also be retained.9 `$ Q( J5 |4 K  ~& M- O
  </description>2 A! l; ?+ h5 x; B0 }. N% ?) G
</property>
% m' f4 s0 L+ r<property>; S+ D# ~; ?7 |9 Z/ D& ~
  <name>dfs.namenode.num.extra.edits.retained</name>+ y: s* Q& @0 c
  <value>1000000</value>( I4 U! h; J; G) k% a
  <description>The number of extra transactions which should be retained
- c9 I4 h: Y/ G0 g- f" r  beyond what is minimally necessary for a NN restart.
3 \/ V3 Y9 R3 u+ K  It does not translate directly to file's age, or the number of files kept,
. b/ Z5 j# a: t9 y  but to the number of transactions (here "edits" means transactions).7 S% h* n7 s" i8 V
  One edit file may contain several transactions (edits).
' C' J; l( n+ d$ {( Q  During checkpoint, NameNode will identify the total number of edits to retain as extra by
9 I* v# t1 g5 M% v# `  checking the latest checkpoint transaction value, subtracted by the value of this property.0 B1 T% n, [* `  M. M: C8 B8 ^
  Then, it scans edits files to identify the older ones that don't include the computed range of( K* Z; V' N) E. A
  retained transactions that are to be kept around, and purges them subsequently.
+ ]# W3 _1 E. w( E/ j) z$ Q  The retainment can be useful for audit purposes or for an HA setup where a remote Standby Node may have
, y) U- F# |# p4 X  been offline for some time and need to have a longer backlog of retained3 @4 ^2 h# Q; s  n: O
  edits in order to start again.$ P1 e5 x* c; W1 S
  Typically each edit is on the order of a few hundred bytes, so the default
1 y/ @% m9 b4 b6 s, M$ t0 ]  of 1 million edits should be on the order of hundreds of MBs or low GBs.
* h0 s( D9 i% g, x3 P  NOTE: Fewer extra edits may be retained than value specified for this setting) e; `* i' ?9 A% A- K4 m- N
  if doing so would mean that more segments would be retained than the number
+ A6 J. G7 w2 A  Q: k; n  configured by dfs.namenode.max.extra.edits.segments.retained.* G& g. N$ w) X$ B) b3 o
  </description>
- x9 X2 ?) }% ~* x. I</property>
. e) ?% u% W- _9 t: V<property>  R8 j0 |; |6 P0 ~1 Z, d6 D
  <name>dfs.namenode.max.extra.edits.segments.retained</name>
% D& h: r! \# ]8 l) k  <value>10000</value>5 _9 b& P3 I* t) ^/ p( u1 n
  <description>The maximum number of extra edit log segments which should be retained
# p) {. |" d9 Y8 ^  beyond what is minimally necessary for a NN restart. When used in conjunction with; q, Z8 f6 F6 X- v8 p1 ]* Z/ Z
  dfs.namenode.num.extra.edits.retained, this configuration property serves to cap7 S) j& S! Y: r+ p  \6 f
  the number of extra edits files to a reasonable value.
( d# N! A0 x, F4 r, g  </description>' ?8 h* X4 }' D8 x' R( a5 X
</property>2 M/ R2 o3 U6 I3 h1 q
<property>
" J1 L, _9 c: I" L1 I+ A3 |4 a  <name>dfs.namenode.delegation.key.update-interval</name>  }9 x0 w. T1 ?' O
  <value>86400000</value>9 P+ c  e+ o3 ?! Q; N" _9 _
  <description>The update interval for master key for delegation tokens
3 m$ g" g1 q& V% q       in the namenode in milliseconds.6 f9 v9 J" {* Q' p" q! H
  </description>
# N* e; ]$ c4 l2 O& }</property>! a! J# }$ Q0 `* z! \; u( K
<property>
$ Z; b9 c# ]9 Q9 K9 [1 M, J  <name>dfs.namenode.delegation.token.max-lifetime</name>, l) N7 h2 p/ p
  <value>604800000</value>
5 }1 H4 W9 i$ u& R  <description>The maximum lifetime in milliseconds for which a delegation
3 L1 D% \' |& X& w: q# g8 p      token is valid., e  k& B2 j! n
  </description>
& P4 z; ?: ~/ o& ^9 ], b/ g</property>- ~9 z: j% ~" h
<property>
5 s; `1 Y9 J( Y3 P+ ^7 a- B  <name>dfs.namenode.delegation.token.renew-interval</name>  z7 q& [1 s, N5 x# k
  <value>86400000</value>
0 M& z  J" E2 Y- T& F/ D! b  <description>The renewal interval for delegation token in milliseconds.
' m) q$ y: F, G  </description>
. ]- I7 x* A, t</property>
( e; r" _0 I+ O3 r, c' y0 p7 L2 k$ R<property>, M4 o6 U* W) j. \8 D5 O; k9 G
  <name>dfs.datanode.failed.volumes.tolerated</name>! o0 P( @1 l7 |; V
  <value>0</value>
4 L3 M0 ~2 R, h5 ]9 c  <description>The number of volumes that are allowed to
; [! @, E* h) r7 ~# f  fail before a datanode stops offering service. By default! \+ J1 |1 D5 U- Z: \1 K$ ~/ _' Z- Y( G
  any volume failure will cause a datanode to shutdown.
2 W! |2 }1 D4 s4 b% l  </description>
: A7 o/ L* J1 F</property>
* ]( Q( P. E9 W. Y<property>
; X7 x: U& y( P' H  <name>dfs.image.compress</name># R6 q% t- s$ p
  <value>false</value>3 t8 l3 X  `0 X1 k( t+ h1 s2 L
  <description>Should the dfs image be compressed?' X9 I  \# M$ R$ ?
  </description>
6 |% N; b  B3 s. u" z; v5 l</property>1 A( m$ `  G* r$ d/ D. ^
<property>9 N) A& X! u7 t0 j. F- `- H% n6 Z! [
  <name>dfs.image.compression.codec</name>
0 R# d2 N- |) s% V! i# F! |  <value>org.apache.hadoop.io.compress.DefaultCodec</value># \# b+ D, k8 D1 Z) @
  <description>If the dfs image is compressed, how should they be compressed?
8 q" v% R& o8 A" t" |$ ]# G               This has to be a codec defined in io.compression.codecs.4 s- _4 L; |. b) P  P8 Y/ |
  </description>
4 z( t! L, s* `  A: R6 L0 E</property>4 s; ^! L1 f  b3 {; o' m" o( V
<property>' v: I0 s& M: L
  <name>dfs.image.transfer.timeout</name>( {6 X1 |4 q/ P/ d3 Q  w
  <value>60000</value>
' y$ E' d, I. r) C0 O  <description>
/ d3 C# s8 d7 e        Socket timeout for the HttpURLConnection instance used in the image0 e+ g- _+ T) S6 a
        transfer. This is measured in milliseconds.. s2 |! j' k, u
        This timeout prevents client hangs if the connection is idle; n& [: r/ ?% B5 F  }. o  H
        for this configured timeout, during image transfer.3 R* b7 J" w- d
  </description>
. c; y3 m1 |7 j7 f2 q</property>
* F. Q' L, _5 O* E8 P) R<property>6 l, `2 L0 R4 [3 ?* r$ d
  <name>dfs.image.transfer.bandwidthPerSec</name>* ^& D1 i6 E, i. H5 Q$ M" }
  <value>0</value>
$ O9 P2 C$ w- O# a# n- `1 ~  <description>1 ?" F, l0 ^. H- d) K' f
        Maximum bandwidth used for regular image transfers (instead of" U( _: X; T, [% \
        bootstrapping the standby namenode), in bytes per second.; j5 ?& [" g4 `2 [. d7 g% S6 O5 {
        This can help keep normal namenode operations responsive during2 [3 Q8 w( j: f) T) v
        checkpointing.
* b' E+ Y3 p% x& M, L+ E( M  J        A default value of 0 indicates that throttling is disabled." p8 L3 |* d* X9 i
        The maximum bandwidth used for bootstrapping standby namenode is! A- K0 o6 D" G( h
        configured with dfs.image.transfer-bootstrap-standby.bandwidthPerSec.- w, Q& ?7 m( i" y+ @9 ]; q
  </description>4 T: w% X2 q7 M: n# r( M, \) k
</property>3 t( E0 w% |! m  V1 }5 X' s7 O
  <property>
9 f6 ]+ G' n8 [4 \4 L) X    <name>dfs.image.transfer-bootstrap-standby.bandwidthPerSec</name>
5 y3 t8 @1 `1 d) c+ _    <value>0</value>6 [$ Q9 p6 v2 a/ X6 G) W) D  b
    <description>3 t1 o! W! Y% l( ^3 m& u7 R  f) b
      Maximum bandwidth used for transferring image to bootstrap standby5 _1 B9 C% J8 i
      namenode, in bytes per second.
% j; a! X6 M! f$ \* v      A default value of 0 indicates that throttling is disabled. This default; m6 @6 b: Q4 J% H$ @
      value should be used in most cases, to ensure timely HA operations.! ^5 r" a. X7 i2 b8 t; A2 n
      The maximum bandwidth used for regular image transfers is configured: e4 K/ Q  \- p- M" }. e8 B+ k
      with dfs.image.transfer.bandwidthPerSec.
8 ?/ v: N/ T  M& h8 G    </description>7 X2 f' Y* P5 }/ `; {
  </property>
& i# L. i8 V- }6 _' k& _4 s5 ^<property>
* s! B  |% R+ y& q0 H  <name>dfs.image.transfer.chunksize</name>
& a" c* D# W+ r  <value>65536</value>
) S9 z9 V& Z( f6 t  <description>" w7 C# m! [) M- X
        Chunksize in bytes to upload the checkpoint.$ {) |+ _  m' s( F6 R
        Chunked streaming is used to avoid internal buffering of contents
" i7 m9 C& |# M; i9 P# `        of image file of huge size.
: j6 A5 n$ f) T3 Y5 r% [$ c  </description>% G& _7 ]. a3 a1 A6 [
</property>
$ Q1 P9 W9 U& q+ n; s5 R+ N) ~$ ~) r<property>
9 b( N. E% v: H; y. U  z, C  <name>dfs.edit.log.transfer.timeout</name>
, f) q9 _0 G- \3 q0 \7 B  <value>30000</value>
; i) i) |2 ]- _' t, b5 R  <description>
( R0 A& s& ^* i- R3 }    Socket timeout for edit log transfer in milliseconds. This timeout
8 T2 c6 _" w) |$ o    should be configured such that normal edit log transfer for journal; Q& @* B" X9 ~, Y& H8 r3 ~
    node syncing can complete successfully.
/ X$ c9 g* l) H4 y) l* g4 T0 d7 t  </description>1 ?% [1 n  Z. v7 _  @6 m0 Z6 ~+ z
</property>8 E1 Y0 h( r) h3 W
<property>5 X) E8 F2 H$ k2 Z' a
  <name>dfs.edit.log.transfer.bandwidthPerSec</name>
" N( D- M  R& g+ r$ u  <value>0</value>
/ E% v6 `* p5 @: W$ Y) I  <description>6 v) Y6 @! E4 V/ j' z" I. l
    Maximum bandwidth used for transferring edit log to between journal nodes
/ ]; C& b. T  e  H% b. s: I    for syncing, in bytes per second.& x6 n5 j$ `) e7 k6 Z
    A default value of 0 indicates that throttling is disabled.
8 r5 y9 Z0 L  V# d6 t8 N  </description>3 o( r7 A+ N( _& ~0 A5 x/ {
</property>1 j4 q+ N7 G# {! S4 z; @5 V" k
<property>
* `  {* C! `* D2 }$ c  <name>dfs.namenode.support.allow.format</name>
$ V& L& V( s/ B1 M. s$ Y, Z  <value>true</value>: n9 h: H( g, @8 d
  <description>Does HDFS namenode allow itself to be formatted?
- z6 ?% b+ y& {& d0 [- x               You may consider setting this to false for any production- g7 f8 E# `7 A  e6 k
               cluster, to avoid any possibility of formatting a running DFS.) D: E- H! [9 V
  </description>9 I! |+ R0 F3 }; @4 W$ Y, A
</property>3 i! A9 I# b& I0 {9 T% Z
<property>3 h: c  N) l) V# a2 R/ Q
  <name>dfs.datanode.max.transfer.threads</name>
$ T4 J8 l( T  l! P! x* C: g  <value>4096</value>/ Q5 H/ `: G1 T1 Z
  <description>
* P, X9 U& e; W/ X        Specifies the maximum number of threads to use for transferring data
8 z0 s  L, p) _  L7 x* m        in and out of the DN., n2 ^+ ~: u8 F$ S) G/ \& t
  </description>
% v( n% X, V7 q$ f$ l( }0 {</property>" t" d* S# t9 r2 I
<property>/ r0 ~8 q. \6 F7 F
  <name>dfs.datanode.scan.period.hours</name>6 {1 \- ]  n7 v" x4 O$ D
  <value>504</value>
" G' O) p  l9 F& ?% o  <description>
# K. q% c- C! ~6 j        If this is positive, the DataNode will not scan any
! e& o( x# j2 I0 V        individual block more than once in the specified scan period.% U4 C- ~# `# T; t$ f/ i, j7 t
        If this is negative, the block scanner is disabled.
; c6 D$ P. i$ H5 b+ j( z% G% {4 e        If this is set to zero, then the default value of 504 hours$ h6 p) Y7 L2 ?" O, M) E2 s
        or 3 weeks is used. Prior versions of HDFS incorrectly documented
* Q1 }' j& D0 g1 T0 a+ j2 C) T- e5 y        that setting this key to zero will disable the block scanner.1 _  S9 P6 Z7 A: B& g" E
  </description>
6 l' c5 j7 D8 I</property>2 n( ?  A9 j' N
<property>' ~0 a+ M: \7 }3 u/ [1 h, l$ j* y
  <name>dfs.block.scanner.volume.bytes.per.second</name>8 @# Y& d* {2 s* {5 K
  <value>1048576</value>
  p4 D, l$ V" C2 V. ]0 P2 Q8 c$ r  <description>
  s: W# e- J; P5 z( J1 e        If this is 0, the DataNode's block scanner will be disabled.  If this
, {% U. t3 j1 ^% S* a        is positive, this is the number of bytes per second that the DataNode's6 R! [; v; Z4 t# Y5 U3 K
        block scanner will try to scan from each volume.* R( ?2 r3 t" f+ R
  </description>
3 b- [! |% ^; Z</property>: V; l: s! ]% n0 J/ E4 \9 x2 b) f) @
<property>
8 _1 X! ~7 ?( o/ R2 x  <name>dfs.datanode.readahead.bytes</name>
( ?8 B8 ~1 X0 D8 r3 H9 b  <value>4194304</value>' E! e$ I" j5 e4 J4 c3 {5 ^& I% _
  <description>( l5 N5 h9 v, w$ n5 _- B
        While reading block files, if the Hadoop native libraries are available,; {0 i0 T- a/ e/ H  t, @
        the datanode can use the posix_fadvise system call to explicitly
; R) ~' O) n# r        page data into the operating system buffer cache ahead of the current5 h9 m% Z2 ?" f8 f# [' F
        reader's position. This can improve performance especially when
+ K; l  [* ^5 w, q' s6 j        disks are highly contended.
1 M; ^! `: l3 q! E1 r+ B% g' O* r        This configuration specifies the number of bytes ahead of the current
7 ^9 x: X# k( ]' h7 P        read position which the datanode will attempt to read ahead. This
( l  Q% s) n, d: {2 H        feature may be disabled by configuring this property to 0.
7 V9 r7 U/ U8 H" @1 d, C        If the native libraries are not available, this configuration has no
- v. a5 K5 n7 I, v$ \- K; a) v! s        effect.
" y, B# A% e9 R: w. m  </description>, t) N# o6 r' t! F1 q
</property>, R! N& x" t) j
<property>
. Y1 d1 @6 R1 q) ^/ R" O0 m  <name>dfs.datanode.drop.cache.behind.reads</name>
) S" _; U+ A* T4 `8 X  <value>false</value>8 b& Q" y1 _+ p$ F8 V& g) e
  <description>
4 I  f- M' [8 |- @4 M' c        In some workloads, the data read from HDFS is known to be significantly
1 _( u0 q* P+ k) ^6 x9 F        large enough that it is unlikely to be useful to cache it in the
) x+ q6 O* B: O; B/ M        operating system buffer cache. In this case, the DataNode may be
- b/ i$ Q2 U. Z        configured to automatically purge all data from the buffer cache4 C+ o+ e% b& I5 L6 h2 O. t
        after it is delivered to the client. This behavior is automatically
' C/ {* ~5 i. ^7 g        disabled for workloads which read only short sections of a block
5 w2 h5 e+ D) S" z        (e.g HBase random-IO workloads).) W9 B7 d0 a5 a% A$ a8 _
        This may improve performance for some workloads by freeing buffer) \7 j( A7 d% G1 _0 Q
        cache space usage for more cacheable data.+ m' U1 J: x9 I0 \4 }
        If the Hadoop native libraries are not available, this configuration
. I6 ^! [/ t3 y0 A- `' w  q        has no effect.
) P' s$ b% `& \  </description>( \; p6 S1 K/ g' F! o
</property>- K; S8 d2 {; R9 |3 T) A
<property>
4 e) B4 u: @) p# n; U* S* ?) m  <name>dfs.datanode.drop.cache.behind.writes</name>
3 R$ H) b& m0 o9 z4 m  <value>false</value>6 J- F- j# a6 ?" U. c
  <description>$ @7 V2 Y! _4 T/ A
        In some workloads, the data written to HDFS is known to be significantly
5 R8 N$ Z. `; E4 q: x! P        large enough that it is unlikely to be useful to cache it in the) h) w, {, q) b2 i1 C
        operating system buffer cache. In this case, the DataNode may be$ y8 y+ V9 T- ~1 S- G! Z- p
        configured to automatically purge all data from the buffer cache, i* ]' u2 M' a9 W/ X8 O% B
        after it is written to disk.
& C; z) T/ S7 P$ O. @        This may improve performance for some workloads by freeing buffer5 x& A* I9 b# L8 J& M1 v! u7 Z& o3 P
        cache space usage for more cacheable data.
& V. H; \% L: n) p        If the Hadoop native libraries are not available, this configuration# V9 x% n1 p5 d& F  P; P
        has no effect.3 W( u& W# ]7 w6 V9 h$ A. k
  </description>
% K8 l& r% ~# O! L, L" M6 s. w</property>+ o5 G5 P$ f: m, Q$ e
<property>6 ^* ?. s7 I0 R
  <name>dfs.datanode.sync.behind.writes</name>
) o4 N3 t! Z$ s2 G' k* C( f- b  <value>false</value>* X" R3 ]  \  i9 |0 @
  <description>
( o& c% s4 o3 K! `! U# W, ~        If this configuration is enabled, the datanode will instruct the
$ W0 m% X% [. a9 J  X9 ?+ a        operating system to enqueue all written data to the disk immediately
( K" P7 B2 E" h1 w( x$ @7 Q        after it is written. This differs from the usual OS policy which
5 O% d7 l' _3 `" e: Q6 [: V        may wait for up to 30 seconds before triggering writeback.: D9 m1 m' ?1 E
        This may improve performance for some workloads by smoothing the
$ n4 e. G( X2 Q/ @8 F* _        IO profile for data written to disk.
$ x5 n4 |' i4 I4 }8 Y        If the Hadoop native libraries are not available, this configuration2 e) A" r+ T9 @5 M
        has no effect.+ L9 U% H7 ?: R; b' V: W
  </description>
$ F. g2 P6 c) N* B/ V</property>
1 n7 t; j. e# b3 d  G! i! L1 Y4 Q<property>4 E# u, |  h6 m2 _& H, v9 b, p+ Y
  <name>dfs.client.failover.max.attempts</name>
+ t, ]6 V# e  _' K  ?# r  <value>15</value>, A. i5 c' t& Z; t5 p
  <description>
# w: \) H0 @" C( L: ]$ J& g, i0 m    Expert only. The number of client failover attempts that should be
+ ^3 {7 X9 O" L: ^3 _2 Q    made before the failover is considered failed.
3 c+ A! [: O) \5 }  </description>) {) N: n* O. \0 P3 r, r
</property>: S' ~# `5 O8 n& \7 ~6 A* U
<property>$ Y  s4 @$ d! {+ O) Q' s% ]" q1 i
  <name>dfs.client.failover.sleep.base.millis</name>
' u2 l; p2 ~& z. g8 z8 H  <value>500</value>% D' x, ~1 q& O0 M; D& c( L
  <description>5 f7 a* I. l6 x1 A+ ^
    Expert only. The time to wait, in milliseconds, between failover  e- s& {; W# Y5 a1 h( t- {
    attempts increases exponentially as a function of the number of
+ q2 S, e. S' p, h- F    attempts made so far, with a random factor of +/- 50%. This option
2 h" k6 n4 d0 O5 }3 ~0 Z    specifies the base value used in the failover calculation. The
6 o9 e: w: b  T/ V% |: m% U2 O8 }    first failover will retry immediately. The 2nd failover attempt/ t2 n. b' `2 ^) x  Y5 b8 K
    will delay at least dfs.client.failover.sleep.base.millis
# c3 t& L7 z" V) z1 s' ~    milliseconds. And so on.
; R& Q) j" w. W$ n/ _  </description>
0 p! Y. C3 b; z: ~</property>
/ Z  @+ ^8 Y+ `6 S8 P  w$ P<property>
* ~0 S  k9 n, B  u2 o  <name>dfs.client.failover.sleep.max.millis</name>
+ [# K) |, E! k4 a  <value>15000</value>
5 L# t  d- E" ]+ ^. B5 ^  <description>
5 s6 P' o, h3 k    Expert only. The time to wait, in milliseconds, between failover
7 K  z; D( v$ z$ i# L    attempts increases exponentially as a function of the number of
9 ]* Y" @6 I: u' ]6 G    attempts made so far, with a random factor of +/- 50%. This option7 F& O1 e; g8 ~9 d3 q: T
    specifies the maximum value to wait between failovers. $ i1 q$ }0 }0 O* b5 j7 t' f0 |( u
    Specifically, the time between two failover attempts will not
5 v  J, _% ~$ t$ z    exceed +/- 50% of dfs.client.failover.sleep.max.millis5 H+ K5 o' n5 i) w" f* l
    milliseconds.
# i  }0 ^7 p2 A4 N9 E9 s7 K7 I  </description>/ N# k% u' z- a3 g4 E
</property>
% X/ E  M$ E8 @- v  X<property>
& ^; i+ `% L1 c" Z; j5 @+ K  p  <name>dfs.client.failover.connection.retries</name>
; m: ^- ?7 h0 L5 I- _- a  <value>0</value>! \, d# T$ J7 O, r
  <description>
0 ~7 Q2 {& w+ m1 z: a/ E    Expert only. Indicates the number of retries a failover IPC client3 `( V% \! f% x/ p
    will make to establish a server connection.
7 w+ Y8 \% e  d; m! |; ]/ I  </description>6 d6 C  J# V9 T; L+ e" e0 i' ]) I
</property>$ D- o" ~& o* x8 P2 {& P
<property>' b5 V, r8 @2 {0 ?( ^
  <name>dfs.client.failover.connection.retries.on.timeouts</name>
7 v# y, c; y2 l3 a( F  <value>0</value>4 ^5 ?0 f! N0 y6 I1 S/ y2 A! U! E
  <description>
1 N1 A4 j5 @( a    Expert only. The number of retry attempts a failover IPC client7 b1 u) x; F3 m) @0 J' t
    will make on socket timeout when establishing a server connection.
3 d- U7 `8 }' \  </description>% p4 u1 H( y  M8 t, j
</property>
, E0 K7 A2 _! A) G8 p7 ]<property>
1 k0 L6 Z; N. C9 I$ d/ e7 }  <name>dfs.client.datanode-restart.timeout</name>9 l( r( n* }( L3 j( s% H
  <value>30s</value>$ t, |, ~* z$ d7 F$ n
  <description>
" ~% j) T1 f2 Z% v+ i+ ]    Expert only. The time to wait, in seconds, from reception of an5 k7 ^: Y6 d1 `3 j
    datanode shutdown notification for quick restart, until declaring
5 h3 W0 |3 `. d0 Z0 T    the datanode dead and invoking the normal recovery mechanisms.
7 s4 e" K: p7 q1 f0 O7 N9 {- j    The notification is sent by a datanode when it is being shutdown- q9 \) V+ h0 |, v; ], i9 \9 f
    using the shutdownDatanode admin command with the upgrade option.
) U9 K) a: u/ S* u* }0 y    Support multiple time unit suffix(case insensitive), as described
9 w# A( n7 b% X( h( Q5 K' k    in dfs.heartbeat.interval.
  j# {3 w3 [* t( M4 W1 T7 b( A  </description>! S6 ^, J# u5 c) ^" N$ Q1 `
</property>" Y7 c( P/ l  ]
<property>
+ P1 x# o7 j3 n. ?$ k. B0 u  <name>dfs.nameservices</name># J$ W' |' o( A7 U+ w( x: Q8 y5 g2 o
  <value></value>
2 i6 X- E1 I. P$ ?" e7 o* g  <description>
  z6 g: n2 e/ E1 i5 E) M! E    Comma-separated list of nameservices.
- K6 _. h% q0 L1 _  </description>
5 D: F7 B5 Z. P, l, r) x: V</property>! E1 e& l. G3 @3 U( G6 i
<property>
( _. m3 ^* n% [! K5 x; ]  <name>dfs.nameservice.id</name># f: T% c. V3 V5 f) _
  <value></value>* L. K. J& Y8 [
  <description>  L5 f  p% x. v8 r. t/ O  |$ V3 l
    The ID of this nameservice. If the nameservice ID is not
8 _& A$ ^6 Q. I6 M  {3 V    configured or more than one nameservice is configured for# x3 w7 K) m! ^# O$ D
    dfs.nameservices it is determined automatically by
" s( C; j8 u! Y; [    matching the local node's address with the configured address.
6 m& q* b- b7 [( p0 J) h  </description>) _! Y0 K8 [: ~  Y/ V" k- M
</property>
/ s: ~1 P2 A, V0 ~, S4 w<property>
, |+ G; g% Z! D: C  <name>dfs.internal.nameservices</name>
' J. H- W4 m, ]/ V+ N- w' e  <value></value>2 P' U9 y6 c& {6 M) \
  <description>
1 O& e5 `. d, ^5 \8 w& S    Comma-separated list of nameservices that belong to this cluster.7 a# G. ~1 a& f  X- {
    Datanode will report to all the nameservices in this list. By default, u9 s+ a- V% e( X: _# G
    this is set to the value of dfs.nameservices.
8 S5 |/ i' l* m5 C  </description>1 I6 u. c7 q6 p! F
</property>
# L! n- }9 a7 z6 E  _<property>
8 H8 x" w" a4 X4 v  <name>dfs.ha.namenodes.EXAMPLENAMESERVICE</name>
9 f$ d: w8 R& @/ ^  <value></value>' e7 x$ I% P; R: E+ c0 R
  <description>
1 }9 W/ q1 }& P2 f$ E- A    The prefix for a given nameservice, contains a comma-separated
  c6 W- l8 ^& @2 Y( w    list of namenodes for a given nameservice (eg EXAMPLENAMESERVICE).
7 p$ J% d- a" l* o7 ~    Unique identifiers for each NameNode in the nameservice, delimited by9 g) }& O' P/ F! Y2 P0 r% h
    commas. This will be used by DataNodes to determine all the NameNodes3 k( n. ~/ @! z
    in the cluster. For example, if you used “mycluster” as the nameservice" Q' H4 M, x! c& R& Q
    ID previously, and you wanted to use “nn1” and “nn2” as the individual2 S2 y8 x# ]6 {7 R  t" w  ^1 J
    IDs of the NameNodes, you would configure a property
* d6 r2 ~+ k: \5 c$ i# N$ V1 T    dfs.ha.namenodes.mycluster, and its value "nn1,nn2".5 ~% a* [, g  c! g  @
  </description>1 f' N- C0 N% Q; Q) L
</property>
) w3 I. n1 @9 v- _<property>
; ]" k  k5 ?* H6 q7 @. F- x4 W  <name>dfs.ha.namenode.id</name>2 z2 `$ Y+ D1 [% B+ S* h* ^7 b
  <value></value>
' S( M% @) m- n2 z3 S% z8 W1 y; b  <description>
7 y+ X' Q, x* V. m3 f$ j* V    The ID of this namenode. If the namenode ID is not configured it( I' U, r* w. ?: W0 f* [
    is determined automatically by matching the local node's address4 I9 _$ M3 K1 D7 M
    with the configured address.
7 a! W5 B7 i" m( N$ l; }+ Y  </description>
: p& S( ~) s7 X</property># [/ X" `+ ?9 \
<property>  `; ]- f" |7 h+ d4 J
  <name>dfs.ha.log-roll.period</name>
/ X3 I9 O1 L% o; W7 n6 n$ y  <value>120s</value>- a& c) X+ N- o7 F( I
  <description>- Y+ t: D2 h. X1 v/ e
    How often, in seconds, the StandbyNode should ask the active to
- I: O1 d1 C1 A  o9 T2 s    roll edit logs. Since the StandbyNode only reads from finalized
- \% S6 K' I* ]; y8 C    log segments, the StandbyNode will only be as up-to-date as how; Z0 _; X) K; f3 ?9 a
    often the logs are rolled. Note that failover triggers a log roll
/ d& n5 u! _1 M; f" n    so the StandbyNode will be up to date before it becomes active.
! V) B1 ]4 m' p+ x8 ]* `    Support multiple time unit suffix(case insensitive), as described/ f, v. R5 G" L& ~, {3 n
    in dfs.heartbeat.interval.
& O4 Y7 V3 d7 p- b: i  </description>* e- R5 I1 E7 P' D
</property>- _) |& J! v% R! M. M( l
<property>6 d5 t, T# Q8 M* m. c
  <name>dfs.ha.tail-edits.period</name>
( i* W$ X2 g4 _- ^  <value>60s</value>
2 J- V. q1 R+ }/ r  <description># K: c0 I$ c. p; E4 P% S# S. _; O: r
    How often, in seconds, the StandbyNode should check for new
2 F( }+ l- B7 k" z7 m/ P' J" u    finalized log segments in the shared edits log.  W% L* o. P! M3 h' }# [" L
    Support multiple time unit suffix(case insensitive), as described% z- l2 q, `3 J5 J# G
    in dfs.heartbeat.interval.
. L. L4 \1 e. b  </description>
, C2 v3 h" B6 L</property>
) f8 r$ p6 e  H. W9 @( |% P1 o<property>
. I& Y, @2 R2 R8 u6 Q  <name>dfs.ha.tail-edits.namenode-retries</name>: P7 D7 t, [8 f4 c" c
  <value>3</value>! `& O* U( a- f3 u- [9 |- [5 m
  <description>; S. z0 Y: B9 G. |# Q
    Number of retries to use when contacting the namenode when tailing the log.5 G6 a+ r/ r! z. S$ j. e+ B5 c
  </description>
- H" u/ t0 |% @6 ?</property>! O6 B: `# {7 i" Y" Q2 c
<property>* m. s( T+ W" A& g2 l
  <name>dfs.ha.tail-edits.rolledits.timeout</name>
1 N! Q* r, X; }0 C( C  <value>60</value>$ T- o& i+ @2 R
  <description>The timeout in seconds of calling rollEdits RPC on Active NN.# ~9 c( c3 l# `# y( _
  </description>8 r, a/ b; i* G# L: m8 M
</property>" q9 N8 n3 Y9 L: C5 _! y. H/ C
<property>
7 r$ s9 O: G8 F9 ^  <name>dfs.ha.automatic-failover.enabled</name>) ~. s4 J7 `; C2 n5 c7 ]
  <value>false</value>. ]) v$ J: R6 J$ Y0 l5 ~$ A
  <description>% T; B. b" Q: y
    Whether automatic failover is enabled. See the HDFS High- n( h9 f$ j# x/ N8 Z2 t$ D& V
    Availability documentation for details on automatic HA
/ H* i  s- {! F1 W  k+ Q: J2 q6 [    configuration.
& D# y- t" b5 r- g  C  </description>( ~3 l$ L1 P6 W; K) d. p2 |
</property>  h$ L1 n/ b; Y
<property>5 ?2 j% r& l+ i$ s/ v% S+ ?
  <name>dfs.client.use.datanode.hostname</name>
  _  q+ \9 P( a8 z( P& l; G5 t; A. {  <value>false</value>
4 n& Q5 u! ^: h, E  `* T4 C9 x" E7 D  <description>Whether clients should use datanode hostnames when
* o% b) [9 \5 h. e; a$ {' M    connecting to datanodes.
4 {- ^3 H8 z5 B$ U  </description>
3 U, u  m9 ?, c& U! Q</property>
' _6 z6 P! E& |4 A# ?: v  Y# V<property>$ M% P+ p& p0 \- K( F% e* n
  <name>dfs.datanode.use.datanode.hostname</name>2 [' n* [( Q6 D. ^& i" c
  <value>false</value>( Z. }" M% K$ X6 y1 k; I
  <description>Whether datanodes should use datanode hostnames when9 c1 o9 I$ t5 r( Z6 D
    connecting to other datanodes for data transfer.
0 B0 T& j$ v8 y" t+ g  </description>' B0 h* F: ~1 |7 n; C
</property>
/ L3 f  j8 P4 `/ A<property>
5 h2 `$ X: h# H) D  <name>dfs.client.local.interfaces</name>
1 m9 ]/ l6 q3 d1 ?1 z6 e  <value></value>
! f8 u- V; v8 }, ]& h  <description>A comma separated list of network interface names to use
# k; P( l) [; d$ x  o  R* D    for data transfer between the client and datanodes. When creating
8 M& f1 x0 S2 B% e7 f" X8 E9 ?& R    a connection to read from or write to a datanode, the client9 U" c& G1 l  q% u
    chooses one of the specified interfaces at random and binds its
, N" o! f1 [" Z* L' Z    socket to the IP of that interface. Individual names may be4 {6 ?/ i2 ^$ |; {
    specified as either an interface name (eg "eth0"), a subinterface
5 F8 ]1 K- s! d4 u* Y    name (eg "eth0:0"), or an IP address (which may be specified using
5 h# z* S. Y4 ~    CIDR notation to match a range of IPs).6 X/ _" d- R$ e1 J
  </description>! E, e3 x3 e0 P9 k  }
</property>. R6 i+ p3 A3 n1 K' g
<property>; q, \6 B* C* u, r" c0 `* M7 U
  <name>dfs.datanode.shared.file.descriptor.paths</name>
8 K* k# P6 I& Y+ P" c3 k0 J6 ^  <value>/dev/shm,/tmp</value>
' k- d4 K1 L" h  f$ a8 `5 K- K$ _  <description>
7 s' r) b9 C/ ^$ O; g# B4 {/ W    A comma-separated list of paths to use when creating file descriptors that: _2 i7 x: N& t- H  t$ Z8 P" b2 C6 R
    will be shared between the DataNode and the DFSClient.  Typically we use8 ~2 n- q5 R/ m" b
    /dev/shm, so that the file descriptors will not be written to disk.8 W" B' n- @1 u! b/ g0 @5 W
    Systems that don't have /dev/shm will fall back to /tmp by default.
3 d5 Q# z. v2 a1 X6 }  </description>
1 S  }$ C* L! j7 ?1 W</property>' {' z: B; _7 a- F+ L" ~, c  `
<property>
" o% f( H$ Z! n3 W$ E  <name>dfs.short.circuit.shared.memory.watcher.interrupt.check.ms</name>
# ^/ e) n  N/ v2 i- v' I7 j  <value>60000</value>' ?+ j8 L% L  [( q7 r6 d7 \. Z
  <description>) W: Y! L( l9 s) L: ^
    The length of time in milliseconds that the short-circuit shared memory
5 d# L/ Y8 L) ]( r% q0 V- _    watcher will go between checking for java interruptions sent from other1 b  K, ?! }7 b1 v; h6 w0 i
    threads.  This is provided mainly for unit tests.9 \+ P* u" A3 K& a- M& S) p, Z
  </description>
' s3 e% _+ n" h' T</property>3 \  x; H% r& K8 F% t. ]
<property>' W; D! R8 I! d4 e/ w; a& X
  <name>dfs.namenode.kerberos.principal</name>1 P: X3 C+ p9 b) C& |' K, r& z
  <value></value>
( B% x3 D0 T4 A( o3 h! P8 |: B  <description>
' c. F, ~7 L6 o$ F$ D. w' @    The NameNode service principal. This is typically set to
4 x/ ^/ Z- _2 j7 x' I, C    nn/_HOST@REALM.TLD. Each NameNode will substitute _HOST with its! V+ z6 p# I0 r5 k  k# D
    own fully qualified hostname at startup. The _HOST placeholder6 J6 t, q  A2 b
    allows using the same configuration setting on both NameNodes
5 M; G, V" w, m9 @* Y, m    in an HA setup.
9 Q* C% U% P! O& {4 b  </description>" V8 J* k, J; A  ~  Z% n- |! k
</property>
) R. u, x2 c5 @5 x, A! E$ a<property>
, F9 z4 a% h) L0 `7 @6 _& g  <name>dfs.namenode.keytab.file</name>: L* Q+ h3 p: I& ]0 s4 {1 o
  <value></value>
' R, D3 e5 a# \+ h& q  <description>, q  e! ]" k; ]# D$ U2 O3 @8 t  W
    The keytab file used by each NameNode daemon to login as its/ e+ V+ P$ Z( G* n- e. j8 v$ j
    service principal. The principal name is configured with
' f% q7 D9 h: K) f  T    dfs.namenode.kerberos.principal.
. H# t6 g4 E7 _" O% H) Q$ Z1 G  </description>$ U4 q) [+ v) e+ }4 T5 S% P
</property>
# K8 ^8 ]4 r/ p+ M1 ?5 }7 B: A<property>3 ~4 F6 s7 z3 C2 B# r3 l3 f7 Z
  <name>dfs.datanode.kerberos.principal</name>5 }9 q) ?, e4 T/ V8 h7 D" D
  <value></value>
. c9 o7 Y6 t6 w( g; t  <description>; ?( u+ H; S$ p% E  v2 s- x* u
    The DataNode service principal. This is typically set to/ ~+ i: h' Z3 G' J9 H! v- n4 z
    dn/_HOST@REALM.TLD. Each DataNode will substitute _HOST with its
( f+ t! \! Y! H4 V    own fully qualified hostname at startup. The _HOST placeholder( X# u9 F2 a2 Q  G  q1 M- ~  B
    allows using the same configuration setting on all DataNodes.
# s+ A+ [  r! W9 }$ v, P0 r  </description>
, \; M* s) O: V' O' B</property>
* I% Q7 H9 E; a5 w8 i4 L<property>
- U- u: C) Y4 i/ l0 U6 |  <name>dfs.datanode.keytab.file</name>
/ `  c, n4 ^# F  <value></value>
$ T/ R+ D8 Y$ J0 p+ D  <description>
/ q& y' a9 ^: v* l! ~    The keytab file used by each DataNode daemon to login as its" a3 u" T4 b1 w0 V# U. D
    service principal. The principal name is configured with. I% W/ X4 n6 Q$ _. o
    dfs.datanode.kerberos.principal.% \  O: N0 Z. w! _0 i
  </description>
9 y# w+ @' [  l* b! i! ^</property>: P, H- |& J' J1 v4 w
<property>
' }! C; X+ W+ ?1 o  <name>dfs.journalnode.kerberos.principal</name>
2 J/ E9 O4 p0 u, ?. `  <value></value>
0 n$ B- y' `, z& b: C6 e  <description>- C0 a& r- Q7 s9 c
    The JournalNode service principal. This is typically set to
7 K/ B; ^/ ~) w. S    jn/_HOST@REALM.TLD. Each JournalNode will substitute _HOST with its
- H1 w+ N: o7 m; z    own fully qualified hostname at startup. The _HOST placeholder
  `( n+ P. O4 p! ^' [, N5 B& f    allows using the same configuration setting on all JournalNodes.
' }3 y3 Y, z8 [4 U' u  </description>
; ~' Y! T+ R0 P$ a3 _& k</property>
, l: U1 m, [6 C- ~2 z# {* h! h<property>
+ {, S" j/ ?9 p9 a4 m/ q: N  <name>dfs.journalnode.keytab.file</name>: ]6 ~3 M5 H$ N: k% W
  <value></value>
8 r/ P* p  Z; G( I9 r  <description>2 ~( a& _) |+ l3 a4 t- X
    The keytab file used by each JournalNode daemon to login as its
  i. N& g& @( e: N% a% q6 t+ s    service principal. The principal name is configured with# l( ~6 H: `, b  m! o5 [
    dfs.journalnode.kerberos.principal.3 g- P/ C( l6 F0 F
  </description>
$ H! k% @6 j5 X% j$ o( {</property>
  F! O+ Q" M" K2 E<property>% D9 ]$ I- ^: U5 t
  <name>dfs.namenode.kerberos.internal.spnego.principal</name>6 ?6 `1 u1 \9 D- \3 y9 O/ z
  <value>${dfs.web.authentication.kerberos.principal}</value>8 r; z" F7 W9 y' `# f, b
  <description>
$ @* E; k* Q+ G; G' M, a4 |    The server principal used by the NameNode for web UI SPNEGO
# q% Q* }/ p) j2 i1 Q% c3 G: m' L    authentication when Kerberos security is enabled. This is
( l0 h1 V+ F2 }0 m+ x% I    typically set to HTTP/_HOST@REALM.TLD The SPNEGO server principal
# |  \$ H. j4 a9 X% e    begins with the prefix HTTP/ by convention.- g( y1 K$ P' M6 v$ O3 N
    If the value is '*', the web server will attempt to login with
1 ?( Y: \$ {" l& w% N& Q' A    every principal specified in the keytab file. V  d! }( |& i% P
    dfs.web.authentication.kerberos.keytab.
7 k" Y( V$ `4 f, ?, }2 g$ b1 @) x</description>
( a0 Z3 K% J' n: Q) ]  Q' x  X1 h</property>
4 \! [" e/ U8 d+ v- V<property>
1 m" c  m+ h4 Q4 L6 p  <name>dfs.journalnode.kerberos.internal.spnego.principal</name>: O) h8 [; t. X# N2 \
  <value></value>9 p7 m" Z* s3 C3 k* G5 X
  <description>
: k2 u) O- k2 O% F/ V. a) y    The server principal used by the JournalNode HTTP Server for
, ^! ^4 `6 \6 `8 \0 Z& i0 ?    SPNEGO authentication when Kerberos security is enabled. This is
# P6 _; l0 s. c" O1 j4 e    typically set to HTTP/_HOST@REALM.TLD. The SPNEGO server principal
1 {7 a; `- q( L. \/ m    begins with the prefix HTTP/ by convention.
0 W) s. Z; I; `& D* l8 B    If the value is '*', the web server will attempt to login with' H  I8 |5 @! ^5 P5 z- |
    every principal specified in the keytab file
8 Z% x2 j6 T7 O) R7 K; s5 E5 k    dfs.web.authentication.kerberos.keytab.' |9 ?- l& a* E( a! G
    For most deployments this can be set to ${dfs.web.authentication.kerberos.principal}4 @( \- |- q1 v8 X- k
    i.e use the value of dfs.web.authentication.kerberos.principal.3 w; g' I* v$ E% g
  </description>4 E1 e7 o3 c! l
</property>) h% q3 ^& Y: k0 p1 ]( k
<property>+ i6 W$ A4 j. ]
  <name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>! \6 Q4 l0 f- X6 w
  <value>${dfs.web.authentication.kerberos.principal}</value>9 p+ a2 N/ E! f  n
  <description>' _- N6 f9 N0 N5 u
    The server principal used by the Secondary NameNode for web UI SPNEGO. K& d* C  s; L2 k
    authentication when Kerberos security is enabled. Like all other
, F7 `' X" ?. |; Q/ k% i9 B3 ]! w1 F! w    Secondary NameNode settings, it is ignored in an HA setup.
- \) w" Z& P2 \! i& r. p3 V    If the value is '*', the web server will attempt to login with
2 h( E0 d2 |' ^  g" Q, j    every principal specified in the keytab file
" g4 u; D# N& w( n9 Q- w/ q    dfs.web.authentication.kerberos.keytab.; d! g+ H1 h' v! f6 h5 l
  </description>7 J9 y: G) K1 {# w
</property>
8 t# J, a; _. g* p<property>2 U8 X% \( K, k! }
  <name>dfs.web.authentication.kerberos.principal</name>
) _- f; U* l# b, [$ {  <value></value>
/ {( Y1 `! L: R' [; R  <description>
9 z/ \! _* N: M& ~  S7 Z1 A    The server principal used by the NameNode for WebHDFS SPNEGO
5 c' c  C1 {2 H, |- y    authentication.
: }; z' m8 O8 R  m+ s    Required when WebHDFS and security are enabled. In most secure clusters this
/ ~7 l0 R# W6 O* M+ J3 q    setting is also used to specify the values for
! R) Q7 U/ W# O; k! V6 {- m2 Z) h    dfs.namenode.kerberos.internal.spnego.principal and
# o4 j3 p3 u# ]( u7 T    dfs.journalnode.kerberos.internal.spnego.principal.
. A% G5 R* P8 E) V  N  </description>, u. l' a, u' f- {: c" X+ [
</property>% X/ J6 [+ n9 C6 A& t
<property>! @8 y$ i$ M/ q3 i1 a: }: r" ~
  <name>dfs.web.authentication.kerberos.keytab</name>
3 [  x' A6 }8 I( ^; t* o3 X9 U  <value></value>
7 n2 }, P0 C4 [) Z. ~$ t$ y2 c  <description>
) q6 `, C1 L, J6 {; @9 y    The keytab file for the principal corresponding to
; u6 q6 D3 M- H: }/ t+ r% t  g    dfs.web.authentication.kerberos.principal.
0 N1 d! S% I: s$ _  </description>
. n' e: {8 ~4 ?3 p: l3 u</property>
$ r+ p) W2 K) v: P% F* T<property>
. H+ J2 n. A4 [; F  <name>dfs.namenode.kerberos.principal.pattern</name>
/ W4 p) g- _" N  <value>*</value>: L% c6 R. c1 Q
  <description>
+ E7 E: p) u7 j- }4 p    A client-side RegEx that can be configured to control( y! w/ |) t/ n- a9 R- V* |. k
    allowed realms to authenticate with (useful in cross-realm env.)
6 N  Z% Z+ I, g- p, s4 l5 @4 J. n; o7 F  </description>
8 j8 J" ^1 T  U5 s</property>2 Q( c/ H. i) F+ _) p
<property>
& ~1 @) i8 w" ^- ^; ?  <name>dfs.namenode.avoid.read.stale.datanode</name>& Y  V" p( y" Y3 I9 L: P& w+ R3 k
  <value>false</value>
, v. M7 c* n( O: X5 N  <description>
/ k# t6 F. ?# a/ L7 ]6 {    Indicate whether or not to avoid reading from "stale" datanodes whose
) P% y, v, ^! ^& D    heartbeat messages have not been received by the namenode   Z0 t4 i9 z7 ~8 R3 G0 n4 R
    for more than a specified time interval. Stale datanodes will be
3 ?7 I' j- G; \: M    moved to the end of the node list returned for reading. See  `  Q  ]" ]9 M% ^$ V& {
    dfs.namenode.avoid.write.stale.datanode for a similar setting for writes.1 T8 s2 a: ^; f1 \6 F
  </description>" u9 A2 M8 X: ~
</property>
7 Q5 j2 {0 A3 e" r* L1 j<property>
( S: F8 s% \( f  <name>dfs.namenode.avoid.write.stale.datanode</name>, j3 o" i' j! K+ Z3 V6 u" ]  T
  <value>false</value>
. }1 E  @$ G2 ~$ R- B8 H' I  <description>
) @1 ], K  F' Y0 ~" {    Indicate whether or not to avoid writing to "stale" datanodes whose
" N/ i3 L: _5 E* k8 {4 W# v0 \    heartbeat messages have not been received by the namenode ( I2 c6 R8 j+ J! i
    for more than a specified time interval. Writes will avoid using
9 L* f4 L  D/ ^# t    stale datanodes unless more than a configured ratio
. d' N. w' W8 G1 b9 _1 R" A* C' }    (dfs.namenode.write.stale.datanode.ratio) of datanodes are marked as 3 i# c5 e/ Q1 t$ o) l9 a
    stale. See dfs.namenode.avoid.read.stale.datanode for a similar setting
) V$ t2 y2 R, Y    for reads.
  t! j  b1 s# H$ V" g  </description>1 b& y' v0 F0 u* n4 |. [, e
</property>3 \& b& f3 \* u: }9 O$ ~% r) C# s
<property>' f, ~* }, ]3 e
  <name>dfs.namenode.stale.datanode.interval</name>
& u- ~, o! K+ ~$ E+ \( ~  <value>30000</value>
/ \1 M: D- ?" f- M2 a5 D  r  <description>5 n/ H" S0 x/ J
    Default time interval in milliseconds for marking a datanode as "stale",2 t( ]% v# J9 Q
    i.e., if the namenode has not received heartbeat msg from a datanode for5 t2 }3 E0 r. l6 j# y7 v% \* r
    more than this time interval, the datanode will be marked and treated
9 p% w, l/ n" @    as "stale" by default. The stale interval cannot be too small since   K0 F7 [1 m  J' A2 i4 J; \
    otherwise this may cause too frequent change of stale states. $ z2 @* ?9 v9 l0 Y* g+ T- w, @
    We thus set a minimum stale interval value (the default value is 3 times # E+ b; q! [3 ?8 d" {! M
    of heartbeat interval) and guarantee that the stale interval cannot be less
& A- [) U0 x) X    than the minimum value. A stale data node is avoided during lease/block+ ]; w4 n: X5 A+ t3 j. D
    recovery. It can be conditionally avoided for reads (see
4 k( F7 x/ j- y' A  k& s    dfs.namenode.avoid.read.stale.datanode) and for writes (see
2 S; c, P% }9 v# v  K) i    dfs.namenode.avoid.write.stale.datanode).0 c1 `  {, l+ Z& v; c+ I2 Y; l# ]
  </description>/ L! r1 T0 r, ^: B3 y0 e, x3 z
</property>
6 x2 P3 H: k' |1 }8 N<property>4 V8 l4 M% R8 l. `: Y* u
  <name>dfs.namenode.write.stale.datanode.ratio</name>- g# g( W+ d0 y6 [  L, I
  <value>0.5f</value>
& [( ^6 b: j! d7 A( B, ?6 V) u  <description>
, o5 E1 S( ?2 a9 ]8 A- _    When the ratio of number stale datanodes to total datanodes marked; o1 V* o3 O" g- x4 @1 S& R3 H3 T
    is greater than this ratio, stop avoiding writing to stale nodes so3 C3 _$ T( }3 ^
    as to prevent causing hotspots.! V- t) M* Y  Q( W8 S
  </description>
9 J0 I% D, E- y4 r</property>
" K- P1 `8 `+ B/ g$ ~: ~6 F3 @<property>
8 `$ F  F' R5 y  <name>dfs.namenode.invalidate.work.pct.per.iteration</name>
0 N- Y$ Z* X. x0 n* }3 V  <value>0.32f</value>
/ L) V; F9 Y& G. r4 z% j3 V  <description>
9 D! ^5 B' b: c$ H4 n# N    *Note*: Advanced property. Change with caution.: ^& ]7 u; C2 {3 |
    This determines the percentage amount of block
! |$ a! [$ j% F! D& u/ D4 j    invalidations (deletes) to do over a single DN heartbeat
9 s" f' Q" [+ L8 u- j  ~$ a    deletion command. The final deletion count is determined by applying this% r" Y4 A2 N7 s, T+ ~, h
    percentage to the number of live nodes in the system.
  |, C( x6 d3 w! [! o$ W    The resultant number is the number of blocks from the deletion list/ g- O2 H  R- p3 D0 F7 t* N
    chosen for proper invalidation over a single heartbeat of a single DN.( ~. D1 u9 a6 A& ~& m0 p2 H' I
    Value should be a positive, non-zero percentage in float notation (X.Yf),
# d, @/ @2 x9 r* |0 t4 c6 k    with 1.0f meaning 100%.
( p' o6 T1 [( r: w9 T3 e5 i  </description>
. I5 c2 Y8 y! I8 C* {% ?</property>. F6 [9 O* i" x& ]) P: m$ x
<property>
) F- P+ u, O# q8 r7 f  <name>dfs.namenode.replication.work.multiplier.per.iteration</name>
2 [: ^, I$ g; N3 z1 h  <value>2</value>
9 E( M. y( D$ g- Y  <description>+ \& o3 {( X# i* r2 ?7 \9 Q& k0 d
    *Note*: Advanced property. Change with caution.
8 o' v( P% ?" D% T, p; o) O! ~    This determines the total amount of block transfers to begin in7 j3 |/ ~" w; a, ~* p5 ~8 F
    parallel at a DN, for replication, when such a command list is being6 a/ J% h- X8 O- o3 R
    sent over a DN heartbeat by the NN. The actual number is obtained by
9 K1 b/ P, s. S% g8 t    multiplying this multiplier with the total number of live nodes in the/ ]& ]( R* \3 o" N/ n" Y) x+ W* B5 t
    cluster. The result number is the number of blocks to begin transfers
! T5 O7 g2 O$ m0 N' G$ U" q7 }; N    immediately for, per DN heartbeat. This number can be any positive,  i. ^* U) \7 @6 K
    non-zero integer.
4 p5 F5 |+ a9 v) X8 a( o$ }% V0 Z  </description>
2 e+ I2 P4 z! F* `7 V; G</property>- V( V7 T% v! \, z; v( C
<property>: k  `6 }2 Q; i/ ~: O
  <name>nfs.server.port</name>
& w4 Z" l" s1 }1 F4 l( k  <value>2049</value>
6 J! {8 E* a" R  <description>; u0 y  D/ Z+ C% T& Y  R5 Y
      Specify the port number used by Hadoop NFS.6 Z" t7 Z, o3 @7 u
  </description>- u2 r1 l7 {( U+ ^( G
</property>
6 @$ T1 \4 u  e+ O<property>
; D: ~7 R6 c7 f3 Y0 j* R8 F  <name>nfs.mountd.port</name>* z; F( ~. h2 v* L: h2 s3 C
  <value>4242</value>
" Z6 C5 D9 n) ]  <description>
( L8 e) N6 d/ ]# O      Specify the port number used by Hadoop mount daemon.
3 ^0 t7 o7 @0 Q! F  </description># J) E3 A0 ~8 \3 c
</property>( o7 X5 F: [6 P2 R7 i0 w
<property>   
' y  C6 J/ X' n  <name>nfs.dump.dir</name>
2 p, X2 M% U$ `. L  <value>/tmp/.hdfs-nfs</value>
: b2 e9 d! c; V6 l" ?0 g% Y1 c  <description>6 c! W1 Z4 @$ V3 `0 B( r- |: E
    This directory is used to temporarily save out-of-order writes before2 j% r' a8 `9 Z5 l* F; f7 b' H
    writing to HDFS. For each file, the out-of-order writes are dumped after
9 @2 R& ^4 B' k' A    they are accumulated to exceed certain threshold (e.g., 1MB) in memory.
% R1 k( l6 h$ Y5 d    One needs to make sure the directory has enough space.3 n- B+ o6 c3 E, }
  </description>+ _) A' F; M7 u
</property>
* C# K1 G  w/ a' t<property>3 o, h$ B/ o0 }  P0 I
  <name>nfs.rtmax</name>
5 d4 o1 q7 ^9 C3 {# b3 {! T% t  <value>1048576</value>
! W4 r7 A8 F" {: J) a$ j+ H  <description>This is the maximum size in bytes of a READ request
4 l% ?! j+ M' {    supported by the NFS gateway. If you change this, make sure you
: E7 \. T( B7 g: l3 {    also update the nfs mount's rsize(add rsize= # of bytes to the
( s; e5 ~2 p% A# O/ m7 K# S6 _    mount directive).; r# S$ q9 O3 D+ l# C  U+ j
  </description>
; y5 D: y& U, H- C</property>
* I; J4 `5 Q/ V% L0 m% C<property>- |/ ^6 X9 H8 B5 T
  <name>nfs.wtmax</name>
6 L6 _7 q: T& f% t: o7 E1 h  <value>1048576</value>' C4 E' A, u  L! e
  <description>This is the maximum size in bytes of a WRITE request
6 S9 p5 z; D; }2 i# o7 m    supported by the NFS gateway. If you change this, make sure you. I9 Q0 L. _: _
    also update the nfs mount's wsize(add wsize= # of bytes to the : ]4 Y$ M* o- {0 e, k, e
    mount directive).
' l0 |& a. [  t1 L, c3 N0 ]/ h  </description>
: x( R! q9 k/ W! A0 p  H</property>7 [* Z. m1 b7 g' P! {2 ^
<property>
! o) _3 C0 D. _9 {8 A  <name>nfs.keytab.file</name>
7 ~9 ~# |9 H- n  <value></value>0 m) X! t% ?4 t
  <description>, b" E3 x' \  \% y  C& t  |
    *Note*: Advanced property. Change with caution.3 R8 k" b. s* T, V
    This is the path to the keytab file for the hdfs-nfs gateway.
+ z; g6 t0 {, J! }& ?$ R0 g    This is required when the cluster is kerberized.
/ q- L% H6 [/ v4 J- a% K0 Y: o  </description>
5 ^2 ?* M* w; m7 c8 v* E</property>
1 @* a, y$ y( @. r<property>, K' ~  U' v& Z) B/ e
  <name>nfs.kerberos.principal</name>, d8 K. a& K4 i% D
  <value></value>$ @5 j; |, T$ T, ~
  <description>: Q& t5 h" o. z5 q" y2 i
    *Note*: Advanced property. Change with caution.
' V, t2 M4 m3 n    This is the name of the kerberos principal. This is required when
3 D, K# B$ m( j0 z; C5 ~    the cluster is kerberized.It must be of this format:7 I" x7 }6 Y5 C  r& b. G  ^( [
    nfs-gateway-user/nfs-gateway-host@kerberos-realm! z/ k, Y7 w. A- q
  </description>- I" d- z0 ^* c$ V4 e3 v- ^
</property>. D1 ]& e5 {  t3 r$ o' J! f0 u$ a
<property>
: c3 m- A5 |2 q. ]2 b+ y  <name>nfs.allow.insecure.ports</name>
. Y" \1 e4 ^0 K/ D- g  X  <value>true</value>% e+ B% o- f) U4 J! J# f
  <description>
3 K' e" ~  I% `4 v5 u+ D    When set to false, client connections originating from unprivileged ports
: F1 E" }% q. t8 ~$ D8 z" N    (those above 1023) will be rejected. This is to ensure that clients) p2 {4 Y6 t3 v' p+ j6 K6 N( o% J
    connecting to this NFS Gateway must have had root privilege on the machine2 l7 g$ T8 u6 y2 v4 L+ f2 ^
    where they're connecting from.$ h5 _6 P5 [- @8 C% v
  </description>
- K% A4 B' ]2 N8 n( i0 H' ]9 Y</property>" D5 ]! r' @( b+ ?: F6 k/ U/ F
<property>2 {# x) T+ {6 ?. y
  <name>hadoop.fuse.connection.timeout</name>, y& ~* Z# p7 f6 S1 ]' E  t
  <value>300</value>7 K% }3 H+ {0 W# i* a0 H
  <description>
4 x. m# |9 V+ |8 U3 w    The minimum number of seconds that we'll cache libhdfs connection objects! {, m! r0 {9 Y( R. I# Q
    in fuse_dfs. Lower values will result in lower memory consumption; higher& Z* s; }) |& `% `/ ~! v
    values may speed up access by avoiding the overhead of creating new
% R1 @1 q% c8 `! D; M# `    connection objects.
0 m% H) H2 O- Z; V9 S  </description>7 f7 J1 I" A) @% I1 [  G8 a2 [( D1 X
</property>; B* M( q, k1 a+ g/ B: v, R; [9 K1 Y
<property>! w& C4 f! t/ C& J# r& ^  O
  <name>hadoop.fuse.timer.period</name>
" q9 M5 ~/ J7 ]9 l5 C1 O  <value>5</value>
0 a# }- l7 p% }! E, K+ C* a+ Z) N9 l  <description>" r8 h  |5 q5 X" ?# I$ Q  c3 m
    The number of seconds between cache expiry checks in fuse_dfs. Lower values  X; G9 u% [0 ]- k) j
    will result in fuse_dfs noticing changes to Kerberos ticket caches more9 I4 ?. D8 [) n4 k
    quickly.
. L& f* g$ j1 A  q: r  |$ |# v  </description>0 P# H& q/ S9 I* [1 P5 D. z
</property>
9 M+ m" \6 k. P: `! ^; m: h<property>
. ]8 V2 n0 v& I! A' }  <name>dfs.namenode.metrics.logger.period.seconds</name>
6 O" E0 h) }: F  N. T1 [5 M* f- h" q  <value>600</value>
& e+ {' D+ U$ |% P# N  <description>' [; W- ~! Y% O
    This setting controls how frequently the NameNode logs its metrics. The; V2 v/ z3 R5 @$ @' C: D' A3 N
    logging configuration must also define one or more appenders for0 }( L2 B* E/ u# @  E
    NameNodeMetricsLog for the metrics to be logged.' X8 r8 [0 x: {7 y" D3 L! h% ^" j
    NameNode metrics logging is disabled if this value is set to zero or: A( v& r8 }" Y) V+ M+ ?5 j
    less than zero.
* M3 t% t# f$ p& t( m7 V  </description>
4 E/ H2 H( o. d2 a</property>, W: q2 g9 E+ F5 r7 N, |
<property>
) P% M$ |+ T. b3 z: C3 S; t( Z  <name>dfs.datanode.metrics.logger.period.seconds</name>7 j6 x8 [; f1 @" @, O
  <value>600</value>0 a+ S: B0 R% T; Z3 K. j/ W" ^
  <description>
' R; L2 ?; t4 x    This setting controls how frequently the DataNode logs its metrics. The
  ~( p+ p" T  h7 L& I    logging configuration must also define one or more appenders for
4 R4 P. }# N/ o' @' K! B* Y5 F    DataNodeMetricsLog for the metrics to be logged.: ~" t& Q1 U% k4 `; p$ H
    DataNode metrics logging is disabled if this value is set to zero or
' j! ?* I8 Y1 b8 [  B    less than zero.
  E4 v1 l/ B9 O  </description>
  F+ k0 i; X4 K1 Z; Q9 h# D</property>% D* l5 o, y, z8 f4 h
<property>& W, A# s$ I0 E' {
  <name>dfs.metrics.percentiles.intervals</name>- K- k2 Z, z4 R
  <value></value>
  l3 V) m+ e/ a3 n0 P  <description>3 O2 R7 k5 t% m, _6 I( g9 P6 ?
    Comma-delimited set of integers denoting the desired rollover intervals 6 I2 E) }5 t( v
    (in seconds) for percentile latency metrics on the Namenode and Datanode.
5 q" Q  D# I; t+ h( [    By default, percentile latency metrics are disabled.
) [# Z4 f* A4 M2 c* V  </description>
. ^7 k7 a$ Y: b7 N$ |% \</property>
5 A$ \7 Y. J, a2 o5 o' b% _<property>' T& G7 K! e7 S( ~
  <name>dfs.datanode.peer.stats.enabled</name>1 ?% G5 Y4 S; F0 m
  <value>false</value>" a  G( C8 }9 X3 g9 I5 s! L
  <description>) a1 R' V8 T; ]4 S( O
    A switch to turn on/off tracking DataNode peer statistics.
3 I; x5 w( @' }3 [  </description>
& j; P; u8 j. M9 b; s  C2 o: ]</property>7 A" C. x( K" ?+ m) Q
<property>
- ^- o. L  n; ]) [- Y5 f/ u  <name>dfs.datanode.outliers.report.interval</name>5 P9 c6 E6 V# Q# Z/ F
  <value>30m</value>
$ }. N  n0 m- ]1 J5 I' J  <description>
6 A. [$ _* @% I9 t    This setting controls how frequently DataNodes will report their peer4 K9 M: O5 `4 u) G% E6 B# K
    latencies to the NameNode via heartbeats.  This setting supports) d: F/ f  b5 k! r
    multiple time unit suffixes as described in dfs.heartbeat.interval.
3 N9 @) A( p5 y! b    If no suffix is specified then milliseconds is assumed.
( H+ S( N5 f" l! Q; j' R: ^    It is ignored if dfs.datanode.peer.stats.enabled is false.
7 Q/ S. Z- ^3 E/ O  U: K  </description>" P" M$ E% W3 `7 x# O9 G
</property>
5 L; i9 {# s) h) ~5 T+ T( g<property>5 G8 E) ^* p+ u4 S2 D! d
  <name>dfs.datanode.fileio.profiling.sampling.percentage</name>. W6 @7 l  s1 c5 Q
  <value>0</value>
( t: N' c5 v. G+ c7 c! {% E8 G  <description>
" F2 n$ e5 K' ^* V$ z; @    This setting controls the percentage of file I/O events which will be- s" k4 \' X* _" ~2 b
    profiled for DataNode disk statistics. The default value of 0 disables
9 d* r+ {- M' O' E: @# Q    disk statistics. Set to an integer value between 1 and 100 to enable disk+ b* a6 Q- \1 Y- [2 Q" _; N0 o& {$ u
    statistics.
+ _  Y8 J- t( |  </description>
. L' ]2 A6 p, h% L% j# m</property>
+ }2 Y; s8 a2 a7 |<property>4 [/ c9 `- j+ {3 c( w: [6 H
  <name>hadoop.user.group.metrics.percentiles.intervals</name># ?0 v# {; j) F3 F. H
  <value></value>6 o2 z0 a( F( }* Q4 V
  <description>8 j# U  o1 U: W, f& [9 b  U6 M
    A comma-separated list of the granularity in seconds for the metrics
( N; D5 B$ s9 M) ^    which describe the 50/75/90/95/99th percentile latency for group resolution4 t% P4 ~0 K- c" v1 ]" t
    in milliseconds.7 x# t& j- Y4 _3 \! l
    By default, percentile latency metrics are disabled.
7 L) e/ F- \9 f) o: m$ ~  </description>1 p5 Q) ]6 [. |- ]2 a
</property>/ v) Y- c+ S/ _- t
<property>8 _8 m7 H) S8 l: n8 C' K4 H2 n2 }$ ^
  <name>dfs.encrypt.data.transfer</name># Z1 o* J$ [! O1 N. c
  <value>false</value>
. R4 g  v4 Z; Y+ p  <description>
% W  P; Z0 |- k/ o) T/ x    Whether or not actual block data that is read/written from/to HDFS should
" K0 t3 S3 c- R    be encrypted on the wire. This only needs to be set on the NN and DNs,& R: v$ d" G4 E; ^4 ^6 g
    clients will deduce this automatically. It is possible to override this setting
% T! _. S) L( o4 J! G% s$ m- b    per connection by specifying custom logic via dfs.trustedchannel.resolver.class. " D) r1 k3 |- @6 ^3 y0 ]
  </description>) f: V3 Q! v( v" C' F/ R
</property>+ q. b$ |( R3 Q" }
<property>
( H! R& ]; t3 x/ J& F  <name>dfs.encrypt.data.transfer.algorithm</name>
3 S5 k; S, b1 h' f$ s0 W, V! Z9 M  <value></value>5 s; M# W2 c! \& I& ?+ [) V
  <description>
2 E  C+ t% |$ K+ m- F1 {    This value may be set to either "3des" or "rc4". If nothing is set, then( B+ w6 }( @! A( e
    the configured JCE default on the system is used (usually 3DES.) It is
4 }2 v5 f, M4 F' t4 \    widely believed that 3DES is more cryptographically secure, but RC4 is
' |$ ?, _, o& u5 }! g8 \    substantially faster.5 d$ _# B0 r, Z1 I0 T- r
    Note that if AES is supported by both the client and server then this
: Z- z8 l2 @0 u  C    encryption algorithm will only be used to initially transfer keys for AES.+ b$ P4 W% H, Q$ W
    (See dfs.encrypt.data.transfer.cipher.suites.)$ v' T9 a' ~5 o5 ?, R- S. g
  </description>
/ b3 R% A4 G2 d  s" |  w</property>
5 ?% K& G$ I, j3 A# H! V0 K0 i<property>
# r: v! K" I  O0 q& H) B$ z8 N# H  <name>dfs.encrypt.data.transfer.cipher.suites</name>6 C- V7 E& Y. q* _+ S2 z6 \% ?; S
  <value></value>
6 @% X- J2 w: {5 e0 t  <description>
# t+ n/ F& q( ~3 e& D- Z& n6 a    This value may be either undefined or AES/CTR/NoPadding.  If defined, then" {# Z9 J  o9 f" p! ?$ p
    dfs.encrypt.data.transfer uses the specified cipher suite for data
1 i4 J% @$ N7 w' A& [9 y5 v% J    encryption.  If not defined, then only the algorithm specified in
8 a6 n: l1 Z# t. F8 T0 m  h+ |    dfs.encrypt.data.transfer.algorithm is used.  By default, the property is
# j1 K; i8 V4 M& \( z    not defined.
5 Y! B" g8 ^8 G4 o+ F  </description>2 I/ v; V5 E5 L0 h6 w+ F
</property>
$ H3 Y0 l9 b+ ]5 v8 Y3 y<property>
0 l, c" C/ X6 j  <name>dfs.encrypt.data.transfer.cipher.key.bitlength</name>& {0 k+ X9 U4 z1 q( i
  <value>128</value>
; e* i) C5 K/ a" m  <description>& Q0 g7 r6 Z; x5 I9 Q) M0 s
    The key bitlength negotiated by dfsclient and datanode for encryption.
1 l) R6 D+ |; ~$ O. J5 K    This value may be set to either 128, 192 or 256.
4 j' l1 F* x/ a( e% `5 ]" g  </description>5 y/ q" K) \# h" I: s8 d
</property>
- b; N9 G& g& u  _9 y, T8 u9 [<property>
) e% w: O" [6 O  <name>dfs.trustedchannel.resolver.class</name>
- n/ y' s% R7 |1 d  o- N  R  <value></value>
7 b1 q: B3 P4 v  Z  C2 o  <description>& C; K) H) Z3 x, u  X7 l
      TrustedChannelResolver is used to determine whether a channel
% C0 H$ F  A; d( A5 Y- x! ^      is trusted for plain data transfer. The TrustedChannelResolver is
7 ]7 K$ B+ Z+ d# J      invoked on both client and server side. If the resolver indicates 6 l# E5 r* o1 S" S
      that the channel is trusted, then the data transfer will not be
# v/ Q+ d+ L# V  Y1 s      encrypted even if dfs.encrypt.data.transfer is set to true. The' W  }: @3 E* t
      default implementation returns false indicating that the channel ; @1 w( |" C* ~
      is not trusted.# p" z4 l9 W  ~- s/ O- _
  </description>
4 @  {2 j( a' R3 E, z! |6 k</property>
& e+ Z& h) e% Q1 e9 ~2 @5 g<property>
0 R/ P: l+ }' @! B  <name>dfs.data.transfer.protection</name>5 f3 H" T7 E0 m/ M1 a9 ^7 h0 ?! B
  <value></value>
$ u1 v: U' V" y2 e/ X( k/ b) q: H) X  <description>5 k5 w# t9 O; a
    A comma-separated list of SASL protection values used for secured2 u# f# E4 p5 J$ m' `5 f
    connections to the DataNode when reading or writing block data.  Possible9 g7 h* ^& H6 H( f( _3 x6 w
    values are authentication, integrity and privacy.  authentication means
* i# u8 [4 Q8 l: W    authentication only and no integrity or privacy; integrity implies& v& t  p, G. \$ a8 D) A: J
    authentication and integrity are enabled; and privacy implies all of
" F6 _! z$ D8 @2 p* B; T. M# Q    authentication, integrity and privacy are enabled.  If/ r0 ?8 _1 j9 z1 y% f# v5 z
    dfs.encrypt.data.transfer is set to true, then it supersedes the setting for7 ~0 F4 Q8 B- _# ?
    dfs.data.transfer.protection and enforces that all connections must use a( \/ O: j& Z3 p! Y, l3 _0 d
    specialized encrypted SASL handshake.  This property is ignored for  [8 ]- |7 [! p% u, `8 i
    connections to a DataNode listening on a privileged port.  In this case, it8 u6 z, j! @# H
    is assumed that the use of a privileged port establishes sufficient trust.1 B" m1 G, ]* D2 w
  </description>" W2 }$ ]3 X* u0 |* Q+ s/ E1 n
</property>
5 C) C% Z. H; h5 ~; I5 h- q<property>
. \5 u* |' x# W+ p+ B' h2 s  <name>dfs.data.transfer.saslproperties.resolver.class</name>
7 }' C% r! o3 A0 [7 @  <value></value>+ _$ C( ]# }- c0 u" N: r5 Q" w
  <description>7 Z% \! C  o0 p. \: J
    SaslPropertiesResolver used to resolve the QOP used for a connection to the' W7 T, `+ h0 x7 y
    DataNode when reading or writing block data. If not specified, the value of2 T( ]) f/ J7 @: X* z( C/ U3 c
    hadoop.security.saslproperties.resolver.class is used as the default value.
$ p" {, ^0 R8 f; T  </description>
1 _& z+ d% u; C/ I$ \" [</property>6 S0 q0 ^3 y. }5 o3 p
<property>
- L! s- F$ h1 p/ c. g. q* T  <name>dfs.journalnode.rpc-address</name>- \# T! M# j5 o  M
  <value>0.0.0.0:8485</value>
3 ]! G! i+ x+ Q6 D$ E! \8 p8 {5 ?  M  <description>
9 g0 C) }2 Y) j+ o" O5 S/ a8 h* }    The JournalNode RPC server address and port.3 }( H) W; n) f. Q' Y
  </description>5 p- x$ i& N: a# h% t
</property>& v4 @4 w, a: `8 k* D3 U  a
<property>5 U0 l* \+ {2 \6 _+ J( M0 A
  <name>dfs.journalnode.rpc-bind-host</name>
$ b* d+ j8 h3 J* n  <value></value>
( N* k/ M: q) F, I2 w  <description>. m# g. n; L5 p: e% ^; `
    The actual address the RPC server will bind to. If this optional address is, D+ v5 S- w( f8 E, @
    set, it overrides only the hostname portion of dfs.journalnode.rpc-address.
- X( R: K# f& t" u    This is useful for making the JournalNode listen on all interfaces by
3 a8 w/ @7 {' w, |6 P+ f    setting it to 0.0.0.0.9 ~( }4 o/ \3 ?! b8 k  {6 j0 E
  </description>$ h" j$ m% a. L* v; v& n% @: B) B
</property>
' V% Y. J2 Z$ B2 q<property>
% v- q& p5 x9 d2 ?/ M  <name>dfs.journalnode.http-address</name>
1 s9 ~5 s6 m8 B3 Q1 b" E' d  <value>0.0.0.0:8480</value>2 D* ?* |) \0 k  T0 q; Y& F& h
  <description>( c/ e. E% Y% P
    The address and port the JournalNode HTTP server listens on.: M& Q- H- p4 y, L2 u
    If the port is 0 then the server will start on a free port.
/ X  z3 H6 C0 g. G# F6 f  </description>0 e8 k/ B2 K' Q2 [$ t/ t2 T. P. G2 i
</property>) a& F2 t7 h# ~; k0 m4 }2 {
<property>1 C5 T  _3 u% X  b9 V9 T9 M
  <name>dfs.journalnode.http-bind-host</name>/ n, f/ A' P1 g) |# v
  <value></value>! v# z4 X* e' q' r  y% ~' D
  <description>
$ y4 C1 P3 \8 O4 }    The actual address the HTTP server will bind to. If this optional address
$ D/ l# z- w! r; N    is set, it overrides only the hostname portion of# J$ p4 O5 F/ q0 D: e5 L0 G  F0 A/ f4 k
    dfs.journalnode.http-address. This is useful for making the JournalNode0 c" y4 O4 n4 O- o# G+ b# g8 p$ P/ {
    HTTP server listen on allinterfaces by setting it to 0.0.0.0.
& ^! p- I, T5 j9 M  J  </description>
' O9 |3 v" r( @+ i</property>, ?# ]1 ~* f2 ]$ u
<property>
7 H* x9 S& [* V" d  <name>dfs.journalnode.https-address</name>/ m" s; H0 }2 m
  <value>0.0.0.0:8481</value>
$ B+ R" h/ w8 K) Z+ e% C  <description>
3 P) Y1 X- B, U    The address and port the JournalNode HTTPS server listens on.  O& a. }% `6 o: Z  f8 L9 Z
    If the port is 0 then the server will start on a free port.
7 X' o6 t2 k6 o2 B  </description>
1 ], E+ c) C2 `6 M! s" ?* r  j7 D3 w</property>8 I/ C, F2 B9 s" F& }+ R
<property>" M. {% ^% X: S) E( L, ?" n( M$ N
  <name>dfs.journalnode.https-bind-host</name>
6 t& i) _9 D9 A5 A2 b  <value></value>5 A: r/ m7 I7 h" s9 ^: c. c7 \9 Z
  <description>! p! m9 ^; q1 [* J) E9 a
    The actual address the HTTP server will bind to. If this optional address
# H" M9 }7 J4 v9 z! ~8 ~( L    is set, it overrides only the hostname portion of
) d7 A  r; g7 i+ h    dfs.journalnode.https-address. This is useful for making the JournalNode/ M* z2 A. h& l5 @
    HTTP server listen on all interfaces by setting it to 0.0.0.0.& g+ ?' I: I* t. U; z& a7 l* W
  </description>: }: I4 _1 _. T0 I8 }% Q8 {% G
</property>6 K; R" h( ^- B/ R# t. ^3 q' v# z
<property>
/ o* k! |6 y) ?  <name>dfs.namenode.audit.loggers</name>
. T: s% Y1 E% @# j: N6 `6 M1 b  <value>default</value>
1 `3 L8 Q* j9 m0 N) ~9 r' d  <description>$ \" G, X/ q$ Y; b! c; ]
    List of classes implementing audit loggers that will receive audit events.+ n! M9 c* |( {5 G6 Z6 x# {$ ^
    These should be implementations of org.apache.hadoop.hdfs.server.namenode.AuditLogger.
- W6 p$ W0 c: m& F- Y8 k* X- f    The special value "default" can be used to reference the default audit; T$ z/ }0 D7 h& p9 Z
    logger, which uses the configured log system. Installing custom audit loggers1 v  M6 t# _0 }! Y
    may affect the performance and stability of the NameNode. Refer to the custom8 l/ T9 I  D! |9 P+ m% q
    logger's documentation for more details.
/ m* W% m* H6 _) Z4 q* m# r  </description>
! \3 v$ Z% F" x. W; z. T* t</property>
2 u* ]: `& {5 i7 z<property>9 [# [1 m0 x& `9 m
  <name>dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold</name>
! G+ |# B) F/ |! G# x1 R  <value>10737418240</value> <!-- 10 GB -->, R- P* A) }7 @. C7 M
  <description>
' z2 v8 M( w) `; F5 ?    Only used when the dfs.datanode.fsdataset.volume.choosing.policy is set to, Z3 D' D4 \% p+ [
    org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy.( O. H% `1 o9 h8 ]" p' y$ F- ^, K
    This setting controls how much DN volumes are allowed to differ in terms of
7 B/ J, h9 e, l; _2 ]: ~! B    bytes of free disk space before they are considered imbalanced. If the free5 X7 f! w8 O& {( X# {% g
    space of all the volumes are within this range of each other, the volumes
: b' [! j# k' a0 }2 o, n& A4 x9 O    will be considered balanced and block assignments will be done on a pure
9 e. f0 x/ Z4 ]    round robin basis.
  P! c! Z! \$ h6 D; d  </description>
! ], w3 y" H& U$ _0 @( M! t$ `</property>" C6 S8 G+ T4 ]$ J) |0 S
<property>
1 I# J& W% F6 R' k  <name>dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction</name>
" {: g5 h" h- \2 Q, k; u  <value>0.75f</value>! X8 X9 {4 w4 ]$ @- B9 q* U
  <description>  p- ~% D& h) ~# ?+ Q
    Only used when the dfs.datanode.fsdataset.volume.choosing.policy is set to
; H4 k' O+ |' P  N# P3 s5 o    org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy.
5 D7 b# I* ]7 ~/ q5 L# I; K    This setting controls what percentage of new block allocations will be sent
/ P' t3 m# w+ a0 W4 D    to volumes with more available disk space than others. This setting should
! T3 S7 H% R5 V% e" u5 Z' n: X    be in the range 0.0 - 1.0, though in practice 0.5 - 1.0, since there should- r6 p& {1 t- D
    be no reason to prefer that volumes with less available disk space receive
1 \5 T. o: L! X* \    more block allocations.
8 A8 F; z* q) ^, h8 V  </description>
9 h1 f! _; k+ M% B* Z</property>
. G) P# i) \" B+ W5 X3 S<property>
2 n( O. u" `- g# f( v! t$ u  <name>dfs.namenode.edits.noeditlogchannelflush</name>
; K: y! f! L) l4 T. o  <value>false</value>  e/ {5 x4 R' d) l
  <description>
& t+ b$ t. ]* y0 q    Specifies whether to flush edit log file channel. When set, expensive
$ Q& N) V# a6 u# _) j/ N6 v+ P3 e    FileChannel#force calls are skipped and synchronous disk writes are3 u- P. U8 G" ^5 f+ `7 J
    enabled instead by opening the edit log file with RandomAccessFile("rws")2 L, v  d  F) t! g4 {
    flags. This can significantly improve the performance of edit log writes6 H* E3 R/ Z, x9 W
    on the Windows platform.
7 B& v% J9 O: H+ P9 @    Note that the behavior of the "rws" flags is platform and hardware specific
& q# N/ }; ~% g) \    and might not provide the same level of guarantees as FileChannel#force.
5 O% x- M8 O/ T5 ]. X" \    For example, the write will skip the disk-cache on SAS and SCSI devices
) |6 h5 S9 A8 J    while it might not on SATA devices. This is an expert level setting,
, U5 C; p  x* M2 J    change with caution.
: I$ h* q/ w% I; l4 T  </description>5 u2 D, I: q$ ^1 l+ |
</property>
0 B0 i3 O$ t) p  Q  f<property>
& x1 l( y5 \; h; L8 M, b  <name>dfs.client.cache.drop.behind.writes</name>
# Z6 B1 @. ^' o  <value></value>- v7 ^9 H2 l% o. r0 T3 G$ C
  <description>/ V" F8 Q0 ?( O( ~2 a
    Just like dfs.datanode.drop.cache.behind.writes, this setting causes the
9 U" K6 w) T' k    page cache to be dropped behind HDFS writes, potentially freeing up more
  E! V0 ]8 u; \" h    memory for other uses.  Unlike dfs.datanode.drop.cache.behind.writes, this
% X5 ]% n' p# ?: ]( Q1 w    is a client-side setting rather than a setting for the entire datanode.% n; e7 m5 P* d( ?
    If present, this setting will override the DataNode default.
) k) k7 _# o8 b  I$ @' }5 e    If the native libraries are not available to the DataNode, this
6 l+ T( T0 ]$ {% P0 |    configuration has no effect.
0 m7 I" a" W2 H3 o' z  </description>
3 j6 X# a4 i+ h3 N8 w% u: _- x</property>
, ^8 n7 F' v) i9 S0 W' `& i0 b<property>
9 x0 A0 Z( m' X. |/ `; J' Z  <name>dfs.client.cache.drop.behind.reads</name>6 s: r/ X8 ]3 o% X# O
  <value></value>3 @, A" i: T0 y8 _7 f2 m
  <description>
2 ?  }' ]& ~: a/ i1 k    Just like dfs.datanode.drop.cache.behind.reads, this setting causes the
9 n: C9 G/ q/ j& }8 r- h    page cache to be dropped behind HDFS reads, potentially freeing up more
+ U. H/ A5 l% I. ?) P    memory for other uses.  Unlike dfs.datanode.drop.cache.behind.reads, this0 N! T* g9 Q/ V$ g! T9 J
    is a client-side setting rather than a setting for the entire datanode.  If
/ s- p5 R2 r3 W7 j) I" m9 m    present, this setting will override the DataNode default.
0 u$ P8 X9 S1 G! d3 L    If the native libraries are not available to the DataNode, this
9 r; [% k  [1 U    configuration has no effect.7 z# d7 i6 Z( e. t  R* {
  </description>
/ ^  x4 p% r# `1 S, B</property>2 F- Y- G- B+ f6 n& |) Z
<property>, w8 J2 K" ^& K; A8 x$ _
  <name>dfs.client.cache.readahead</name>
- M; l7 V, X$ ?4 ]# ~( [3 Y1 N- a  <value></value>) j: Y! a+ k4 h$ f9 U3 w- ^
  <description>  ~! f  b& z4 p. [8 C$ g6 I
    When using remote reads, this setting causes the datanode to, u: X" ~8 ]/ _* p9 o& v
    read ahead in the block file using posix_fadvise, potentially decreasing
3 @2 G+ |" c, r1 b6 n% E  c    I/O wait times.  Unlike dfs.datanode.readahead.bytes, this is a client-side
; w$ y$ y4 m9 p    setting rather than a setting for the entire datanode.  If present, this2 A. V! T4 P3 s/ k  v
    setting will override the DataNode default.
; z: B/ X0 W3 P* R. y2 a    When using local reads, this setting determines how much readahead we do in
; @4 t& I; {. Y# U/ h2 W    BlockReaderLocal.& F8 w: ]2 e% b3 k1 k6 g
    If the native libraries are not available to the DataNode, this& W" y5 e2 Z# b" w- z
    configuration has no effect.
% t7 m5 f* @( w! v6 f4 ^) U* e  </description>
. G9 v( o1 }% ]! v: Q, \: ?4 A% J6 k2 R</property>
& g- Q( @" [# K: B<property># u3 N+ ]! @% c
  <name>dfs.client.server-defaults.validity.period.ms</name>
5 w  v' Q$ F, [  <value>3600000</value>8 B' ]  l4 J# W' r' K% I
  <description>1 w2 i( ?0 t9 r2 I3 G7 d
    The amount of milliseconds after which cached server defaults are updated.
- o) @6 k' U2 c/ e    By default this parameter is set to 1 hour.' i( j; [/ j5 D8 R5 I! U  Q) ]
  </description>6 w" q3 G  j+ x* j# n% X6 ]) e
</property>
# K* V. ~) K, t/ b/ V<property>
) X6 C( L0 g$ y6 P% x! g  <name>dfs.namenode.enable.retrycache</name>9 X6 i9 K4 B& t# a9 h
  <value>true</value>/ T+ ?: c. t- i- V' ~( M
  <description>9 @3 d2 r$ @5 Y9 L! j- [% T2 W
    This enables the retry cache on the namenode. Namenode tracks for
1 t2 L2 b* D, H    non-idempotent requests the corresponding response. If a client retries the
2 B  p1 [9 p' a. s7 K    request, the response from the retry cache is sent. Such operations6 r9 q! v4 w2 e
    are tagged with annotation @AtMostOnce in namenode protocols. It is  u% \" C# ~- [7 u( u
    recommended that this flag be set to true. Setting it to false, will result% d  t! L( m- I6 ~
    in clients getting failure responses to retried request. This flag must
& n2 Y7 v0 X1 r' p. m# [0 ^    be enabled in HA setup for transparent fail-overs.
' e( B) M' c. K+ p! q    The entries in the cache have expiration time configurable
( ?4 Y$ n; T- }) l  e1 t/ h    using dfs.namenode.retrycache.expirytime.millis.. s& K9 s+ A, W- p# X3 {& \& w
  </description>
6 x# M6 V4 {, z% N# i& I* r, R+ p</property>
8 x, W; X! k( k6 \- u* `( ?# i: I<property>
7 a7 |9 p9 l3 G) k& p* v2 ?  <name>dfs.namenode.retrycache.expirytime.millis</name>
9 K8 g. Z! P1 t7 {7 d, c  <value>600000</value>3 A6 \. B7 I2 P" q+ e, m" Q/ M
  <description>
$ i; K3 X) u0 Y    The time for which retry cache entries are retained.
- l; g' B- b; ]5 i; F  </description>$ j& f3 _& U$ B1 v- _2 e
</property>
" f9 i4 u, u# f+ A% s% |<property>
1 K( _. f# F3 G/ ?9 e  <name>dfs.namenode.retrycache.heap.percent</name>
5 @9 [6 d& K$ O& l& k: Y( x( m1 n# M  <value>0.03f</value>
* [0 k* I3 V6 z4 [  <description>& s+ a  _  y: C' x8 ?* Y
    This parameter configures the heap size allocated for retry cache
# M+ u" S9 ~" K1 K9 j7 J/ B& i    (excluding the response cached). This corresponds to approximately
) B& {6 m( i7 S, g& m    4096 entries for every 64MB of namenode process java heap size.; M3 N  k% l, e; z& z+ V  G
    Assuming retry cache entry expiration time (configured using
7 j4 d: I( c4 K3 x. {- a    dfs.namenode.retrycache.expirytime.millis) of 10 minutes, this
0 ~: S/ x" f+ k, \- y$ A7 ~& S    enables retry cache to support 7 operations per second sustained' r( H9 _$ u; ^  J; K# B2 {- d% d
    for 10 minutes. As the heap size is increased, the operation rate4 x, m/ ?/ ?! j! |6 ?9 J
    linearly increases.
* B; ]. F/ I5 Q( o( a- ?  </description>
: w$ m+ }$ T/ T/ o</property>6 W0 ~/ @9 \* |+ S) c0 d) Q
<property>
2 Z( r8 [2 M6 z1 n  <name>dfs.client.mmap.enabled</name>
: l+ n0 }* S) R/ u/ g4 x' G0 G0 b  <value>true</value>3 }: D: h' p8 Z- y- T3 K" Q2 m
  <description>
8 Y) K0 s7 x9 N: r; w! ?# m    If this is set to false, the client won't attempt to perform memory-mapped reads.' J, ^4 c* z5 h& {( w4 N
  </description>
: R7 c: v" N- o) T; S" g8 p</property>2 P/ ]4 X. ]- T) [2 D" d
<property>
- g, f8 v6 r- f1 \7 i  <name>dfs.client.mmap.cache.size</name>
2 p, G6 y6 C" v7 k  <value>256</value>
* m$ w+ U) H. A7 C3 `  <description>
+ E0 b9 R- |" ]' R) o3 {    When zero-copy reads are used, the DFSClient keeps a cache of recently used
  u/ t  p, o8 t6 t    memory mapped regions.  This parameter controls the maximum number of
8 q" q# Q1 ~1 g$ O0 b& t" N    entries that we will keep in that cache.
5 v8 ^' _# }" h; V9 ?$ Y- L8 y2 O' {    The larger this number is, the more file descriptors we will potentially" P% w) Z5 K8 q$ ?3 n& T
    use for memory-mapped files.  mmaped files also use virtual address space.
3 d! R2 m: N: F4 q4 y# v# A: ]    You may need to increase your ulimit virtual address space limits before
0 I$ s. E) l- d; e5 A    increasing the client mmap cache size.
: P/ G7 W) Q8 \    Note that you can still do zero-copy reads when this size is set to 0.% W7 m( ]" p' \$ |
  </description>
/ p9 o# M- W7 I8 }7 v</property>$ J- w3 a  o, }$ a* R. q" N
<property>6 Y. Q9 D# o; G) {9 L4 Y4 w5 ~
  <name>dfs.client.mmap.cache.timeout.ms</name>" q4 i. o7 a' D" ]; b' I
  <value>3600000</value>& l2 O  N" I/ Q+ @7 L" C# M
  <description>
: y. q8 w% t- I: x9 I    The minimum length of time that we will keep an mmap entry in the cache5 h( F8 O9 w1 ]6 F( E% L
    between uses.  If an entry is in the cache longer than this, and nobody
; _  K- N) t! Y! t( F6 x: E: ^! _    uses it, it will be removed by a background thread.% {6 t* f6 `1 ]# }% m, `
  </description>
+ |4 I6 i! B2 I, P7 V* T</property>! m. z% {" l; a- Z. s
<property>4 K+ q+ p$ u0 a/ y0 D) ]
  <name>dfs.client.mmap.retry.timeout.ms</name>- F, ]+ e2 m) n& D
  <value>300000</value>* g4 _! k( j' _, i0 H& }
  <description>
1 K* T9 f$ ^9 v! h3 P$ d    The minimum amount of time that we will wait before retrying a failed mmap6 f! `( I' q. u2 V5 ?" e6 n5 r- ^
    operation.: L, v5 s1 a$ r4 W% f7 n( c
  </description>0 F) D; v& v3 O7 Z" M. B# q3 p
</property>
* P; \% _' S- L0 X<property>
% ?( w9 r" A% \/ P& `, F8 b  <name>dfs.client.short.circuit.replica.stale.threshold.ms</name>
6 ~6 K4 d0 e$ X4 l6 n  <value>1800000</value>  @. c/ t' Q8 h
  <description>; @( [$ D: U  X# ~* Y
    The maximum amount of time that we will consider a short-circuit replica to; y; R" \; O0 E0 R
    be valid, if there is no communication from the DataNode.  After this time8 k# T. k9 G2 r- U
    has elapsed, we will re-fetch the short-circuit replica even if it is in: q* I8 V8 L; ?
    the cache.6 A' s$ L" m# f, `2 s$ U! S6 T
  </description>
% `9 t" f" K; z3 R</property>, A, o" F4 x+ i7 E3 e1 u
<property>8 q* q& O1 X. j& j' V8 a$ k- h9 Q7 N
  <name>dfs.namenode.path.based.cache.block.map.allocation.percent</name>1 Y! c7 p% i; I' x
  <value>0.25</value>- J1 B* s2 {0 k) h1 i7 i  r
  <description>
# k5 t; N" {' Y# T    The percentage of the Java heap which we will allocate to the cached blocks
) P# }+ c' x5 E' U& j6 q( A, |    map.  The cached blocks map is a hash map which uses chained hashing.& N0 i1 ?+ g# p" a
    Smaller maps may be accessed more slowly if the number of cached blocks is* Q" ]5 w  K* z
    large; larger maps will consume more memory.
/ |9 S( H( n' N% V  </description>
/ |, j8 \/ h! t, ^, i</property>5 l( S/ ^- F, O* x
<property>
7 w! y9 z& x! g) x% y  <name>dfs.datanode.max.locked.memory</name>1 M5 |+ \& ~5 I. e
  <value>0</value>
! d7 d$ u. o& w9 ?4 P: b* E5 n  <description>
' Y. ?  ]7 [/ h& s: w    The amount of memory in bytes to use for caching of block replicas in
( L& G$ s; d/ ^2 ?1 g! d1 X; R    memory on the datanode. The datanode's maximum locked memory soft ulimit
2 p2 s. t! m3 e9 \) `6 I) ~" Y    (RLIMIT_MEMLOCK) must be set to at least this value, else the datanode/ ]. \+ ~7 I: y1 T+ ~
    will abort on startup.9 C0 n6 H" P# \" L( h
    By default, this parameter is set to 0, which disables in-memory caching.7 C1 f+ o% K. L, `+ W$ {1 Q
    If the native libraries are not available to the DataNode, this$ g! d4 q* C1 b% ~' K' S& c
    configuration has no effect.
( M9 G; ^8 U$ @0 C- Q9 j9 R. x  </description>' y! m. i& i* F/ i
</property>
7 \/ A/ y: d9 F<property>7 Z# w! l; S# m* a3 i  o
  <name>dfs.namenode.list.cache.directives.num.responses</name>
5 {2 n( }7 b$ X3 g  <value>100</value>9 a9 T6 v$ ^2 d% R6 _
  <description>
6 y: x9 w1 [1 r3 k( v    This value controls the number of cache directives that the NameNode will
0 k7 Z. {) ~/ W& [- D: ]; A    send over the wire in response to a listDirectives RPC.- t9 v8 H+ R" Q& S7 l
  </description>
3 s9 f3 p# C& \</property>
$ b3 i# T- v; f<property>
" m6 O* l& I8 P1 K) p3 u  <name>dfs.namenode.list.cache.pools.num.responses</name>
+ {: {. _$ E) N% _  <value>100</value>
) V9 ?% ?3 P! n' [  <description>7 l( s' N# R) H- R5 k8 N1 `
    This value controls the number of cache pools that the NameNode will
6 |8 s$ }" ^1 D: b    send over the wire in response to a listPools RPC.+ H; c5 B0 ^" M! U+ w
  </description>
  z, [1 ~! B2 T- E" S7 e</property>2 G1 u$ c0 J" Z3 s5 Y4 V# S
<property>4 q( d$ h, \, l! C: T
  <name>dfs.namenode.path.based.cache.refresh.interval.ms</name>
4 X0 `5 {" Q" B" j5 ]7 t5 f  <value>30000</value>) x! y9 k( |3 v  [1 x/ L
  <description>6 r$ F- w2 A6 x, Q8 n
    The amount of milliseconds between subsequent path cache rescans.  Path. S, K% X/ H7 c* A6 I4 l1 f
    cache rescans are when we calculate which blocks should be cached, and on
5 [8 v; s; F, U3 {; S/ J( H. `5 f    what datanodes.- Z; o1 `, ?) r: N- m* k. L
    By default, this parameter is set to 30 seconds.5 Z, @; }* T  b# N- `+ w+ P
  </description>" Z7 l" [* @" ?0 v* w& C
</property>' r0 `, k; D) V6 e/ {/ y
<property>
8 W% w$ f' |+ p6 Y0 d- F  <name>dfs.namenode.path.based.cache.retry.interval.ms</name>
- C* E1 w/ _/ g& {# t& i+ P+ z% B  <value>30000</value>. [; n/ Q- Q3 d2 u% \
  <description>
5 S1 C  e. |3 [$ D7 W    When the NameNode needs to uncache something that is cached, or cache
5 T( N! o/ H* h; b4 z3 M. c    something that is not cached, it must direct the DataNodes to do so by' G7 e4 Z+ P/ H; y
    sending a DNA_CACHE or DNA_UNCACHE command in response to a DataNode2 |; J$ B+ d  M1 t6 M( D
    heartbeat.  This parameter controls how frequently the NameNode will
. A% c+ V' e9 p, ^  V    resend these commands." h& D( w% G0 [" Y( l
  </description>
  i4 z1 d- C, s0 A6 z</property>
5 C) i! k" g$ h+ h7 R- J* ~3 T<property>
  w& A- Z1 Z( m% h- z5 ~! b  <name>dfs.datanode.fsdatasetcache.max.threads.per.volume</name>7 ~9 w3 d7 U! L! {( \. t% c4 `
  <value>4</value>
, Z- M( @) \6 e' B7 A! |  <description>
  I4 h# r7 I5 T1 c+ |; o  ~    The maximum number of threads per volume to use for caching new data6 k8 `/ U' }1 O6 v6 G
    on the datanode. These threads consume both I/O and CPU. This can affect
; [- @) w* \9 v+ L, J  F8 G    normal datanode operations.
) i; n6 E8 p; ?2 W/ D, z( }  </description>
) k. O# L% I/ x1 L. p, w8 ?2 x/ M</property>
; D) ]# S5 X  i<property>
4 [. Z2 S3 L( s# y; }6 U  <name>dfs.cachereport.intervalMsec</name>
1 M' Y+ |  Y$ X  {" T3 b) N  <value>10000</value>
7 b! T4 g1 p% n3 o" c  o" R  <description>, e) J7 }) Y" t7 C9 k6 o
    Determines cache reporting interval in milliseconds.  After this amount of( `# e2 x6 a# F  k* V/ p! w
    time, the DataNode sends a full report of its cache state to the NameNode.( O. w# U( K% P/ E" g7 b5 j8 X' J
    The NameNode uses the cache report to update its map of cached blocks to
4 x. h9 J/ q! m' @5 U* V8 C    DataNode locations.1 v2 A4 w) i5 X1 \8 Z
    This configuration has no effect if in-memory caching has been disabled by' j! G( a9 P$ q! v9 A7 V& A4 g
    setting dfs.datanode.max.locked.memory to 0 (which is the default).
" d2 F# d* u6 K: H" ~! A7 x    If the native libraries are not available to the DataNode, this
: r# b, _# Q- C    configuration has no effect.0 j* D: g; y. m- m1 z
  </description>2 i% }, ^4 R' K4 d  e" ~1 Q% M" d
</property>. O- n* F' y$ j& n
<property>
" o: _0 _& e2 I  @  <name>dfs.namenode.edit.log.autoroll.multiplier.threshold</name>
  o5 o/ Q! T; S0 _) y! C  <value>2.0</value>, Q- ^, l0 ]& S- _. p
  <description>2 i$ ~3 T8 M2 @& b5 P3 Z) `
    Determines when an active namenode will roll its own edit log.6 }: l6 C8 i0 v; C/ k( \
    The actual threshold (in number of edits) is determined by multiplying' w# Q" M- f6 o& p$ C+ `# W2 _
    this value by dfs.namenode.checkpoint.txns.
" p& ?  y  v" W  Q8 o2 U7 R8 F    This prevents extremely large edit files from accumulating on the active
: X/ T! r% [, A2 {3 ~    namenode, which can cause timeouts during namenode startup and pose an
& q9 J6 @, L' T  X' f    administrative hassle. This behavior is intended as a failsafe for when
& l/ r) q* u1 C' w8 m    the standby or secondary namenode fail to roll the edit log by the normal' O6 ?7 F9 x* t  W9 O3 S1 y
    checkpoint threshold.
$ q6 J) t: u* h" P" V  </description>
# a' J4 x; u; R5 _% @* h9 G</property>
, o( n8 M  |! x3 C<property>
) f( z% K) N  K0 e  <name>dfs.namenode.edit.log.autoroll.check.interval.ms</name>
% e; b  Y5 {7 O  <value>300000</value>
. G3 }) @. f: H% F  <description>
+ G( I! S. H# M9 H+ t- N    How often an active namenode will check if it needs to roll its edit log,5 n' a, Y& ^9 f  z! r
    in milliseconds.
6 A* N4 O! }, H# M" T  </description>1 D4 X5 y( c& G7 Z
</property>
7 o; b" J4 c, E% z$ e. ^<property>% e0 r+ u& A4 r& {( x/ V$ w0 f
  <name>dfs.webhdfs.user.provider.user.pattern</name>
; l# t+ c0 p6 r# N! }  <value>^[A-Za-z_][A-Za-z0-9._-]*[$]?$</value>" K8 U" |! r" N- [8 M# A
  <description>
# b. ]+ s' |3 K$ O7 [: a( f3 g    Valid pattern for user and group names for webhdfs, it must be a valid java regex.
6 X. i, j( m- B- f! B: I2 A) X2 q  </description>/ B6 W+ R: ?2 M3 N
</property>
( A1 d0 w3 @! J7 b<property>" o) [8 {; d8 _5 z/ ]
  <name>dfs.webhdfs.acl.provider.permission.pattern</name>
) R8 q# |  e) H0 u; L2 m& N. U  <value>^(default:)?(user|group|mask|other):[[A-Za-z_][A-Za-z0-9._-]]*:([rwx-]{3})?(,(default:)?(user|group|mask|other):[[A-Za-z_][A-Za-z0-9._-]]*:([rwx-]{3})?)*$</value>
: `$ P  T# J9 O5 @' i' D, p  s  <description>2 J; {4 H! u" z$ R' P! Z
    Valid pattern for user and group names in webhdfs acl operations, it must be a valid java regex.9 N1 e: S. O% Z* x* R5 _) m/ T7 g* [
  </description>
! T0 G" Q0 T' d0 L0 V. Y</property>1 f7 G+ h; H( M; g) w7 [
<property>
8 M) {# G3 e; B$ V2 U6 }  g  <name>dfs.webhdfs.socket.connect-timeout</name>! u1 \, S# i9 t# s( d" C
  <value>60s</value>5 c9 S# w! i# o8 S  t, Q5 w* r
  <description>
  u0 B6 X" n4 S/ L, G% t. M' \5 A    Socket timeout for connecting to WebHDFS servers. This prevents a1 ?: @( \. T" N8 G, |# ?
    WebHDFS client from hanging if the server hostname is
3 h0 u2 @' K- e$ G    misconfigured, or the server does not response before the timeout
1 \4 e6 |/ \+ D: E    expires. Value is followed by a unit specifier: ns, us, ms, s, m,8 m  ]- D) p- A0 c' B! T+ C
    h, d for nanoseconds, microseconds, milliseconds, seconds,
. E5 h3 m+ ^  p& N# D; B" f    minutes, hours, days respectively. Values should provide units,
) ~: y( w6 l; j* o/ D2 J    but milliseconds are assumed.) L1 ~" \" N& V9 y% i# X& p' _; Z
  </description>
  D" q9 f2 W/ i- M4 p</property>
8 ?; T6 t( ?$ Z$ C9 i<property>
( Z* \: A' k- r  <name>dfs.webhdfs.socket.read-timeout</name>
5 K! m  F' v& W- n; j2 P  <value>60s</value>
8 o0 ^7 t: G. F- s0 `7 u* ]2 Q& e$ c  <description>5 s: K; i$ W4 S* a
    Socket timeout for reading data from WebHDFS servers. This
+ K9 j- b1 H. O# |7 W4 g    prevents a WebHDFS client from hanging if the server stops sending
7 {8 n; _5 L% Q6 _& [. W2 T    data. Value is followed by a unit specifier: ns, us, ms, s, m, h,
. f( Y7 ]6 N1 h: [$ r    d for nanoseconds, microseconds, milliseconds, seconds, minutes,
  Z% |& E$ Y) t7 c7 }7 W% O    hours, days respectively. Values should provide units,
! U8 b' S9 {- D    but milliseconds are assumed.4 _; a! c; p+ f. n
  </description>
2 o+ G2 c1 U2 Q</property>
: P$ J3 ?* O& d; d1 u<property>2 \, w2 M1 g( N, p2 S6 D
  <name>dfs.client.context</name>, q, [( G' w* x, B+ x* v. n
  <value>default</value>4 s/ p& R9 A8 n; @9 J+ c
  <description>) |3 y- `8 O# D( t$ D& x4 \
    The name of the DFSClient context that we should use.  Clients that share+ e+ U! s9 N  `4 w. L, c
    a context share a socket cache and short-circuit cache, among other things.
5 X6 V3 S+ L& j# _) E% U' G    You should only change this if you don't want to share with another set of8 R% j6 A1 c3 K( z; Z
    threads.
, C8 i4 G; p; C$ i, B' v. A0 J  </description>6 K4 ^) A) Y% E3 n6 z. @' I
</property>
$ V7 N3 t$ ]8 {( _1 F9 c; N- W+ c<property>
$ Z+ S7 ?" H6 D! g  <name>dfs.client.read.shortcircuit</name>+ d* |2 b  Y8 w# [
  <value>false</value>- }0 ?3 |. A' w& |  W
  <description>; a$ i% Q) p* t3 G
    This configuration parameter turns on short-circuit local reads.
) e4 M3 N% @# \9 l( ~  </description>/ K. n/ v& `0 v5 e" }" ~
</property>( V. A5 e7 }! n, e& C8 b2 `
<property>
" s" O& w$ `9 a, Z7 H* R  <name>dfs.client.socket.send.buffer.size</name>$ O9 E2 s0 ~; t+ r5 k
  <value>0</value>2 B, r! V( U( [' C
  <description>, r$ c- C7 ?: d+ D7 H# N  W- M
    Socket send buffer size for a write pipeline in DFSClient side.! j' V8 W, ]3 B, o- \
    This may affect TCP connection throughput.
/ O# [9 u, f4 f1 W9 Q. r2 V6 Z1 m    If it is set to zero or negative value,
4 i4 A4 K1 V( n" x5 u    no buffer size will be set explicitly,: M& l& T0 Q' N! L5 u
    thus enable tcp auto-tuning on some system.
- q3 k  I( C& d    The default value is 0.
: q$ P, ]' }5 p; \5 z  K( X* Z6 ~  </description>3 j9 Y& M. l3 s
</property>* [2 U' }" b3 j4 R
<property>- w6 Y2 M7 ]5 j6 ?
  <name>dfs.domain.socket.path</name>
+ S& I& j6 T% @5 I4 v  <value></value>
; ]6 o# U, W+ G+ p$ f" J* T  <description>
7 e: `: E: Q$ |" N% I8 w    Optional.  This is a path to a UNIX domain socket that will be used for* k0 T) a$ ]! L$ _& q7 T
    communication between the DataNode and local HDFS clients.
. P% L% o# M6 v8 F; }    If the string "_PORT" is present in this path, it will be replaced by the
+ Z5 S! O6 X% s: Q3 H8 |$ o/ p$ F    TCP port of the DataNode.
7 R2 p' T( @/ R$ D$ U  </description>/ q: H, X% ~2 P6 w  l% I
</property>+ k& i  S3 E. C7 F' V; `# z2 _7 X) n
<property>5 l7 y/ }5 S9 l0 |/ q% T  z
  <name>dfs.domain.socket.disable.interval.seconds</name>
; O5 v0 l# S% ^0 d' A7 G  <value>600</value>
. o; a& B3 e3 X9 d, g6 z  <description>9 N3 @' R* p1 f, T0 q+ t6 K6 y' m: _
    The interval that a DataNode is disabled for future Short-Circuit Reads,) P" m, q( x% a7 K5 ~
    after an error happens during a Short-Circuit Read. Setting this to 0 will& S3 }; u8 n/ ?9 R& a7 [1 B
    not disable Short-Circuit Reads at all after errors happen. Negative values0 c1 h1 P2 [* Y  p
    are invalid.
* c1 h) J; \8 E  </description>( y- L( R* F  A. p% N
</property>* V4 f" l1 L+ Z2 P5 }' n5 O
<property>
  Q: }" ]8 {0 N  C  <name>dfs.client.read.shortcircuit.skip.checksum</name>" n6 V+ g) K( r3 x0 {5 M; M4 H
  <value>false</value>
$ ], \! Q) \! J* c/ L$ Z) Z  <description>! _6 r0 ]6 a& M5 B7 A3 {
    If this configuration parameter is set,
4 x- J6 a! m7 P1 p2 c. _3 O    short-circuit local reads will skip checksums.
# k3 Y* G( O+ G" G" l6 d    This is normally not recommended,
- H: S- T1 z# X" g3 Q* l; t$ `7 V/ y    but it may be useful for special setups.
, C* u; n# b# r' N  k    You might consider using this0 l8 i) z! m" O9 v
    if you are doing your own checksumming outside of HDFS.6 J* Y$ \* P( k. q( o) Z; F$ t0 H
  </description>  v& F6 R1 N. b  g5 E7 M( a0 D) @- j
</property>
7 n5 w# `' T: g. `9 O3 u, Q<property>
' y8 s0 t) D/ X. @9 G  <name>dfs.client.read.shortcircuit.streams.cache.size</name>9 ~+ S! I# Y$ [! q; h
  <value>256</value>
" H- D7 {& {9 D7 i  <description>
; P4 G3 j3 z; v$ ]8 l    The DFSClient maintains a cache of recently opened file descriptors.& V5 p9 f2 x, ?% y$ N' W
    This parameter controls the maximum number of file descriptors in the cache.
: E$ _6 v2 {% N4 h    Setting this higher will use more file descriptors,8 ?7 j- g: m/ g+ k: n( t3 T
    but potentially provide better performance on workloads# g7 g8 a5 Z7 e
    involving lots of seeks.. L+ O' [3 D/ w1 v
  </description>
' C( y9 ^# g/ q, H: b, S- e, u1 H</property>3 p. O4 R0 ~% Y- d5 ]" g
<property>
" j( {7 s2 D" u9 e) _& f  <name>dfs.client.read.shortcircuit.streams.cache.expiry.ms</name>
$ @! Q  L6 o; }6 n% @- \; ~  <value>300000</value>
3 U+ ]3 \" f& g) r& H  <description>
9 |" C) k2 a% }7 n4 V+ h    This controls the minimum amount of time% y: j& o+ G+ J
    file descriptors need to sit in the client cache context
2 D/ I. V8 @- b# [    before they can be closed for being inactive for too long.
9 [5 Y) M1 C5 }7 `  </description>7 S, E6 c' i# J; X  a
</property>
- G" @0 ]% N  U1 S0 m$ G<property>
4 }( B7 \& e& f, `) x" x) J; j  <name>dfs.datanode.shared.file.descriptor.paths</name>
: s5 C9 n( Z% s3 ^  <value>/dev/shm,/tmp</value>" F8 P5 G' ^6 H( _2 [0 T
  <description>
5 F1 s( M7 u% \3 ]    Comma separated paths to the directory on which- x8 J& K1 \: z4 a0 i7 h* ?5 A# a/ G
    shared memory segments are created.! j" p- \: [5 ^) [6 b9 \3 q
    The client and the DataNode exchange information via
$ S2 |; o0 [; _    this shared memory segment.
, E- H; C- y# p7 M    It tries paths in order until creation of shared memory segment succeeds./ p1 u% z( C8 D3 _
  </description>2 F( A! N1 p: |( X* \9 T, a
</property>/ G5 i' J/ K6 j, H9 M! q4 S: c2 _
<property>0 O. S3 ?8 r1 S) ^- @' R$ A& x
  <name>dfs.namenode.audit.log.debug.cmdlist</name>' p4 B6 O) U4 u
  <value></value>& Q$ k( X! W8 f* X9 ^
  <description>/ I) B& x# N) {6 {
    A comma separated list of NameNode commands that are written to the HDFS2 g" v6 R9 F& h8 k2 U; G/ b
    namenode audit log only if the audit log level is debug.' w  L5 h' n4 n# u1 h8 v
  </description>6 a" G- e& n2 V, S" o* _
</property>1 u: ~" u" J3 x5 F% l9 q
<property>
  F8 P! ^! b" v0 w; r  <name>dfs.client.use.legacy.blockreader.local</name>
7 _: F8 N" Z0 H; }  <value>false</value>
9 K8 \" i, |9 V  <description>' A+ q& i4 ?' n0 T) G* v
    Legacy short-circuit reader implementation based on HDFS-2246 is used
4 e  H1 Q2 k: o; T    if this configuration parameter is true.
. Q- N, r' h, J    This is for the platforms other than Linux
! z3 j6 J+ X5 p& ^# U    where the new implementation based on HDFS-347 is not available.  N" ^- u. r9 i8 n
  </description>
0 K& t, c1 Y0 f' h7 h: m, h& @</property>. {& m9 W+ K2 t1 d* |
<property>4 ?6 {, l% x! h  Z. D2 ]
  <name>dfs.block.local-path-access.user</name>& [9 D2 J9 q- p8 x
  <value></value>0 m0 _+ u, f; o' O5 m& c
  <description>7 x# ~# p7 Z8 X# \
    Comma separated list of the users allowed to open block files
) ]6 |4 w6 y8 Q8 @    on legacy short-circuit local read.
1 A) o8 G, O+ L3 U- @5 R: ^$ Y  </description>
! g, P& g1 B5 M& f  U& P- r3 ]</property>3 F/ o, z( j, O; p$ r! B
<property>  b! p% c  u) {9 H
  <name>dfs.client.domain.socket.data.traffic</name>6 [  m* s, M, x: C8 i# d
  <value>false</value>
. _8 h4 V3 ~2 w/ p% q' y  <description>
4 [1 f' H# Y; p    This control whether we will try to pass normal data traffic
* \! L& p! ~6 r. M0 S3 y- m! I    over UNIX domain socket rather than over TCP socket* U: T" \! e# T7 n* j
    on node-local data transfer.
5 g( [9 T8 q, ~3 X3 t9 t. R    This is currently experimental and turned off by default.- G+ T0 v$ Z+ ?7 O
  </description>! h6 H4 n+ e. }+ W
</property>
: `" \1 d  f. M( U6 L<property>: S5 t& o: d# K7 P, a$ y% Q
  <name>dfs.namenode.reject-unresolved-dn-topology-mapping</name>$ c1 n. A7 k3 J6 Q( L! a2 }2 ^
  <value>false</value>6 b, ?# v" i8 v7 B; \
  <description>8 z+ `1 r, K* H
    If the value is set to true, then namenode will reject datanode
2 j# ^6 x+ E7 `% T# l    registration if the topology mapping for a datanode is not resolved and
0 E3 c/ S* j. v" C2 p    NULL is returned (script defined by net.topology.script.file.name fails " D+ ?7 T* b8 y: S) G3 }7 F% n
    to execute). Otherwise, datanode will be registered and the default rack
6 A4 J' b  @: o# r) l  e2 T    will be assigned as the topology path. Topology paths are important for 1 s2 k+ X- B, c8 i5 ^0 Y
    data resiliency, since they define fault domains. Thus it may be unwanted
/ L; S0 ^5 l# e1 y6 Z    behavior to allow datanode registration with the default rack if the
8 C) c% u- O% H& S3 Q$ ]    resolving topology failed.! p# v1 G5 ^& l/ O
  </description>
/ c0 T% m8 {; f# ?5 c& }) Y</property>
4 M! T6 u) P$ y# N<property>! Z1 m) ?9 \- U5 Q
  <name>dfs.namenode.xattrs.enabled</name>
5 a- a: m  X5 X  <value>true</value>
$ D6 T7 b3 g5 }0 u  <description>
. p/ U2 |# a6 R& N- s& Z    Whether support for extended attributes is enabled on the NameNode.
$ b# t6 W9 l5 e' }: b3 s6 N7 R- l  </description>
! V& Z3 Z  c% I  X& l6 s</property>
( O" j$ a" o% n  p8 t0 K<property>% g0 L8 g7 I* u, q. ^! l5 a9 x% u7 b
  <name>dfs.namenode.fs-limits.max-xattrs-per-inode</name>
0 m5 b+ g: d) l+ u- E; f9 h  <value>32</value>
+ R" n8 S6 l7 _  <description># o  Y- M) l/ K. k- ^
    Maximum number of extended attributes per inode.
" O! x9 u# B& q3 m( G& V- G9 H/ Y  </description>) |5 `  Q7 s$ Y. i
</property>
5 v/ G0 V' d+ j( v" {5 x, m<property>
) n2 i/ L4 V& m# U: T% L1 x  <name>dfs.namenode.fs-limits.max-xattr-size</name>
$ Q/ _( R7 P# h; E) r  <value>16384</value>2 N# X! G2 q0 N! T
  <description>0 T" I% S: w4 ]9 u
    The maximum combined size of the name and value of an extended attribute" K" @* w+ d) t! T
    in bytes. It should be larger than 0, and less than or equal to maximum
; n4 E) K2 x( b5 T& U$ Y    size hard limit which is 32768./ z2 R) p  t& L7 }
  </description>0 S" o' ^6 M( F6 D
</property>+ c- ]  X1 l/ W! Z3 b+ A6 p* k
<property>
7 u; u; @! G! q" x: c* z, J( R! _  <name>dfs.client.slow.io.warning.threshold.ms</name>- [! R8 P7 v9 M+ ]
  <value>30000</value>9 `. z! G( |4 b# O% M  R3 Y8 S
  <description>The threshold in milliseconds at which we will log a slow( d/ q3 }9 Q. m% K, i$ n3 j
    io warning in a dfsclient. By default, this parameter is set to 30000" G1 [  J* k: _2 J, \1 b+ ]8 Y
    milliseconds (30 seconds).: s4 |- F! S4 `4 W) q
  </description>
( v- ~4 K' l4 Q2 A</property>
' {2 I. c% p, @9 i<property>) F/ {1 A/ ~% C3 w6 a! c- ^7 j
  <name>dfs.datanode.slow.io.warning.threshold.ms</name>
9 v5 {5 N- b- \. Y! j/ U- G% z, B  <value>300</value>
0 W2 Q# F0 _8 m+ o1 v/ O  <description>The threshold in milliseconds at which we will log a slow/ A: B- P; L4 P' {1 r( ?& x# b, |7 u7 W
    io warning in a datanode. By default, this parameter is set to 300
3 \2 _: Q% g3 J    milliseconds.
( ~/ J2 f) c4 S" G0 P& f$ H+ P  </description>) a* S9 u1 L; i
</property>
6 p1 D% ^9 Z. w<property>- z& @. y- A& @5 E' a& n  g( [
  <name>dfs.namenode.lease-recheck-interval-ms</name>
4 g: U" n0 k) t+ Y8 v  <value>2000</value># ^/ g; v! L6 c9 ]) P
  <description>During the release of lease a lock is hold that make any$ j7 b: E4 d3 @: i. x' k
    operations on the namenode stuck. In order to not block them during
4 V& K7 ~' B7 W, J$ H. k7 F! s    a too long duration we stop releasing lease after this max lock limit.
! ?# D& |; t# c( f  </description>
& i' W7 o( D  [$ n4 u! c& B</property>
4 c& ^1 g7 d8 O* D! d6 U2 |7 l<property># W3 Q) q' _4 a& h
  <name>dfs.namenode.max-lock-hold-to-release-lease-ms</name>
, I( j( N- Z2 H; w3 ?  <value>25</value>1 X: h+ E8 }1 L9 `
  <description>During the release of lease a lock is hold that make any4 b/ W: Q$ O5 D' }
    operations on the namenode stuck. In order to not block them during  ]) {, I2 l: I: @( Z3 ?
    a too long duration we stop releasing lease after this max lock limit.
; u. ]6 `, U: V  </description>
( ~( _/ L1 z3 E$ w( L$ s3 @' ~</property>+ O3 y% P" k! H% z/ W: ?5 S
<property>9 U6 `( X; G+ c
  <name>dfs.namenode.write-lock-reporting-threshold-ms</name>
$ G$ z; }" z# g& m6 Q: w  <value>5000</value>. F8 y& H$ k6 U( Y! R7 W) U5 F- I
  <description>When a write lock is held on the namenode for a long time,( }  R$ r- N0 u
    this will be logged as the lock is released. This sets how long the
' H' a6 ~- M+ u8 ^' f+ L* J7 c    lock must be held for logging to occur.9 z% B6 _2 t* ^& Y, G
  </description>8 o  {# X  Y' D9 {9 y/ s+ s. i3 n4 c* ]
</property>
" ^8 A0 T/ B* K- L2 s- t<property>1 z. G3 k  a/ |# M, M
  <name>dfs.namenode.read-lock-reporting-threshold-ms</name>
: C& m1 E+ f) e) k  <value>5000</value>- d! d6 S; c. H( v
  <description>When a read lock is held on the namenode for a long time,
3 V' Y3 X8 W# A" M; n1 p& ~6 z    this will be logged as the lock is released. This sets how long the
- Z! ~$ b6 ?- E) k( t3 a    lock must be held for logging to occur.
8 F* |% N1 Q7 P9 ?  </description># ]7 I9 ?9 V8 T% J
</property>2 z3 L# L9 n; {- ^$ T5 K
<property>+ C2 Y( G7 h1 A8 y3 T
  <name>dfs.namenode.lock.detailed-metrics.enabled</name>
) e# S7 r" d# `4 F% D/ X  <value>false</value>2 t1 d! @: \: T- ^$ U/ D8 j$ J* W
  <description>If true, the namenode will keep track of how long various
$ W4 r, n% I5 e2 F/ o    operations hold the Namesystem lock for and emit this as metrics. These/ ?; T% H% l/ t7 p+ v
    metrics have names of the form FSN(Read|Write)LockNanosOperationName,
+ L" W& n& V; X    where OperationName denotes the name of the operation that initiated the# }* k  H2 q  X/ x7 W* m
    lock hold (this will be OTHER for certain uncategorized operations) and
! E# w. [% U2 v7 u( E    they export the hold time values in nanoseconds.* s$ ~& K3 Y2 T2 \
  </description>6 D3 G  Y, V* B* g3 n2 ^+ F
</property>: ~# m: {5 N, q* I2 Q' E
<property>
  P/ |/ {! ]9 O& t  <name>dfs.namenode.fslock.fair</name>' `2 i/ c7 E/ v
  <value>true</value>& u7 e- N/ l/ p% L7 S
  <description>If this is true, the FS Namesystem lock will be used in Fair mode,8 @; Z" t( H2 L2 x9 l$ v6 l
    which will help to prevent writer threads from being starved, but can provide
# y( _. ^1 o: j5 E5 \3 V    lower lock throughput. See java.util.concurrent.locks.ReentrantReadWriteLock
! @. [7 M1 Q3 y+ @6 i' P- F    for more information on fair/non-fair locks.
3 h4 Z1 ~% S9 k* L1 k  </description>
# [$ e# m& v& r" w: e8 |" W8 ^! }</property>( R3 }: g8 ~/ J1 G
<property>
; d3 `8 M- l1 _" N0 A- u7 v; q& [  <name>dfs.namenode.startup.delay.block.deletion.sec</name>& C5 O/ c- u2 p
  <value>0</value>0 ?! w1 g: j) x, p0 q% R# T9 h
  <description>The delay in seconds at which we will pause the blocks deletion8 r7 b1 `) ]; x$ c% x& k4 t
    after Namenode startup. By default it's disabled.! i2 j$ \3 D4 j1 }2 ^
    In the case a directory has large number of directories and files are9 V# K6 ?+ ?4 f' o6 b2 j
    deleted, suggested delay is one hour to give the administrator enough time4 P* ?! n! j" F5 v5 W
    to notice large number of pending deletion blocks and take corrective
6 M& }3 r, A4 g8 b* _    action.) v9 C6 }3 \$ i; [- z
  </description>* o% x6 X' U# P6 b9 o! _
</property>6 @; M3 k7 _- i/ S$ S% A
<property>
1 r( o- b& g. n" \+ h* A' P  <name>dfs.datanode.block.id.layout.upgrade.threads</name>
; V4 A; B1 ]; _0 A  <value>12</value>8 p* v$ Y0 C3 }0 B: S
  <description>The number of threads to use when creating hard links from. M# d" c# z9 d9 L. O
    current to previous blocks during upgrade of a DataNode to block ID-based
* |7 k# A6 Z: X! S) u# d2 N  p2 y9 M3 N    block layout (see HDFS-6482 for details on the layout).</description>3 k1 m, F* X* ~- J! o$ E$ X
</property>! Q8 P; O2 C; F2 `# h4 n
<property>4 C! K; F+ ?; ]4 m# A/ }
  <name>dfs.namenode.list.encryption.zones.num.responses</name>
- |/ B" H! Y$ w/ {, j  <value>100</value>! i, W/ I) ~; D. ]: `- C
  <description>When listing encryption zones, the maximum number of zones! \0 g: H/ ^6 T8 l) H
    that will be returned in a batch. Fetching the list incrementally in
. K( ^/ ]" ]* W' H: Y/ g    batches improves namenode performance.1 N; C3 v) F0 f$ t. Q
  </description>! b& g9 [+ f. P- F5 |' m
</property>& k5 Q$ Z3 u9 D* P) s$ n
<property>: i4 N9 V8 w% \8 H5 a9 W5 G9 k
  <name>dfs.namenode.list.reencryption.status.num.responses</name>
  h! ~2 S9 B! e9 k2 J3 i  <value>100</value>- {# q3 b' n9 W; d3 F
  <description>When listing re-encryption status, the maximum number of zones
- p& R) T7 V9 @  I8 P! Q! F4 A% Q( O    that will be returned in a batch. Fetching the list incrementally in* ?! Z* t  I5 N$ {: W/ ~' ]
    batches improves namenode performance.& N' S* N+ T" `+ G
  </description>! N; W* V" x9 t+ X2 B1 s
</property>/ }+ D2 b$ V! W% |5 H
  <property>
# i5 B" ~6 p$ d5 S9 {    <name>dfs.namenode.list.openfiles.num.responses</name>" m: i% t' ?9 t0 Z/ \% n( [2 M
    <value>1000</value>
6 K& H- v# ?7 i8 G8 [, Y7 x    <description>, p* A9 b: x# X* W5 Q: s
      When listing open files, the maximum number of open files that will be/ Z* u- p5 h- _$ o& G" E+ V& _) c' X
      returned in a single batch. Fetching the list incrementally in batches# t6 l) Q6 ]' W9 k+ M3 M
      improves namenode performance.5 m1 Q! b2 X9 B+ o8 ^& [
    </description>1 ?+ B9 n9 m: x, u4 j. K
  </property>) ~  U+ ^" z4 M" t! W$ _
<property>6 b8 P. w  I' @/ L7 W# l+ i( b
  <name>dfs.namenode.edekcacheloader.interval.ms</name>
1 o: F' P+ j2 \' s1 }: ?0 u  <value>1000</value>) u) D1 `' L0 I( d( Q% B& D
  <description>When KeyProvider is configured, the interval time of warming6 l" N) N7 z' }+ V
    up edek cache on NN starts up / becomes active. All edeks will be loaded, W: \8 O0 _1 E- Y' ]  M
    from KMS into provider cache. The edek cache loader will try to warm up the' ]5 Q( b) ?$ Q6 x9 q+ j- _
    cache until succeed or NN leaves active state.
" a4 n6 v. O. K7 N# M) M+ ~  </description>9 S' u$ c8 l2 V# t
</property>
( Q6 D; r1 v5 o) G: S1 v9 y<property>
. |" o! y+ n' v% P, o' V5 K  <name>dfs.namenode.edekcacheloader.initial.delay.ms</name>
7 j/ K0 k$ d5 n: b" Q  <value>3000</value>
; \# D% [% k8 f; t, R( Z. N: X, G  <description>When KeyProvider is configured, the time delayed until the first
# i: G& Z+ [; j) ?3 z    attempt to warm up edek cache on NN start up / become active.
# |# ^6 C! r6 B% j$ g7 i& k. ?; k5 s/ \# Y  </description>
8 R9 v- x: }9 D</property>
3 E1 ?/ B" k' g) r, J* N<property>
/ w" G# p/ C# ?- h( U6 V  <name>dfs.namenode.reencrypt.sleep.interval</name>
) t- b( _. ~& K% P  <value>1m</value>
% {+ _7 d' c4 O* F  <description>Interval the re-encrypt EDEK thread sleeps in the main loop. The. N. N% o* O" M3 d: |9 S3 C
    interval accepts units. If none given, millisecond is assumed.' ~4 f4 f; R3 d! z
  </description>8 d  j4 @, B  }
</property>" r' Z- b5 R$ V8 [( ]9 t: p8 u
<property>
; X- _. y. u, E& H  <name>dfs.namenode.reencrypt.batch.size</name>
- E6 g* _3 {0 {5 S  <value>1000</value>6 Q6 i- \6 Q4 {9 ~; @% g
  <description>How many EDEKs should the re-encrypt thread process in one batch.( S$ N  o' U6 ]  ~- M1 P
  </description>4 ^; b1 {! F5 @% a. U
</property>8 x5 i- P5 b8 `" j! N
<property>
& f" \+ B) b9 l* h- s  <name>dfs.namenode.reencrypt.throttle.limit.handler.ratio</name>, w# D0 y) {) Z7 P) m- X' k" i3 [6 a
  <value>1.0</value>0 H; |2 j* n4 A/ [$ J
  <description>Throttling ratio for the re-encryption, indicating what fraction7 C9 O2 P. Z4 L2 m# k! Z" E
    of time should the re-encrypt handler thread work under NN read lock.
, ]! y- x( o" [+ f8 o9 P4 D    Larger than 1.0 values are interpreted as 1.0. Negative value or 0 are/ Y# N9 e& H4 x: g3 u) j
    invalid values and will fail NN startup.
! V' r2 W$ k" h  </description>( \. E  s  k8 X! K3 R* F' ]
</property>
* `: g0 K4 k/ V# R0 ?1 g<property>" ^* `. M0 Y- b" M
  <name>dfs.namenode.reencrypt.throttle.limit.updater.ratio</name>
5 j3 @. g6 o' `) R4 f" U2 O5 }2 A  <value>1.0</value>. T# H$ W8 P' S" R# t5 ]
  <description>Throttling ratio for the re-encryption, indicating what fraction% P4 Y& ^. X4 j0 y6 ?9 U- Z+ W- K
    of time should the re-encrypt updater thread work under NN write lock.
$ r+ _' H8 x, l# F3 P7 v    Larger than 1.0 values are interpreted as 1.0. Negative value or 0 are
) E9 m. b4 t- p* S+ E    invalid values and will fail NN startup.3 X1 D$ \/ }+ U0 k/ O4 _/ i. {
  </description>
! X$ X3 ^) I" P% [</property>
! [0 L3 i* X* p' p: N: \<property>3 c0 O) m6 ]" B2 V
  <name>dfs.namenode.reencrypt.edek.threads</name>& g) s; @9 I. h0 C* Y! A5 \! Z
  <value>10</value>7 s  s' q- l' R' V! \, a1 R) ?
  <description>Maximum number of re-encrypt threads to contact the KMS) Z, q' m2 T0 l
    and re-encrypt the edeks.; q: c( K3 F4 G6 v5 J% P  N: u2 ~
  </description>
4 @9 X# A1 F% l" h& f: T</property>
1 W! s$ `. I) W# ~1 l' x<property>9 m, o4 w5 [" ]4 o; e0 a
  <name>dfs.namenode.inotify.max.events.per.rpc</name>+ i: L3 a& h7 f
  <value>1000</value>
$ b1 G" U" c% B. }: x) O+ S: N( d/ ]  <description>Maximum number of events that will be sent to an inotify client
4 k: [! y3 C9 B, ]' l8 t    in a single RPC response. The default value attempts to amortize away6 s6 b0 {/ u: @$ P0 J2 H
    the overhead for this RPC while avoiding huge memory requirements for the
$ a& _8 ^* N8 F  _% }0 }    client and NameNode (1000 events should consume no more than 1 MB.)
" t! Y7 T2 ^* _; o  </description>
. [4 I2 p/ O# O& C2 \- l</property>
" a' S& W& M8 A) |$ `<property>
1 Z1 D: ~" Y8 s$ z' X  <name>dfs.user.home.dir.prefix</name>
+ y" O3 P+ U4 g  <value>/user</value>- @& l! C( c" l% w& q# A! g
  <description>The directory to prepend to user name to get the user's
7 M* s0 x5 p5 q- _    home direcotry.  r$ S- ]& d5 c1 f
  </description>8 |) s. P' b8 |, y. ]1 O  P) i
</property>
. k1 i3 o7 |0 c' S<property>* j/ A: G# |! |+ g, [# [0 k
  <name>dfs.datanode.cache.revocation.timeout.ms</name>/ j4 `1 Z1 M" M  S; M" I
  <value>900000</value>
* T) g1 R  f7 ^6 ]5 N* ^  <description>When the DFSClient reads from a block file which the DataNode is9 X* O# p1 w  u6 G/ s4 v& L
    caching, the DFSClient can skip verifying checksums.  The DataNode will3 y5 Y8 e3 n, t$ m
    keep the block file in cache until the client is done.  If the client takes* O2 }* m& r% s& ~( V% U; W( }
    an unusually long time, though, the DataNode may need to evict the block
- T8 V% u1 }$ M7 K8 P  V% Q    file from the cache anyway.  This value controls how long the DataNode will" Y/ M7 z/ T' _" s0 y6 J
    wait for the client to release a replica that it is reading without
9 P- l* `2 n9 Y3 g    checksums., L& {' g9 H5 W
  </description>2 X9 t* z0 }. t
</property>
4 P  N2 I& @( L<property>
; D. C7 d* d& q. R$ A  A9 q  <name>dfs.datanode.cache.revocation.polling.ms</name>. y; i% V$ v1 R8 v8 F/ F
  <value>500</value># x4 R- A( x' [
  <description>How often the DataNode should poll to see if the clients have
6 f4 O& c7 U! d( z  i/ B9 u) {    stopped using a replica that the DataNode wants to uncache.+ L3 o/ J, W. r) o
  </description>
: H0 F" I% H4 L( I" t3 A</property>$ S7 B* M2 A$ @% E( Y5 I
<property>, P+ I( S7 h/ _
  <name>dfs.storage.policy.enabled</name>
3 K) z# c/ Y. ]  <value>true</value>5 X. t5 a0 Z' q  k  i& s+ Q! O4 @
  <description>; s8 U( R# }9 c
    Allow users to change the storage policy on files and directories.5 Z4 T' ?5 t) c5 `* E
  </description>
) |5 n: p& J/ i</property>
' R! v+ u- B/ S<property>9 \4 i/ B% E2 l+ b$ b( C4 {
  <name>dfs.namenode.legacy-oiv-image.dir</name>  y0 i$ G" m* I7 s$ q
  <value></value>
' C+ N3 @% G& F# j$ L  <description>Determines where to save the namespace in the old fsimage format8 F! D. {5 c: _; g) Q# J$ c
    during checkpointing by standby NameNode or SecondaryNameNode. Users can
. Y% K5 Y! |1 K' ~) t% x    dump the contents of the old format fsimage by oiv_legacy command. If
3 K0 Z6 R) l5 C1 b& ?    the value is not specified, old format fsimage will not be saved in
! u+ [' x4 Q7 O/ i% Q    checkpoint.
9 m$ c& ~6 P. d9 v  </description>- ?8 r' g) f$ v
</property>& f) F* d4 U9 b& I4 h) A% s
<property>
3 ]" L2 @' Q% b) R! Y  <name>dfs.namenode.top.enabled</name>" p0 T" o- g' j$ v2 f: g
  <value>true</value>
( w. h9 v8 X; @# O* L9 Z4 J  <description>Enable nntop: reporting top users on namenode! [1 Z$ U5 B) _- c
  </description>
( P; Q1 T% M* P( b5 y  G</property>8 v3 }8 l( G4 f( F. S, P( V3 n* D1 d
<property>
5 D/ |+ v$ \- t* s  <name>dfs.namenode.top.window.num.buckets</name>' ^6 L. Z' m8 e" M; U* z' S
  <value>10</value>& S! E+ q: k0 i  P0 t7 }& A
  <description>Number of buckets in the rolling window implementation of nntop* k. z( X4 s0 T7 j/ R( [
  </description>1 d1 ?/ c' o4 P$ F+ l) w
</property>; c) H6 ?+ |( M
<property>
+ M# v, e) O$ N- j0 p  <name>dfs.namenode.top.num.users</name>9 |4 \# O/ V: m( R! v
  <value>10</value>" M  g+ s! {; L' \& I: f( Y; u
  <description>Number of top users returned by the top tool# v1 G6 ^8 R; M* _
  </description>7 b3 j9 n6 V4 w
</property>
* |: ^" B) e# O6 K<property>% Y" ?' s; I& W9 h; Y: M
  <name>dfs.namenode.top.windows.minutes</name>+ M4 j1 \. o# G
  <value>1,5,25</value>9 I- h; u( q, u* x2 N1 ?
  <description>comma separated list of nntop reporting periods in minutes
6 P+ A' _% b' }5 h+ I% k  </description>
0 s$ u! F' z- B) x8 g</property>
7 f1 z; ~! Z) U0 \<property>
8 j' H) ]6 o+ b. U    <name>dfs.webhdfs.ugi.expire.after.access</name>4 J1 D- f% c4 t. C
    <value>600000</value>
9 w- c) Q/ Q/ x, M+ a# c    <description>How long in milliseconds after the last access; U. x- Z5 @! k( P3 p. K
      the cached UGI will expire. With 0, never expire.1 K% B9 [4 B7 q! C9 }
    </description>1 C  A: q2 p% g% d! r2 n* n
</property>- {0 k6 F& E* U5 u
<property>
: K3 P' ?3 a6 L+ q' A8 t  <name>dfs.namenode.blocks.per.postponedblocks.rescan</name>0 V) }* \: [$ q
  <value>10000</value>( d0 Z3 e1 I! K
  <description>Number of blocks to rescan for each iteration of
# B4 t7 j, D$ c6 c' W    postponedMisreplicatedBlocks.6 o. h% I) x; m/ U
  </description>
: X9 h' ]& G. ~1 \. v: A7 Y</property>
7 W5 V( h' X  a* o! P+ U0 b9 `<property>6 m$ s' p# |" O% R0 h- c3 ]
  <name>dfs.datanode.block-pinning.enabled</name>
% i, X7 O7 _, b) Y$ E4 `  <value>false</value>
% _) l1 b! N: X- k& O  <description>Whether pin blocks on favored DataNode.</description>
4 D1 H8 [" `7 _) P</property>8 T+ Q, v& t! v' Y3 V. P
<property>
# U9 a! O# O4 w+ X, j  <name>dfs.client.block.write.locateFollowingBlock.initial.delay.ms</name>
7 M' F8 e4 m' d, Y1 Q, g* z  <value>400</value>: _; j% H/ ]. B, t& c: S( I* Y8 E
  <description>The initial delay (unit is ms) for locateFollowingBlock,. d: V2 e) O; {6 r6 @) f$ c: {  q
    the delay time will increase exponentially(double) for each retry.
& e" t' d4 {" G$ G7 M( q  </description>3 Y2 m0 E9 T% o% `2 n
</property>: c1 [/ c1 W1 @" `7 _
<property>
/ f- T. G1 ~& Y9 a  <name>dfs.ha.zkfc.nn.http.timeout.ms</name>
' e. Y( g7 o% A/ d; y  <value>20000</value>
! z2 Y5 D2 {! ^  <description>
. Q/ ]) x7 W$ Y" |4 m5 [    The HTTP connection and read timeout value (unit is ms ) when DFS ZKFC
. h5 f: c0 H7 L& O    tries to get local NN thread dump after local NN becomes
4 K, o+ Z9 D6 X  c! S. P  c    SERVICE_NOT_RESPONDING or SERVICE_UNHEALTHY.) Y4 g# y" Y, _
    If it is set to zero, DFS ZKFC won't get local NN thread dump.
9 I  j; N: W- Z- }" {) p, W  </description>8 c' w' Q' e* z$ n( p
</property>3 c, K) i# r) J: }& W; r$ g8 }/ @
<property>
4 u+ c! q# u* E$ ^; S5 w  <name>dfs.ha.tail-edits.in-progress</name>
- j/ J4 Q! R( e0 U: i  <value>false</value>
; e' Y6 K  Z  _  z. ]  x  <description>
$ z/ f  K# E: w1 D0 N% A    Whether enable standby namenode to tail in-progress edit logs.6 q  B: ]) c1 E, h3 A0 m8 c
    Clients might want to turn it on when they want Standby NN to have$ O4 c  j# ^+ R8 a8 C
    more up-to-date data.
% Z  I& M: ^7 ?0 E* h# z# E  </description>
, @0 S6 ]7 T$ h* `+ e7 L, D</property>2 |! Y5 V; T  i; k
<property>2 T6 Z1 d- F' p2 i
  <name>dfs.namenode.ec.system.default.policy</name>+ w; f, ]( d4 P0 y: e
  <value>RS-6-3-1024k</value>
2 Z5 i0 b& A$ |$ r  Y  <description>The default erasure coding policy name will be used3 g9 [  p; Z( c& |
    on the path if no policy name is passed.
( u( \" ^: ~8 |$ n3 E  </description>
( Y( I4 H- Y, w& t</property>8 z" t# ]4 N$ j2 y) z
<property>
8 P. c! h' O$ I& |% K# g: j  <name>dfs.namenode.ec.policies.max.cellsize</name>
7 ?) Y9 u$ o1 Y& y: n: z+ c* w  <value>4194304</value>( H% H! o  O4 {
  <description>The maximum cell size of erasure coding policy. Default is 4MB.
- o! r+ ^; A, O  </description>
3 N4 F. z6 O, @$ w1 Z. c8 l7 ?</property>
7 N8 x4 Y. h4 R5 d- F0 N% o* j<property>
' i) }: {4 n9 M. c  <name>dfs.datanode.ec.reconstruction.stripedread.timeout.millis</name>( ]$ ]  r/ e" _
  <value>5000</value>: i4 }5 d8 l) o/ K+ `" A7 U
  <description>Datanode striped read timeout in milliseconds.7 P3 u0 b  x( ^% a, f
  </description>4 k4 H# b( a( u  c
</property>! H$ V+ V6 i5 e$ b7 y, r! P
<property>
- D0 I) a# [) ~8 h  <name>dfs.datanode.ec.reconstruction.stripedread.buffer.size</name>& O! m; H2 U) `' M, u! o
  <value>65536</value>' `. ]: N$ _* ?, z4 A( {9 L3 ~7 i
  <description>Datanode striped read buffer size.
% j7 I$ ^/ p- G3 _1 Q& K$ }: x  </description>: D0 Y  j) A: [+ q/ C7 T
</property>
) V  g/ e! T8 |( Y' u<property>
4 W" U$ E3 l: v- {" v+ d! a  <name>dfs.datanode.ec.reconstruction.threads</name>
) v+ A0 ]" j4 w5 m, X" n# V  <value>8</value>
$ V2 X, T0 I4 D  <description>( G) E9 @8 y8 k- K6 s1 L
    Number of threads used by the Datanode for background( |, _' R# D; o* Z( b1 s3 [
    reconstruction work.3 \! `7 ]& n* `: d; T! z2 G6 F
  </description>
) g% P: U; q4 Y8 c2 D: p</property>8 D* a( q2 S! A3 ~" [
<property>
: a% m+ S: ]5 W3 O* F  <name>dfs.datanode.ec.reconstruction.xmits.weight</name>
, l% l" U6 R1 d! F  <value>0.5</value>
) `% j- d' B. D& \' T& L2 z" b0 @3 X  <description>" X' s! c0 Q5 C
    Datanode uses xmits weight to calculate the relative cost of EC recovery" [$ \3 M" c8 g
    tasks comparing to replicated block recovery, of which xmits is always 1.
: }) R) ]6 l# Q, w7 Z    Namenode then uses xmits reported from datanode to throttle recovery tasks/ d' f9 b) i: t$ s% f' i7 W# Y
    for EC and replicated blocks.5 D* J9 i% H$ J+ F5 B6 t% x8 h0 o
    The xmits of an erasure coding recovery task is calculated as the maximum
4 \" M- ^7 P! \+ E$ Y( b( k    value between the number of read streams and the number of write streams.
6 d3 A9 `6 I* i  </description>
; Q! ^/ @! p2 M; ~: Y; s/ z, U2 z9 j</property>* m: x) k' m  ]9 Q+ x& b
<property>9 r& m7 |* n* u$ e
  <name>dfs.namenode.quota.init-threads</name>" j2 |$ w+ `0 V3 e1 `$ [
  <value>4</value>' H5 m7 I: P% ?! D! s
  <description>
0 h& l4 I' ?) ]9 I) [5 S8 Z" o    The number of concurrent threads to be used in quota initialization. The- f" A! ~' ^* L
    speed of quota initialization also affects the namenode fail-over latency.8 |+ w( p! k+ l! S  b# w9 K
    If the size of name space is big, try increasing this.5 c! G) S0 G/ k: L" o$ j  K
  </description>4 ]3 I: h4 E" d6 b3 M$ e
</property># d) T: E( O1 P0 R
<property>
1 u5 V9 m4 O& A7 P5 j! e  <name>dfs.datanode.transfer.socket.send.buffer.size</name>
: X, Q8 D, T9 ]  N) \9 T  <value>0</value>
& W0 Z; m; d0 L/ W& F- L) K9 C% O- o  <description>7 c) a* q( ?8 L
    Socket send buffer size for DataXceiver (mirroring packets to downstream
+ d$ G5 m8 d, j& ]( y: t5 E    in pipeline). This may affect TCP connection throughput.0 @1 G" H( S1 k% N! s! S: ]
    If it is set to zero or negative value, no buffer size will be set
; _& e$ E* f: b/ f7 z" @& z    explicitly, thus enable tcp auto-tuning on some system.; @8 u8 q2 M0 o& D# x5 s
    The default value is 0.
+ O5 P/ s8 U9 ]  </description>
4 _9 ?. e$ J, ~7 f</property>3 v9 y/ C* F& r
<property>
" H8 K4 d8 S, c1 F  <name>dfs.datanode.transfer.socket.recv.buffer.size</name>
5 A( e) k+ k5 g2 y& d. i9 b6 D; }, U  <value>0</value>8 r' j4 X- {. J/ _/ I9 s# w) P% a
  <description>
0 l$ j. Y9 r# R* ]    Socket receive buffer size for DataXceiver (receiving packets from client
$ m2 L- Z2 _' @9 [8 N# C) R    during block writing). This may affect TCP connection throughput.9 c( j/ L1 S* ^0 X: ?
    If it is set to zero or negative value, no buffer size will be set( }2 ?0 e" @! Q9 c$ v; X$ _
    explicitly, thus enable tcp auto-tuning on some system.7 V* e6 |; f0 }' n! ~: p
    The default value is 0./ O3 F) O+ C5 Q" W6 m4 \
  </description>
1 r5 c& r5 L" v3 z5 {</property>/ e' @! c7 E" |
<property>
4 t& M8 E: r0 s6 f9 q) ]6 x  <name>dfs.namenode.upgrade.domain.factor</name>
3 |  A: l# X* I! ]2 V" ~/ v  <value>${dfs.replication}</value>3 `5 ?3 l+ {4 D1 u3 y
  <description>. N* j, I4 c/ x( s6 O& Y+ w" G+ W
    This is valid only when block placement policy is set to3 T& q  a/ p4 s; ?! S. a
    BlockPlacementPolicyWithUpgradeDomain. It defines the number of
2 o% ^/ i2 a  p. D9 \2 V- W& |9 \    unique upgrade domains any block's replicas should have.- h+ H8 K! O, {9 q) w: n) w# x
    When the number of replicas is less or equal to this value, the policy+ s2 q- A' M/ f/ P
    ensures each replica has an unique upgrade domain. When the number of
9 @% c! ]) H2 {9 u3 F" d$ ?    replicas is greater than this value, the policy ensures the number of3 M% j- z, O9 Z! h! `% l
    unique domains is at least this value.
" D5 ]- b* N8 q; f0 d* i; e  </description>
7 \% s) t0 U  J& ]* o% G, K</property>4 J* h7 J; L; g  l: u- \
<property>
) n- x# Y& Q3 K# |  <name>dfs.ha.zkfc.port</name>
2 U* ~( C! ^7 X- y. t* }' H  <value>8019</value>/ ?/ q" O( b2 \% G% n4 W4 @5 A+ l
  <description>
+ D! i1 v/ m: @/ e    RPC port for Zookeeper Failover Controller.) a. \  p3 c, W' I$ B
  </description>
2 u# n2 b4 i7 Z( o+ g" B' d2 i; G- q" j! |2 Z</property>
& }+ j* K0 F9 s+ W<property>) e* G  U" S% b0 {; Q
  <name>dfs.datanode.bp-ready.timeout</name>
  V3 }" b  ]7 s1 |7 x  y: `. r  <value>20s</value>
# E' e5 ~3 o, M! \% z% E  <description>
5 F, `9 v# j. j8 X" s' W    The maximum wait time for datanode to be ready before failing the
; ?2 W6 m- f1 s1 h: `! K' |6 d3 ~    received request. Setting this to 0 fails requests right away if the
6 Z; _! v$ J0 h8 d8 Q    datanode is not yet registered with the namenode. This wait time% T2 o7 k/ b2 G% h' j
    reduces initial request failures after datanode restart.
2 ?$ D! t5 Q& c    Support multiple time unit suffix(case insensitive), as described/ d9 ~7 s. W+ H4 R7 ~3 `
    in dfs.heartbeat.interval.
0 M4 I9 i+ v, ~4 ~4 R+ u  </description>2 [+ _( C8 N- o. R! p( K* d8 t
</property>/ O. A% c' C0 R3 W
<property>
& h- b1 M- X( |1 ]; e5 I  <name>dfs.datanode.cached-dfsused.check.interval.ms</name>
, t9 h0 {/ q5 H% {  <value>600000</value>: }' V6 a* }: L$ N! X) I/ P7 e- f- g. o
  <description>
6 ^/ z2 K- y% o- c& o    The interval check time of loading DU_CACHE_FILE in each volume.# A2 r& y' X0 }7 B: g* W3 \
    When the cluster doing the rolling upgrade operations, it will. L$ r, b* z* e3 w7 y, P
    usually lead dfsUsed cache file of each volume expired and redo the
* p- J7 Q4 i8 `: b1 K+ g    du operations in datanode and that makes datanode start slowly. Adjust( B5 J# E: T) j) ]/ w& V
    this property can make cache file be available for the time as you want.
8 h) B; v% v: y( _" V% G  </description>
+ y+ T8 g6 t  j9 \% _4 Y% p* T9 X! I</property>9 H; k- R7 d' U0 D0 h, l
<property>0 D; W$ `+ B& Q8 b# b; W
  <name>dfs.webhdfs.rest-csrf.enabled</name>& D( w9 ~" z' |4 ^; J/ L% @- u) S3 ?
  <value>false</value>
$ M5 P& H1 M& g  <description>( E+ U* y& p, c/ _1 U! ^
    If true, then enables WebHDFS protection against cross-site request forgery
) H2 A4 Y4 k) R4 \0 p& O. x- a0 d    (CSRF).  The WebHDFS client also uses this property to determine whether or/ W0 b) x' ^6 K- t+ x4 f  M/ k
    not it needs to send the custom CSRF prevention header in its HTTP requests.4 e$ s0 H2 R4 V0 m) `
  </description>
* j+ f2 `: S" a+ }* G8 j( y</property>
) M3 A6 A1 h! |0 v% K+ _4 u<property>* e4 `% e9 t- E4 N, Z
  <name>dfs.webhdfs.rest-csrf.custom-header</name>
- x& d) k" r3 k+ J0 Y  <value>X-XSRF-HEADER</value>
# O. M0 t# D, }. u5 u  <description>
1 K2 i3 i: |1 ?1 B$ m8 k/ z7 f    The name of a custom header that HTTP requests must send when protection
6 E+ r8 m* \2 C- t    against cross-site request forgery (CSRF) is enabled for WebHDFS by setting
7 q) E9 [5 N) v6 \, @( o3 [1 `    dfs.webhdfs.rest-csrf.enabled to true.  The WebHDFS client also uses this
: X; y- a9 s5 U+ p1 f0 Z) V    property to determine whether or not it needs to send the custom CSRF2 f9 O8 h! F7 Q- c. b0 ?1 O0 Z
    prevention header in its HTTP requests./ X9 X0 H# r/ Y7 y
  </description>
( O4 G" e0 S% g7 B! M5 a</property>5 B" S4 P- I. J/ v
<property>
* S- b8 Z- g: l3 l  <name>dfs.webhdfs.rest-csrf.methods-to-ignore</name>( l/ ]1 T9 [$ w( i$ m
  <value>GET,OPTIONS,HEAD,TRACE</value>5 K& d, X! T: f
  <description>! ]9 `+ R0 g! q+ c# m( r5 Q# D
    A comma-separated list of HTTP methods that do not require HTTP requests to& |1 D, V3 ]; z4 d  o
    include a custom header when protection against cross-site request forgery
( j: A- K3 a% Q0 @* j- e    (CSRF) is enabled for WebHDFS by setting dfs.webhdfs.rest-csrf.enabled to
% g8 B" @$ m) r& z/ j% m    true.  The WebHDFS client also uses this property to determine whether or9 a; \' {0 q1 W* K) [& g
    not it needs to send the custom CSRF prevention header in its HTTP requests.8 \9 H) y* z/ \; g# k. Z
  </description>
8 ^0 v& d# H. |% O" Q& i</property>. U  f0 h# r7 N% j+ F  _, `
<property>
5 ~7 u8 j8 a8 }; O  <name>dfs.webhdfs.rest-csrf.browser-useragents-regex</name>
# T, l2 @& m3 ~% `1 j  <value>^Mozilla.*,^Opera.*</value>
2 `. A- j3 V  A" ^' @3 D. X  <description>% ?" E4 Y, g/ O- |0 g& N
    A comma-separated list of regular expressions used to match against an HTTP
* Q' Z' r1 s) ~: X4 U    request's User-Agent header when protection against cross-site request
* f* X" l% L' N  f% @    forgery (CSRF) is enabled for WebHDFS by setting
7 Q' ^( a: T0 g, x% N5 g7 d8 u3 P6 f    dfs.webhdfs.reset-csrf.enabled to true.  If the incoming User-Agent matches' M- C/ l3 s- k) e( e, e0 E
    any of these regular expressions, then the request is considered to be sent
) ?% O8 J; P, E1 U/ g+ ?    by a browser, and therefore CSRF prevention is enforced.  If the request's
+ l0 K9 r3 L1 |' {6 x+ q    User-Agent does not match any of these regular expressions, then the request
# n' W8 S: ]+ L( Z/ k  K: y    is considered to be sent by something other than a browser, such as scripted
8 t% z, \7 j$ K! q    automation.  In this case, CSRF is not a potential attack vector, so
! V( z! W# ?4 ?+ M0 I    the prevention is not enforced.  This helps achieve backwards-compatibility
( C* V; _* R8 x$ m* ?  l. U8 o0 U! _    with existing automation that has not been updated to send the CSRF5 K# E" [3 j3 l: }
    prevention header.
5 G  ]& N1 z2 b1 O- Q  </description>% e+ o/ N+ n! q" I; {. P( p
</property>4 b. O8 s6 L. f. w) j
  <property>
# ^- Y" b+ J& E3 H+ T    <name>dfs.xframe.enabled</name>1 _# G; Y+ O0 Z3 p" Q2 c  q
    <value>true</value>, ]; k- v% L6 O
    <description># K. l4 e# f5 {; r2 K6 m
      If true, then enables protection against clickjacking by returning5 G& h  [# o; T1 z% I' d
      X_FRAME_OPTIONS header value set to SAMEORIGIN.
" L" T( m: G. _# v( N: z4 x- h4 H      Clickjacking protection prevents an attacker from using transparent or& Z9 Q. B; n, }! l- f; y2 U
      opaque layers to trick a user into clicking on a button( R. @1 c# r; D  r
      or link on another page.
% k1 ^5 t7 N* y/ A$ N% E    </description>
, ~' v0 P5 D9 P9 r% x! u  </property>
3 G' I: e3 K& i3 w  <property>! g" B: x2 r3 I: N$ p4 u, J- \0 E- e
    <name>dfs.xframe.value</name>0 f/ s5 u) m  x' J
    <value>SAMEORIGIN</value>! U0 Y9 C; ^" R5 ~
    <description>
( ~3 t" h4 G* O( t      This configration value allows user to specify the value for the' [/ B3 g3 g* L& k5 ~7 H4 r
      X-FRAME-OPTIONS. The possible values for this field are
# S4 q3 C5 d- m( E      DENY, SAMEORIGIN and ALLOW-FROM. Any other value will throw an
$ n# J2 z% S! v$ Y. W& K      exception when namenode and datanodes are starting up.
) C" `) ~9 N5 y& J, G    </description>
% u; M  W# G" z; ?  </property>  S# [% c. l* R& E7 j& {9 W2 ]9 U
<property>' Q1 Z; R9 l. P) Z/ M; I; C5 K
  <name>dfs.balancer.keytab.enabled</name>
: o. Z  J9 q$ {0 x' h  <value>false</value>
1 i8 d6 P! U& {: j( F7 k& G$ z/ X3 [0 [  <description>* b1 R/ t  `5 V' u7 E
    Set to true to enable login using a keytab for Kerberized Hadoop.
) o+ E/ `$ `( N8 A" o# ~4 y0 D$ E  </description>9 N* w( P6 X* q, U! W3 _9 j$ Y
</property>
3 G) l& X( r. R% V+ R+ l<property>4 ?, r. m9 J! ~/ R- X1 o
  <name>dfs.balancer.address</name>" D; N& P! ]6 R. t8 |2 ^
  <value>0.0.0.0:0</value>
- T' c% M4 R* H! ]% B  <description>
$ |6 u) U% w( T: A2 U& C7 j" M    The hostname used for a keytab based Kerberos login. Keytab based login2 c: m2 L: n+ W
    can be enabled with dfs.balancer.keytab.enabled.
, B' u) y, R4 ?- `3 A. s  </description>
1 b0 W1 I' `2 z& Y( U</property>
; [& d6 K, k. C! e! s<property># b9 t# ]$ Q5 l/ ]- f3 p
  <name>dfs.balancer.keytab.file</name>
6 z; i( R& _% c2 w+ e- V0 W  <value></value>& M2 n6 n) o6 H
  <description>! n- ~& F7 r+ T. h0 O: e, M
    The keytab file used by the Balancer to login as its
. D- i/ Q4 N( V7 g# W. R    service principal. The principal name is configured with4 G$ {2 O- m# S9 k/ ~' @4 E
    dfs.balancer.kerberos.principal. Keytab based login can be
$ O# e5 `5 E# F/ J5 V% u    enabled with dfs.balancer.keytab.enabled.
/ S9 w2 D8 ]8 E+ J4 e  </description>4 t9 ]7 Y. c# E, r- m+ x2 f5 @
</property>  q( v" }$ k! m1 z" z
<property>. Q( [% w* [& X1 V, p
  <name>dfs.balancer.kerberos.principal</name>
1 k3 G/ f1 q: \+ t0 ?! Y  <value></value>* y0 W; O# p5 k2 p' [9 p# L. A
  <description>, Q  T% P" I- e: S1 y! G; G' c
    The Balancer principal. This is typically set to
  [/ |3 g% @3 F( W    balancer/_HOST@REALM.TLD. The Balancer will substitute _HOST with its
' V* \, y! A# j3 k9 c$ D' U, E& L( Y: U    own fully qualified hostname at startup. The _HOST placeholder
' ~1 Y) c( T6 a# u$ t" r    allows using the same configuration setting on different servers.
/ K& m, i( r5 i9 E5 k. y    Keytab based login can be enabled with dfs.balancer.keytab.enabled.! K- A5 f' B! k3 x- {) A4 i+ C
  </description>. n- |5 R  X$ z% u+ s
</property>: w5 U% T1 p9 Z! a2 ?
<property>: e0 Q* M6 n. Y: p: p( \
  <name>dfs.http.client.retry.policy.enabled</name>7 z7 c% J; I1 v4 P( }
  <value>false</value>
* X, U1 {, D% v5 A& O) d/ f3 H  <description>
% \2 R8 P0 b/ x) h; J) k    If "true", enable the retry policy of WebHDFS client.
. _8 F: i6 [  o8 w7 b! z( z3 d- Y    If "false", retry policy is turned off.! G! c# d1 g4 ]$ L* z% D% h
    Enabling the retry policy can be quite useful while using WebHDFS to7 j6 v4 `* @+ |; K) s( {
    copy large files between clusters that could timeout, or* Y  P& ~. G% y) T% T% r- \
    copy files between HA clusters that could failover during the copy.' c* ]) H+ `) ]; S: g9 A! A" S. N
  </description>& F3 N& I% l9 m
</property>9 z. x: q, Q- Z& E; c( n
<property>( W4 a& B4 X' |8 n+ N
  <name>dfs.http.client.retry.policy.spec</name># [- F+ Y  x. H" ?
  <value>10000,6,60000,10</value>
6 h, U! _3 O! P8 @2 E8 C' Q+ |" V6 F  <description>
, Y) D2 _- B" X$ I9 ^8 {; G    Specify a policy of multiple linear random retry for WebHDFS client,/ |" v0 r0 y9 W5 H9 C
    e.g. given pairs of number of retries and sleep time (n0, t0), (n1, t1),5 N3 W  E9 O, x
    ..., the first n0 retries sleep t0 milliseconds on average,9 R/ ?5 @! ?% A: w  Q
    the following n1 retries sleep t1 milliseconds on average, and so on.- \# `  O+ x: `1 b/ N7 n1 ?/ [
  </description>0 y0 ~3 O" \& \5 Y7 s. a
</property>
/ Q9 ?( _1 E- ]$ }% H<property>
5 m7 B6 r8 M5 q* Z% P. p  <name>dfs.http.client.failover.max.attempts</name>1 |8 ~; ^9 c, x* D
  <value>15</value>8 d- X6 l+ h# ^
  <description>, {. z5 @6 ?/ `. p
    Specify the max number of failover attempts for WebHDFS client
- l6 D8 ^9 ^7 R. v$ _9 c! P$ C    in case of network exception.& l6 r) `+ W& j' w* e% e, a
  </description>- D9 C4 P9 `- Q
</property>
5 W" Z# A/ ~  k: p<property>& O# t: c3 \/ P4 b. h' g
  <name>dfs.http.client.retry.max.attempts</name>
; f: K7 G' y3 y/ L' m' V  <value>10</value># Y/ B3 n; |* g( g5 t
  <description>
- Q0 U) f* l9 q* J0 H    Specify the max number of retry attempts for WebHDFS client,7 q7 R/ O5 y0 X6 j* R6 H/ U$ U( b
    if the difference between retried attempts and failovered attempts is
8 P- }2 d% N* D& d/ @; x8 Z) B    larger than the max number of retry attempts, there will be no more8 J. h4 t+ V- O5 s" _
    retries.
; Q8 F( @& d: y, I$ O( b  </description>
+ Y2 q. O! N6 {9 w) @</property>5 N6 J! n8 N4 L& k8 R
<property>
. I9 s+ s; w  n- r  <name>dfs.http.client.failover.sleep.base.millis</name>
9 P# j9 e8 A+ B0 P  u  <value>500</value>
1 C0 N! t' A5 P  `1 G2 c& D  <description>
3 W$ D* f7 W4 ]7 e0 S3 r    Specify the base amount of time in milliseconds upon which the  c# h7 l! r8 z  h- K0 F" w
    exponentially increased sleep time between retries or failovers
& X- M* q" l! O2 r    is calculated for WebHDFS client.0 G' C1 d$ a, o: ~, \% l; M
  </description>
# }: f! H' v  f. o2 h; Y( m5 G</property>
, O/ J* T" z$ ~; I9 n; p<property>  F. w9 ^' g  Y5 S) J
  <name>dfs.http.client.failover.sleep.max.millis</name>
0 g  s* [+ Q. ~' S; O9 m0 d- D+ l  <value>15000</value>
  ~! v" {6 |- K5 i  <description>
8 S2 E/ W  z! [4 G    Specify the upper bound of sleep time in milliseconds between; ?4 w# t7 v( o% d* R1 l4 ?
    retries or failovers for WebHDFS client.9 @4 {* z  C: q( `
  </description>
+ i. c. ]4 {% i  o8 r: R( E/ ~</property>
+ v/ G9 N: H! H' Q8 [  Y0 W  b<property>2 n3 C: Q6 [0 o9 h+ \3 {% z
  <name>dfs.namenode.hosts.provider.classname</name>  }$ ]' u6 r) m* X
  <value>org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager</value>
' t; R3 L" x$ R, r; p8 P; Y  <description>8 ~# w) I% P$ P4 h9 [4 k, H
    The class that provides access for host files.
3 O" ^; Z; ~7 o3 ]7 A9 S  A5 B# I    org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager is used
' b! v9 q1 n$ V    by default which loads files specified by dfs.hosts and dfs.hosts.exclude.7 s3 w& }0 f  A
    If org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager is
+ R9 U- n; Q2 K3 h8 H9 u5 C9 b    used, it will load the JSON file defined in dfs.hosts.' `% q# [) |. c+ K  a/ S. o: j
    To change class name, nn restart is required. "dfsadmin -refreshNodes" only& I5 f! U! a9 H3 \
    refreshes the configuration files used by the class./ f* E: h# B: b. d" u" c+ A6 y
  </description>% C! i, x9 U: a. ^
</property>
  [% [. v4 T" ~9 e3 g, _<property>, i5 v" _& s9 z5 X* ]& U% }+ T
  <name>datanode.https.port</name>
. \0 x, y2 z2 q6 u) Q9 c* v$ D  <value>50475</value>6 [4 `2 r# B; @
  <description>, z1 @$ Z6 C  g4 _' S, l
    HTTPS port for DataNode.
) F7 x* d7 J8 F+ x) R4 i  </description>
. Y( Z, K" c# ]$ f0 k</property>
3 y- M5 K+ Q# Y, i<property>
9 T3 e  G) A" y7 V+ ^: x) p  <name>dfs.balancer.dispatcherThreads</name>
) \: }7 d9 j/ p3 d. G7 J. f  <value>200</value>5 P/ C. e$ w$ q$ |6 j( ?4 q
  <description>
6 p2 L4 Z, i6 h$ c: j; C- L. a. F    Size of the thread pool for the HDFS balancer block mover.
2 D% _# Y( }) |    dispatchExecutor0 M9 o. ~, f/ j$ d; V. k3 M
  </description>9 Y0 |: K% `3 c5 c( J) {% p5 }
</property>; @7 k/ i- d7 i: p0 m
<property>
7 A. K2 [& M5 G4 E9 h  <name>dfs.balancer.movedWinWidth</name>
/ z5 J; f" w0 [# `5 `/ \  <value>5400000</value># e7 @$ J  O( {0 X  j5 S7 m
  <description>
/ }5 ?- X4 Z. `6 r" H5 D    Window of time in ms for the HDFS balancer tracking blocks and its$ O( d! `1 F( [+ w8 b
    locations.
, ]: N* c/ u# X3 ]5 X- S3 O, _  </description>
' U5 c0 ?( X6 X- L1 E# e</property>8 c: `' r% h" ?% k: M
<property>
/ ?9 e# Z# O( v' _  <name>dfs.balancer.moverThreads</name>
0 o* H) x0 h$ n3 [! F6 j  <value>1000</value>
. m0 z. I) h- O' e2 s) N6 O  <description>
, p$ j- C4 y$ s+ X! P% Q- P    Thread pool size for executing block moves.
$ N1 h; G3 `5 J# D1 D9 r3 {" G    moverThreadAllocator& M. R! H) B! o/ \* `# M
  </description>- v( l& ^* m; y2 r  M1 U
</property>5 d+ E! h  {) d  C( G
<property>( l6 b& d9 A. ]1 Y5 \4 B
  <name>dfs.balancer.max-size-to-move</name>0 g! E5 [& u& ^: Y
  <value>10737418240</value># N6 y9 b  [0 _% I- c
  <description>
" ^8 `  r3 ]/ ~' R( O+ L    Maximum number of bytes that can be moved by the balancer in a single
7 P# c! c5 A' a. @6 I: _    thread.
# V& S0 a4 V' I2 l: u6 i3 r8 O  </description>
4 Y9 F/ n4 k1 l/ u0 |</property>8 k; F* C3 N  O! D
<property>: x& K9 e. y9 ?; o
  <name>dfs.balancer.getBlocks.min-block-size</name>
0 l6 Y, s- m$ P; W( V  <value>10485760</value>
5 M/ d0 G; z0 Z! r5 J4 Z  <description>
* L$ L1 z# H: T" p  N9 E0 T    Minimum block threshold size in bytes to ignore when fetching a source's2 T1 x7 b4 _6 l, l- I( b
    block list.' c+ y+ @! C  u; d! |  J. X, {
  </description>7 m' X$ j( s( L0 F% O
</property>1 C9 r5 G) X$ j+ L) V6 _$ D
<property>
! d/ }4 Q. h' H. p. D  <name>dfs.balancer.getBlocks.size</name>
0 Z! ]4 P2 K8 n$ F. O5 J  <value>2147483648</value>- l2 G1 Y8 H% v# r- B; a
  <description>
/ H; Y* R. V$ g- f2 }4 O# b    Total size in bytes of Datanode blocks to get when fetching a source's
* y- E$ V0 B9 |. z" U. v; Q0 Q    block list.1 M& Y' v% l4 E1 O( q1 p& r& E) V; F, L
  </description>
4 T! p) T  U7 l: A! O5 c) C</property>7 T+ V7 L  ]# \7 N- P9 R7 }& c
<property>
0 H* o4 `0 K1 Q  <name>dfs.balancer.block-move.timeout</name>! y# G/ p) ^, U8 m0 m$ T
  <value>0</value>; v6 I. ~/ H  s$ ]9 o
  <description>
$ H* a9 z; Y: R* q* Y4 }' N; ^$ r    Maximum amount of time in milliseconds for a block to move. If this is set5 |5 H/ O5 }" K% k2 Z
    greater than 0, Balancer will stop waiting for a block move completion8 h, b) @1 {5 D% e
    after this time. In typical clusters, a 3 to 5 minute timeout is reasonable.
1 q% B" Z5 @! T8 k4 o" Q- \    If timeout happens to a large proportion of block moves, this needs to be
9 a5 g- o9 j& P8 l, |( W$ C, l    increased. It could also be that too much work is dispatched and many nodes3 k( y4 i$ d- t3 z- p8 Z9 s0 Y
    are constantly exceeding the bandwidth limit as a result. In that case,6 m. T3 W. \( K% L
    other balancer parameters might need to be adjusted.
# @. R$ A1 I' s: u9 T& u4 Z/ s6 ~    It is disabled (0) by default.$ ?$ V3 s7 K9 y+ U- s+ L( x
  </description>+ i* d) M6 k8 X. d
</property>6 G# z8 b9 }) V. J/ X
<property>
$ [$ `! e$ s8 Q# _0 P! B  <name>dfs.balancer.max-no-move-interval</name>+ I" l- Q- a% G3 U6 J0 [
  <value>60000</value>
9 s7 _$ Y7 N. \  <description>6 ]1 x, ^* v7 t0 R8 j
    If this specified amount of time has elapsed and no block has been moved6 y. g/ U1 T! q# W
    out of a source DataNode, on more effort will be made to move blocks out of
; M3 a6 X& {; N' F- y% A9 z    this DataNode in the current Balancer iteration.
) E/ [2 q! U( N+ F- N7 }' [2 e  </description>1 r' G% H0 {0 C  p
</property>; n3 U6 _8 p9 o5 {- b- ^! \% l
<property>
1 F& ?% P( ^% h, e  <name>dfs.balancer.max-iteration-time</name>
. o* D) r( v" W7 E  <value>1200000</value>* O9 e! A: u( Z: a
  <description>1 k# n0 C; x7 J( v2 {5 s
    Maximum amount of time while an iteration can be run by the Balancer. After
3 ?& L3 R7 `: O    this time the Balancer will stop the iteration, and reevaluate the work
! l) M4 T! L  b/ `: [$ s" V+ C    needs to be done to Balance the cluster. The default value is 20 minutes./ ?5 u4 c( ]6 X1 {
  </description>+ y& S. n# V  K6 g- v4 g. G" @- @
</property>
* h! o! X8 f. z9 l# v<property>
3 q; E9 v7 v3 s6 d  <name>dfs.block.invalidate.limit</name>
6 C# L% _7 U; F  V5 q  <value>1000</value>+ N; G& y$ T: X: m& R
  <description>! B6 ]+ I% g0 d7 X2 k7 U5 ?
    The maximum number of invalidate blocks sent by namenode to a datanode
. |& B" m9 ]' E8 L7 ?4 w    per heartbeat deletion command. This property works with' n+ U8 ]1 L( y' n/ l3 ]
    "dfs.namenode.invalidate.work.pct.per.iteration" to throttle block
" i$ g+ ?" i$ T  o, y    deletions." n! K1 ^9 d+ V
  </description>2 c. k/ n' d. `; M) q# t( V# n' Z
</property>
7 Q# b8 O. w' H* u4 }/ v4 p<property>/ y3 O2 K+ _5 Z; ?: W9 i. H
  <name>dfs.block.misreplication.processing.limit</name>+ s' n9 f; V% e$ i6 X* h/ d2 c
  <value>10000</value>
, g  [% v! P! G  @& ~! ~( H; K  <description>' u, C# ~/ y- t. g6 C# `2 _
    Maximum number of blocks to process for initializing replication queues.* f9 d& B* I! _3 I
  </description>
- B5 X: z3 {) s3 `- A! f8 G3 w</property>
- G. S- r; ]9 O4 i- e<property>
8 k9 O4 P  l$ p& @4 a  p, F  <name>dfs.block.placement.ec.classname</name>
4 p) Z& p% K- N( b$ k  <value>org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyRackFaultTolerant</value>
* @3 X' S" M* N  V# L5 }& r  <description>  ~4 t) [/ H$ e1 G
    Placement policy class for striped files.( a6 h& U* M5 Y- q+ [
    Defaults to BlockPlacementPolicyRackFaultTolerant.class; S5 _; g% h: M' ]
  </description>
) v* F8 _' f* o* ]' m( j</property>
( [8 L* Q5 R. p. P* ~- f<property>
' L  _' D: h0 q" P0 A  <name>dfs.block.replicator.classname</name>' M$ f5 ^7 K+ u! ~2 E: q
  <value>org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault</value>0 E/ y- P' q$ V' Q$ t: P
  <description>, q3 u% V( L% Q, s( T$ Z
    Class representing block placement policy for non-striped files.
) Y3 S, {( n* p3 b1 }& H8 d# i    There are four block placement policies currently being supported:
! }) u2 C7 y+ n8 H# V8 b    BlockPlacementPolicyDefault, BlockPlacementPolicyWithNodeGroup,$ n3 `! J5 C5 p, c
    BlockPlacementPolicyRackFaultTolerant and BlockPlacementPolicyWithUpgradeDomain.$ S2 S2 g* C. n: P" e9 d2 D7 y# H
    BlockPlacementPolicyDefault chooses the desired number of targets' x& s! U- q% x( ~& P
    for placing block replicas in a default way. BlockPlacementPolicyWithNodeGroup
% u4 i$ L% [* I9 S& F1 H5 z    places block replicas on environment with node-group layer. BlockPlacementPolicyRackFaultTolerant
! p4 K; T+ z6 T! Z    places the replicas to more racks.
% j' N. ]( A2 u" X! z7 `    BlockPlacementPolicyWithUpgradeDomain places block replicas that honors upgrade domain policy.1 I' `; y( m" u: i$ o% p6 k7 \& m
    The details of placing replicas are documented in the javadoc of the corresponding policy classes.
* H. q0 M) K' p& @( U    The default policy is BlockPlacementPolicyDefault, and the corresponding class is
: a. s4 X1 p# s  G    org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.
: ]- ?; }. r2 h! s- Z  </description>
$ I) L2 w$ a" M4 X: ]8 t</property>" ^( Z& a4 g8 C* e. p( Q
<property>0 U' s! b2 d, A1 J/ _4 U: {7 {
  <name>dfs.blockreport.incremental.intervalMsec</name>9 h) {$ R+ v  N
  <value>0</value>
8 u( `( ^5 m- Y  <description>/ K! A2 T. y2 l; Y! O
    If set to a positive integer, the value in ms to wait between sending
: u/ L1 W6 C  [$ P5 B: R    incremental block reports from the Datanode to the Namenode.1 @) L: i& r  Y6 d$ c
  </description>! x$ m# t1 \6 x- p6 M
</property>% C" h- x/ {6 E2 H  y
<property>
5 Q! a! h* F  ?' O  <name>dfs.checksum.type</name>
. |" f6 J& |! Z: m  `& ~3 v- N) P  <value>CRC32C</value>
  I7 S. X4 B% ?/ i  <description>% P) `4 z& M7 c4 C1 r0 G$ t
    Checksum type/ b% j* T# L- `$ ?" k
  </description>8 E. X  ?# G& p# o5 _5 |- X
</property>
5 l9 P- x( E2 N  f) a) B. T9 m<property>
, r# x  D( V  r, Z$ _; r5 D7 E  <name>dfs.checksum.combine.mode</name>2 m% c" L" _$ f2 L
  <value>MD5MD5CRC</value>
7 n9 y, I( h0 q: t  ~, ]  <description>1 @8 y9 X2 `3 Z5 g9 Z% [3 u( r
    Defines how lower-level chunk/block checksums are combined into file-level# q- z8 N# J& x7 P, ]8 R
    checksums; the original MD5MD5CRC mode is not comparable between files
0 o/ z1 f6 k! }- r3 a( E3 F    with different block layouts, while modes like COMPOSITE_CRC are7 K( `3 o" N9 U
    comparable independently of block layout.
3 Q7 d2 |$ Q/ c" r/ b  </description># Z! M6 f5 I) J) V0 H
</property>- C, ^, |- T9 |5 l" N
<property>
# U' d' Y* B8 C) l+ m8 h: W5 R  <name>dfs.client.block.write.locateFollowingBlock.retries</name>! {2 D% l" S- G) t8 M
  <value>5</value>
+ y4 z, e4 k1 j, H! b5 J  <description>& A( F( Q4 @4 t
    Number of retries to use when finding the next block during HDFS writes.8 ~+ U6 L+ Y1 M* M. i
  </description>
9 @9 C% k8 ^' G/ V</property>
; R% [! r. Y/ j2 c/ y9 b' A<property>4 M  {- e! f" v- c: i6 b
  <name>dfs.client.failover.proxy.provider</name>7 \$ ~: z7 q+ k* m4 ^  V1 M5 ~
  <value></value>  I2 N- o6 u2 ?/ P
  <description>0 P# L2 X/ S& o) C0 K
    The prefix (plus a required nameservice ID) for the class name of the
; Q3 N" i0 X$ E: C8 I2 S1 n    configured Failover proxy provider for the host.  For more detailed
4 R! F- U" a- p7 _& p. k    information, please consult the "Configuration Details" section of5 S  R3 Z. M, e& B
    the HDFS High Availability documentation.- [6 j* F1 y. G0 Y
  </description>
8 c) ~+ p$ S4 L- c6 {; _  F' p</property>
6 l8 ]2 E# H0 n) Q- l/ R<property>' u# s2 R9 F  `; ]& _, f
  <name>dfs.client.failover.random.order</name>. f- J3 ?1 i$ G! ?. U  S0 [
  <value>false</value>5 ^' b) J6 u+ M; }' t- m1 U5 R9 o
  <description>& n2 y5 X1 X% l
    Determines if the failover proxies are picked in random order instead of the: J4 T0 f6 @; j! f1 Z, f+ A! d
    configured order. The prefix can be used with an optional nameservice ID0 l# A; ]' w* j5 M$ c
    (of form dfs.client.failover.random.order[.nameservice]) in case multiple8 N0 \) Z: }" v
    nameservices exist and random order should be enabled for specific' f! V! |/ D6 p$ e: {' s! R! q
    nameservices.
" V2 j3 C/ u* ?3 {1 I+ G  </description>
9 V( V3 M* E9 m' s3 |</property>
1 }, g- d0 X3 B$ O8 i4 D; k<property>
4 f% X1 ^4 _0 w  <name>dfs.client.key.provider.cache.expiry</name>
0 m( N. d) B8 i7 p: ^  <value>864000000</value>
" r1 `% O! O1 [5 w/ e4 ^  <description>
4 ?( E, y; Q0 _3 E& d    DFS client security key cache expiration in milliseconds./ A6 e  }7 }; o6 I1 A
  </description>. C2 Q& {9 K2 |
</property>
: }7 y* N$ x) ?<property>2 S5 j8 ]# n6 X  |! ~* u1 X4 t; |
  <name>dfs.client.max.block.acquire.failures</name>
( H% J, p6 V( h8 Z: y& B1 N$ ]  <value>3</value>2 j! Y. f  v+ `+ Q
  <description>
5 }5 b( m8 V# a% N% c! i    Maximum failures allowed when trying to get block information from a specific datanode.
& N: k( E7 @" G8 F4 h  </description>
: r, ]( e  @! I, y: |- N' s</property>. {7 i0 p* b- c
<property>
9 m9 [" O! F+ l  <name>dfs.client.read.prefetch.size</name>2 `1 D1 n' b6 f
  <value></value>
* H6 e1 D+ @! s, q8 H2 i+ \  <description>
! H0 G2 l6 ]  n) A( \- |% \    The number of bytes for the DFSClient will fetch from the Namenode
) q: h0 l* p: x1 L    during a read operation.  Defaults to 10 * ${dfs.blocksize}.
7 G- f0 a  i$ T' d" e/ ?  </description>
4 N6 Z) p$ {2 ]/ N" u) f0 J: S</property>
6 m# x( B" o: u  `' \& ]* ?<property>
; i: k! O1 ~' x  <name>dfs.client.read.short.circuit.replica.stale.threshold.ms</name>* T, a9 }3 f/ g6 g7 Y' \0 C9 S
  <value>1800000</value>0 i+ E" j; ?9 w# K
  <description>
5 M- C) u: w6 Q    Threshold in milliseconds for read entries during short-circuit local reads.
' y3 B7 a& F% {" L  </description>
6 K6 g9 V6 z' c8 f% b</property>
! L5 |% [8 h+ u( z0 V<property>
9 X$ j' B9 k, M' @, H  <name>dfs.client.read.shortcircuit.buffer.size</name>
/ F- Z* R) c. ?7 v, E5 N/ a  <value>1048576</value>
' p, C" k" y# C; A: \$ f! ?  <description>
% T% P3 \0 `5 r! L    Buffer size in bytes for short-circuit local reads.
# z: _& h8 |; c: S/ ^4 T; f( |  </description>
7 b& O* S6 z3 f8 t2 a</property>
: m5 [- e  C2 l1 \& z<property>$ ~2 @. c! a6 j0 o9 @
  <name>dfs.client.read.striped.threadpool.size</name>
( `8 @4 a9 F, g# R& B" v6 I  <value>18</value>
( Z2 Y. o* E& L  <description>. ^  p, o9 \6 U/ x, Q) ^; B
    The maximum number of threads used for parallel reading
# v! J. s7 V9 O" F( k( L$ I    in striped layout.
4 N2 F- l, I& P7 X3 \  </description>
* D- c" w% X6 [7 U. p( k</property>% N* {  @  n1 d# `5 |, q
<property>
$ e7 X" M2 H, k0 D/ n& J  <name>dfs.client.replica.accessor.builder.classes</name>
8 `/ x% m) r4 j6 [5 @  <value></value>. c0 C& g: T4 Q- }+ H
  <description>
/ O- |$ C2 B  K* _# d9 e2 @    Comma-separated classes for building ReplicaAccessor.  If the classes# m$ X" D3 I; Y7 F
    are specified, client will use external BlockReader that uses the
* L* B4 d/ P2 y$ {    ReplicaAccessor built by the builder.) I* m4 P  l" f! o$ ?, J
  </description>- e9 O6 `4 ^* q
</property>
" L; A3 a3 G- p$ ^3 x<property>/ H: O- C( Z' H0 ?
  <name>dfs.client.retry.interval-ms.get-last-block-length</name>
* y; K. c9 [) v% `+ Q: ?; r  <value>4000</value>6 w5 z/ b- X& h, i& r; \0 b1 K
  <description>3 |% Q  s" Q: G  C* h1 t( {
    Retry interval in milliseconds to wait between retries in getting0 \7 H+ V& @- X$ D* v- I8 @7 e  N
    block lengths from the datanodes.; A( c' p8 L) o2 s. F$ p/ c8 u2 s
  </description>$ O0 e% N2 o. D  [+ d
</property>
& j4 v  @, Y  {) K<property>
# M0 ?  V% Q: k! _8 h  <name>dfs.client.retry.max.attempts</name>
$ q6 G; _; ^$ [5 Q  <value>10</value>- P. @$ N3 g; e, L) D% M& D
  <description>
4 e' b/ m$ w; t7 L( c  h9 q# u    Max retry attempts for DFSClient talking to namenodes.
* B$ i- {* p. S4 d. b: B  </description>3 T  |4 O' A1 X9 ~- X4 r6 `, b. f
</property>; _/ J' m2 \8 y- `8 b" @
<property>
& F$ b, c7 b* R  <name>dfs.client.retry.policy.enabled</name>
5 I: A+ S. \3 w6 H! a  <value>false</value>
4 D; J8 J2 ~4 |: e0 M% t) L/ |  <description>, x% W  R$ d8 W1 ~
    If true, turns on DFSClient retry policy.# n, L1 g' S/ U7 i% G4 f$ C0 Z
  </description>, {, C5 e* J/ r( B4 [9 t* I
</property>
6 Y  A7 L" t- x# O' F% G- d<property>
# w. j+ V; J) I& r& l; s" X  <name>dfs.client.retry.policy.spec</name>" G# b1 g0 t) p" j2 q8 n4 j
  <value>10000,6,60000,10</value>
0 f4 g- S; W! ?+ r$ c  <description>2 ?9 Z, S1 @4 n& R
    Set to pairs of timeouts and retries for DFSClient.5 l, v+ L: |9 x8 j' n
  </description>
8 k# G' o  O/ l</property>
4 [+ b2 h  ?! i4 g% f3 m<property>
" `: j9 u, y% H% `% a  F0 m/ p  <name>dfs.client.retry.times.get-last-block-length</name>; `- m+ c3 }$ y7 D/ b
  <value>3</value>% f6 a/ Y/ g0 u5 u) `
  <description>
' m, g. T) m) Z. D) {8 k& y. V, G    Number of retries for calls to fetchLocatedBlocksAndGetLastBlockLength().: n+ [/ [9 K! Q4 ?1 g- [2 m
  </description>6 J3 a( g& N  N2 {$ i
</property>
( u2 k+ H+ H# ]) [! n<property>9 X" E% A5 R+ V
  <name>dfs.client.retry.window.base</name>6 n8 a/ N* T+ u- D8 ?- X) X% N% r
  <value>3000</value>
. }$ ~- n2 a- s  <description>7 w2 V( T& b6 P  H  h
    Base time window in ms for DFSClient retries.  For each retry attempt,7 F1 q  Q* q% N6 ^. R
    this value is extended linearly (e.g. 3000 ms for first attempt and
+ k$ L- U7 h% X" W& [    first retry, 6000 ms for second retry, 9000 ms for third retry, etc.).
% }" |% ?" `/ N, D4 ?6 W' D' m% y  </description>; H. V5 b' Q/ @# h9 N9 S
</property>
* {/ V* `- P, z" T6 b% j+ ~7 h- Z<property>9 }( {1 M) ~# m) ^$ d7 J2 y6 g
  <name>dfs.client.socket-timeout</name>
3 l$ N$ o$ d8 W8 B4 ]  <value>60000</value>9 k, @, z/ N/ h* _4 A$ ^
  <description>
( _# |  X5 h3 ]% f. J5 |; Y    Default timeout value in milliseconds for all sockets.% |- s; y  f8 f& H/ x# j
  </description>6 q" J; O, e' H
</property>7 a5 ~' r0 J, a) J0 j6 @
<property>
' o( P8 Q3 T' C$ K6 m  <name>dfs.client.socketcache.capacity</name>! F7 }" x( f7 U( x# U5 ]
  <value>16</value>
1 q* b7 m. j4 I% t+ d. Q% S6 f  <description>7 V# C! [. d, t* ]8 j
    Socket cache capacity (in entries) for short-circuit reads.1 k( y6 m; A- p! W
  </description>
& u# d8 c0 G. F& J' g( m2 ^) P</property>
4 H7 z. T0 E! J6 L) z<property>3 w( T9 m( l1 A7 h( a6 Z$ D. ]
  <name>dfs.client.socketcache.expiryMsec</name>  ~* v/ Q3 ~2 m
  <value>3000</value>
+ |% B1 C# T9 @  <description>" W% _$ w. W& Y  p0 O9 g/ n
    Socket cache expiration for short-circuit reads in msec.. D: \% s$ _- ]7 c% j
  </description>5 b4 D+ l6 F* j, d
</property>; U3 Y9 J  X! M3 s" u5 n- |
<property>
- ?$ w1 b' I# n: J. [  Q8 y( B  <name>dfs.client.test.drop.namenode.response.number</name>
5 w# q! u! F( e4 P  <value>0</value># t1 t& K0 u' e3 i" ~
  <description>
6 J7 Y0 e- h/ w; H* Z7 R. _    The number of Namenode responses dropped by DFSClient for each RPC call.  Used9 k6 z( c, i0 d" {$ u
    for testing the NN retry cache.% X- E4 E- P& A
  </description>4 F" [: O, P( h
</property>; I9 }0 a# D9 P8 y
<property>6 u3 b$ `* g9 q9 l
  <name>dfs.client.hedged.read.threadpool.size</name>" x2 x& v2 Q  h6 t8 n
  <value>0</value>
5 i; m/ p: e8 S$ I  |! W  <description>  e/ Q4 H% q8 v' v" ^1 B0 N  N4 w$ s" O
    Support 'hedged' reads in DFSClient. To enable this feature, set the parameter* ~( [: p' J4 Q8 \$ _
    to a positive number. The threadpool size is how many threads to dedicate
4 V; ^( V8 A* p4 k    to the running of these 'hedged', concurrent reads in your client.
1 Y& [2 D0 K" }  </description>
( m& Z2 N8 o2 Z3 a1 I</property>4 i6 _1 V& ~% N7 e6 V, H: q6 n3 M
<property>
* F7 R0 q! B+ h7 o$ d  <name>dfs.client.hedged.read.threshold.millis</name>
. f+ D4 h: C( h  <value>500</value>  N6 K; Y3 ]3 ]/ u; ^) C6 \
  <description>
: m* @/ k* F1 }% s& Z    Configure 'hedged' reads in DFSClient. This is the number of milliseconds
1 g8 w( j" O9 x% c6 j! H$ m' [9 X& f    to wait before starting up a 'hedged' read.
& a' p- d' \) O& P! F' V  </description>  Y7 r7 w9 |( J, O9 _+ m6 W# h
</property>( N5 E# J. Y; s* D9 @7 l
<property>% j7 p; @6 c+ C; b+ u
  <name>dfs.client.write.byte-array-manager.count-limit</name>
' s6 t( m: Z  j3 G+ S- @  <value>2048</value>/ o7 s: K- }2 q( D: d
  <description>
$ D) `$ S; f6 C) m" y# D: l5 w    The maximum number of arrays allowed for each array length.3 P9 A  s* C) C- {: r( Z* z
  </description>0 Y: o8 ]! C+ z( d( [9 H
</property>( n' J7 k0 l3 A
<property>
& |" F: R( Q# w: i  <name>dfs.client.write.byte-array-manager.count-reset-time-period-ms</name>( H& p8 O; z8 w0 j2 Y* S& w/ W
  <value>10000</value>
% {, Z( f" z) l+ l  <description>4 X' B9 w: A2 j* P( z7 k7 Z& y
    The time period in milliseconds that the allocation count for each array length is
+ i3 k3 U! l6 |! r: u3 F& e3 x- Z, g    reset to zero if there is no increment.  x& b1 H+ Y4 ^+ _0 t- n" C7 I+ Z
  </description>
. ~# ]: v0 d2 |2 z# W$ G3 D0 u* {</property>4 q* c  l, Q  M+ V( x
<property>
) U/ s2 f) c' c2 E- `  <name>dfs.client.write.byte-array-manager.count-threshold</name>! T' c) l% j+ O  I( J) N5 x
  <value>128</value>
2 Y7 l0 p1 O- o0 t0 d  <description>* v: C( N* a; [* D
    The count threshold for each array length so that a manager is created only after the& S  j* T0 `( ~4 ~
    allocation count exceeds the threshold. In other words, the particular array length
. T$ P8 O$ N3 M, R' l/ ^3 q    is not managed until the allocation count exceeds the threshold.' y- {6 ~; Y4 O5 w% c# l
  </description>% K8 e' H( j. |  @2 o/ S
</property>% x) [3 R0 ~- D& l/ `0 B
<property>
3 S: ]! h6 o. G$ ~# q+ @: _  <name>dfs.client.write.byte-array-manager.enabled</name>; c* n) o  v1 k5 @3 J/ B% f
  <value>false</value>
  L7 {6 M3 l# W& G1 w- C/ K/ p  <description>
" E6 ^+ w* N$ B& b3 g  [6 t9 R    If true, enables byte array manager used by DFSOutputStream.4 N4 [: ~1 @0 v7 Q1 d4 {9 J
  </description>3 L% J; K! p3 n7 t# t
</property>
! j1 E& M! Q8 {* I3 Z' y<property>
, U9 l# O7 O# U2 s4 M4 _& f' W1 t  <name>dfs.client.write.max-packets-in-flight</name>$ O3 N% z' n' q  x/ r& x
  <value>80</value>
7 N" L* J9 _3 Q* N) v$ J  <description>
- k- I7 P1 |) [/ p" O" M5 _    The maximum number of DFSPackets allowed in flight." Y, U: B% }3 C% g! u
  </description>
; c! S$ _. U2 }& A+ p  ?0 A$ g</property>+ I3 t: L$ W# ]" ^
<property>' u( R+ U9 Z2 x) R1 c  |) b' L
  <name>dfs.content-summary.limit</name>
3 ]. {9 o. v$ f0 c* |  <value>5000</value>& m, r. P- n# ^! w$ J
  <description>/ a9 Y2 m# n8 @6 T
    The maximum content summary counts allowed in one locking period. 0 or a negative number
( n2 p$ i5 k. X! x, r, X2 B    means no limit (i.e. no yielding).
# ?& a/ G& A1 w' y  </description>. E" v* p. J8 U  |3 X
</property>/ R# j0 A2 J, c" y7 a' @
<property>
! v( Y- S' c5 g: K8 a) _1 S. B  <name>dfs.content-summary.sleep-microsec</name>
, v8 J! ]4 ?; q) m' e  <value>500</value>
( O. h$ b) q; Z( o; b8 ?  <description>
2 R0 A" ~" X9 D9 K% q    The length of time in microseconds to put the thread to sleep, between reaquiring the locks
: \% w; H! y6 y' i& r    in content summary computation.
0 j7 _0 C& M6 E! _3 B  </description>
" l+ p" I  r7 I/ J1 y! }. r</property>; \4 {: `" U. s/ w! B4 r
<property>
& r8 c, o$ Z* K# {  <name>dfs.data.transfer.client.tcpnodelay</name>
6 U: l6 l& S/ f% u0 u; g* j  <value>true</value>
8 A  @  N7 f) q% b" f# @2 w0 ?  <description>' |2 ~$ }, ]; t* ^* Y- m1 \
    If true, set TCP_NODELAY to sockets for transferring data from DFS client.
) p9 ^/ f" f8 P+ U9 ?1 l$ f/ d; O  </description>5 {. _$ e% Q" b
</property>
7 U% W( P6 b! E: c<property>% j4 |# L$ k. p( X
  <name>dfs.data.transfer.server.tcpnodelay</name>3 w1 m) m& [# w. k: b
  <value>true</value>
! D9 y3 l, }+ o- f, x: Q- A" t  <description>" K1 C2 G; x" a  {! b# p6 g
    If true, set TCP_NODELAY to sockets for transferring data between Datanodes.
1 ?3 t$ @/ f! T  </description>- C( b2 I3 P5 x' K4 D
</property>
: C/ o7 b; o4 F2 h) p, o9 O<property>
2 S3 u9 \( m0 m" w9 ^$ T% l2 f  <name>dfs.datanode.balance.max.concurrent.moves</name>$ Y3 s% |1 |( ~. o* p
  <value>50</value>$ ]5 @7 D0 x* }  d4 n
  <description>8 Y3 _& X) f! V/ @& ]
    Maximum number of threads for Datanode balancer pending moves.  This" n# `9 Z0 W5 z% ?0 e1 C) W# M
    value is reconfigurable via the "dfsadmin -reconfig" command.( y/ S7 D7 e4 u6 w9 Z2 H/ L7 |. x
  </description>& Q5 p$ j$ J; M$ C; U
</property>
' a! J$ t5 [3 p. B' w6 h<property>
( ~$ s6 {) B5 S. c5 C# Z) c' I  <name>dfs.datanode.fsdataset.factory</name>
$ ^! b* [4 T" U( C! V  <value></value>: q! l" f, G0 \" o  w
  <description>
: h5 q  q9 P8 H* {- C% V    The class name for the underlying storage that stores replicas for a
# ?1 f& V: Q+ X% N    Datanode.  Defaults to
. a' f( o. N/ Z9 c# j; D% A    org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory." l. e3 B0 a( p8 T0 R% S
  </description>
4 v- M2 P) j2 H* t</property>
9 M7 {. c2 h- \( j<property>
  Y, W% ~9 F1 ~* x. I* o  <name>dfs.datanode.fsdataset.volume.choosing.policy</name>
$ ]3 [  k- i; ~$ n/ d  <value></value>
- }: {* Y9 b/ ]$ \1 N  <description>2 t1 Q2 e( }4 V: y7 u4 g
    The class name of the policy for choosing volumes in the list of
4 Z9 x* V& D+ M) g& F1 [0 i    directories.  Defaults to
: q4 b" p& U# k+ Z/ ~    org.apache.hadoop.hdfs.server.datanode.fsdataset.RoundRobinVolumeChoosingPolicy.
$ @& d/ w$ ~6 n, \. l& U" J" u$ h    If you would like to take into account available disk space, set the2 n! Y" L. D! p7 e
    value to1 c% @4 V6 ?- X% q3 f# P% E% L; R
    "org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy".; e2 a1 m7 ], t, \
  </description>
& M, E& r; w) l2 D$ ?</property>
  W: G. X0 t- w5 W<property>
$ x. c' p4 d% D; x6 y  <name>dfs.datanode.hostname</name>
' |; D" @1 S7 P' Q! _9 Y. e  <value></value>
2 M8 l4 X) V$ j3 z7 E  <description>+ ^5 A& h9 T/ {8 T1 ~# H4 f
    Optional.  The hostname for the Datanode containing this
7 E; d* V. F: {% c3 i# D% L    configuration file.  Will be different for each machine.* f7 ~- B( |" T
    Defaults to current hostname.
3 }$ l" @9 F. {* m+ o) {  </description>5 ?. A& R9 x4 I! C+ @
</property>
8 x4 {$ L3 c5 b* G/ p) o<property>7 W. i1 U9 v  S: z8 q% T) W" v( n
  <name>dfs.datanode.lazywriter.interval.sec</name>; t- e, x! W+ N) P" n* m* f1 S
  <value>60</value>( W' Y3 X& N+ e) D% l3 @
  <description>% h1 L* z+ d3 v# ^
    Interval in seconds for Datanodes for lazy persist writes.
4 v. m* J+ H2 T$ t2 o% h% a  </description>
- q$ v! r! o8 c, k; i5 M/ t</property>
& y% H! o/ ^% e" m3 z/ b<property>: O- G, g3 d* ~( j9 [
  <name>dfs.datanode.network.counts.cache.max.size</name>
, [2 {4 B4 U9 G" j7 N0 ~  <value>2147483647</value>
/ n5 \5 L- }8 l2 ?  <description>2 k  z, W- ^2 V
    The maximum number of entries the datanode per-host network error: E% M% Z# ~! }3 v* M
    count cache may contain.5 W* O; ~: w8 Y0 p" e
  </description>( p" @7 x6 ^9 d, b3 x
</property>5 ^7 G# i7 Y4 {; o1 N
<property>1 ~. z  c$ P: J, H( T: }: Y2 K
  <name>dfs.datanode.oob.timeout-ms</name>
- G  _6 g# ]! W, M% J5 d/ e  <value>1500,0,0,0</value>
4 f% k% b# O/ k* L: R7 k' R  <description>" j, Q' C$ z* s$ z  E. d1 i# p' d  }
    Timeout value when sending OOB response for each OOB type, which are
. C, @" r# H9 b& d5 ?    OOB_RESTART, OOB_RESERVED1, OOB_RESERVED2, and OOB_RESERVED3,
  Z' r. ~! I2 D* |% N* ~    respectively.  Currently, only OOB_RESTART is used.
- t* `2 U5 g5 h' A$ ?0 T  </description>! s1 I' ], C- B
</property>
7 c7 N" F, _5 w$ V: M: m<property>
) T! B3 s$ \. P) g  <name>dfs.datanode.parallel.volumes.load.threads.num</name>: S. ?7 j6 v" i* R7 `
  <value></value>
; l( s6 C4 n+ ?% f; I# F8 W: A  <description>1 a# i" k; a7 `& v( G" L
    Maximum number of threads to use for upgrading data directories.3 ?2 ?2 |  R0 R; P
    The default value is the number of storage directories in the
. F( Y8 l: e' T; ^    DataNode.
1 `$ H* |7 o5 f) z* Z& e  </description>
- S/ c, @* j  ?7 U$ s; l</property>) B$ w" a$ k0 M) `0 t" ]
<property>
6 u" W9 G. A9 G& \# C2 J  <name>dfs.datanode.ram.disk.replica.tracker</name>
4 h. S: m& O+ U' K  \  <value></value>
0 |4 c( [" ]+ W2 Y& t- ?  <description>, r* C) b6 p! _5 \& @9 a7 [8 T2 K
    Name of the class implementing the RamDiskReplicaTracker interface.
& R% A5 b# K' A    Defaults to* L% m9 c0 S! W
    org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.RamDiskReplicaLruTracker.
; ~" i5 L; `9 o1 }8 y  </description>
; G( s, _: e* S& U- u5 _' y</property>* w* h8 v3 ~( M. @$ _' H
<property>
9 j4 F' U& }* A/ j  <name>dfs.datanode.restart.replica.expiration</name>: h9 |( }* R: F0 J5 X
  <value>50</value>
9 C" P$ C4 ^4 h4 d  <description>! Z' B3 ?6 U( d+ t& l# ~: a% \
    During shutdown for restart, the amount of time in seconds budgeted for
1 q/ ]7 ^$ I. R1 i' C# Y4 B    datanode restart.
% n9 o* E1 Q8 u' D+ C5 t8 L  </description>- j" ?. n5 M& y2 B7 }
</property>
; ~9 o" q. a4 d  `# T5 I$ y/ z2 K<property>$ o8 }* j8 g  n( K, v
  <name>dfs.datanode.socket.reuse.keepalive</name>
4 y4 b* j( ?; w, f6 u/ d0 d2 B3 m! F" }  <value>4000</value>. ~" N5 V: i3 z: a0 P! E2 ?' F
  <description>$ [- k, L% C, ^+ q
    The window of time in ms before the DataXceiver closes a socket for a- x9 _, `' O  B. W1 D1 A
    single request.  If a second request occurs within that window, the# A8 s& I8 H) N, R7 v+ a" y
    socket can be reused." y0 K# j8 o& {5 m
  </description>1 c) I% m0 S( ?# g& u9 I
</property>* e0 n# b! J; w+ ?8 L8 }
<property>; T- P1 y4 B) e
  <name>dfs.datanode.socket.write.timeout</name>
4 I9 P- L* X/ V, G; D  <value>480000</value>. K6 N0 J0 k; J. r* `( s
  <description>
8 a+ x2 \4 `6 Y1 y    Timeout in ms for clients socket writes to DataNodes.
1 ?+ r: G; |) J$ M  w8 Y  </description>$ M* E5 B2 a1 d1 i# [$ z
</property>1 p% }, f! C1 j2 P! X
<property>. R6 E: T  H. D, s
  <name>dfs.datanode.sync.behind.writes.in.background</name>
( q% T% A! T# u/ Z  <value>false</value>
$ l  O7 u2 c8 ^  ?7 P  D4 B4 Q  <description>6 i- n! j/ s# F; \5 ?- D3 `" A
    If set to true, then sync_file_range() system call will occur5 v8 z& K$ V0 N: Q' ]2 D# X* d
    asynchronously.  This property is only valid when the property
, s, K/ ]: `6 T  }    dfs.datanode.sync.behind.writes is true.
2 Q8 a, s- m( ]4 c+ F  </description>+ R' D9 V  @7 ~* N" `- V
</property>
. g8 b# S3 ~3 |- q8 B' D9 F, I8 g<property>6 X& ?+ {7 E/ u' F; G
  <name>dfs.datanode.transferTo.allowed</name>
; E8 G& n# X8 m: y  <value>true</value>
9 W! c6 P7 Q  y- m/ A  <description>% U# _. y" ?5 l$ z8 W- c4 X! S
    If false, break block transfers on 32-bit machines greater than
( T7 I5 Q  Z- {2 x    or equal to 2GB into smaller chunks.
8 o7 ~$ G- M) K+ K  </description>9 x+ s  u4 M1 S, k
</property>
8 p1 p8 p0 P' e4 a4 n<property>5 a( Q' t( O7 g7 a0 a
  <name>dfs.ha.fencing.methods</name>
/ E: A( H1 v7 ^" q' ?5 E* C  <value></value>
5 E/ K$ z" X, v, [. {  <description>
, ~3 S( e9 U  k    A list of scripts or Java classes which will be used to fence
7 J2 W0 a- T" v    the Active NameNode during a failover.  See the HDFS High
; ?9 O3 C: {4 L    Availability documentation for details on automatic HA
6 a7 C* b3 H' A" T4 H    configuration.& \2 j3 K: p. W$ m) Z' t
  </description>$ k3 d: ]8 i5 m5 E' ]; N
</property>2 e( d2 b' `- ~, X/ ^
<property>
5 c( O' ]% ]8 m8 F/ G2 f  ]  <name>dfs.ha.standby.checkpoints</name>
0 X, F- o* _% R7 _# m1 P- I  <value>true</value>! y' H- y1 Q3 F4 V/ n# q
  <description>$ a6 Q3 T) R# c( _$ l
    If true, a NameNode in Standby state periodically takes a checkpoint
8 W  i2 f0 Q' A% L# O" R    of the namespace, saves it to its local storage and then upload to, e% j9 ^# H: G' {: L
    the remote NameNode.0 o6 M9 m& H% D0 C8 U
  </description>
' G5 r& u% ?, U1 B0 s( r0 j# W</property>- D7 _  j1 A" E" M! M, @
<property>
$ e* E0 h3 @4 M4 S" M3 p5 m  <name>dfs.ha.zkfc.port</name>
7 r2 X1 `0 U9 o  <value>8019</value>4 X% c  a8 Z" [* N. J
  <description>3 @2 m" L* [. O& w% z
    The port number that the zookeeper failover controller RPC+ o( a0 w) H" l# V* @
    server binds to.1 t1 m! I9 C7 z' h7 ~
  </description>$ m2 g# `9 U/ ^: E+ v6 P" X! `
</property>) Y% c( B- t+ }) x' _8 q
<property>& ~/ z9 T( N' M# z. u  ^: S) N
  <name>dfs.journalnode.edits.dir</name>2 K$ a# G. A/ U9 s/ S
  <value>/tmp/hadoop/dfs/journalnode/</value>) \" k) `8 b; U7 W, D2 ]/ T
  <description>; p, {* X! Y# l( h
    The directory where the journal edit files are stored.) B( y9 |" E/ g
  </description>4 J6 A. P$ ~" O
</property># }+ }! H7 |5 Y+ h% B
<property>; |: w" ], o7 z5 z3 o; X
  <name>dfs.journalnode.enable.sync</name>9 l5 l; n) R0 P1 N5 p1 l- R9 E. N8 J2 o
  <value>true</value>% F  ?# i7 b) V6 [0 P* X5 @$ ?
  <description>
0 c0 Y- {' j) s# v    If true, the journal nodes wil sync with each other. The journal nodes
3 H: K# T8 T; o* i- S' E8 P    will periodically gossip with other journal nodes to compare edit log
  v  \; p7 L  ~7 X9 W+ v8 A    manifests and if they detect any missing log segment, they will download1 T; p* O5 e2 f
    it from the other journal nodes.
/ W! M9 Q5 Y8 Z  </description>
1 a% u: a- Q  R8 \4 T- h3 n$ C! c</property>0 D+ U5 p7 q/ o, O& H2 P; ?5 a8 T
<property>& D4 }, |3 \* h1 r: ?2 Z3 h
  <name>dfs.journalnode.sync.interval</name>0 n* n. o; s  T4 {. h7 @
  <value>120000</value>+ p; z8 P- C! T$ Q+ S% r
  <description>5 D& s+ x! B- s! f. X' d& ~% L
    Time interval, in milliseconds, between two Journal Node syncs.4 T2 A( ]( b1 M( `) t; @
    This configuration takes effect only if the journalnode sync is enabled
; R8 O. v) |/ S    by setting the configuration parameter dfs.journalnode.enable.sync to true.
9 c; y9 N: J8 O  </description>
1 _* a8 Z0 S0 t* ^3 d& x</property>& M  I/ F/ X* x) S0 a  \" D
<property>
. z; v; h4 f7 j0 Y0 V  <name>dfs.journalnode.kerberos.internal.spnego.principal</name>$ i6 L; w/ r" X
  <value></value>2 Q, b4 c# K; H8 `
  <description>+ p2 h) q7 O# u2 ~
    Kerberos SPNEGO principal name used by the journal node.. Z5 I9 f( O' r: U
  </description>3 j/ W* M2 h5 E4 C; Z
</property>3 ?8 ?4 |, v& g
<property>
8 |1 x  R% Q9 Z# V# B) Y  t# d1 G  <name>dfs.journalnode.kerberos.principal</name>
5 r  Y4 E; Q8 R& V% q  <value></value>7 N, e8 E$ z8 H0 o, V, d
  <description>
( {# j9 Y$ @! f# R    Kerberos principal name for the journal node.
! c: V( ]( F4 @8 i  </description>8 p7 [1 |7 |1 t$ d# E! `
</property>
3 q* U, ?. R0 T2 @<property>
# q- S, c# r' s+ B) q  o  <name>dfs.journalnode.keytab.file</name>
& |. D! p: Z1 K. Z/ [" E, r  <value></value>
. P/ S7 }! a' e8 _/ k  <description>
  Z: _7 R7 s- \( {! t% G    Kerberos keytab file for the journal node.. w# d0 X% B$ E) U6 w7 ^! P! W
  </description>
# x5 ~, h3 f; r3 X& ]! X</property>
* ]9 L. W7 W. A, {7 r1 d9 \<property>
  I: {) D( K4 \* {' B5 K. k5 \  <name>dfs.ls.limit</name>7 @# q. D8 O. T0 t. @$ h, E
  <value>1000</value>
$ H) N- G3 N$ R" o8 A  <description>
6 D- c; V6 I# N* L: z2 ?    Limit the number of files printed by ls. If less or equal to
7 z" d3 b' G- m( m9 W0 ?    zero, at most DFS_LIST_LIMIT_DEFAULT (= 1000) will be printed.
; L- L7 A' J% Y: Z* S# a# P- R* S/ ~) L6 f  </description>7 g% F* \" X" p7 Q$ J
</property>
0 O8 ^- e' \' d% B<property>( d$ c, c* R# N" u( W7 v; D2 o- m
  <name>dfs.mover.movedWinWidth</name>9 r, X8 c, S! n# L) D3 i
  <value>5400000</value>
3 g; @9 {. J: I3 J. J8 z! Q  <description>  o3 W- ?6 a) I1 V, P; F- s
    The minimum time interval, in milliseconds, that a block can be
& l! x5 w. l* J' m    moved to another location again.' i3 b' {. w/ y' I$ w. ?& W# G
  </description>: [4 r! F$ u8 m6 E, r6 i
</property>; H1 @# M5 |% F. O
<property>, Z4 Y8 H5 P* m. x. m+ G9 L
  <name>dfs.mover.moverThreads</name>
( R" R& T) M' ]" P0 N  <value>1000</value>7 g3 u' u( l" N- O5 O4 `
  <description>
. R. a2 R& a8 S6 ]1 O    Configure the balancer's mover thread pool size.1 x: z# u0 {7 g* E8 u2 ?! g
  </description>6 {9 ]" [) U- b$ P" }# a: _+ k9 O
</property>3 F! d  x* ]7 E6 v! L% R' _
<property>
! f4 O) _3 p  {' X$ U3 w  <name>dfs.mover.retry.max.attempts</name>3 @' s9 L' `0 |! E! w: \5 I/ m
  <value>10</value>
% v, i3 N; B7 ?) k  <description>
2 N4 a8 ^- b0 D; J5 l* g    The maximum number of retries before the mover consider the6 T: ]/ D1 B3 U# D  e* w
    move failed.
) B% E1 N' N5 B& D4 [% ]: h  </description>- p: H' e9 @  M2 s
</property>
1 c% y/ y( N. n& i! l/ r<property>
% @, e5 N. W  [  <name>dfs.mover.keytab.enabled</name>
5 H& d; r. _  ^: i  j  <value>false</value>) w" a; @$ m$ `8 L, \; e
  <description>- ^% S+ N0 [7 f' {/ y
    Set to true to enable login using a keytab for Kerberized Hadoop.
2 q5 p$ o8 z4 g, z+ Y  </description>
. H, P4 e1 G- Y3 [  T+ s8 W+ q</property>
5 x' V, u7 U7 _1 M; Y/ b2 t<property># ^- T9 U& s) {6 F7 `4 Z
  <name>dfs.mover.address</name>
/ z7 l9 n/ h! ?# q  Y( Z  <value>0.0.0.0:0</value>1 P, l1 Q, M0 `# C% ?
  <description>
+ ?7 r- V( ~# ]- `    The hostname used for a keytab based Kerberos login. Keytab based login
9 W: L8 L% K6 T- @0 k* v4 n5 E; Z' n    can be enabled with dfs.mover.keytab.enabled.# }$ N' l6 A$ b* `* {7 G
  </description>
0 A& [; f5 V! y# P7 o$ {</property>
1 _/ x- K) M( s! i1 `) ?) B<property>
, W$ R. _! u8 c/ x7 u# D" C  <name>dfs.mover.keytab.file</name>
$ [, [* Z6 k8 a# J  <value></value>. }: b2 s4 g- V( r- r
  <description>
& a# ]2 B/ v2 Y; \1 t7 O; O    The keytab file used by the Mover to login as its! p, g: v4 Z2 P' N6 }
    service principal. The principal name is configured with
1 a1 x1 M: a( G+ r# M' \    dfs.mover.kerberos.principal. Keytab based login can be
' e1 [' Z$ |+ r5 |& d% _2 Z    enabled with dfs.mover.keytab.enabled.. U, {" S# a: [) Q3 ]9 C- l6 N
  </description>
- [8 i6 \9 c# k% q</property>
$ Z5 X" X2 F  [! o$ t+ y<property>7 f, B7 L3 t) f# d! M
  <name>dfs.mover.kerberos.principal</name>
$ \- @. r: v! ]  Z( e( i" p7 {  <value></value>$ M- b! _  B0 _
  <description>
2 y7 l/ N+ j1 O/ T5 H    The Mover principal. This is typically set to
- m+ }; p1 L( q) |$ s  V    mover/_HOST@REALM.TLD. The Mover will substitute _HOST with its0 P0 ]% U! N/ E' E
    own fully qualified hostname at startup. The _HOST placeholder
0 [8 V1 ]5 p# o- D' _7 b. q7 N    allows using the same configuration setting on different servers.1 G3 E0 h! j5 T0 C0 e0 @
    Keytab based login can be enabled with dfs.mover.keytab.enabled.
# ?# N# g* U5 j- h9 w, n+ U: x, E+ l  </description>$ M" V0 v$ J9 B4 R; `' z+ A: F) V0 r
</property>
# j" ]% j: E" L  N/ z; I<property>
9 Z# ?% Q6 x0 f7 [  <name>dfs.mover.max-no-move-interval</name>
/ ?! H3 a7 X0 N7 N* k" F. u  <value>60000</value>
. K, S0 M* Q6 v/ k8 Q0 E0 X9 }$ V  <description>2 b" N, \& c. f- D6 j# S
    If this specified amount of time has elapsed and no block has been moved/ |( Y  u/ g* F! C) k2 p3 N/ \
    out of a source DataNode, on more effort will be made to move blocks out of
% C9 V3 N/ m2 s/ a+ ^9 a# ?6 x    this DataNode in the current Mover iteration., }# A( s2 e! c  M# y/ Q% [4 b
  </description>& b& E$ y! r- |1 V
</property>
6 T! P, t: @. ]# O( e<property>8 L; `! ^, b% S! k
  <name>dfs.namenode.audit.log.async</name>( I$ N5 w" K1 f; o% \: Q9 `+ I
  <value>false</value>/ [" [+ G4 S; I6 C
  <description>6 G- @) I! P0 V  }, q
    If true, enables asynchronous audit log., ]6 {7 W: y: `, H8 E" k
  </description>8 ?6 ]9 E4 Z6 a
</property>
' l1 h" Q, Y2 {" a6 h/ b<property>" w) R9 ~! H: r8 j; a
  <name>dfs.namenode.audit.log.token.tracking.id</name>7 S; O, L" b! a! t) R, Z
  <value>false</value>
/ @$ v% J  Q% \5 x; \  <description>
4 j4 L  J7 c0 j5 K& S2 n  L* C% X+ Q    If true, adds a tracking ID for all audit log events.6 p8 Y$ L  R2 ?. b8 I
  </description>6 z8 y  Y; }6 s5 f0 Q- M
</property>
, q7 V$ \8 {8 X( I; V: |# B<property>
  N6 p* n/ Z9 g9 V0 l  <name>dfs.namenode.available-space-block-placement-policy.balanced-space-preference-fraction</name>6 b5 L8 L" Z8 O. B5 D( R
  <value>0.6</value>
" L- c9 r0 I6 q3 w. l/ ^  <description>: M7 B+ m. h: g$ D: _) W# P4 Y
    Only used when the dfs.block.replicator.classname is set to
" D6 d  n9 I3 t    org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceBlockPlacementPolicy.
0 X# R- l, C% C( Y2 u4 K/ W    Special value between 0 and 1, noninclusive.  Increases chance of3 V4 C, z! D% [# Q
    placing blocks on Datanodes with less disk space used.3 o) [6 W. ~8 d4 W: R4 y( I
  </description>$ a" D8 G: V" G9 Z/ h4 d
</property>0 O# f3 U5 {0 E: N' {
<property>
$ h* U& V4 ~% k' w; E! e  h  <name>dfs.namenode.backup.dnrpc-address</name>$ u! s- s- w* G/ P, K7 Q
  <value></value>
  x- n" H; |- E5 j  <description>
% M2 A6 b. _6 a, d    Service RPC address for the backup Namenode.
( K$ O3 x9 y# l/ x7 q; Y8 W' F4 r' w  </description>; M, Y) L4 }7 d& ~
</property>
; Y4 N* N  @! A! \% J* s<property>
8 _( q; B; x4 |' O* l+ L% E  <name>dfs.namenode.delegation.token.always-use</name>
7 Q+ }* M5 Y4 q( e3 N, b# I: p  <value>false</value>
" e7 d& c" I% _! [2 u( m3 J3 j  <description>
5 ]1 U) ]- c; h" j, [5 i1 S, W, P    For testing.  Setting to true always allows the DT secret manager1 A$ T) R  |7 ~6 d5 z/ w
    to be used, even if security is disabled.& |* X6 Y# [, C- R7 ~0 r# d" {( W) s
  </description>
) q4 K) L* H) Q6 e</property>
7 m$ j+ ^9 _% [1 H<property>2 N: k0 |4 K2 D+ Z* v
  <name>dfs.namenode.edits.asynclogging</name>
/ u) }5 E5 f; [2 r& d5 `2 {  <value>true</value>
" e! d! j% |6 H3 t& ]7 }7 o5 |, b) A  <description>
4 a7 y! E! m0 Z; U$ Y    If set to true, enables asynchronous edit logs in the Namenode.  If set% X2 `0 e, w9 u( \; o
    to false, the Namenode uses the traditional synchronous edit logs.
# Z2 q% c1 ], h* i  </description>
" }4 X, |$ r3 Z) ^</property>
# e3 |/ c( q# g. j7 q9 i" V" B<property>( K7 S9 H$ ]/ y8 N
  <name>dfs.namenode.edits.dir.minimum</name>
* n. h1 [+ Z9 s; b: [  <value>1</value>6 b0 h' B9 Y, J/ ^" @* H
  <description>; e% e8 i& V5 M2 a/ t
    dfs.namenode.edits.dir includes both required directories, i% i* \, m" I. ]2 _5 X7 B$ V
    (specified by dfs.namenode.edits.dir.required) and optional directories.
9 K$ _, k& ^2 p    The number of usable optional directories must be greater than or equal
4 }/ S: ?) C2 a% L    to this property.  If the number of usable optional directories falls1 b5 C, w1 ]( y2 ], T" B, f
    below dfs.namenode.edits.dir.minimum, HDFS will issue an error.
; C9 U9 g. z3 |7 ~% A    This property defaults to 1.+ h7 L( f  ^9 U) q% }: ]# N) j
  </description>
) p: N4 n! j1 Q' i' A; e/ v</property>9 z6 W6 W8 X; V: C. f# L! o, M
<property>& p1 }1 x: o- \. o
  <name>dfs.namenode.edits.journal-plugin</name>
* }- n6 p" w  m, p5 s  <value></value>  c8 ^& Q' ^% t
  <description>" K8 z( w* x: K; ]9 e: ^, ~; J% T
    When FSEditLog is creating JournalManagers from dfs.namenode.edits.dir,! Q/ J* i! ]# P0 P4 C
    and it encounters a URI with a schema different to "file" it loads the
# J4 j4 n0 \5 J& R+ K, P: u    name of the implementing class from  v' i! u: x8 J8 i
    "dfs.namenode.edits.journal-plugin.[schema]". This class must implement
3 G4 d, i% w3 M    JournalManager and have a constructor which takes (Configuration, URI).0 ~/ w  ~$ K; V/ b
  </description>2 h! k: b* Q6 D% K5 `4 g
</property>+ C0 Q; S1 g" A) F* J& r3 T
<property>( t6 O4 _7 ^& ], Q4 \. N! S6 j
  <name>dfs.namenode.file.close.num-committed-allowed</name>1 ?2 u& `: q3 N  v) G* B1 q4 ?' D
  <value>0</value>
4 s5 ]$ W4 a6 H1 w$ m: E  <description>6 x7 _9 [4 v5 N
    Normally a file can only be closed with all its blocks are committed.
; s7 s7 T$ K# K8 U; F    When this value is set to a positive integer N, a file can be closed& V6 |% T0 Z( l5 O- W. ~5 z
    when N blocks are committed and the rest complete.9 w) Y2 ?: M* D) N9 H2 u
  </description>$ t' j' k  _; a) W6 ?
</property>/ E6 I9 D% s* M- ~: Z" L
<property>
5 a. i8 w* E- a! J5 D  <name>dfs.namenode.inode.attributes.provider.class</name>, G1 p/ W5 d1 O8 c( |6 v  k
  <value></value>
* n1 w1 C3 A0 Y  h3 f  <description>
& p' U1 U* ~8 {/ [3 R    Name of class to use for delegating HDFS authorization.' g5 Q" D3 X9 _! n9 m0 Z5 Y7 ^8 M* b! W
  </description>% \; ?& k2 C3 r9 T
</property>
1 q: ]" B  ^0 |3 f! v+ k<property>
4 w: ~* n! c$ t+ M$ [  <name>dfs.namenode.inode.attributes.provider.bypass.users</name>$ v% k) Y" l( M& Q- h3 t
  <value></value>
# b7 c2 I/ p8 F* P6 R$ l$ {  <description># a. ^0 E( ]2 O4 C
    A list of user principals (in secure cluster) or user names (in insecure
6 R! [) p- @- |" u    cluster) for whom the external attributes provider will be bypassed for all+ S' `9 E' y, |' v& [& X* D
    operations. This means file attributes stored in HDFS instead of the- x+ y+ F) [5 ^7 R4 X- z, c: G3 c
    external provider will be used for permission checking and be returned when3 V) i& Y$ W, [, A' }, c
    requested.
- O& d. H  N% ^( h) H, |  </description>
: J- v) C0 j" i9 m% F/ g# I</property>
+ \- _: R. ~, U/ k8 D6 Y( {2 A<property>- n& x8 l. n6 \
  <name>dfs.namenode.max-num-blocks-to-log</name>3 W( X/ ?% ?, @: G  N  v) ^
  <value>1000</value>
6 }' }/ d; [) B* v! y1 e* X. w  <description>  [- C5 v$ d+ g  N
    Puts a limit on the number of blocks printed to the log by the Namenode" U2 Q7 C  p- u* w4 ]% T) M$ S! ~2 R
    after a block report.3 P6 D0 x) ]4 U7 [
  </description>2 u; ?. @9 }8 p( Z
</property>% h. k" \  t) X
<property>7 j- S8 i/ p- Z8 q: k
  <name>dfs.namenode.max.op.size</name>( S- q2 J( F( z. K5 R- m
  <value>52428800</value>( o2 ^  w' b" a, N' J9 _9 @7 g9 s
  <description>3 T8 C: ^8 m8 k3 v, `
    Maximum opcode size in bytes.
  d, H1 i# o5 d: f2 ?7 ]' Z+ s  </description>4 X! j+ {9 J, s2 S- o/ ~2 c. x
</property>
! U1 q8 c/ B# ~, J! d2 O& I; v<property>
9 B1 s0 m$ F9 p  r% h/ `/ }  <name>dfs.namenode.missing.checkpoint.periods.before.shutdown</name>
, h1 C! k  y) `' x. }( I2 L  <value>3</value>) G1 [5 g( s8 q
  <description>
4 W7 P5 ~' H0 f' |) b    The number of checkpoint period windows (as defined by the property, E6 l* \1 ~- U
    dfs.namenode.checkpoint.period) allowed by the Namenode to perform8 p5 S. W% [+ o$ D& ?; y4 k
    saving the namespace before shutdown.
$ m0 c; @$ M1 J  </description>
) m; `. F; a% D1 S9 m</property>
. g( k) _7 k( h$ w- }2 D) t9 Y<property>: J9 j, Z+ G) e! J+ e5 w
  <name>dfs.namenode.name.cache.threshold</name>
( R- j+ {/ M* V! e  <value>10</value>9 y0 s& I% ~* I5 D
  <description>; Y9 n$ f2 k1 k( I1 g5 L% Z% Y
    Frequently accessed files that are accessed more times than this7 R/ q2 }) M' V2 p
    threshold are cached in the FSDirectory nameCache.; a0 F4 M# q% X
  </description>0 T3 K( b9 |: u  U5 m$ d. f  D- E
</property>! Y- \2 `3 G  S
<property>
, z( R1 Y. z& W+ u- I  <name>dfs.namenode.replication.max-streams</name>& X% A( a8 e# v3 N' l( `/ s4 z
  <value>2</value>, P+ s/ V% F4 e6 ]
  <description>: ~9 ~& B8 d1 b
    Hard limit for the number of highest-priority replication streams.
+ E  x( N2 p1 ], b, e3 ~  </description>
3 c- g+ ~  y' A7 g</property>! n+ ]* e" Z$ E$ d" |3 j; l0 _: i) K
<property>
; L, G5 p- t& J8 R9 i: T* N  <name>dfs.namenode.replication.max-streams-hard-limit</name>
4 h2 F0 r& m0 _8 D* I6 g, k* ~  <value>4</value>
( z5 Q# X3 y( B' h  <description>
8 {; e3 D+ A: ~+ r9 G    Hard limit for all replication streams.
: j9 `; l2 j4 d) b( v  </description>. y2 Q  y3 ^: T* A& l4 A: V
</property>
; L+ \  O( `' k4 _<property>5 c! t0 k0 x* h  ?% n- Y  p9 q+ Z- F
  <name>dfs.namenode.reconstruction.pending.timeout-sec</name>
; q5 ^+ f! i: T4 P" c: r/ a% Q0 X  <value>300</value>$ ^4 f- Q$ P0 T. a/ [% Y
  <description>3 w/ U; z: \( C7 A- ~! E4 E$ o+ B
    Timeout in seconds for block reconstruction.  If this value is 0 or less,. o6 L; a0 ]6 I# V
    then it will default to 5 minutes.& N# y+ j+ a( h" K" K
  </description>
8 v  x% s) b# e; {' E</property>7 a9 d3 y8 w8 x( E( }
<property>
* G8 F0 ?8 k- Q" g5 r  <name>dfs.namenode.stale.datanode.minimum.interval</name>
) Z- M% p- t5 G: Q% F6 E  <value>3</value>6 }9 @9 P8 Y" q$ p; h$ V
  <description>! ?" \4 B" _3 V
    Minimum number of missed heartbeats intervals for a datanode to
+ X5 K0 y- H2 `6 W/ m1 x    be marked stale by the Namenode.  The actual interval is calculated as
# p) V9 j" k7 u# Y$ d" |    (dfs.namenode.stale.datanode.minimum.interval * dfs.heartbeat.interval)
; L8 W$ `  Y7 V! p    in seconds.  If this value is greater than the property7 M' a- w2 n  j3 M8 g5 D  _
    dfs.namenode.stale.datanode.interval, then the calculated value above
/ v, L% S; X3 a    is used.0 j, h* M. P  {4 e2 t# W" U- X* J
  </description>$ R: K1 m2 W4 V2 v' s% W
</property>2 a1 `. P/ p: z) y0 k' f' N- h
<property>
! a. E3 f7 i9 L( C  <name>dfs.namenode.storageinfo.defragment.timeout.ms</name>6 d* M5 A1 l; Z7 v$ a! D
  <value>4</value>
( U" Q% z  n) [5 q# }; h9 q5 V6 c  <description>
3 V/ l2 }$ R! e* f    Timeout value in ms for the StorageInfo compaction run.: _8 y% ?# w' a9 C
  </description>+ Z& O9 @! t' G1 {# h4 J9 b
</property>7 A2 s! N( T3 g
<property>
2 u$ _: X2 U6 j  <name>dfs.namenode.storageinfo.defragment.interval.ms</name>
+ t# |8 D+ a& n: E# Q: W  <value>600000</value>" Y" n; X/ o5 O+ F" y. _; G
  <description>
$ N  W8 o$ I: n5 @1 W3 k    The thread for checking the StorageInfo for defragmentation will! u/ [7 Y' t$ \% Q0 V+ m
    run periodically.  The time between runs is determined by this
& B! }% _9 G# R$ z& O4 x' B& Y    property.  C! i( M* z% d( n3 _" j
  </description>9 ~8 ?$ Q7 a: Z& |4 K. L6 f
</property>, M4 l4 e$ U- q$ s, v8 [5 C
<property>
: b- ^$ z0 m4 z5 q( H  <name>dfs.namenode.storageinfo.defragment.ratio</name>
- I$ {' n5 l! r% v% s  <value>0.75</value>
9 f* N* I+ ~5 ?: ?* W  <description>( C. }1 W4 C1 h5 H
    The defragmentation threshold for the StorageInfo.' E+ b+ W! A* q2 d- f3 h6 w7 D
  </description>
' N2 M7 O- w4 B1 |1 y4 K, q8 s7 f</property>
0 L% O) y- Y' E4 ]<property>8 u  A. @# Q. E. V0 K: t, n# F
  <name>dfs.namenode.snapshot.capture.openfiles</name>2 h, ?0 T# {) B* q- S6 \
  <value>false</value>: g- S6 P+ @9 y4 J, F* C2 A! l& P
  <description>+ ^# h  ~6 h. ?/ K' [
    If true, snapshots taken will have an immutable shared copy of
' p" S( a  c. b& U  Y    the open files that have valid leases. Even after the open files8 ~% O2 x: D+ |/ U# _. i' B
    grow or shrink in size, snapshot will always have the previous
- l# f, @; z# B: D+ e; R- R9 ]9 o    point-in-time version of the open files, just like all other
2 d  p: A! d! v* f' e  r    closed files. Default is false.7 J. _7 c& S/ ?/ \' o
    Note: The file length captured for open files in snapshot is
. \3 ^5 V$ d1 Q9 K    whats recorded in NameNode at the time of snapshot and it may
, X3 b4 T. W; U2 d: p2 B# |8 b+ @    be shorter than what the client has written till then. In order2 _5 j" S# E0 e# H" o3 X
    to capture the latest length, the client can call hflush/hsync* {5 G+ [8 x0 g* j6 @% |! k
    with the flag SyncFlag.UPDATE_LENGTH on the open files handles.; n, ^* o& X' O) P* i
  </description>* F9 G9 ]+ `0 a9 J$ X
</property>
9 p  H% b! J$ B. t! n+ g5 Y  W<property>
9 X; ~4 m+ b7 d* p5 ^1 N, T  <name>dfs.namenode.snapshot.skip.capture.accesstime-only-change</name>
/ U. \/ D+ b1 B6 v/ ~0 w  <value>false</value>
; m) I" f2 G/ s( z- H2 z  <description>( |; o# r) ], {/ G, S9 k
    If accessTime of a file/directory changed but there is no other
5 c: I. ~# O% Q; G' X& p0 _    modification made to the file/directory, the changed accesstime will
3 Z9 t# x5 ~1 Q0 j9 o% _    not be captured in next snapshot. However, if there is other modification
. f! O( L! Z* T    made to the file/directory, the latest access time will be captured5 g; M; t/ ~7 @7 X% l9 o5 `
    together with the modification in next snapshot.
. _$ T3 t; W& A+ g! [6 z) l  </description>
3 z# q3 r( o; a; |! K1 [</property>
8 U) d" |2 D" l& k" d# Z<property>
1 j& `0 }7 s" k- [' |+ R! T  }  <name>dfs.namenode.snapshotdiff.allow.snap-root-descendant</name>5 c9 w4 r$ ]* [0 d4 H9 X
  <value>true</value>
% B8 ^/ C0 r% o+ I2 B  <description>8 a/ ^6 ?* J: }2 Z. Y/ H
    If enabled, snapshotDiff command can be run for any descendant directory
5 q' l  K* F" O/ ]    under a snapshot root directory and the diff calculation will be scoped
# Z# n7 V: B& D5 e    to the given descendant directory. Otherwise, snapshot diff command can/ X; a3 d6 l1 Y1 S3 }6 [; I
    only be run for a snapshot root directory.3 v% d( E5 T' s$ G: Z
  </description>4 x- U7 y$ _  c. A$ h3 e; r+ h
</property>
0 d# V( C* }3 E& C% v0 M6 B<property>
) q/ G6 G5 _2 C. x2 [3 C  <name>dfs.namenode.snapshotdiff.listing.limit</name>
2 @1 O) b0 g3 G$ s+ k# I  <value>1000</value>/ e# x4 E8 ~5 J( p+ N5 S
  <description>; @* K% B$ O6 _0 X2 d! M
    Limit the number of entries generated by getSnapshotDiffReportListing within
* E3 S2 N+ W. }6 J, {4 G    one rpc call to the namenode.If less or equal to zero, at most
/ p; [4 U8 Q& c# E7 A    DFS_NAMENODE_SNAPSHOT_DIFF_LISTING_LIMIT_DEFAULT (= 1000) will be sent; J! i+ G3 t, M# U" H3 [: l1 s
    across to the client within one rpc call.
0 B% v* s* Z  N, ^  </description># x# u2 T. R" j% o" S0 _
</property>
+ C+ a6 H/ v& d' O<property>4 G" ^' Y% ]* q, T
  <name>dfs.namenode.snapshot.max.limit</name>
' e4 ^6 G$ U  Z) c) t/ D) }  <value>65536</value>* _# |1 U: C) ?( {
  <description>
$ U% o' @- N  _9 I    Limits the maximum number of snapshots allowed per snapshottable! `) }  O8 ^5 ~5 `+ l. L
    directory.If the configuration is not set, the default limit7 e1 p4 n% N" X7 \: i# M
    for maximum no of snapshots allowed is 65536.- Y4 u/ E( l' T3 n$ O
  </description>) Q, Z; M- |! B  N9 E9 E7 r7 t
</property>
, |, o, I: ?' I* T, X<property>+ y7 E& ~4 D- t& H( }/ W  _
  <name>dfs.namenode.snapshot.skiplist.max.levels</name>! A- d" c7 A1 d2 x( f9 V
  <value>0</value>; a' R0 `2 |5 `. a
  <description>
/ a" N) M, p$ d    Maximum no of the skip levels to be maintained in the skip list for" i9 t: P" h! B# P$ n
    storing directory snapshot diffs. By default, it is set to 0 and a linear
( u+ s8 _9 O" T3 C+ t0 _    list will be used to store the directory snapshot diffs.1 S. u  a0 Q3 {. h
  </description>
* k, A; F+ h0 S  ?: \+ m</property>
5 u, T3 A. {5 b7 S. U/ D<property>5 Y2 h8 v! a2 M. P: O
  <name>dfs.namenode.snapshot.skiplist.interval</name>. Y# s! a* D' i; J
  <value>10</value>- J# G" L/ E4 @. D; b4 S0 `4 \( q
  <description>* K0 c! R; a9 e. g. G1 `
    The interval after which the skip levels will be formed in the skip list9 `5 p6 E' r# e1 n& Y( F/ Q8 V. u4 q5 p# s
    for storing directory snapshot diffs. By default, value is set to 10.7 }# g7 ]1 \3 A* i  t6 ^  T+ L
  </description>
) `* W5 H' Z9 V+ r</property>
3 M; M% f# n5 c" C! E2 a. H<property>  |4 H: A' ?6 N5 p* Q
  <name>dfs.pipeline.ecn</name>
) x0 k3 t2 D& G  <value>false</value>
7 k$ g* P- \( b  y2 c  <description>+ O8 ~0 o. f  M& s6 S3 o2 v0 ?6 ~
    If true, allows ECN (explicit congestion notification) from the
! W9 q' Y# b: H, i7 ~& b" `    Datanode.2 T: Z: Y/ {1 Y8 M
  </description>
7 [+ @0 Q% }) {- ~. h, c5 {</property>
% u& _5 y6 J) y4 M, M<property>; w2 o8 H4 |& k" Q4 W
  <name>dfs.qjournal.accept-recovery.timeout.ms</name>$ Y+ s( a# T, Q( G; i
  <value>120000</value>2 x5 Q+ h. e, X( J8 H2 S. J: j/ u6 w
  <description>1 X/ s5 W: J: A$ w, _+ j
    Quorum timeout in milliseconds during accept phase of
4 J' V) N: |$ f- L    recovery/synchronization for a specific segment.
- E9 j( Q$ Q+ h0 W! j; t  </description>/ e. G$ Y. _# S1 h
</property>
4 M& y- l1 H; N8 f& N* @+ ^<property>8 N1 M0 t: H/ }
  <name>dfs.qjournal.finalize-segment.timeout.ms</name>' M9 u9 v  C- Y; @' V9 e# c) k( y
  <value>120000</value>
4 s/ m. p1 {' d+ h- J5 Q  <description>. J# V+ w! \# W5 n) l# \
    Quorum timeout in milliseconds during finalizing for a specific
8 q% E2 x7 c; n5 ~! K* X0 @' ?/ t& L    segment.4 y$ Y/ C  y, s5 ]: d5 `( G- q8 h
  </description>
2 C' f6 H2 h3 l</property>1 X$ B6 j4 ^8 B( \4 S5 h
<property>  U- _/ m) g$ m% X- i; m
  <name>dfs.qjournal.get-journal-state.timeout.ms</name>1 n6 F6 e5 d$ ~; E
  <value>120000</value>& v9 L. ?/ T* v& A2 \
  <description>' E( q0 K# ~; |: V  p( T
    Timeout in milliseconds when calling getJournalState().
8 X! M$ M$ x1 `    JournalNodes.$ y. w( P$ |: I3 F6 b' n$ @( c
  </description>
0 }2 I; R+ U" [) Z</property>) n5 \( i) f) x3 u" i8 G
<property>2 ]5 j6 P/ b7 T) n6 X& m1 A
  <name>dfs.qjournal.new-epoch.timeout.ms</name>7 u9 l, g6 H. o6 A! J
  <value>120000</value>
* h; e5 \% s5 H8 \3 _. W( G- k8 P  <description>
9 K$ Y  s& h0 p    Timeout in milliseconds when getting an epoch number for write% u; r+ V, b2 D/ o% X
    access to JournalNodes.
: j, A3 x  w: m! w- p1 B5 o* ?  </description>+ v2 W; Y& b$ u, p
</property>
0 ?# t! u1 A' z3 [* R, n- X<property>3 R; N) h, i1 a, e, ^3 I8 P
  <name>dfs.qjournal.prepare-recovery.timeout.ms</name>+ W% \! Q( N- z) A+ q
  <value>120000</value>
# Y1 G9 f! x; z  <description>
+ j1 R' Q# m) i+ {  @/ q' {& u1 h    Quorum timeout in milliseconds during preparation phase of
6 b& u. Y5 @0 Q    recovery/synchronization for a specific segment.
# B; P, A( J% ?0 f0 H# \  </description>
" J1 u6 i$ z* ]8 B- \; O2 d</property>
' a- n* r& x4 H0 b( h2 c5 Z<property>
3 F  H5 i3 N' l9 _' ]* _  <name>dfs.qjournal.queued-edits.limit.mb</name>4 p5 N- A: Q- g- [5 |. G
  <value>10</value>! W' z. b: j8 d
  <description>: G3 w" ^2 |  Z
    Queue size in MB for quorum journal edits.; o  c$ k4 M, O, ^4 z. n3 g
  </description>0 u5 k7 _  O- d% H
</property>
# ?, q: x2 ^: u! A" g/ z' M5 E! T<property>% E6 }: h  ]8 K
  <name>dfs.qjournal.select-input-streams.timeout.ms</name>1 J" ~$ r! s4 ^; F- n, d& A2 S
  <value>20000</value>7 G& g7 _+ }6 P- l% }- ~( C
  <description>
& g, D. h) l2 M; \  F( L6 s+ V- o    Timeout in milliseconds for accepting streams from JournalManagers.
. @7 C& F7 e# `% A/ l  </description>
: v# M( O3 F( `1 ~& t; x# y</property># r1 H. p  i4 w8 d
<property>
: w' w, b7 M/ ]2 L+ Q0 ?; N* ~$ T  <name>dfs.qjournal.start-segment.timeout.ms</name>: `+ ^- S& ]( t; {
  <value>20000</value>
% P- F. d" }' g+ e0 M6 A  <description>
) \; C0 E4 j1 n" c1 [# a    Quorum timeout in milliseconds for starting a log segment.
4 w1 k! O1 @: d) g; Y! V+ b& D  </description>
& N( L( I! c9 N* e: D% I</property>
0 Q0 j& o/ W7 N6 _4 v4 H<property>
2 t0 P* I  P- Y( K  <name>dfs.qjournal.write-txns.timeout.ms</name>
6 A5 U  l* c! R! N# k- y  <value>20000</value>
8 z& R" h2 q% ]- Y, T  <description>
+ S% |+ r1 w0 ^$ B' D0 P# c    Write timeout in milliseconds when writing to a quorum of remote; n; Z# o0 ^3 H
    journals.
/ {( T2 F) v/ @% C  </description>
% q3 b' Y9 b  W. Q9 W/ _% B</property>
7 j7 K! j7 i; q7 J; w<property>
& Y/ V# J* _, f9 C, s# G9 S  <name>dfs.quota.by.storage.type.enabled</name>
# E6 {- N9 E6 |- u+ R  <value>true</value>; _5 V$ s  O1 }; b! j% M9 t3 |
  <description>  Y1 D" d: ?! t& o" j% H
    If true, enables quotas based on storage type.& S  ^4 h- M; L: Y5 U0 _& }
  </description>
& f6 Z: _+ Y: {4 w</property>
4 e. m% _& \9 D" w9 j<property># H& }, b" Z  L( n+ K0 z. j
  <name>dfs.secondary.namenode.kerberos.principal</name>  A  N$ N' r/ i5 D6 v
  <value></value>+ d' h6 I$ Z7 r) O3 R' h
  <description>
" H  q, h  K8 h- G' @9 b3 o    Kerberos principal name for the Secondary NameNode.; t9 t) E: W: k5 G0 V4 g* S' P  q
  </description>. B* Y8 e1 n4 n# k; r
</property>+ E7 P+ k; j1 F) P" t- e, q: H
<property>
9 I7 X. {4 M" ?, Y0 R% n# S. {  <name>dfs.secondary.namenode.keytab.file</name>2 _0 f: a: I. L4 j" N! @# t4 L
  <value></value>6 u/ m9 |8 U% _6 [; y% B
  <description>
% N  g# }* Z$ }( W    Kerberos keytab file for the Secondary NameNode.
: f: o8 t8 k% n, w2 q( t  </description>
! P" @* j6 Y: O3 l</property>/ r; X" L8 S2 A7 I& T. l6 {  ^. M
<property>
/ R2 z9 K; R+ F* K: V  <name>dfs.web.authentication.filter</name>
6 N/ C0 r: T4 h$ w  <value>org.apache.hadoop.hdfs.web.AuthFilter</value>
' n( L7 r# }: ?$ R& |$ C  <description>, Q5 H" |$ D0 }$ d
    Authentication filter class used for WebHDFS.8 |% ?6 T/ ?. W- w& X) M
  </description>, {) D2 z0 o9 C5 G( c( _, Y
</property>3 Q9 d% B3 W2 U8 y- q1 V' J% k
<property>
5 p- U( M& J4 s  U$ R  <name>dfs.web.authentication.simple.anonymous.allowed</name>
' v& \, m/ `" f1 H  <value></value>  X# K( J+ m. {6 T2 k0 G7 o
  <description>
) z) _( Y1 i* K8 @! V    If true, allow anonymous user to access WebHDFS. Set to  R* U6 D; w$ q" c/ d/ ?
    false to disable anonymous authentication.# e& d. [9 F( m+ f
  </description>
2 ^8 ]. S, m7 t</property>
- u3 P; P- v( \5 {5 g9 q5 \4 e<property>
# O( @/ Q0 k+ [& X7 T; }  <name>dfs.web.ugi</name>
- @) ?0 P! z# ]* e  <value></value>& ?9 W( d9 t& u
  <description>- H5 O  b( L7 B7 j" }
    dfs.web.ugi is deprecated. Use hadoop.http.staticuser.user instead.6 `, [8 O# E6 y: b5 e& Q
  </description>5 D2 m" Z1 {0 n3 S) s+ O
</property>/ z% h' U, W/ K9 \9 V& m. Z
<property>5 i4 `  H0 a+ @. b+ e& _
  <name>dfs.webhdfs.netty.high.watermark</name>4 o1 v8 W' I5 B" R/ C! t2 i& |, u. U
  <value>65535</value>; o7 [$ W- }/ A9 ^! k  }4 y
  <description>
7 g# Q3 P7 ?" P/ Y; b4 D    High watermark configuration to Netty for Datanode WebHdfs.
$ k: t: p- T4 G% \# B' L- i  </description>
; m! n6 k$ `/ s1 U4 |/ {1 |5 F</property>
- B, o7 x" z/ b! H0 q<property>
5 b0 i1 c/ |- K& I8 U- A" ^  <name>dfs.webhdfs.netty.low.watermark</name>
. P0 j/ J, a# n; W- }; t  <value>32768</value>
) U' n' I% q! \' \: L  <description>
2 B5 Q- r7 E# e% {; L( v1 I- W    Low watermark configuration to Netty for Datanode WebHdfs.0 H* A2 K* E! r. p8 n
  </description>: T$ H/ ]% J! o) j1 N% b
</property>
2 J* o& V7 i9 k! p! z/ v8 W/ N# A7 \<property>8 v) X/ y: z3 G5 T4 J
  <name>dfs.webhdfs.oauth2.access.token.provider</name>) o1 L, E3 K# `& R
  <value></value>
$ ]3 A. R3 Z' x9 x  u8 X) }" K  <description>
. p$ ]: {- \" a8 E    Access token provider class for WebHDFS using OAuth2.
$ \9 p6 [$ J5 F: a    Defaults to org.apache.hadoop.hdfs.web.oauth2.ConfCredentialBasedAccessTokenProvider.
9 X: w0 P' P, |" Q% ?  t  </description>- Y; n+ ~- r( t) e5 x0 G$ \/ V1 x
</property>0 t3 Z. t( p  X8 W
<property>
6 @# P# ?8 }7 F1 G! [  <name>dfs.webhdfs.oauth2.client.id</name>
* H% Q1 g) n; a* k$ b  H' B) K  <value></value>
1 V; ?; R& D4 r: Q9 N  <description>4 S2 Q8 c3 u$ l9 J
    Client id used to obtain access token with either credential or
2 D! G- I0 j) `- }; f% k    refresh token.
. v3 p5 ?* _! C; L8 d7 A0 t  </description>
+ i. ]9 R8 ]+ H0 v% e</property>
, u$ x+ s# D' `0 }- I2 U<property>
4 E0 ?( J$ \/ Z7 r3 t* u2 V1 L  <name>dfs.webhdfs.oauth2.enabled</name>$ C8 [0 Y: B, @7 j% t; C
  <value>false</value>0 D# P2 o+ Q* X3 ^7 ?% l+ u
  <description>& O$ v% H0 B, ^5 U: \0 Q
    If true, enables OAuth2 in WebHDFS
- B( @0 E5 h  b3 C  </description>
$ I  ~4 X& A; W1 b. F</property>
* {' b9 F! v- K- |) n; C2 f<property>
8 \" l& Z  D. W/ u% e  <name>dfs.webhdfs.oauth2.refresh.url</name>5 l9 M. Q) v3 R' n, t% M9 g, ~
  <value></value>
5 q. P: Z; ?# ^/ X5 g# D$ }8 G  <description>9 C2 [  j! }5 e, ]; p2 H, [
    URL against which to post for obtaining bearer token with- ^# \( |$ x$ L0 R* R, k0 [; v
    either credential or refresh token.6 ?7 l! O( R4 d, d- k) t, k
  </description>  Q+ p- b  C, W; {
</property>4 Q" _5 d3 j& ^6 `
<property>
  ?& S" X& K) q) _, B  <name>ssl.server.keystore.keypassword</name>+ D! a/ h* _! a5 [6 y- l$ x
  <value></value>
( Y/ u& ^% G4 |* u, @- r' T$ t  <description>
, u5 L5 _& y  X: z7 @1 b    Keystore key password for HTTPS SSL configuration
6 V3 D" J  R( b5 G% C) V% E  </description>
& M" L7 c' C) v- v( G! v$ O</property>
, `" z  {4 W8 H' D+ k<property>) X% x2 z! b, X# y
  <name>ssl.server.keystore.location</name>8 D/ j( s: v8 q4 X' s0 w% S! e
  <value></value>
' Q/ _: V/ S& E* P9 _1 N  <description>$ U! x* [/ {* E' v3 ]" q
    Keystore location for HTTPS SSL configuration  e$ @) t* X. M3 J) [
  </description>+ h" u$ C  w* J  _  }5 A8 ^& G7 V
</property>- W$ k' k! i& L9 X! Y. z
<property>9 C* n  u1 Q2 b8 H4 x6 x; ?
  <name>ssl.server.keystore.password</name>
$ _7 @3 z; s% K, M: |  <value></value>
* U& z, C! i# n4 C, |3 r  <description>+ K, d( K+ _0 ]; M3 L- r% h
    Keystore password for HTTPS SSL configuration8 d8 j# c8 p! l5 [
  </description>; T; g( d: w  C( I! U/ U
</property>
5 Y2 b" X/ S4 h& h, l* f7 u<property>" M, d4 R( Q8 }2 j2 f6 f
  <name>ssl.server.truststore.location</name>
" }0 G+ l  |3 u" D. `  <value></value>
3 m4 ?& G, x$ @* a0 Z% H# S  <description>7 V+ h! X+ c. p( @/ C% G/ b0 H9 U6 j
    Truststore location for HTTPS SSL configuration
6 `6 k  z' V! t9 \5 `  </description>
3 b) K# R1 b4 N/ I</property>
$ B; M6 L; Z) s<property>- K7 L8 M3 q. i) E" @
  <name>ssl.server.truststore.password</name>  s/ Y+ V9 l( w8 U; L* t3 F% V
  <value></value>: e* s% l8 {( e, F
  <description>
& p7 C  F/ ~) e$ u* m* j6 N    Truststore password for HTTPS SSL configuration
8 S, l3 e$ ]( f  </description>/ `4 a- c4 d" x9 j. `+ l
</property>
- L! t# X9 H3 A2 E) A( \. ?<!--Disk baalncer properties-->
% `6 ]4 c9 P, s' J5 ?  <property>) t" @  b" M* y/ h( \8 p. d
    <name>dfs.disk.balancer.max.disk.throughputInMBperSec</name>! X, n0 a- O( I
    <value>10</value>  k# |: ^. T2 J) r2 p; W0 V2 c# m# ?
    <description>Maximum disk bandwidth used by diskbalancer
2 g. M' _8 `, g7 R, u      during read from a source disk. The unit is MB/sec.
! U* w) H# U. S# F& \- X) j7 J    </description>
6 u- q# u4 M: Y) y; {: n. O' }9 |  </property>9 G9 c" w; {5 P- U5 ^2 [) ]
  <property>4 W4 }& L. ?3 N* `3 F# m% s. U! `
    <name>dfs.disk.balancer.block.tolerance.percent</name>
( p) j1 I& _7 h1 J) `    <value>10</value>2 E* A) W* [  y% o: g# v
    <description>
% G" L, \) r5 c8 j) T  x- _      When a disk balancer copy operation is proceeding, the datanode is still
# P, G, ~5 B0 Q& |      active. So it might not be possible to move the exactly specified0 t" y1 Z5 p) F7 Q' B  J
      amount of data. So tolerance allows us to define a percentage which% L( h% B. e. [
      defines a good enough move.
4 w5 V- r' k6 I% N2 b    </description>
0 V9 M3 j" h2 g5 [  k  </property>; P3 @( x# ?4 u
  <property># O2 p3 t3 A2 C, \
    <name>dfs.disk.balancer.max.disk.errors</name>" u' _9 q- b9 K4 |
    <value>5</value>
" \1 j6 F5 n; }+ j. J# n    <description># M2 Y; p' I) u7 s, P3 |1 Z7 p
      During a block move from a source to destination disk, we might
& o# r7 u8 @" y4 {      encounter various errors. This defines how many errors we can tolerate& S/ t0 B0 B, x. K6 m, H% ]
      before we declare a move between 2 disks (or a step) has failed.; |, c2 T; @% B  N
    </description>: {5 n' W1 \9 O# B2 X( ?9 [* I
  </property>+ v/ N) e" O& L" Z$ @9 I
  <property>$ ^; `/ ^' [+ F; u8 y7 W2 J# x
    <name>dfs.disk.balancer.plan.valid.interval</name>$ p# H; [" ?1 L( k8 _( M/ N
    <value>1d</value>- \# C6 l. K: ]: d2 ^+ W
    <description>  f0 v4 k1 @  j( k8 [7 }- G# y# G
      Maximum amount of time disk balancer plan is valid. This setting
4 g9 @  _& o! o1 L- K      supports multiple time unit suffixes as described in8 U" |' z3 Q6 [" q$ z1 U& r9 F. [$ Q
      dfs.heartbeat.interval. If no suffix is specified then milliseconds
' k, R6 ~3 ]3 s      is assumed.8 [% Q% I5 ^8 [
    </description>9 k( B2 E* G  k: p: t5 ?7 _
  </property>
4 H( q: c: N) {: T' D) j; {. b  <property>3 t) b1 F, R) T. J
    <name>dfs.disk.balancer.enabled</name>& S4 R) `& f' ]. Y9 j6 X/ B, J" _
    <value>true</value>
; R6 d; @7 t8 \+ e    <description>
! P7 F) `! Q) z        This enables the diskbalancer feature on a cluster. By default, disk% v9 @% N0 F. T( V8 X) u0 v, \
      balancer is enabled., x# ?' o% h( R: Q
    </description>( I% k: S$ }& U+ y5 _1 i$ q
  </property>$ D) [1 S0 ]3 V- |( Q
  <property>6 P- h6 P5 D1 v8 {/ n
    <name>dfs.disk.balancer.plan.threshold.percent</name>2 l% |" X: D( g0 f! n% Y3 f
    <value>10</value>
! h5 X0 c& v6 u2 o* x# Y    <description>3 @0 c5 J2 H2 J4 Y% ^, _
      The percentage threshold value for volume Data Density in a plan.
" e/ W5 h6 J  }  U      If the absolute value of volume Data Density which is out of: p5 n' a' u1 a6 {6 l) V
      threshold value in a node, it means that the volumes corresponding to8 G) j  ~* @& ]; a9 w% A6 L
      the disks should do the balancing in the plan. The default value is 10.
/ }# B# q( t# g1 n! l3 ^8 o+ W    </description>, O8 h+ v! A- `" g% e0 Y
  </property>
, R* ~6 Q6 J) ?3 Q. h( ]$ `7 G  <property>
9 I# ?3 c( k% {    <name>dfs.namenode.provided.enabled</name>
0 g7 ^5 {! }1 E! U    <value>false</value>- ~) _* g" g( f% ]0 b
    <description>0 |% d# B8 X0 L* l
      Enables the Namenode to handle provided storages.
+ X# W! ^0 d; N/ l( \    </description>7 i5 a6 S2 Y. I5 T7 Y$ n
  </property>8 ^0 K7 l, d& z/ r  X" u
  <property>
1 g0 D6 @0 G  R3 i1 P/ s    <name>dfs.provided.storage.id</name>
3 m" t- s$ Z/ @. ?    <value>DS-PROVIDED</value>
; B( r8 j7 H. G0 {. v* s) s3 ?    <description>& m4 H! M3 q4 y+ W7 U- [# I
      The storage ID used for provided stores.2 `. y- y, Z. i# @3 ?
    </description>
8 H9 w+ z1 t6 B( t( Z  </property>1 t! W) p) v( s7 f$ S7 q
  <property>' W% @3 ^7 @+ X( i4 P0 s! H1 L) o
    <name>dfs.provided.aliasmap.class</name>* P! {2 W  K2 e& C
    <value>org.apache.hadoop.hdfs.server.common.blockaliasmap.impl.TextFileRegionAliasMap</value>7 f. p4 x  J( R( V/ ~' G1 f. e' ~
    <description>
/ k+ N4 G& b: {: u  V      The class that is used to specify the input format of the blocks on
- r  |' E4 W" I$ J6 R# H8 O      provided storages. The default is% \* |: O* |' }  R5 l
      org.apache.hadoop.hdfs.server.common.blockaliasmap.impl.TextFileRegionAliasMap which uses
  _. g  m- B7 h      file regions to describe blocks. The file regions are specified as a( v6 c( o" s. V2 k- M5 Q0 g4 K( M
      delimited text file. Each file region is a 6-tuple containing the
9 Z$ G2 C; u. ]% B: q4 p) O      block id, remote file path, offset into file, length of block, the" a0 U* F( o) q7 h, i$ j2 j
      block pool id containing the block, and the generation stamp of the
/ ^$ w+ u) }/ g! a! G9 O# t( G0 ~      block.
6 i3 l/ O$ x3 Y4 Y! M    </description>. P. }- S2 k8 X7 w7 s( u2 v
  </property>& ]' j3 G3 k0 M  ?9 [
  <property>
  C  x* C! s7 M9 k: G4 E" U    <name>dfs.provided.aliasmap.inmemory.batch-size</name>$ g+ m0 v: h. B9 L* o! d
    <value>500</value>
+ S' t- y- ]6 J) a    <description>4 f/ j; {/ q- Q7 y9 C
      The batch size when iterating over the database backing the aliasmap
4 q, k$ A" N- n1 h    </description>
* U( U/ S6 f4 l8 i9 W4 [& N% V  </property>0 [' c" ^% h- ^& n1 `% S* K
  <property>7 b8 s3 R! ~9 D- i! f* d. H! i* l$ w
    <name>dfs.provided.aliasmap.inmemory.dnrpc-address</name>
& x# I5 |; Z" {! s    <value>0.0.0.0:50200</value>
( x( R) \4 T& u& E, b1 Z& v    <description>3 h6 @3 Z- c# T0 J0 y" n! }
      The address where the aliasmap server will be running  R3 o' v2 j+ x* B' r! a. l
    </description>
2 @! C! Z6 p1 N0 }% E0 \* N5 \: H  </property>( [' ?9 P0 j# Q* W8 W) s( r
  <property>
0 F2 w6 z  S. y" m1 f0 f    <name>dfs.provided.aliasmap.inmemory.leveldb.dir</name>& ]$ e6 {9 [. P' f4 I2 x- j& [
    <value>/tmp</value>) P" l: ^% a5 U0 K' H+ D) g
    <description>" }1 m: p' L' f& A) H( @* D
      The directory where the leveldb files will be kept7 b1 q" K! i/ J
    </description>: q, ~, X( D* W+ X0 M/ R
  </property>/ r/ R5 X' u5 {9 ~/ C
  <property>
$ {( l/ T( A( @- ]# N3 {    <name>dfs.provided.aliasmap.inmemory.enabled</name>. y. I( q# W8 t! h0 F
    <value>false</value>
# l! ~0 K7 l$ j$ ^( q    <description>
! {% u3 M6 w) X6 y4 o      Don't use the aliasmap by default. Some tests will fail* X* R7 F# M, l0 @
      because they try to start the namenode twice with the
' j" x$ [8 E. A      same parameters if you turn it on.
( d: w" ^% E/ k9 Z    </description>
* @. _4 U2 U& T& {  </property>2 P, W5 c4 T$ K& Y) j" {
  <property>6 r7 B& V9 D# L5 B5 y9 `& C
    <name>dfs.provided.aliasmap.text.delimiter</name>
' t0 k5 K" i3 y! u! Y. N# a3 _    <value>,</value>
! d, j- Y5 N/ g    <description>
3 x( \2 I9 i9 J) Y! ]0 U; H  Z        The delimiter used when the provided block map is specified as
) ?' `$ ?/ Q4 J: T, u$ h8 q        a text file.
3 @, N! g1 C# F$ E    </description>; D8 Y/ a: O& X- h3 A7 d& g) y
  </property># [* a2 u6 }* l0 y& x6 C
  <property>
4 Z/ |' M3 R: a6 S    <name>dfs.provided.aliasmap.text.read.file</name>
2 P5 t+ v- O: _8 N+ U3 `+ E    <value></value>
' V0 O8 Q) B! ]: g7 q; S    <description>
9 f: }( K3 e" r: t        The path specifying the provided block map as a text file, specified as+ g2 r3 d( q$ N+ V
        a URI.
8 p0 d) p0 z2 y9 F( m6 N    </description>, N/ q' t5 V8 t: j$ \
  </property>
0 D8 L2 S. |: `2 p7 _. Z6 _  <property>
1 Y: z( v. q$ A3 K$ ^    <name>dfs.provided.aliasmap.text.codec</name>
4 c+ o# j9 s  ]! h6 H. w+ e9 z    <value></value>. D! F5 I) Y1 j- ]0 M2 }( X
    <description>- Q' g) B" `" }* u* b+ l. l- h( P% W
        The codec used to de-compress the provided block map.- c; }' D8 E( j" u
    </description>8 U( \& l7 E; W1 @( k6 d- O
  </property>0 {" t6 K) }! }9 J. S
  <property>
; K' f2 c: V! l! G' v, c( o    <name>dfs.provided.aliasmap.text.write.dir</name>4 a) N$ X' a& @  S0 F* B
    <value></value>
) ?3 F8 Z$ z# S( G6 y    <description>- m% q  h$ e3 S( J+ i' U) s
        The path to which the provided block map should be written as a text
$ ?$ O( ]4 \+ d" `# d: O2 y+ t        file, specified as a URI.4 r  t( w$ m$ ?1 p. @# B, r- ~
    </description>
, E. k; i" o7 B- e+ ?1 W( }  </property>
8 C5 ^* P1 I9 J# i  <property>
# h: F  v5 w8 J' W1 w    <name>dfs.provided.aliasmap.leveldb.path</name>8 A# d/ k" e4 O
    <value></value>9 W' t1 H0 {8 {6 {8 C/ C& z
    <description>
- L0 G; l  x( i" V" r  ^& _      The read/write path for the leveldb-based alias map1 l* y4 `& `) `$ O& s3 ^+ T
      (org.apache.hadoop.hdfs.server.common.blockaliasmap.impl.LevelDBFileRegionAliasMap).2 e4 U0 M7 \1 [( L  u: ~
      The path has to be explicitly configured when this alias map is used.2 N7 L6 Q6 l( s# V3 {
    </description>) ~" t  E3 `; E  Q6 I' U0 U
  </property>
; O$ V* M, B# s6 }  <property>* k7 J/ ]0 D" s5 z& |
    <name>dfs.provided.aliasmap.load.retries</name>% |6 L# v7 Z; b/ L  J
    <value>0</value>
- l: f. h: C1 F6 b5 M7 m; y+ Q5 c2 |7 K    <description>0 B, E3 D& C; t8 ?: y1 k1 p: h- y$ m
      The number of retries on the Datanode to load the provided aliasmap;
3 r7 A+ K7 \& T  S1 K' y4 L      defaults to 0.
  l' J: y. `7 P- [6 l/ V. i$ c3 ?    </description>- Y# e% }: D( T: g) |- E3 ^; |
  </property>
/ V% h$ `/ n( P- r0 g/ A  Y  <property>5 X2 Z: n) V- @# A9 P% v
    <name>dfs.lock.suppress.warning.interval</name>- F* I+ F4 u8 k
    <value>10s</value>8 B( O6 I1 ~2 l, i+ F* c* l' d
    <description>Instrumentation reporting long critical sections will suppress* w9 r# ?1 p# k5 _& s
      consecutive warnings within this interval.</description>  ^8 e2 i# }$ W9 S9 x( O
  </property>
  l9 a- _9 ]' x8 ?# l- e  <property>
6 f# q. ^  y/ f& ?0 G+ u, M    <name>httpfs.buffer.size</name>
8 S; E. {" |& K& Q+ J4 {: ?    <value>4096</value>
9 x* f! ~& E7 }# B    <description>6 E" S$ |. ]# x8 A+ G- h
      The size buffer to be used when creating or opening httpfs filesystem IO stream.
+ ?# q0 n+ d' s' w) _    </description>- n0 @9 q* C/ D- }# p3 v3 v
  </property>
6 C; x7 Q, G' J) S; y. N  <property>
) m7 l  Q/ j4 r  l  ^4 F5 D    <name>dfs.webhdfs.use.ipc.callq</name>8 r5 w4 D3 D6 `6 ^$ s
    <value>true</value>
3 M0 ?0 n' n6 J, m, p" Z    <description>Enables routing of webhdfs calls through rpc
7 I7 ], P) _' A1 z      call queue</description>% s' J9 u3 s) v! J: u4 }8 E( _
  </property>1 J2 P7 b# T% f) _3 Q8 U6 J
  <property>1 V4 S6 \( j9 x0 B# Q
    <name>dfs.datanode.disk.check.min.gap</name>
' a* N$ @4 p/ r* M    <value>15m</value>6 u  z! ?' v( }/ u2 D+ w' I' H
    <description>' x5 f/ Q6 w! r2 v
      The minimum gap between two successive checks of the same DataNode
4 s7 d& Q1 k' B! v      volume. This setting supports multiple time unit suffixes as described
. M2 k  q, r- E/ j4 C" _      in dfs.heartbeat.interval. If no suffix is specified then milliseconds' s# e" L2 h1 p7 V8 _
      is assumed.; _; ]7 J$ E' _* h
    </description>2 S6 B! a$ G* `+ [  P! P* S) B$ k
  </property>. G6 ^! r9 P6 W1 @2 |1 `
  <property>
4 u. R5 q- H; c8 J    <name>dfs.datanode.disk.check.timeout</name>
% s4 A. }" f0 R# N" Q    <value>10m</value># L1 ^2 F( ]- B: ~# ~4 H0 q
    <description>
2 u. }9 c' O  s7 i6 O6 E' N' @4 D      Maximum allowed time for a disk check to complete during DataNode* X( z: C& J) {: G3 y" r
      startup. If the check does not complete within this time interval
" }$ ~, Z, Q9 M% o# q% z$ t      then the disk is declared as failed. This setting supports8 T$ {- C5 J0 z5 P+ }' H; G- w$ c
      multiple time unit suffixes as described in dfs.heartbeat.interval.- x) E. q9 [9 z
      If no suffix is specified then milliseconds is assumed.
" P" n& I- ^/ f) \+ b    </description>% W% d- C4 }9 F3 R& C0 ^! A: J. O3 P
  </property># i& e! k5 f; Q& N6 _
  <property>
5 @1 f; U) b9 W    <name>dfs.use.dfs.network.topology</name>- P. n( p" Z" k# ]1 W6 a
    <value>true</value># p" {! k6 z; B8 Z
    <description>( ~7 a0 l9 l0 g
      Enables DFSNetworkTopology to choose nodes for placing replicas., N- q: c; p7 o. F) [! X$ [  L+ l
      When enabled, NetworkTopology will be instantiated as class defined in: F. @! z1 x( i1 w# W9 B& b
      property dfs.net.topology.impl, otherwise NetworkTopology will be
2 g' W2 m  ^2 D& c      instantiated as class defined in property net.topology.impl.( ]4 l# F, ?* c( ]& o/ s+ }
    </description>
$ N6 t2 L* y. U  </property>1 v& W3 M  y% ]- G& |) ~
  <property>6 f$ [! l8 a' @2 ?
    <name>dfs.net.topology.impl</name>( \- B: Y$ a  {2 x4 X+ G
    <value>org.apache.hadoop.hdfs.net.DFSNetworkTopology</value>
- H! e5 U1 r1 ~+ b: C    <description>
5 a8 e: t+ k& C* O      The implementation class of NetworkTopology used in HDFS. By default,, r- ^* C( Z3 S; \( |: S' H4 H
      the class org.apache.hadoop.hdfs.net.DFSNetworkTopology is specified and
# D9 b9 X! v- g      used in block placement.  w& q" `9 \& s: K1 v3 w2 p
      This property only works when dfs.use.dfs.network.topology is true.
* \) I6 }( E! t3 M    </description>8 @! o/ `' F$ z2 q1 k
  </property>' \+ l: Q' K) j) O9 H
  <property>
, v# H' L3 _& q' F8 l/ F9 R    <name>dfs.qjm.operations.timeout</name>
) |5 l3 B( Z/ K! u( V% X5 w9 b$ B5 P    <value>60s</value>% b$ I9 G/ M2 D$ ^
    <description>7 Y- K6 d* Q" R- o7 Z  F
      Common key to set timeout for related operations in
# t; r* r+ r, c; I7 [      QuorumJournalManager. This setting supports multiple time unit suffixes# f- l' p1 t: u7 k1 G3 x3 z& {
      as described in dfs.heartbeat.interval." S  W, Z9 N7 c; q2 }, u  ~
      If no suffix is specified then milliseconds is assumed.
% _, J" E) e* Q; K( D, a9 }    </description>
: z! g& |; U* Z" F8 I" b  </property>; V+ l0 o% G, K; {
  <property>
2 V/ L, Q4 m! K6 u8 [( ~! G    <name>dfs.reformat.disabled</name>& U" l- J0 U; \2 y9 t4 Y) P# u3 F1 S
    <value>false</value>
8 s  f& L  e4 W    <description>& q( S/ [. q- k) ^% p
      Disable reformat of NameNode. If it's value is set to "true"" V8 c2 _* a$ F" W5 g8 v5 t. j
      and metadata directories already exist then attempt to format NameNode
$ X4 n& J: A' b; b* F      will throw NameNodeFormatException.
1 L; i& W! J0 `    </description>
: m7 F* G9 z! G9 Y  }& M& Y" d4 J0 n  </property>! K6 E" L* p# _" x/ I2 N
</configuration>+ M8 w! L5 b+ U$ U4 ~! r$ f8 w
3.mapred-default.xml6 s' T0 n/ U$ Y% A) g
3 }: }, k* _+ J/ L0 R5 m
<?xml version="1.0"?>
' G2 E: ]( h' W# d<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
4 Z8 L+ R4 o# A- [0 R; F<!--
  }: T  H9 l  q   Licensed to the Apache Software Foundation (ASF) under one or more
$ @+ M' c5 E, a3 n. t: S3 i) e   contributor license agreements.  See the NOTICE file distributed with
# d+ S6 W& g/ A# \   this work for additional information regarding copyright ownership.% O4 m! u4 o, v2 ^: `
   The ASF licenses this file to You under the Apache License, Version 2.04 C% O# H. @% o# X! R# @! l" Y
   (the "License"); you may not use this file except in compliance with
) G& N* W4 q. K9 {  a   the License.  You may obtain a copy of the License at
5 W8 L/ ]2 _, j4 g3 x3 `4 M# V       http://www.apache.org/licenses/LICENSE-2.0
. h1 e6 o2 c# K! i/ u   Unless required by applicable law or agreed to in writing, software- N  q% `6 Z' s& e: e/ `
   distributed under the License is distributed on an "AS IS" BASIS,5 v9 J; ]3 d" r+ W( [6 Z
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.* b7 `% I. j  ?9 M1 N2 v
   See the License for the specific language governing permissions and6 H6 ~5 Q' B( m0 e: u
   limitations under the License.3 P2 i8 h1 h& r4 r1 Y/ X
-->2 s1 ~7 Q. t; B( }" ~1 s
<!-- Do not modify this file directly.  Instead, copy entries that you -->. j4 U# v/ V5 h2 D' T% T# O: L* g
<!-- wish to modify from this file into mapred-site.xml and change them -->( A$ e1 N+ F+ l8 J# {
<!-- there.  If mapred-site.xml does not already exist, create it.      -->
* H' f! t/ K2 d8 J  I<configuration>
2 J$ C4 v; Q+ H2 p4 I3 g<property>
" G( b, \, f0 I' U7 c  <name>mapreduce.job.hdfs-servers</name>
8 _: W6 S3 j( l# t1 x  <value>${fs.defaultFS}</value>9 F' j% I) a: Z! |
</property>
, Q7 [1 {+ i- f6 }! T<property>0 x  Q3 n0 _- W2 `3 U7 Q
  <name>mapreduce.job.committer.setup.cleanup.needed</name>
$ }3 L( B* l6 v  <value>true</value>
% M2 l: R5 Q$ |4 E1 y& v  <description> true, if job needs job-setup and job-cleanup.
3 D0 r( o5 v- ]/ c- D                false, otherwise
% p6 V8 L$ ^4 |  </description>7 ?3 z  ^: i# |# x' s1 b
</property>" d2 \, I4 z( @: }
<!-- i/o properties -->
/ @8 o7 l! F4 e, `7 D: O$ t<property>
: O% e$ L+ H$ o* K7 t' H! F  <name>mapreduce.task.io.sort.factor</name>
+ f- ]( E% E% Z8 z7 w  <value>10</value>
; S* C, R. m' R5 q2 l' F  <description>The number of streams to merge at once while sorting: n& p3 @) d- l& x+ [
  files.  This determines the number of open file handles.</description>
" q) z, H  c8 B+ |  i6 o0 m1 K! t</property>
' z1 b9 n" A0 l+ @<property>
* a- F) [* ?) {% i0 E$ m  <name>mapreduce.task.io.sort.mb</name>
1 t& c' N/ F/ ^0 \& D  <value>100</value>
! |3 D7 O  \: K/ S5 m. ?  <description>The total amount of buffer memory to use while sorting
( ^* Z) Q$ o, A0 Z4 @  files, in megabytes.  By default, gives each merge stream 1MB, which; Y) d% h! V! l" @. z1 Y
  should minimize seeks.</description>
6 d2 q9 i5 e6 ?1 j3 ?9 c9 c</property>
+ K( Z4 g) i, m. R; m+ |. g<property>9 V. T- K8 O1 o: F* [. j( U
  <name>mapreduce.map.sort.spill.percent</name>8 g/ U6 y9 G/ |2 H9 T( R
  <value>0.80</value>
1 D( O7 ]$ Y) e8 a  <description>The soft limit in the serialization buffer. Once reached, a
! S9 v2 J2 N" M, {1 @1 R$ _4 _  thread will begin to spill the contents to disk in the background. Note that
: S2 F2 c% x. m4 A7 s( @8 c  collection will not block if this threshold is exceeded while a spill is# W+ h4 l6 k0 a( U( K: R9 w" M8 M
  already in progress, so spills may be larger than this threshold when it is; h0 a- D) V1 e+ \2 Y
  set to less than .5</description>* p8 T9 Q- P& d, V- Q) |
</property>/ m6 D" e' T% A5 B+ P$ x, f8 e2 x/ e1 G
<property>
% \) w1 ?. F! d5 h3 D  l' a  <name>mapreduce.job.local-fs.single-disk-limit.bytes</name>  r' x+ w/ o) v) ^: n3 H5 r" U
  <value>-1</value>
2 @0 C5 v: w5 p7 O! v4 m- a  <description>Enable an in task monitor thread to watch for single disk
- T8 D4 k# G9 b( W# O3 e0 L4 \& ?    consumption by jobs. By setting this to x nr of bytes, the task will fast
8 U6 M) p4 x8 r6 W3 p    fail in case it is reached. This is a per disk configuration.</description>  z! `: d) W2 C) j" |# z
</property>
4 a+ C0 ~+ d7 f; O% g<property>& B" C" _. A+ V- M' G5 E; g  U
  <name>mapreduce.job.local-fs.single-disk-limit.check.interval-ms</name>
% f8 [/ M4 w8 u/ w! L# D+ F3 D  <value>5000</value>! o) x! t& d$ x8 `; J, p
  <description>Interval of disk limit check to run in ms.</description>
2 b1 u4 f7 ?6 Q0 i8 E4 G  n</property>
$ m& D; ?6 B5 p" a! i$ F; E. Z<property>" z3 U0 P) m- E  `4 {8 f
  <name>mapreduce.job.local-fs.single-disk-limit.check.kill-limit-exceed</name>0 |8 x  t) l+ N' j! s
  <value>true</value>
, W' M( `/ ~' S) i, [  <description>If mapreduce.job.local-fs.single-disk-limit.bytes is triggered! W4 }! W9 `% z
    should the task be killed or logged. If false the intent to kill the task
( T5 g; n2 M0 H8 E, r6 k( G6 |    is only logged in the container logs.</description>
6 P0 K+ M7 u5 ]; }/ x</property>" F" V, r2 @9 u, y3 Q( q) Z. e
<property>* W! ^# E* J, j: U; G4 ?' z
  <name>mapreduce.job.maps</name>
/ ?' L4 u+ j! L1 |8 R& x  <value>2</value>
) ?3 w$ p6 c' A3 q( @  <description>The default number of map tasks per job.& I4 ?" r5 N: o; M: k/ r+ {) j
  Ignored when mapreduce.framework.name is "local".4 E- G9 i* v( D% U  E4 q* d
  </description>
9 h. Y! b% k$ z  @# j) z</property>
. X9 Z- m1 Y# v- A5 }<property>
  }7 H! \3 r. G/ W% ~. L  <name>mapreduce.job.reduces</name>
8 w; i3 |4 Z5 }; B4 y0 x  <value>1</value>0 ]" ], ]% V) z+ E  s" p
  <description>The default number of reduce tasks per job. Typically set to 99%
1 G/ b( E$ p, p8 X& ]9 @  of the cluster's reduce capacity, so that if a node fails the reduces can- x7 m0 _  S9 o
  still be executed in a single wave.4 Y5 \: _" q" Z7 Q5 x) y
  Ignored when mapreduce.framework.name is "local".
8 l& u- c$ J' h  </description>
$ o! y. g$ q$ {</property>
: P2 \1 M, T# T<property>
( A6 R3 F" G) E' e* ?  <name>mapreduce.job.running.map.limit</name>
% {9 j) A( a: Z4 K+ b  <value>0</value>
/ U# q0 O! J9 y8 Q  <description>The maximum number of simultaneous map tasks per job.5 P+ u) @; K! Q& m$ s9 M0 A$ v$ z
  There is no limit if this value is 0 or negative.
$ e! r8 G: o; @8 N  x/ X  </description>
( R* t" T6 R! s  o9 X& K+ o6 [</property>4 u3 A9 i$ u2 k9 U- T( H6 \9 s. L
<property>& w% d7 E* f3 s0 o) H. k
  <name>mapreduce.job.running.reduce.limit</name>
  K7 u2 A6 \6 a  <value>0</value>
6 q4 X" T! f4 l* U" C  <description>The maximum number of simultaneous reduce tasks per job.: Y4 x, i* m' [. k6 ~" R
  There is no limit if this value is 0 or negative.& i/ b) m$ [9 b; t) X. K( _: I
  </description>
: U# R. ~2 C& R! a9 B" E</property>
9 W; ^, }2 ]% s' S7 f<property>
, H# x5 m7 h9 i/ v  <name>mapreduce.job.max.map</name>* l2 @0 M4 }% v. N; l! x6 l* V
  <value>-1</value>
4 T  t# [* |; k0 k& }$ }0 g; D  <description>Limit on the number of map tasks allowed per job.; g! Q# D: Z5 I
  There is no limit if this value is negative.
! w1 @( V9 T$ @- y  </description>% B, M8 j# z0 G! T* \
</property>/ \$ s  |; q+ j8 y1 [, U4 u9 a
  <property>
1 g. \9 x5 D( V5 |( l0 g3 z- u    <name>mapreduce.job.reducer.preempt.delay.sec</name>& M. S* f+ M: ?: _( e, N" K
    <value>0</value>; \5 N4 P( @2 w1 H& R
    <description>The threshold (in seconds) after which an unsatisfied0 e. T; t: n: E, q7 x, x" f! z  Y
      mapper request triggers reducer preemption when there is no anticipated% \  o( x1 \' X- f: d* W
      headroom. If set to 0 or a negative value, the reducer is preempted as" i9 c1 q, @4 m8 @  i1 G  Y7 V6 {' N% x
      soon as lack of headroom is detected. Default is 0.
: u0 C. g! V% H. \* M6 `+ V    </description>$ U& u& ?" `% S) b0 q
  </property>7 Y0 @; `' k) X  [
  <property>+ W4 b% F& k6 ^+ P$ r
    <name>mapreduce.job.reducer.unconditional-preempt.delay.sec</name>/ Q. T* c6 e$ G: I
    <value>300</value>- }3 T5 R& U1 j; O" O6 Y
    <description>The threshold (in seconds) after which an unsatisfied0 E2 N! a$ |# l. v( @' n- v
      mapper request triggers a forced reducer preemption irrespective of the
/ j7 `' L) ~# z( f      anticipated headroom. By default, it is set to 5 mins. Setting it to 0* m  F% F( S1 w; \/ F- r+ Y4 ]9 ^
      leads to immediate reducer preemption. Setting to -1 disables this
2 x; ^- N/ m  `/ b7 \      preemption altogether.
0 f, V& w; Y2 T5 B  r0 y    </description>
5 I1 ]2 b. l( k- d3 Y  </property>
: m- ^4 h* m& d; u% @  <property>
( W* ~8 |1 M" \* m$ v0 G  M" `, D    <name>mapreduce.job.max.split.locations</name>8 L* Z  N  M6 l2 Q: r% L
    <value>10</value>
1 i0 u* N; j4 S: g/ E9 q" X    <description>The max number of block locations to store for each split for
: _( o' W/ B2 t    locality calculation.0 C: e: w3 R5 i8 Y, w, |" S
    </description>& k, ^5 m: `- L' e5 B
</property>+ \, U' [) h7 |9 q" V3 {' U
<property>
2 O' K0 n, W! m: L) t6 D2 e# Q+ I  <name>mapreduce.job.split.metainfo.maxsize</name>
5 \" L! w% n/ c3 v5 a  <value>10000000</value>
0 r6 W# C& I# U. A% O8 _$ M  <description>The maximum permissible size of the split metainfo file.: H# B% i" v7 }" a
  The MapReduce ApplicationMaster won't attempt to read submitted split metainfo7 M4 j5 Z. j, Q6 h. W) f7 T1 i8 v6 E
  files bigger than this configured value.
1 Q# {/ S& B5 i: Z. O' Z  No limits if set to -1.
# i5 N) Y7 `" v, h; r1 J  </description>
9 s% m% B9 }2 P  H</property>
  |$ j3 Q% M- u! [  c$ E$ I<property>
4 A. X$ |4 @+ C& S  <name>mapreduce.map.maxattempts</name>1 D6 a: p+ T; J! M" ^
  <value>4</value>
% H$ J! ~' s9 V( o  <description>Expert: The maximum number of attempts per map task.- \& p! d+ l! v5 u
  In other words, framework will try to execute a map task these many number' d' V6 k8 z" B% ?3 `' h
  of times before giving up on it.- y1 O8 F4 k1 R# f7 Z5 W: c
  </description>7 F/ n& n9 Y) d* D. S1 b
</property>5 o' d; K" b- F. P# a9 e: r
<property>
% @) _! T. X8 W  <name>mapreduce.reduce.maxattempts</name>. B4 P! M! e: Q6 Q! o! V
  <value>4</value>
5 _% _7 l  @/ }9 X8 R7 f, U2 l  <description>Expert: The maximum number of attempts per reduce task.( [/ c# R7 L5 c  k0 L. Z# O
  In other words, framework will try to execute a reduce task these many number- ^0 g- W- O6 H, d% f+ R- Z* q2 h# C
  of times before giving up on it.$ J: i+ s+ f" s; G
  </description>
( X7 U' O. R0 z' \  s</property>
) Z% V- Q$ A  r5 C, f- f<property>
4 Y% M5 x/ t. I& H8 ~  <name>mapreduce.reduce.shuffle.fetch.retry.enabled</name>
6 z  Y# [3 e- J& v$ C  <value>${yarn.nodemanager.recovery.enabled}</value>
9 _8 G! K6 b0 ]) Q0 ~* I& d/ c( ^  <description>Set to enable fetch retry during host restart.</description>+ Z8 w, ^' z; I: ~% S# T
</property>% `' m/ K3 ?' i
<property>$ d/ s8 D+ }& A' R
  <name>mapreduce.reduce.shuffle.fetch.retry.interval-ms</name>
6 D2 m) S+ Q6 T  <value>1000</value>2 n4 ]7 w+ I- y9 t# v+ _' O
  <description>Time of interval that fetcher retry to fetch again when some
5 O5 U9 |. s9 c, T; ?  non-fatal failure happens because of some events like NM restart.0 Y1 A- Z& g2 U) J! ]) a; O: }
  </description>
' o4 J% G* J& p1 c! r</property>
0 ^2 f+ I7 Q7 R0 o& b3 R<property>7 @) u$ R) V# v% O5 O. v1 E6 Z
  <name>mapreduce.reduce.shuffle.fetch.retry.timeout-ms</name>  N1 l( g% F5 K0 c$ B2 ]
  <value>30000</value>0 `( u2 o1 u+ l6 ^8 ]" N* f
  <description>Timeout value for fetcher to retry to fetch again when some
. j+ k+ B4 L" ]1 L* T  non-fatal failure happens because of some events like NM restart.</description>
2 P" b6 s9 H1 [2 d* m0 J( S6 m</property>
) M2 q: n+ a' U4 b9 d+ @* Y<property>
9 S5 F2 C4 ?3 d  <name>mapreduce.reduce.shuffle.retry-delay.max.ms</name>
- y5 ^/ `, e2 O2 c) f  <value>60000</value>
3 q0 h9 i5 k* H+ C9 V  <description>The maximum number of ms the reducer will delay before retrying
/ t; a4 P- `- G. }. C' j# W) ?  to download map data.
& \- H# U$ ?: V$ C4 y  </description>; P, r3 D* r! o3 S
</property>
* u; ^& C) f9 J( ?% h4 n8 k  h<property>
* }& w, Y0 J; k1 H& n  <name>mapreduce.reduce.shuffle.parallelcopies</name>
, c! S. P# ^5 g' Z! O" |- Z( @0 h  <value>5</value>" {1 z! u8 S- K4 T7 k2 Q5 j/ B& D
  <description>The default number of parallel transfers run by reduce
& l7 a' c& T9 R: b( V- z  during the copy(shuffle) phase.
( A7 t5 v/ ^) K" S  </description>& }' Y' ]  K. n) d# F
</property>+ ]1 |  k$ x/ g6 i. I% d
<property>$ d5 a+ z4 ?  N$ f+ v! E8 }7 N7 F8 Q. w
  <name>mapreduce.reduce.shuffle.connect.timeout</name>+ t6 K( o0 [1 q; a  U9 {
  <value>180000</value>* d" I9 f  L1 M* Q( E7 U. c/ `
  <description>Expert: The maximum amount of time (in milli seconds) reduce
& C8 S4 i) @% t) Y/ T  task spends in trying to connect to a remote node for getting map output.) z4 O1 V6 _4 F
  </description>7 s  v+ Y/ x' Z# x: p4 Z* Q7 N
</property>- R' F, ^+ w+ z' b
<property>$ U5 c3 e! x6 T7 E* U5 @
  <name>mapreduce.reduce.shuffle.read.timeout</name>* Q8 N6 C0 ~) Z8 q+ @
  <value>180000</value>
7 C" N8 a8 E6 _) o' i1 O  <description>Expert: The maximum amount of time (in milli seconds) reduce
. r9 d9 i0 ]1 A: |: p9 q  task waits for map output data to be available for reading after obtaining; A; D0 d- W& b' k' M" `7 W
  connection.7 @  r) `3 J6 b* A+ c) s( C. \6 I3 ^
  </description>
* H2 J7 x- v$ y3 X1 K' A' W: \</property>
# W8 w! V1 `  K( B3 j7 z# P<property>0 l7 t, n$ o/ z- u# q( y% V" n3 ^9 m
  <name>mapreduce.shuffle.listen.queue.size</name>9 d# `% Q! S3 Q8 n+ o# r/ {
  <value>128</value>+ X" @! Y9 D. d% R1 l* t
  <description>The length of the shuffle server listen queue.</description>1 F: A- w: B# u" F) c
</property>
# V) ~7 T: Z" j4 m<property>
; E$ U. Z  w, h1 ^4 r  <name>mapreduce.shuffle.connection-keep-alive.enable</name>
$ z5 U& ?7 |! V' y; J& e  <value>false</value>8 x) q) ]& h& p& G$ v  a. ~& J8 o/ Y
  <description>set to true to support keep-alive connections.</description>
+ C' E2 U. O: j5 ^</property>
( ^. ]9 ]' y- ]; P3 J<property>" L. A+ Z! x8 q1 E& }5 R! N5 ^. }
  <name>mapreduce.shuffle.connection-keep-alive.timeout</name>
* g0 w! A) J& C9 |* x  <value>5</value>/ a2 z$ n4 b8 Q+ t
  <description>The number of seconds a shuffle client attempts to retain
, D$ u  w2 q3 B: Q, z4 N   http connection. Refer "Keep-Alive: timeout=" header in3 a- _6 I7 B- j+ G" L3 x( l
   Http specification9 |. |+ F3 @) K: z" G- \
  </description>1 p! @& r- L# u- G- J
</property>
+ R9 H- h4 Z2 J) w  d<property>
" @$ V2 c5 D) j) @  <name>mapreduce.task.timeout</name>
) k' p: V( K0 j$ k  <value>600000</value>
9 C  ]/ m% x' `  I  <description>The number of milliseconds before a task will be# r4 H- \. H! l1 X
  terminated if it neither reads an input, writes an output, nor5 Y( u* J4 M' t2 {% k$ I
  updates its status string.  A value of 0 disables the timeout.- n* [" o2 k% k  g" p
  </description>6 h# Y/ f2 e: J0 V1 X0 s8 d
</property>; ]2 [. Q$ {- X5 f! I
<property>- D# U* ]7 F5 l3 ~, r- N8 p
  <name>mapreduce.map.memory.mb</name>
8 ^- i; w* Z  h  <value>-1</value>
/ ^" L- E" q- F- ^+ X0 f/ l$ Z  <description>The amount of memory to request from the scheduler for each
  d; B( p1 ^# l: Y5 ~    map task. If this is not specified or is non-positive, it is inferred from
/ q# {# S3 V: s    mapreduce.map.java.opts and mapreduce.job.heap.memory-mb.ratio.
/ ]' z0 s' z& g+ k: T8 u9 x    If java-opts are also not specified, we set it to 1024.
. I6 a9 A2 b* Z3 m  </description>
/ [/ Y! o. {; j, b</property>3 T1 p, T8 m) x. _; A. g# _
<property>7 q* m& I# U+ s/ `) A  x9 \5 w; v- q
  <name>mapreduce.map.cpu.vcores</name>6 _7 ^* ^' v- s! ^# V- Q
  <value>1</value>
0 F* y- ]# F( P; D. j2 c  <description>The number of virtual cores to request from the scheduler for
% Z) ?! d- t! D6 o) d  each map task.
# w. s, F& e+ N  </description>3 N" ]7 ^9 X  p8 u6 Z) J+ k0 {
</property>
* q3 J; l/ G: ~" w8 J<property>
5 e- f3 w( a$ P# ?2 \  <name>mapreduce.reduce.memory.mb</name>1 |' Q5 |% [" C1 L3 }$ z" i
  <value>-1</value>
; \' k/ q( k" r& Y+ b9 d  E0 S7 f  <description>The amount of memory to request from the scheduler for each4 _! v5 h; P1 _- e+ M! y
    reduce task. If this is not specified or is non-positive, it is inferred# B& j* Q2 U% O5 ~6 P
    from mapreduce.reduce.java.opts and mapreduce.job.heap.memory-mb.ratio.; ^9 }" i# u0 d$ {) [
    If java-opts are also not specified, we set it to 1024.
: ^; u1 S; ]% ^/ K4 r3 q* R  </description>
0 v8 E: i) M5 {0 \2 N</property>3 U8 a* S  [$ U. L% B+ M
<property>
( {9 o8 Y2 [4 y8 K9 x9 v  <name>mapreduce.reduce.cpu.vcores</name>" U, s" L% s& |8 J7 }; t( |
  <value>1</value>5 N9 i5 \! Y: C3 O+ Z" j) f5 m* P
  <description>The number of virtual cores to request from the scheduler for  J  H1 @5 y- b& c2 `( X4 ]
  each reduce task.) k& o1 x3 M( I' P
  </description>" n- u# O8 _- Q% ?
</property># @. q+ {. k! R3 R6 b
<property>9 N7 H3 D( _  ~0 d
  <name>mapred.child.java.opts</name>7 }6 k' C% C6 \# e1 r
  <value></value>
  }3 `1 c1 `# _, u' e8 w1 F  <description>Java opts for the task processes.
2 q6 P4 @, c# o7 f3 w  The following symbol, if present, will be interpolated: @taskid@ is replaced
2 W" w: C! V9 X- x  by current TaskID. Any other occurrences of '@' will go unchanged.
4 ~" m4 [: q0 P' v  For example, to enable verbose gc logging to a file named for the taskid in
: m3 N" ]3 d/ x  /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:# T! ]8 h5 Z: M7 @
        -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
# i  x) p# l/ g- b9 ]' K8 X, l$ u  Usage of -Djava.library.path can cause programs to no longer function if5 ?. p9 Y/ A  R3 v
  hadoop native libraries are used. These values should instead be set as part
5 n4 T9 G: `% }( v  of LD_LIBRARY_PATH in the map / reduce JVM env using the mapreduce.map.env and4 h( i, w6 k5 e9 y8 I* }
  mapreduce.reduce.env config settings.
1 w) V! R7 a* |9 r+ r) @- Q  If -Xmx is not set, it is inferred from mapreduce.{map|reduce}.memory.mb and( `: O2 R& ^% {5 V$ Y
  mapreduce.job.heap.memory-mb.ratio.5 G0 H' i! l! v6 y4 [
  </description>
7 T. Y& i  K+ t! ~</property>7 u! ?- y- k7 Y0 M7 e
<!-- This is commented out so that it won't override mapred.child.java.opts.
' Q8 Z- J, y! O* ]) [: ]' b<property>" a) J# E0 P- e4 O6 e) j, H
  <name>mapreduce.map.java.opts</name>
  }, t6 E% L# r2 S  <value></value># o" Y: h% E, ]. K6 i% \
  <description>Java opts only for the child processes that are maps. If set,
1 {6 @0 P! `% ?: O  this will be used instead of mapred.child.java.opts. If -Xmx is not set,
' B  ]. N2 @7 I! D3 o# b  it is inferred from mapreduce.map.memory.mb and( i8 C( |7 c2 \) m- i* l5 J
  mapreduce.job.heap.memory-mb.ratio., x( D" }; i" [1 h* p% o0 N1 w( `" T
  </description>0 H; V* U( A$ @7 e& y
</property>( K' |( y# }' E  Y, G
-->( }7 Q- @% \: X  \7 M) h2 T- F" l
<!-- This is commented out so that it won't override mapred.child.java.opts.
2 r1 ]8 c) r2 R. M3 @<property>0 X0 G; k& T- n. `
  <name>mapreduce.reduce.java.opts</name>
/ [+ @" x, f4 I0 S  <value></value>
! G& [' D1 u* H/ p  <description>Java opts only for the child processes that are reduces. If set,
' x) n3 }: B+ P9 \) a; {- r  this will be used instead of mapred.child.java.opts. If -Xmx is not set,
2 h; X$ J# r5 |/ I  it is inferred from mapreduce.reduce.memory.mb and- J; l. f+ X4 ]0 r8 `; W9 b
  mapreduce.job.heap.memory-mb.ratio.
% K. ]/ B" F7 v# D# \! y  </description>* I: G6 K, A& {6 c- N
</property>
* H( G5 x6 a  I: e-->) t- o3 P1 o; |# f6 f
<property>
+ X) K! J3 P# J  <name>mapred.child.env</name>5 T/ H" H( q& \3 h0 Z
  <value></value>9 }5 {" i0 f5 q; c
  <description>User added environment variables for the task processes.) r( H- O. m' ^4 z7 M' ]. }
  Example :) N+ H; B0 Y% y- T) A) N& {
  1) A=foo  This will set the env variable A to foo! e2 j( W9 R: v1 k# L% h% N! [1 z. q
  2) B=$B:c This is inherit nodemanager's B env variable on Unix.* \$ ^/ O  h. z
  3) B=%B%;c This is inherit nodemanager's B env variable on Windows.
4 L+ z0 Z1 g/ w8 W3 S; o0 z8 \  </description>; e2 Y) t6 X3 ?) s
</property>' m: c) \  f2 }% `1 ]
<!-- This is commented out so that it won't override mapred.child.env.
% H2 \  X9 `( d; a: j# {<property>, b  j4 @0 C/ N' F
  <name>mapreduce.map.env</name>
( i  x% h7 N2 _  <value></value>( s& X9 G2 Q9 }
  <description>User added environment variables for the map task processes.
6 ]% a' u% J0 ?0 M, y8 O" i7 q  </description>" I# d- c9 I8 J5 e4 [1 u( b
</property>9 C1 I2 ~2 B# ~* Q+ E# l1 \
-->
  N! b& J& d  J  _<!-- This is commented out so that it won't override mapred.child.env.: D; c0 v0 }) q( l4 \  Z  h1 ?) G/ i
<property>
- M9 q9 B. F/ w: y* {# ]8 B  <name>mapreduce.reduce.env</name>. Q+ j; Y0 c! s' E5 ^
  <value></value>& R% M, F+ h9 k+ _. Y6 N5 n! J
  <description>User added environment variables for the reduce task processes.4 O/ L4 {) ^* g$ [
  </description>8 m$ E% y* ]2 g
</property>
( m! L& G! h0 q0 q-->3 G4 x+ E/ f; x5 T6 O: I
<property>8 h: L& n- k8 [! E: O- M5 H. n
  <name>mapreduce.admin.user.env</name># u# H; V# Q0 _' {& _$ x; V
  <value></value>
+ g' g; m7 Q+ }+ M. |  <description>% F* o1 X: \6 `9 F; y! u( m/ U
  Expert: Additional execution environment entries for5 `( W7 v$ W$ ]5 t- q- e  f/ e% u( f
  map and reduce task processes. This is not an additive property.# s; F& M5 x- b$ J0 c
  You must preserve the original value if you want your map and
- @) [* q* I! X# ^# b5 h  reduce tasks to have access to native libraries (compression, etc).
4 u9 V* J( [& C9 v  When this value is empty, the command to set execution' v1 M; p2 M6 T3 c4 t
  envrionment will be OS dependent:
3 d+ ^- @2 y: o/ |  For linux, use LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native.
8 _* x% k, G! G7 G! y: J9 Q$ h, k- a  For windows, use PATH = %PATH%;%HADOOP_COMMON_HOME%\\bin.0 M. R+ @* z$ a% t
  </description>+ ]" H& \& i( F$ v& M
</property>
5 E4 B) \( f2 K8 j* G  Q  C<property>+ T! w. L/ a+ U% Y: ^
  <name>yarn.app.mapreduce.am.log.level</name># U5 J0 {$ R- U
  <value>INFO</value>
$ N3 u# ]* m$ w% G  n  <description>The logging level for the MR ApplicationMaster. The allowed+ k' w! @. U) N5 Z' x: a  g& d  X
  levels are: OFF, FATAL, ERROR, WARN, INFO, DEBUG, TRACE and ALL.
9 J- [" l, g  K3 s: H+ Y0 Q. R! J2 K  The setting here could be overriden if "mapreduce.job.log4j-properties-file"
2 R( `1 |: p1 l, j4 Y  is set.& e: M) N" [# ?6 i5 o2 Y9 ^
  </description>
) @  {5 T8 m# P; U" q' o</property>
4 m) ^5 q1 R6 n6 a: @( `8 g<property>
" q% ~5 C3 j  _6 |4 z$ t, x  <name>mapreduce.map.log.level</name>
1 s6 m0 c* J; H% z; Y" V  <value>INFO</value>
- ]. d) r( v! V$ W: ~8 ]7 W( u  <description>The logging level for the map task. The allowed levels are:
1 U' E7 U. d1 }" B  OFF, FATAL, ERROR, WARN, INFO, DEBUG, TRACE and ALL.8 w1 B$ |! j$ C  z
  The setting here could be overridden if "mapreduce.job.log4j-properties-file"
% P# Q+ K- A3 Z7 N3 I  is set.
8 p7 l) X* k" D! |  </description>
+ \- r8 |9 b# r</property>' i  K* q$ v# d+ e  f7 o
<property>2 ?/ {% s. V( b% N# |& p$ n6 ?% u; j
  <name>mapreduce.reduce.log.level</name>
: _3 U; [0 U7 g" Z) C2 g/ ]7 y  <value>INFO</value>
6 p2 d5 z) G8 O, i) U  ^' d$ n  <description>The logging level for the reduce task. The allowed levels are:% P! {& E& Z  b1 [
  OFF, FATAL, ERROR, WARN, INFO, DEBUG, TRACE and ALL.* V; y" \0 _  Q& a: M
  The setting here could be overridden if "mapreduce.job.log4j-properties-file"- p* R( _7 n5 G
  is set.
+ l# t8 a2 x: A/ N6 ~- {+ [8 [: K  </description>
2 t) Y* M1 ^  f% S6 U</property>
; S- ]* J( d# @7 R/ f1 C& N! N5 h. V; [<property>  w7 C* ~7 Q' `' x" q$ o8 ?8 O3 L4 q
  <name>mapreduce.reduce.merge.inmem.threshold</name>
. d" v& ~6 U( t) g& K' p8 T! F7 J  <value>1000</value>
) z! K  X! d9 k' U+ D1 A  j  <description>The threshold, in terms of the number of files
7 N3 `7 A: Y5 r5 }" D7 h, M  for the in-memory merge process. When we accumulate threshold number of files; ?0 R. l6 F5 X6 k1 |& |9 Q
  we initiate the in-memory merge and spill to disk. A value of 0 or less than
% \6 {0 N9 s& ?# A! w  0 indicates we want to DON'T have any threshold and instead depend only on4 {& y+ |  i4 \5 m
  the ramfs's memory consumption to trigger the merge.) Z! T9 a# N. f, k9 R2 J
  </description>
* V* ?! S' P/ v$ [" b/ D</property>7 d4 {7 P4 L9 I; b' |
<property>+ s4 A9 X" |% A& T. T
  <name>mapreduce.reduce.shuffle.merge.percent</name>
. g* k0 V, V) L+ ~8 W  <value>0.66</value>
) @, H/ R# u" u: I  <description>The usage threshold at which an in-memory merge will be8 ?+ n' Z' U2 k" u% V
  initiated, expressed as a percentage of the total memory allocated to
0 U" \4 A2 x, Q6 Q( |  V# U  storing in-memory map outputs, as defined by* L" s5 C+ }) s& [% D8 B- q
  mapreduce.reduce.shuffle.input.buffer.percent.& v. b: }) |8 W; [, m3 _, r3 n
  </description>+ e; Q+ v4 H3 C) N% a
</property>! D' x' g; j* ~/ A' L
<property>
! k; x- e' \5 |( p, M7 K6 v5 x/ p, P  <name>mapreduce.reduce.shuffle.input.buffer.percent</name>6 _" o& K4 G4 j' O
  <value>0.70</value>
/ ~& D6 c1 z5 i9 |* R* W8 t  <description>The percentage of memory to be allocated from the maximum heap" L! N, W8 M- b
  size to storing map outputs during the shuffle.$ x, U3 g8 v$ e, n: ]0 a$ Z" E
  </description>9 B8 c  p+ z  j
</property>- h2 {2 _6 X( s+ m/ \* `  r. \" g1 f# {
<property>: T- ?1 ^! |% K: `
  <name>mapreduce.reduce.input.buffer.percent</name>
3 G4 d- i/ W/ P% r  <value>0.0</value>
& }; A# p: x8 t, G0 I  <description>The percentage of memory- relative to the maximum heap size- to, ^  @) v' S9 {
  retain map outputs during the reduce. When the shuffle is concluded, any
. E3 q) }9 F: o, W' I* V  remaining map outputs in memory must consume less than this threshold before4 J) o3 X. a' v2 g2 h" x
  the reduce can begin.8 Q+ t" s+ Z6 a" H5 N( c1 ^% H- V9 f
  </description>
( m3 h6 }* w- R</property>
: V( H" N/ c; \! s' S9 z<property>" b0 E8 k) X/ Q( H% [3 c7 k
  <name>mapreduce.reduce.shuffle.memory.limit.percent</name>
) M2 ^  H9 a, |8 K' U; ~  <value>0.25</value>
9 o+ u! @0 A0 `  o  <description>Expert: Maximum percentage of the in-memory limit that a- W& ~% }3 A- |3 L6 x- L+ i7 V; ]
  single shuffle can consume. Range of valid values is [0.0, 1.0]. If the value
! E0 s, d* Y/ L- d! Q: U  is 0.0 map outputs are shuffled directly to disk.</description>1 R/ m& S( |( ?( ^1 {# K
</property>6 t7 n' r; K: `1 @& \; @
<property>
2 g' x. |2 j( p% J8 ]  <name>mapreduce.shuffle.ssl.enabled</name>
3 c) i; Z5 \" a& g& z, i9 }  <value>false</value>
# B1 X! X* |6 N  G, ~, F* [1 m* i  <description>& @5 T) {3 i4 Y& m
    Whether to use SSL for for the Shuffle HTTP endpoints.; X% K: Z: i) R. n" w
  </description>
7 `4 [0 Z7 e) _) l$ `# f" P</property>1 w2 R, i% p- P! A. e
<property>' C0 _8 t! j6 A, P) P
  <name>mapreduce.shuffle.ssl.file.buffer.size</name>
6 y/ S- }" F4 L. U+ g3 Y4 G  <value>65536</value>2 _: G" F3 J* m& u9 F+ L0 H
  <description>Buffer size for reading spills from file when using SSL.
# K1 f+ K0 ^4 ^+ E0 C  </description>
) ^' o0 J# p  j/ ]' h) r2 d</property>4 u) _0 t" a. G) K
<property>
8 s  ?" t) G, {! f  <name>mapreduce.shuffle.max.connections</name>
1 U+ ^8 k+ N0 k" _' p) L: v  <value>0</value>8 |+ ]5 G9 E% Q6 x, L
  <description>Max allowed connections for the shuffle.  Set to 0 (zero), |% k4 P3 b2 p: j# {/ Z
               to indicate no limit on the number of connections.
2 O2 ]$ D0 U; n2 Q% H  </description>
' I$ ^1 D3 ]" L3 V6 I- c! ^; `7 b1 s</property>$ X1 R9 M& G7 }& ^" t5 I$ J6 r
<property>
0 }# o+ W8 w, t  <name>mapreduce.shuffle.max.threads</name>
$ ~% C7 u9 O2 Y2 |6 j  <value>0</value>& G; h  w( |, Z5 r
  <description>Max allowed threads for serving shuffle connections. Set to zero9 l: b: Y7 ^; M6 k7 N4 |* X7 B
  to indicate the default of 2 times the number of available/ i' n: Q, A" {: \
  processors (as reported by Runtime.availableProcessors()). Netty is used to3 m* W( T  _' p3 k8 P3 P
  serve requests, so a thread is not needed for each connection.
" q& y+ s3 k) u6 O$ d  </description>
+ j3 `9 P% h& [</property>
3 [! o1 e- f/ D- U( P<property>
2 ]& m8 l5 d$ X) }/ |3 K, x6 H; J. _  <name>mapreduce.shuffle.transferTo.allowed</name>
7 X0 c$ W) p8 R. B- j  <value></value>
5 \0 C3 _# r# P" X  <description>This option can enable/disable using nio transferTo method in
  u; b0 I3 w) s+ ^0 ?* s  the shuffle phase. NIO transferTo does not perform well on windows in the: V: V# f+ O7 B  O7 x) [
  shuffle phase. Thus, with this configuration property it is possible to
, }% T$ W6 z$ O9 `7 X  disable it, in which case custom transfer method will be used. Recommended
) J" _( g: f, i3 w9 ~: l. |* Y  value is false when running Hadoop on Windows. For Linux, it is recommended; M/ b, m- L/ d: S
  to set it to true. If nothing is set then the default value is false for
/ K% ~  p( Q  ^  w  Windows, and true for Linux.4 z1 U7 W0 T$ ]) }9 m
  </description>0 T1 I- D: |3 t6 I
</property>9 h* [: f6 c" |3 g
<property>
6 G6 x/ n6 M4 X  f4 A  <name>mapreduce.shuffle.transfer.buffer.size</name>
6 V' Q! E+ p6 o7 h. r  <value>131072</value>
" F/ ^1 v8 |/ |$ t5 e  I3 ~  <description>This property is used only if
/ \) C$ ~; z: ?  mapreduce.shuffle.transferTo.allowed is set to false. In that case,! s$ f3 k  S: G
  this property defines the size of the buffer used in the buffer copy code
& _6 a7 o" R$ H# X/ S( u  for the shuffle phase. The size of this buffer determines the size of the IO
1 F9 [) s7 y  ^5 T; Z% X( v  requests.
  j) w- e+ }7 C$ w6 Z  </description>
6 J# L( C; _. b+ |</property>: S: i9 F- f+ G# [3 k6 a
<property>8 X3 S) c6 `! a/ Q' `3 `
  <name>mapreduce.reduce.markreset.buffer.percent</name>
  o8 [6 ?; y7 g4 ?  y5 C  <value>0.0</value>0 `# {/ l' W0 {8 T# J$ {# b, E9 I8 c
  <description>The percentage of memory -relative to the maximum heap size- to5 b$ m' f3 _) d, k' t2 Y
  be used for caching values when using the mark-reset functionality.5 F6 m) @" Z8 }: \, |" {
  </description>
9 L" b3 U* e) D* m; w! h; j</property>+ {) v* Q0 w& w3 j
<property>
' S4 I7 {7 A3 s( y/ C- e  <name>mapreduce.map.speculative</name>3 x- v9 Q5 I1 R8 S( {( v* J  N
  <value>true</value>
0 {. S9 }$ ]6 q, X3 w2 l  <description>If true, then multiple instances of some map tasks
) w( @4 W) Z. m4 l- }. }               may be executed in parallel.</description>
" I! L' S# _+ v1 ~</property>
5 W+ H5 @. V& Z8 ^  o% ^! D<property>
" s( ]/ h# g; c) V0 O# k: S  <name>mapreduce.reduce.speculative</name>
$ y+ R7 w; V8 {  |  <value>true</value>
) u5 u% Z5 {6 |, b3 Q! [% ^6 U- r  <description>If true, then multiple instances of some reduce tasks
) t* h, d8 t' B$ ^1 D* X/ v               may be executed in parallel.</description>7 J6 \' N3 S4 ]9 G# _0 ~
</property>6 w7 E6 x" _- r7 F& _
<property>
7 o: d" I: J8 A4 v' O( S  <name>mapreduce.job.speculative.speculative-cap-running-tasks</name>. T) ^  r1 h; s( g( G. Z
  <value>0.1</value>
( q" W/ |  w# }  <description>The max percent (0-1) of running tasks that* o5 R; C) S: I" E0 G. p0 s
  can be speculatively re-executed at any time.</description>6 e/ D) x1 p/ |  n' o: l
</property>
9 _; p6 Q) c* D/ r6 P( m<property>, |4 u$ `& |6 x1 [
  <name>mapreduce.job.speculative.speculative-cap-total-tasks</name>! B( M) t, X0 q9 a' L& i4 F
  <value>0.01</value>: h4 ?  [1 \8 i% i& G
  <description>The max percent (0-1) of all tasks that" C% {; Y, U1 ~5 e' K
  can be speculatively re-executed at any time.</description>
) e8 u+ i- @% y# \& j) G2 e</property>0 [! C3 ~, {+ d+ Q- h! V  w
<property>
) \  Y$ p" J7 t" v* {7 [1 e7 O! z  <name>mapreduce.job.speculative.minimum-allowed-tasks</name>
# k% Z: }6 ]" ?4 O5 A  <value>10</value>
# Y  }2 O( a$ _( ?: `8 X  <description>The minimum allowed tasks that, e' d- C1 d7 }$ M
  can be speculatively re-executed at any time.</description>0 R4 H5 W, Z- ^
</property>; Q1 D- O2 P4 P) G
<property>3 k  Y& P7 f- c+ W+ ^. Z
  <name>mapreduce.job.speculative.retry-after-no-speculate</name>/ P. S, s/ B$ G/ U/ n0 }; i
  <value>1000</value>
9 C- m/ M* v5 z* l) P; t  y  <description>The waiting time(ms) to do next round of speculation
6 O4 `6 A3 [) m& j0 k9 B7 V  if there is no task speculated in this round.</description>
( \! f0 n. c  Z& e$ C. ?4 v% a" O$ }</property>, v9 L' e% f( n# G4 T3 `
<property>
6 i+ T2 B; T. C* ]% V  <name>mapreduce.job.speculative.retry-after-speculate</name>
3 X& F) _2 v  L# \" I; H# S- g  s5 ]4 `  <value>15000</value>
9 _( ?2 q0 ]; B8 P  <description>The waiting time(ms) to do next round of speculation2 f8 m" A& H9 c+ K: S( h) b
  if there are tasks speculated in this round.</description>6 B0 ]7 d+ U+ C  z0 z0 ]; k
</property>& l0 y3 p7 h3 N- X( V* J. M7 y+ @
<property>
* n# q9 v2 u& `5 q1 @6 H4 r  <name>mapreduce.job.map.output.collector.class</name>
! S7 ~% x. a+ P# J7 R0 k/ B  <value>org.apache.hadoop.mapred.MapTask$MapOutputBuffer</value>
& D' D# \: H+ [6 d5 ~  <description>9 t5 |& s  g/ Z4 ?6 h
    The MapOutputCollector implementation(s) to use. This may be a comma-separated8 ~+ r$ [" V' q7 V: j, o! z2 d$ D3 m
    list of class names, in which case the map task will try to initialize each8 |. V. {$ i' Q6 D1 v, i
    of the collectors in turn. The first to successfully initialize will be used.
( l# [( ?" K2 V& E6 a: h  </description>
) j- q8 _1 H9 ^& T( [+ ~, w</property>
0 k% ^( W% s) `8 x; T% h) c<property>
0 D7 v! Z. R- n. q# d  <name>mapreduce.job.speculative.slowtaskthreshold</name>
( C/ t" M0 V: @# g9 g# n- p( n  <value>1.0</value>- F, v2 P4 y  N7 Y, ?, [
  <description>The number of standard deviations by which a task's
. y5 e) ~8 i# j9 D  ave progress-rates must be lower than the average of all running tasks'5 u; z" q. X/ m4 }9 E6 E
  for the task to be considered too slow.
' K1 M' d% R/ R$ T  </description>/ j% m, `8 _7 y
</property>
2 Z" E9 ^3 z7 w' [' |<property>
+ P. R3 E7 G1 I( Z: O6 e  <name>mapreduce.job.ubertask.enable</name>7 E# z* \) L7 A4 P$ f: U. O' X, d1 [* x
  <value>false</value>
' K7 y+ M$ v- ~  k  h  <description>Whether to enable the small-jobs "ubertask" optimization,5 V3 F, P' f5 y3 p
  which runs "sufficiently small" jobs sequentially within a single JVM.
1 Y3 L& v& s3 }3 M) U6 R# h  "Small" is defined by the following maxmaps, maxreduces, and maxbytes
2 V0 \2 V! u; t$ }/ p, n  settings. Note that configurations for application masters also affect% K; F! Q  Q0 g/ N2 u' s
  the "Small" definition - yarn.app.mapreduce.am.resource.mb must be$ y2 H5 U' Y9 x( W& z
  larger than both mapreduce.map.memory.mb and mapreduce.reduce.memory.mb,( u$ M1 p+ C2 `1 g" X7 M
  and yarn.app.mapreduce.am.resource.cpu-vcores must be larger than
- H' i; y' G/ D  both mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores to enable
! g6 m9 f7 j1 f& C) d  ubertask. Users may override this value.+ X4 H! z+ w  F
  </description>
- L; a2 k; O: e2 ?</property>  t+ O: N! v/ ]% H7 J4 t
<property>
; ?0 S: n* ?+ L. u1 r4 A/ R% N% Y) G% _  <name>mapreduce.job.ubertask.maxmaps</name>
3 m7 S: q  o8 \1 U  <value>9</value>
+ c1 P; O* _: q  <description>Threshold for number of maps, beyond which job is considered
1 l$ ~0 D3 F9 N  too big for the ubertasking optimization.  Users may override this value,
/ M+ ?2 e  E- L8 `' t. ^  but only downward.( w3 W# F3 }% m* I, |" p2 W
  </description>$ C! F+ L  T7 v+ k8 p9 A
</property>/ Y0 `8 y5 n! a( {! e
<property>
2 ^; T7 X$ b# E; b  <name>mapreduce.job.ubertask.maxreduces</name>
/ |& E! n3 Y7 ?. N% p  _& U0 O  <value>1</value>) v4 K! h4 {" u8 q: p# `2 p  e7 R5 g* _
  <description>Threshold for number of reduces, beyond which job is considered
) q$ K% @/ g7 }2 F  too big for the ubertasking optimization.  CURRENTLY THE CODE CANNOT SUPPORT
. G+ V0 D4 C- w) }& p3 Y( f6 ^  MORE THAN ONE REDUCE and will ignore larger values.  (Zero is a valid max,4 F/ M% D- l5 e( p, h' S$ K4 P
  however.)  Users may override this value, but only downward.
% e+ ~; A! o0 ]4 w8 W4 K5 _# I; C  </description>7 a! V0 c1 K3 P2 \0 u, i, A7 Z
</property>6 A6 t* T2 \1 l6 q
<property>: B) [$ E2 Y6 F) |, O: |, ?
  <name>mapreduce.job.ubertask.maxbytes</name>
. R% {5 G$ \& z! K- }  <value></value>
' `1 b' u8 h. n' n4 t) N( ]- a, i$ D8 g, s  <description>Threshold for number of input bytes, beyond which job is
* f6 @! @. Z5 Q3 D9 E3 {  considered too big for the ubertasking optimization.  If no value is
- F# h8 v0 J& J6 N4 v, [$ y* ^3 s  specified, dfs.block.size is used as a default.  Be sure to specify a# o) `. [. r: h, ]
  default value in mapred-site.xml if the underlying filesystem is not HDFS.
. ?: r! y9 E4 b. i/ c- i  Users may override this value, but only downward./ I4 @$ H0 L# T8 }  K1 ~+ M
  </description>
) v; P" [  f5 F6 h. u</property>
8 U* W$ w: V2 S$ x) Z0 F' \<property>' M( j" }" H! ]7 x
    <name>mapreduce.job.emit-timeline-data</name>- d3 J3 |7 v0 J3 s$ z
    <value>false</value>; v' d" H! x3 N9 s6 D
    <description>Specifies if the Application Master should emit timeline data
# h7 V* n" Y3 J3 L& _    to the timeline server. Individual jobs can override this value.
9 D  F' x. ~) ?/ E1 H- @% K    </description>
# w. \5 t" Q; q</property>
4 T! U7 q& L% m, h1 }# x; D<property>
- F- b8 g1 D3 ^8 q' T  <name>mapreduce.job.sharedcache.mode</name>
3 d/ p7 p5 C" v& z8 m9 S2 P# V  <value>disabled</value>
2 o+ P1 W  a' Q3 [  <description>2 ?' g4 M  S1 ~' O9 M+ o1 P- r  P
    A comma delimited list of resource categories to submit to the shared cache.
1 k9 j- P  c* z    The valid categories are: jobjar, libjars, files, archives.
+ B+ J# B' {6 \+ k    If "disabled" is specified then the job submission code will not use
, Q2 g8 x9 X9 X- i: }9 j3 \    the shared cache.6 ]5 t3 Z& ^) ]9 B* P/ j" F
  </description>% L+ U% u  F: O2 s& M6 ^
</property>
0 u! L, n$ F$ }' X* N- p" \/ B0 b6 h<property>
# x2 {; w/ p4 a& P7 d7 @. ?  <name>mapreduce.input.fileinputformat.split.minsize</name>
& B) J9 o$ @9 w" u  <value>0</value>* l8 q8 K% V8 L' b, ]! h
  <description>The minimum size chunk that map input should be split
  y8 M9 j  E5 |5 x  into.  Note that some file formats may have minimum split sizes that
( e+ B& |" w, d1 G: h  C+ F' S  take priority over this setting.</description>' o5 G2 g: O) _* J1 l$ A4 @4 U
</property>9 C$ k9 A4 o$ U1 |, L
<property>+ S! i# C* p* a- u) x8 @- m5 |: A
  <name>mapreduce.input.fileinputformat.list-status.num-threads</name>* [" o8 m3 K1 l( ~9 d( U; ?
  <value>1</value>
3 l$ i! K$ ^; W5 ]: s- o: Q- b  <description>The number of threads to use to list and fetch block locations! j, j2 H' y, c' V( I
  for the specified input paths. Note: multiple threads should not be used
0 a4 I  J3 r7 X7 s  M2 f! [, _  if a custom non thread-safe path filter is used.
% G' v" y7 s8 A( q& x8 ~/ X: R' T  </description>
7 g/ n( \6 A3 ~: Z</property>- D8 I" Q/ S3 O% s6 @2 `! F
<property>3 U9 I; {# R& @8 G, n  E
  <name>mapreduce.input.lineinputformat.linespermap</name>9 v2 J% ]3 ]# Z  O; F! ], i
  <value>1</value>9 s0 Q5 e% O2 r* p( \) O
  <description>When using NLineInputFormat, the number of lines of input data2 s& H+ P+ D$ E. i5 m6 R  ?
  to include in each split.</description>* W& Y4 y9 ?) o6 j& h8 j
</property>
) h7 S9 a' F6 [<property>
' V0 {5 s; s. o! J* @4 i  <name>mapreduce.client.submit.file.replication</name>
! J' k9 t$ K# e  <value>10</value>: g+ v( l2 b0 h4 T7 N; t5 b; m
  <description>The replication level for submitted job files.  This
# `! k- D! d- k$ {. S  should be around the square root of the number of nodes.
; Z( t5 ~* z; \5 y3 Z( X  </description>
/ g3 A/ I7 p7 m7 j2 k) t</property>/ ?0 X; i: v( j) X" B# h3 J
<property>
0 H; M2 B$ _# L6 r2 v$ L/ U  <name>mapreduce.task.files.preserve.failedtasks</name>2 T- B  l8 M0 u* a% K
  <value>false</value>
  h0 f3 @$ Y8 k2 m  |& L' _8 Q  <description>Should the files for failed tasks be kept. This should only be
2 F/ r" i  e. m               used on jobs that are failing, because the storage is never
* t$ M" I, N' F$ ^) l               reclaimed. It also prevents the map outputs from being erased
$ b5 N; x0 V  o* e+ E) O9 D               from the reduce directory as they are consumed.</description>
7 a0 q1 R. n: p' h</property>
/ Z1 V4 H7 [4 S<!--6 \' c  B2 e  i6 I/ ~8 U1 B
  <property>
3 ~: ]6 T# {- [! Q3 Y  <name>mapreduce.task.files.preserve.filepattern</name>3 G9 H9 u# n+ _) x1 k+ A
  <value>.*_m_123456_0</value>
+ L( P  _8 @1 h1 e  o4 {. H  V7 b  <description>Keep all files from tasks whose task names match the given# C' R7 q5 @5 r6 b0 J
               regular expression. Defaults to none.</description>
* v  V0 H  |- E4 W! Z$ j  </property>6 A! N/ w5 ~% @6 \
-->: v5 c0 f2 S. O, W
<property>2 ^) W( z$ a+ D, C, }" ]6 c" t
  <name>mapreduce.output.fileoutputformat.compress</name>
/ ~( f8 Z8 t& i. a) f2 }2 K9 z  <value>false</value>
+ B; |6 S6 n7 f  <description>Should the job outputs be compressed?: d& e  a5 q5 d# |1 [
  </description>
' b' R  ?) V3 l. J' w</property>
1 U' {+ o) o7 N' A2 ]) \5 A<property>$ [+ b: q  j; o& ^! j. G. J0 o" h
  <name>mapreduce.output.fileoutputformat.compress.type</name>( K7 V% q8 Y' Z8 B* L
  <value>RECORD</value>
1 W9 S+ O7 O2 I, @$ |& t  <description>If the job outputs are to compressed as SequenceFiles, how should
* z* e2 i3 N- q0 ^; B               they be compressed? Should be one of NONE, RECORD or BLOCK.
; c$ s! U3 _% |: p2 ~, `' U) Q, n: \  </description>; ]% U! V: Z/ u/ ]
</property>; W: W8 v' d9 N0 Z8 n& L9 d
<property>
7 j4 ~% R  I5 P0 l7 |/ I% E  <name>mapreduce.output.fileoutputformat.compress.codec</name>8 \2 l6 C, h- R# d
  <value>org.apache.hadoop.io.compress.DefaultCodec</value>
/ X$ n- e+ ?$ X  <description>If the job outputs are compressed, how should they be compressed?2 c; A! L- Z, ]
  </description>
, M$ G6 \0 j5 D; [  }</property>
3 C. h! K$ f/ ~8 _7 h2 \) F<property>
, z% a$ E& \' N) _! p7 e  <name>mapreduce.map.output.compress</name># `0 E6 Q( _$ f2 g/ k( H$ ~# r  S
  <value>false</value>
; R$ J) }& F: E6 T: ^3 T" O, ?  <description>Should the outputs of the maps be compressed before being
- G6 V1 Z2 a8 k- R* X0 m8 _! R. ^               sent across the network. Uses SequenceFile compression.+ s; G- O8 s! Q/ @; \' e
  </description>3 ]( J; H. ?' v, o$ z
</property>8 _* N( u. p- T3 u
<property>! a5 a, C, n& v% F# C
  <name>mapreduce.map.output.compress.codec</name>
5 Q; R# P4 \8 b$ D! K  <value>org.apache.hadoop.io.compress.DefaultCodec</value>$ r' c/ X& E3 x- `
  <description>If the map outputs are compressed, how should they be7 @9 }  Y* a% }0 j4 D- O) _1 J0 C
               compressed?
  V6 D! N  X3 B  </description>
. @* m* |' O* {/ R</property>1 L( j" [8 x" |7 {7 d9 T9 O
<property>
; J6 N0 T, o+ I' ?; g2 u  <name>map.sort.class</name>
' P. R2 c* L& x0 B( V' t  <value>org.apache.hadoop.util.QuickSort</value>
  Z1 N- L( z# a- c1 b8 R9 M. k  <description>The default sort class for sorting keys.
& Y7 P( l9 |+ b/ z) W4 i  </description>; Q# v- K3 A( V/ S
</property>/ U7 o& m* U/ D7 k
<property>1 e* p! E" U7 }" \
  <name>mapreduce.task.userlog.limit.kb</name>3 A- \8 M3 m  M. l% p2 M
  <value>0</value>
8 h+ w& R* P3 M$ x: _2 c  <description>The maximum size of user-logs of each task in KB. 0 disables the cap.
& I2 ~9 F" ~" Y  </description>8 d3 e0 |6 T3 x: W  t
</property>
8 r: {" T# b# g& Q% _  k- D<property>1 v, e9 M/ v* B
  <name>yarn.app.mapreduce.am.container.log.limit.kb</name>
/ Z# R' L/ K$ n! H. `* n0 j  <value>0</value>  {# p5 ]5 c# y! I2 c, e
  <description>The maximum size of the MRAppMaster attempt container logs in KB.
- z- D3 |. F4 Q' e5 e1 l% s    0 disables the cap.
3 i+ H9 N- L4 U  </description>
  r+ r6 c& Z( O</property>2 f8 m) R3 J* X: q/ M4 w7 M
<property>) ~6 D3 E# G* r5 `- e
  <name>yarn.app.mapreduce.task.container.log.backups</name>
! T( w$ Z( ~. S! g  m4 K% r$ K  <value>0</value>
0 o$ U% x. Q% z  <description>Number of backup files for task logs when using
$ F5 I: g+ c( p# z5 _    ContainerRollingLogAppender (CRLA). See
% {6 N! S$ F2 ^0 `* P- h    org.apache.log4j.RollingFileAppender.maxBackupIndex. By default,5 n$ o  a8 H; p0 f: W
    ContainerLogAppender (CLA) is used, and container logs are not rolled. CRLA! b+ d" d$ u. ]" s/ I' A
    is enabled for tasks when both mapreduce.task.userlog.limit.kb and
1 d: f. e' ?; @1 I9 @    yarn.app.mapreduce.task.container.log.backups are greater than zero.4 q$ @* W& s2 V# x5 N1 T6 J# u) y% r2 h
  </description>& a/ i" |; f& H5 |& z
</property>* C6 t# b2 @; R) d1 `! [. @4 |
<property>
! P0 E; K0 A' g; F  <name>yarn.app.mapreduce.am.container.log.backups</name>6 O4 k  j1 d& Y
  <value>0</value>$ {* ]; b( Q% L2 a3 }2 x2 O
  <description>Number of backup files for the ApplicationMaster logs when using
# g1 H  B& h% V9 s* I  {3 J  F    ContainerRollingLogAppender (CRLA). See
, V. _" x% ?* X- h# H0 k2 m  ?" K! J    org.apache.log4j.RollingFileAppender.maxBackupIndex. By default,4 i) b- P/ ^6 J
    ContainerLogAppender (CLA) is used, and container logs are not rolled. CRLA9 y& n7 ~1 J+ e; x) W
    is enabled for the ApplicationMaster when both4 X5 I# g: d0 w/ J
    yarn.app.mapreduce.am.container.log.limit.kb and
2 ?3 j9 C) _8 ~1 {3 l+ g+ H$ X) b    yarn.app.mapreduce.am.container.log.backups are greater than zero.
6 k4 T% o* E* z4 V  ?* p4 z6 @7 y  </description>
0 `. J9 @, k, _7 N2 N</property>- e* `( T/ Q' Y2 e
<property>& Y% i. I9 R; \
  <name>yarn.app.mapreduce.shuffle.log.separate</name>( p+ @5 H8 h( u& z( a( a
  <value>true</value>
/ m8 m. O3 w  k) N# V  <description>If enabled ('true') logging generated by the client-side shuffle9 w2 k& c  ?* u- d
    classes in a reducer will be written in a dedicated log file- m. B3 N6 ^- D5 ?( C/ `6 u9 q, |; k
    'syslog.shuffle' instead of 'syslog'.. j2 G, x6 Z& a* G9 m1 j
  </description>+ L7 \& _4 w7 A8 Q+ v$ p
</property>; G8 J2 z7 N/ x, T7 T/ X2 l- b1 ~
<property>
; ?; h' B$ T4 ~1 Y3 I4 P  <name>yarn.app.mapreduce.shuffle.log.limit.kb</name>+ q4 s  r" z# H4 ?1 f
  <value>0</value>5 k5 `+ V/ y3 ~9 D1 k
  <description>Maximum size of the syslog.shuffle file in kilobytes% A# W5 a. p; U7 ^
    (0 for no limit).
2 |! ^9 H& F3 V/ Y% W7 Z" [& A2 t8 F  </description>+ E6 {/ G. ?. q! _" E/ b. x' i# N
</property>' D7 ?9 q) g4 z. d  Q
<property>
2 B- S% L8 j- L7 @- R  <name>yarn.app.mapreduce.shuffle.log.backups</name>2 F8 D+ n9 n  y( D* s) Y: [. h" M% p
  <value>0</value>
9 b2 |; k! S3 D/ H  T3 B/ {, E2 V1 n  <description>If yarn.app.mapreduce.shuffle.log.limit.kb and
2 T! ]' _2 a: c/ {2 O    yarn.app.mapreduce.shuffle.log.backups are greater than zero3 z& Z( F1 z6 V6 r! ?% j
    then a ContainerRollngLogAppender is used instead of ContainerLogAppender
1 }+ Z1 g1 w9 S6 y    for syslog.shuffle. See
3 p( a6 x7 g) p; i. G1 J    org.apache.log4j.RollingFileAppender.maxBackupIndex% n, q  _0 }. {
  </description>" i7 n9 }1 r' X3 E
</property>
0 ?7 V3 u6 e& c1 B8 W- n2 V<property>
, K  }* c2 l- w" C2 q  <name>mapreduce.job.maxtaskfailures.per.tracker</name># I/ G/ B. t4 M- I) i
  <value>3</value>- @! \+ F$ p; T3 P9 w. a- g
  <description>The number of task-failures on a node manager of a given job
7 [; R# T  C1 B6 Y( y: K' x0 L               after which new tasks of that job aren't assigned to it. It/ [4 \2 F! U- W
               MUST be less than mapreduce.map.maxattempts and/ \+ K2 r' F1 i* ?
               mapreduce.reduce.maxattempts otherwise the failed task will6 j7 s! A; a1 E. K4 `. c
               never be tried on a different node.
; U4 _, r/ t1 B) j# y9 `  </description>6 m; F# w9 J& i) M
</property>
4 I& }. m2 `1 D& A" e<property>1 V7 [+ F2 Z9 ^% g# w( b2 C* m
  <name>mapreduce.client.output.filter</name>
' Q: w9 B3 s6 |$ v. Q  <value>FAILED</value>8 u2 R/ J3 l$ n. d
  <description>The filter for controlling the output of the task's userlogs sent
% l' T8 {9 e, A9 e               to the console of the JobClient.
; g! W4 I6 _% T) x7 z               The permissible options are: NONE, KILLED, FAILED, SUCCEEDED and
+ p& f& R7 z. X. F               ALL.5 F4 J5 O" `+ H1 ~
  </description>
1 O; \; j1 W6 [. x</property>
/ L+ j7 S3 b9 {, M" T! e5 q; _  <property>) R" L- @. s( w) q9 u& D
    <name>mapreduce.client.completion.pollinterval</name>8 @- q. X/ t. S8 k
    <value>5000</value>
9 \# [% C+ k7 k1 G; d    <description>The interval (in milliseconds) between which the JobClient  J- O. K3 W& |: ~4 g
    polls the MapReduce ApplicationMaster for updates about job status. You may want to# ~/ F# N( W# b& ^6 v# T
    set this to a lower value to make tests run faster on a single node system. Adjusting" s' [' T1 |, q- Z: `( Z- I2 W
    this value in production may lead to unwanted client-server traffic.9 D1 }6 O1 \, e6 p; r, T9 x
    </description>9 l6 e: I' y* M7 g& P, K
  </property>! f5 G6 O2 d( S8 a# A
  <property>
. U* M* ~4 }8 Q$ e: _! e* a: Z    <name>mapreduce.client.progressmonitor.pollinterval</name>
7 \1 U. [8 q# W" z; K6 e    <value>1000</value>
- K7 M/ \( ?- s    <description>The interval (in milliseconds) between which the JobClient( S( Q6 O- `' s9 Y
    reports status to the console and checks for job completion. You may want to set this
& r4 K! f; h- w    to a lower value to make tests run faster on a single node system. Adjusting
5 J0 t. n8 U& b    this value in production may lead to unwanted client-server traffic.
3 ]; Z; L* H3 N# U/ D    </description>* N  [$ H3 \3 s, }
  </property>
$ {& d: T9 T6 A8 o+ l5 l  <property>
/ ?, T' L, D" p) p  k) }    <name>mapreduce.client.libjars.wildcard</name>8 i4 h& s& L1 |4 K1 {' l
    <value>true</value>( b9 A3 w3 k$ D; e
    <description>
" U# ?- H4 ^6 H- U        Whether the libjars cache files should be localized using1 S3 Z5 u7 V: R6 x7 r# C
        a wildcarded directory instead of naming each archive independently.* q, j4 X% P# I+ K: r  z& ?$ T
        Using wildcards reduces the space needed for storing the job
6 W; E) F+ y' Z  F        information in the case of a highly available resource manager& }+ g% f$ O  s) n, D7 G- F
        configuration.: _7 G0 M2 B' b: i, p
        This propery should only be set to false for specific
7 [# _* c4 }7 W- y; l! V        jobs which are highly sensitive to the details of the archive
* B) u5 B* m( R+ r1 T5 e2 P        localization.  Having this property set to true will cause the archives
6 n6 y1 t4 l, p+ R7 |! h+ q        to all be localized to the same local cache location.  If false, each% l: _8 _5 e. c' a$ w; g
        archive will be localized to its own local cache location.  In both/ J: Y" G; M0 }8 J& k
        cases a symbolic link will be created to every archive from the job's
8 @. O- Y: s* @7 W; S: b' P8 M        working directory.
' Z. Y% ?) w5 B& ~    </description>
4 q" _# e7 Y2 Q: t# k  </property>/ ?. C5 `) U! m3 |; r* {- y6 R
  <property>" Q% t2 V% v: Q  |. S
    <name>mapreduce.task.profile</name>( H0 K  O) \0 K0 O) [1 o! x+ s
    <value>false</value>
1 z5 T/ P4 {! i7 n. k! |0 ~! ^* r    <description>To set whether the system should collect profiler! Q  w% [9 P; M/ x1 g- U
     information for some of the tasks in this job? The information is stored1 Z9 _. B% w9 L. K# ^; S" @: T+ Y
     in the user log directory. The value is "true" if task profiling
! n/ c. O! D. U- Y; \' k     is enabled.</description>
  a" I) C  _3 j8 g# B5 t( V" P  </property>5 x0 A7 q6 z! K
  <property>
: O& g7 K$ R4 f2 s$ K    <name>mapreduce.task.profile.maps</name>
: G: ]7 ?% H* c; e' ?8 Y    <value>0-2</value>
4 k& \& j* N" A: w1 P    <description> To set the ranges of map tasks to profile.* m/ B* B0 g0 q+ P
    mapreduce.task.profile has to be set to true for the value to be accounted.
9 l/ z# L. I3 d, A% ?, c3 Q    </description>) g$ L2 d3 m/ o! x# _2 Z% K
  </property>8 ?/ Y" T3 a. `4 V
  <property>, J, h$ x& W5 P, ^% o1 y, }
    <name>mapreduce.task.profile.reduces</name>
, {! ~0 G+ M$ _  i    <value>0-2</value>
% a9 U0 p1 [7 V  t5 D6 b8 B    <description> To set the ranges of reduce tasks to profile.3 P# S, m+ o/ g( o: n3 B, d; \
    mapreduce.task.profile has to be set to true for the value to be accounted.: G  }7 i+ T: B" b3 T% O
    </description>
, c. O5 P$ S5 Q( A6 P% z' O+ }  </property>, V% W- E4 b" x( s
  <property>0 _. `8 O: s5 ?$ M
    <name>mapreduce.task.profile.params</name>) |& o1 u" b3 L- c! X
    <value>-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s</value>7 w  F. d- N* N0 x3 R" e. L
    <description>JVM profiler parameters used to profile map and reduce task
4 `3 X; f5 R0 d# J2 R+ k      attempts. This string may contain a single format specifier %s that will
# [/ R1 d/ F+ y      be replaced by the path to profile.out in the task attempt log directory.6 a8 l. A  @% K, @6 z8 L4 n: `" I  C
      To specify different profiling options for map tasks and reduce tasks,( L8 E+ v! U0 ]: [  i( S
      more specific parameters mapreduce.task.profile.map.params and: y& a+ e7 E8 x
      mapreduce.task.profile.reduce.params should be used.</description>' k- K3 L0 j( c+ w5 R0 J
  </property>% M  e9 M" ]9 p, D; \
  <property>
' G/ s) e0 D9 Z1 r6 m    <name>mapreduce.task.profile.map.params</name>
/ G) ^( g+ M' E    <value>${mapreduce.task.profile.params}</value>
3 A  f+ O, Z" V. |9 r- X* ]    <description>Map-task-specific JVM profiler parameters. See3 g% }" R. r) y
      mapreduce.task.profile.params</description>
# B+ M: v" x6 u& U  </property>
) f" e% E5 \, r" i7 e  <property>
: u6 `8 r  T( w# U: ?5 R5 c    <name>mapreduce.task.profile.reduce.params</name>
, V9 N. S3 G3 }+ B$ p3 r3 W+ ?9 ^0 c    <value>${mapreduce.task.profile.params}</value>
3 Z8 ^- v$ G8 D+ o! U& d/ \    <description>Reduce-task-specific JVM profiler parameters. See
  g5 k6 _* n3 O+ n8 R- e      mapreduce.task.profile.params</description>
. S3 F1 k; \# g/ z" K  A( b7 b  </property>! Y) g0 J& l0 C* A
  <property>
: l8 u, v3 l5 V' M6 }& z    <name>mapreduce.task.skip.start.attempts</name>
9 N0 F1 t3 `8 J) k0 M0 b# I; L    <value>2</value>
/ [; y- _- N  z    <description> The number of Task attempts AFTER which skip mode
* m# g7 q1 f' l/ c: c& K    will be kicked off. When skip mode is kicked off, the6 k( s. ]9 o- H# p6 c
    tasks reports the range of records which it will process
, `7 N0 P, L; ^: l+ F    next, to the MR ApplicationMaster. So that on failures, the MR AM
( {. }  f; E- W1 q    knows which ones are possibly the bad records. On further executions,
; m: ?5 h+ Y+ R    those are skipped.
, [  M6 u/ f5 x- v# Y9 V    </description>
9 X) T) S7 o/ O, I  </property>
% e8 u0 _! s. q* W# d  <property>
- E0 M' d1 q# |. `4 e: i: D6 h9 \    <name>mapreduce.job.skip.outdir</name>
4 I* j& v! ?1 c. ^, I    <value></value>
) X: ~4 w; }, ]! `    <description> If no value is specified here, the skipped records are8 a! W% Z+ @0 ^  X/ c( Q4 X' B
    written to the output directory at _logs/skip., u0 C1 q& w9 [8 q
    User can stop writing skipped records by giving the value "none".
; a5 `. ?6 [, x; ?    </description>
; M/ E2 y. H1 N: V5 M+ U( Q  </property>* S) Q" \, N4 G4 E$ M
  <property>
2 W0 t. U' [! s    <name>mapreduce.map.skip.maxrecords</name>1 a/ v* W+ m4 J5 l0 ~1 _: _* l3 Y6 c% I
    <value>0</value>
; ]; S! j9 w5 d$ M, r2 }    <description> The number of acceptable skip records surrounding the bad
8 Q3 w0 r; W( l" c    record PER bad record in mapper. The number includes the bad record as well.
/ q8 f: \  Z7 h. ^2 V! J    To turn the feature of detection/skipping of bad records off, set the5 V" n- S4 x2 W+ I' @: y
    value to 0.+ _# D  a" h1 L" y- |
    The framework tries to narrow down the skipped range by retrying
1 i3 v& y7 F; Z4 s9 Q) C    until this threshold is met OR all attempts get exhausted for this task.' G2 k. x* T0 C$ {
    Set the value to Long.MAX_VALUE to indicate that framework need not try to# ?  Y2 q7 G7 y# T4 I, S7 P) ~* g
    narrow down. Whatever records(depends on application) get skipped are
; Q# e6 S6 X2 ~! ~6 m' N+ b2 d    acceptable.' K* G/ L; [/ j4 L
    </description>
2 @  e4 @" `* Q  </property>
$ k, f1 S/ i+ ?  <property>
. P+ k% v' @% z5 O. c2 k    <name>mapreduce.map.skip.proc-count.auto-incr</name>$ H8 O* I) B  Q' u1 f$ r
    <value>true</value>
! R1 k; |4 s* r! s    <description>The flag which if set to true,( j3 b; h/ p( `$ |  g/ i; P
    SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS is incremented by
& y  @. ^  k; F  P; O+ e0 Y1 R    MapRunner after invoking the map function. This value must be set
$ J# F! ~3 I1 d# D' l% o& P    to false for applications which process the records asynchronously
# [1 W5 X; k$ z8 a) V    or buffer the input records. For example streaming. In such cases% A( n% `3 M: O. Y
    applications should increment this counter on their own.& C) V3 I/ c9 f% h. O" J$ |* _: Q
    </description>8 ?* Q3 N" C8 ]1 s' }8 ~" R% _
  </property>! {  [6 L9 B9 j. k/ Y. g+ T9 D! `! E) W
  <property>' Q9 F7 F  D' b
    <name>mapreduce.reduce.skip.maxgroups</name>
1 K* l' T0 E, Y2 f    <value>0</value>8 H$ i: k4 d% s
    <description> The number of acceptable skip groups surrounding the bad
; J) J# s# f  H& Z8 R- j    group PER bad group in reducer. The number includes the bad group as well.
: X6 N2 v0 D; s; H, I5 h    To turn the feature of detection/skipping of bad groups off, set the& B$ `1 A6 c3 E: U' X
    value to 0.- M. k% o* e+ }7 m7 H! [
    The framework tries to narrow down the skipped range by retrying# {/ j0 ^2 i. J9 r  ^* z2 u" g& g
    until this threshold is met OR all attempts get exhausted for this task.. J( {% z" z' E. B7 t
    Set the value to Long.MAX_VALUE to indicate that framework need not try to
$ V7 n: t! q% P9 ]0 \    narrow down. Whatever groups(depends on application) get skipped are, T9 u: w! P- X1 w' [1 e: L# V
    acceptable.! f' p% S/ d: S5 l5 y) K- O1 p
    </description>
' ]; e; d/ \9 b) g, G/ N  </property>
+ A6 k& N: h" ^  <property>8 J7 {2 O- F7 n6 }0 V2 D
    <name>mapreduce.reduce.skip.proc-count.auto-incr</name>1 O" h. Y/ x' H" e/ r+ h
    <value>true</value>
8 x0 j; K; s. t5 W5 z9 P0 T    <description>The flag which if set to true.' ]1 L6 l' P& b) b) [$ v; ?& C! F6 {
    SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS is incremented by framework
6 j9 [- a( m- B+ z    after invoking the reduce function. This value must be set to false for) {) G0 H$ H+ i. J) J
    applications which process the records asynchronously or buffer the input; D' t- g5 T: W9 H3 ~0 j" ~
    records. For example streaming. In such cases applications should increment
3 l( S( }9 M8 s  t" V; |2 s    this counter on their own.% n/ U# i  X  L  |0 M8 S
    </description>" j: E' n) e) A# ], L( }
  </property>
8 l4 z5 s  Y" X' b. d' l& v% M1 i  <property>  v; v2 y- z- F' q0 `7 e, Q% x
    <name>mapreduce.ifile.readahead</name>
7 D& ]' m* Q5 x7 ~, |* {+ ?    <value>true</value>
% T* c3 _+ Z" i# o  \% D3 y, G4 ?    <description>Configuration key to enable/disable IFile readahead.2 ^  b8 C  s- `, h7 d5 q' }3 x
    </description>
) U7 H0 c  ?* P( P" O+ v  </property>
, h; B) m! J9 x# G  <property>
7 J, E  z  n  C, g3 N    <name>mapreduce.ifile.readahead.bytes</name>
$ j+ g2 V' T" V. y( z* m8 W    <value>4194304</value>" C, C) [9 \; m4 H: P$ V- u
    <description>Configuration key to set the IFile readahead length in bytes.
; S0 l2 b, t, R    </description>8 G/ Q4 K7 a; U8 {# b- {
  </property>! d7 g2 \0 p$ f; j7 T$ A
<property>, ]; z) J9 m) U6 Z5 n& y" F
  <name>mapreduce.job.queuename</name>
6 U7 {8 ]* j, y, d2 B9 L* r  <value>default</value>. H5 t+ ?: W3 a2 v, w3 a8 f
  <description> Queue to which a job is submitted. This must match one of the
; b5 ]% B7 Q! c. s4 E. \    queues defined in mapred-queues.xml for the system. Also, the ACL setup% M" j. Z6 ~' F9 ?3 X( S  w" C
    for the queue must allow the current user to submit a job to the queue.
9 a# w: j- ^6 A! e6 X% C    Before specifying a queue, ensure that the system is configured with
) c. z5 a; p* Q5 h    the queue, and access is allowed for submitting jobs to the queue.5 W: y' m$ [/ D' m0 ^3 t
  </description>) n0 o0 V+ o4 _/ n) V8 Z
</property>
6 p; d$ t* N; d  ^6 \( {  <property>
" J( Z2 W9 L# {  O1 S/ n- `    <name>mapreduce.job.tags</name>
, M# z& T) B; i8 m" t& h    <value></value>
7 z. L% ~7 J$ Q) z5 e    <description> Tags for the job that will be passed to YARN at submission9 a- h5 Q4 W" o) G
      time. Queries to YARN for applications can filter on these tags.
! s( K2 `' A. Y" I2 }* h      If these tags are intended to be used with The YARN Timeline Service v.2,
; O* X  g' K8 N! K! R      prefix them with the appropriate tag names for flow name, flow version and
! k, u/ s4 e, ]$ K5 B      flow run id. Example:2 L- K. B' S/ c* k5 w& a1 I% A
      timeline_flow_name_tag:foo,
& y; z) @2 L& l8 B' @      timeline_flow_version_tag:3df8b0d6100530080d2e0decf9e528e57c42a90a,
0 ~4 J8 Z4 I6 x( ^0 U3 f      timeline_flow_run_id_tag:1465246348599
! x! @$ [+ H7 K5 s" C    </description>
  I1 H) e7 O. R7 Z) U  </property>/ ~- Y  o0 s, H8 I- k4 F! W
<property>$ c% R/ I5 G  a; \* P. f
  <name>mapreduce.cluster.local.dir</name>3 \5 k! _7 c3 `: b: p' r
  <value>${hadoop.tmp.dir}/mapred/local</value>6 c- U2 G6 }  M: [& d9 c  l
  <description>
# ^) X3 ^, p. w1 H/ f9 n      The local directory where MapReduce stores intermediate3 e; u! z- V3 O6 A  K* e" B
      data files.  May be a comma-separated list of) W& g$ A# ?% @8 a
      directories on different devices in order to spread disk i/o., M% R& [- N& N8 h' q2 C% }
      Directories that do not exist are ignored.
1 `5 D; S* v# c5 Q  </description>
3 u, |$ n1 Q* U, w8 l' k' L9 y4 X</property>! \9 V4 K+ f$ J
<property>3 J6 F" a2 N4 j: k) ~! L4 j
  <name>mapreduce.cluster.acls.enabled</name>" ?" l0 [4 J2 L% m* B
  <value>false</value>
, S. X" p$ T. C  <description> Specifies whether ACLs should be checked
+ F, I1 ^# J2 c1 ~) ]/ K    for authorization of users for doing various queue and job level operations.$ ~8 N1 E6 ^; @9 Z# X6 D
    ACLs are disabled by default. If enabled, access control checks are made by
1 s  v4 i, P  y2 X0 Y& J    MapReduce ApplicationMaster when requests are made by users for queue
+ k' H4 n  T3 \    operations like submit job to a queue and kill a job in the queue and job
( k4 r1 I* Z+ C$ _4 j  C    operations like viewing the job-details (See mapreduce.job.acl-view-job)$ v. X5 q6 P+ p& c
    or for modifying the job (See mapreduce.job.acl-modify-job) using# R0 V& ~* V) V1 h0 E) g
    Map/Reduce APIs, RPCs or via the console and web user interfaces.7 t8 Q! q5 A' J) b3 \
    For enabling this flag, set to true in mapred-site.xml file of all+ i# ]3 m) V, B
    MapReduce clients (MR job submitting nodes).9 I; I" ~6 `+ c/ ~
  </description>8 U9 P/ ^3 P) {. ~
</property>% }; V3 I6 b! [, z3 F- p
<property>. g' ]  N6 W6 o# U
  <name>mapreduce.job.acl-modify-job</name>
" B! f6 |2 f" u  <value> </value>' r3 g! E! w. }$ l" j7 ?1 S1 t- K
  <description> Job specific access-control list for 'modifying' the job. It
4 Z: v+ b5 P, g    is only used if authorization is enabled in Map/Reduce by setting the& ~$ X% _1 R3 P
    configuration property mapreduce.cluster.acls.enabled to true.
1 ^0 v1 c" ^9 a# H1 x' P& O    This specifies the list of users and/or groups who c