机器规划：
10.241.95.109 master jdk,hadoop namenode,ZKFC,Resourcemanager
10.241.95.107 h107 jdk,hadoop namenode,ZKFC,Resourcemanager,zookeeper,Journalnode,
10.241.95.110 slave1 jdk,hadoop natanode, nodemanager
10.241.95.111 slave2 jdk,hadoop, natanode,nodemanager
10.241.95.105 h105 jdk,hadoop, natanode,nodemanager,zookeeper,Journalnode,
10.241.95.106 h106 jdk, hadoop, natanode,nodemanager,zookeeper,Journalnode
1:设置服务器的hostname
目标文件：/etc/hosts 对象： 6台机器通用
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 salve2
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.241.95.109 master
10.241.95.110 slave1
10.241.95.111 slave2
10.241.95.105 h105
10.241.95.106 h106
10.241.95.107 h107
2:设置javahome和hadoophome
目标文件：/etc/profile 对象： 6台机器通用
JAVA_HOME=/usr/java/jdk1.8.0_201
HADOOP_HOME=/opt/app/hadoop-3.1.2
CLASSPATH=PATH:
HADOOP_HOME/bin:$HADOOP_HOME/sbin
3：设置ssh免密码登陆
执行：ssh-keygen 生成密钥
/root/.ssh/id_rsa.pub中生成的内容粘贴到 /root/.ssh/authorized_keys中，然后复制到每一台机器，6台机器就是6套密钥
4：hadoop配置文件
对象：6台机器通用
core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns1/</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hadoop-3.1.2/tmp</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>h105:2181,h106:2181,h107:2181</value>
</property>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.http.address</name>
<value>10.241.95.109:9870</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>
<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>10.241.95.109:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>10.241.95.109:9870</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>10.241.95.107:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>10.241.95.107:9870</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://h105:8485;h106:8485;h107:8485/ns1</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/app/hadoop-3.1.2/journaldata</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>
sshfence
shell(/bin/true)
</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>/opt/app/hadoop-3.1.2/share/hadoop/mapreduce/, /opt/app/hadoop-3.1.2/share/hadoop/mapreduce/lib/</value>
</property>
<property>
<name>mapreduce.job.reduce.slowstart.completedmaps</name>
<value>0.9</value>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>2.1</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>2</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>8192</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>8192</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>2046</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>2046</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>2046</value>
</property>
</configuration>
yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yrc</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>10.241.95.109</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>10.241.95.107</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>h105:2181,h06:2181,h107:2181</value>
</property>
</configuration>
5：设置从属文件
对象：master,h107
works.xml
slave1
slave2
h106
h105
6:格式化HDFS
对象master：hdfs namenode -format
然后把主节点的数据copy到standby机器上
格式化后会在根据core-site.xml中的hadoop.tmp.dir配置生成个文件，这里我配置的是主节点/home/hadoop/hadoop-3.1.2/tmp，然后将/home/hadoop/hadoop-3.1.2/tmp拷贝到从节点的/home/hadoop/hadoop-3.1.2/下。
7:初始化zk
对象：master
hdfs zkfc -formatZK
8:启动hadoop
对象：集群中任意一台机器
start-dfs.sh
start-yarn.sh
至此hadoop高可用集群搭建完毕
