hadoop-2.6.0安装（new)，hadoop-2.6.0new

hadoop-2.6.0安装（new)，hadoop-2.6.0new

1.下载Hadoop安装文件(hadoop-2.6.0.tar.gz)

解压hadoop到/home/ld/Lee/目录下

tar -zxvf  hadoop-2.6.0.tar.gz -C /home/ld/Lee

2.修改/home/ld/Lee/hadoop-2.6.0/etc/hadoop/hadoop-env.sh文件的环境变量：

主要是添加java路径

export JAVA_HOME=/home/ld/Lee/jdk1.7.0_67

3.修改/home/ld/Lee/etc/hadoop/core-site.xml配置文件，在configuration中添加内容如下：

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/lee/opt/hadoop_tmp</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://master:9000</value>
</property>
</configuration>

4.修改/home/ld/Lee/hadoop-2.6.0/etc/hadoop/hdfs-site.xml配置文件，在configuration中添加内容如下：

<configuration>
<property>
<name>dfs.name.dir</name>
<value>${hadoop.tmp.dir}/dfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/lee/opt/hadoop-data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>fs.checkpoint.dir</name>
<value>/disk1/hdfs/namesecondary,/disk2/hdfs/namesecondary</value>
<final>true</final>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>

5.修改/home/ld/Lee/hadoop-2.6.0/etc/hadoop/下的slaves文件

slaves文件写入

slaves1

slaves2

6.修改/home/ld/Lee/hadoop-2.6.0/etc/hadoop/下的mapred-site.xml文件

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>Xmx1024M</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>Xmx2560M</value>
</property>


<property>
<name>mapreduce.task.io.sort.mb</name>
<value>512</value>
</property>
<property>
<name>mapreduce.task.io.sort.factor</name>
<value>10</value>
</property>
<property>
<name>mapreduce.reduce.shuffle.parallelcopies</name>
<value>5</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
<property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/mr-history/tmp</value>
</property>
<property>
<name>mapreduce.jobhistory.done-dir</name>
<value>/mr-history/done</value>
</property>
</configuration>

7.配置yarn 在yarn-site.xml中加入一下内容

<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

<property>
<name>yarn.nodemanager.webapp.address</name>
<value>0.0.0.0:23999</value>
</property>

<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
</configuration>

8、在yarn-env.sh中加入以下内容

export JAVA_HOME=/home/lee/opt/jdk

9.将/home/ld/Lee/hadoop-2.6.0拷贝到hbase1和hbase2对应的目录下。将/ect/profile和/etc/hosts也拷贝到hbase1和hbase2机器上。

$ scp -r hadoop-2.6.0 slaves1:/home/ld/Lee/

$ scp -r hadoop-2.6.0 salves2:/home/ld/Lee/

10. 配置hadoop路径

a)在/etc/profile中添加以下语句

export    HADOOP_HOME=/home/ld/Lee/hadoop-2.6.0

export    PATH=$HADOOP_HOME/bin:$PATH

b)使配置文件生效

$ source /etc/profile

11.配置结束后，在bin文件夹下格式化hdfs

./hadoop namenode –format

 

到此hadoop集群配置结束

版权声明：本文为博主原创文章，未经博主允许不得转载。