Hadoop集群优化，hadoop集群

Hadoop

namenode vs RM

端口

A port number of 0 instructs the server to start on a free port, but this is generally discouraged because it is incompatible with setting cluster-wide firewall policies.

HDFS

ECC memory

ECC memory is strongly recommended, as several Hadoop users have reported seeing many checksum errors when using non-ECC memory on Hadoop clusters.

dfs.name.dir

配置成多个路径，FSImage和EditLog会同时写入多个路径，方便以后恢复用

RAID

不适合datanode，如果配置成RAID的话，就不需要副本了：

适合namenode：用于保护元数据信息

MapReduce

slowstart

参数mapreduce.job.reduce.slowstart.completedmaps表示map任务完成多少后，reduce任务才能开始执行。将其设置成0.80，以增加集群的吞吐量