Hadoop2启用cgroup


NodeManager启用cgroup资源隔离（Hadoop2.3）

1、准备container-executor.cfg文件

  • $ cat ${HADOOP_HOME}/etc/hadoop/container-executor.cfg
    yarn.nodemanager.linux-container-executor.group=users 注意：'users'来自$ groups 返回的第一个组
  • 从根目录到container-executor.cfg的user全部设为root

2、设置container-executor权限

  • $ sudo chown root bin/container-executor
  • $ sudo chmod 6050 bin/container-executor

3、准备cgroup目录

  • $ sudo mkdir -p /cgroup/cpu

4、yarn-site.xml新增配置：

<property>     <name>yarn.nodemanager.linux-container-executor.resources-handler.class</name>     <value>org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler</value>   </property>   <property>     <name>yarn.nodemanager.linux-container-executor.cgroups.mount</name>     <value>true</value>   </property>   <property>     <name>yarn.nodemanager.container-executor.class</name>     <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>   </property>   <property>     <name>yarn.nodemanager.linux-container-executor.cgroups.mount-path</name>     <value>/cgroup</value>   </property>
  <property>
    <name>yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user</name>
    <value>username</value> # 这里username可以是运行NodeManager的用户
  </property>

相关内容