【甘道夫】Oozie-4.0.0-CDH5.2.0 安装部署，oozie

文章由LinuxBoy分享于2019-03-27 06:03:21热评（481）

【甘道夫】Oozie-4.0.0-CDH5.2.0 安装部署，oozie

引言 Oozie是Hadoop生态系统中的工作流控制组件，可以调度MR，Hive，Shell等各类任务，并配置复杂的工作流程。今天介绍如何安装部署，使用的是单机测试环境。

欢迎转载，请注明来源： http://blog.csdn.net/u010967382/article/details/41355953

下载解压 下载安装过程需要的相关文件： oozie4.0.0安装包 http://archive.cloudera.com/cdh5/cdh/5/oozie-4.0.0-cdh5.2.0.tar.gz
ext2.2包 http://extjs.com/deploy/ext-2.2.zip 存在在/home/casliyang/Downloads/ext-2.2.zip
将oozie4.0.0安装包解压拷贝到： /home/casliyang/oozie/oozie-4.0.0-cdh5.2.0
这也也是即将配置环境变量的OOZIE_HOME

配置环境变量 向/etc/profile中加入如下三行，注意加完回到原用户后执行source /etc/profile export OOZIE_HOME=/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0 export OOZIE_CONFIG=$OOZIE_HOME/conf export CLASSPATH=$CLASSPATH:$OOZIE_HOME/bin

将ext2.2.0和hadoop相关jar包加入war包，并发布到自带的web server下 在$OOZIE_HOME/bin目录下执行 ./addtowar.sh -inputwar $OOZIE_HOME/oozie.war -outputwar $OOZIE_HOME/oozie-server/webapps/oozie.war -hadoop 2.5.0 $HADOOP_HOME -extjs /home/casliyang/Downloads/ext-2.2.zip

生成oozie数据库脚本并执行 这里使用的是自带的derby数据库在$OOZIE_HOME/bin目录下
执行以下命令创建数据库脚本
./ooziedb.sh create -sqlfile oozie.sql

通过以下命令执行sql脚本初始化数据库 ./oozie-setup.sh db create -run -sqlfile $OOZIE_HOME/bin/oozie.sql

配置oozie 配置$OOZIE_HOME/conf/oozie-site.xml 修改以下配置项，将值指向真实hadoop配置文件路径，实现oozie和hadoop的关联： <property> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> <value>*=/home/casliyang/hadoop2/hadoop-2.5.0-cdh5.2.0/etc/hadoop</value> <description> Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is used when there is no exact match for an authority. The HADOOP_CONF_DIR contains the relevant Hadoop *-site.xml files. If the path is relative is looked within the Oozie configuration directory; though the path can be absolute (i.e. to point to Hadoop client conf/ directories in the local filesystem. </description> </property>
配置$OOZIE_HOME/conf/oozie-default.xml 将oozie.services属性中的org.apache.oozie.service.JobsConcurrencyService值提到第一个位置。

HDFS上创建共享目录 在$HADOOP_HOME/etc/hadoop/core-site.xml中加入以下配置，红色的casliyang为用户名，蓝色的casliyang为组名，singlehadoop为oozie的host（注意不能用localhost，会报错）：
 <property> <name>hadoop.proxyuser.casliyang.hosts</name> <value>singlehadoop</value> </property> <property> <name>hadoop.proxyuser.casliyang.groups</name> <value>casliyang</value> </property>
启动HADOOP，该步骤不详述。
在$OOZIE_HOME/bin目录下执行： casliyang@singlehadoop:~/oozie/oozie-4.0.0-cdh5.2.0/bin$ ./oozie-setup.sh sharelib create -fs hdfs://singlehadoop:8020 -locallib $OOZIE_HOME/oozie-sharelib-4.0.0-cdh5.2.0-yarn.tar.gz setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0/libtools/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/casliyang/oozie/oozie-4.0.0-cdh5.2.0/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] the destination path for sharelib is: /user/casliyang/share/lib/lib_20141121184647

启动oozie 在$OOZIE_HOME/bin目录下
前台运行oozie： ./oozied.sh run

后台运行oozie： ./oozied.sh start

查看oozie网页： http://ip:11000/oozie

停止oozie： ./oozied.sh stop

推荐文章：

【甘道夫】Oozie-4.0.0-CDH5.2.0 安装部署，oozie