Maven编译hadoop-2.2.0-src

Maven编译hadoop-2.2.0-src

Maven编译Hadoop-2.2.0-src

1.环境

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /opt/maven
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_45/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.18-371.3.1.el5", arch: "amd64", family: "unix"

2.执行

mvn package -DskipTests -Pdist,native,docs -Dtar

3.插曲1

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

打补丁

Patch :https://issues.apache.org/jira/browse/HADOOP-10110

方法：修改hadoop源码目录下的hadoop-common-project/hadoop-auth/pom.xml

添加或修改如下部分内容：

      <dependency>
          <groupId>org.mortbay.jetty</groupId>
      <artifactId>jetty-util</artifactId>
      <scope>test</scope>
  </dependency>
    <dependency>
      <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty</artifactId>
        <scope>test</scope>
    </dependency>

4.插曲2

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/opt/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory -> [Help 1]

没装cmake

人家就是想要这个地方 这可不行  果断

yum install cmake

5.插曲3

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version

没装protobuf-2.5.0。

下载protobuf-2.5.0.tar.gz  解压  3部曲编译安装。

6.继续(这次学能了，带着debug)

mvn package -Pdist,native -DskipTests -Dtar -e -X

等了好久好久

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [2:18.165s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [50.526s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1:14.522s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.760s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [5.983s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [13.161s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [10.647s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.010s]
[INFO] Apache Hadoop Common .............................. SUCCESS [4:32.249s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [18.927s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.134s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [5:29.380s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [1:43.219s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [44.693s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [13.217s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.219s]
[INFO] hadoop-yarn ....................................... SUCCESS [1.957s]
[INFO] hadoop-yarn-api ................................... SUCCESS [1:36.233s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:31.987s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.334s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [26.932s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [49.536s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [7.882s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [1:01.936s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [6.642s]
[INFO] hadoop-yarn-client ................................ SUCCESS [11.930s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.437s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [12.064s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [1.363s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [58.193s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [11.262s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.328s]
[INFO] hadoop-yarn-project ............................... SUCCESS [17.453s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [43.479s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [6.048s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [46.449s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [9.770s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [17.249s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [9.929s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [12.193s]
[INFO] hadoop-mapreduce .................................. SUCCESS [6.817s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [10.957s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [32.407s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [4.950s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [18.817s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [10.909s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [6.662s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [11.594s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [3.737s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [3.623s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.127s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [2:55.243s]
[INFO] Apache Hadoop Client .............................. SUCCESS [25.929s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [2.516s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:38.762s
[INFO] Finished at: Sat Dec 28 21:59:12 CST 2013
[INFO] Final Memory: 68M/200M
[INFO] ------------------------------------------------------------------------

注：第一次运行，一般要下很多jar包

有时候会出现下载的时候下面这样的东西引起的一大片错误

 Connection to http://repo.maven.apache.org refused

只要重新执行

mvn package -Pdist,native -DskipTests -Dtar -e -X

就能接着下，一般是网络问题。

Maven权威指南_中文完整版清晰PDF

Linux 安装 Maven

Maven3.0 配置和简单使用

Ubuntu下搭建sun-jdk和Maven2

Maven使用入门