数学之路-分布式计算-hadoop、python与R(3)，-hadooppython

数学之路-分布式计算-hadoop、python与R(3)，-hadooppython

二、ubuntu 12.04hadoop2.6配置与安装

1、java7配置

#su

#addusermyhaspl

下面增加su权限

使得myhaspl用户属于sudo组

#vi/etc/group

sudo:x:27:,myhaspl

然后重新启动

#reboot

 转到myhaspl用户，开始下载解压java。

本博客所有内容是原创，如果转载请注明来源

http://blog.csdn.net/myhaspl/

#sumyhaspl

$wgethttp://download.oracle.com/otn-pub/java/jdk/7u71-b14/jdk-7u71-linux-x64.tar.gz

myhaspl@aaayun:~$tar -xzvf jdk-7u71-linux-x64.tar.gz

myhaspl@aaayun:~$sudo mv jdk1.7.0_71 /usr/lib/java

root@aaayun:/home/myhaspl#vi  /etc/profile

exportJAVA_HOME=/usr/lib/java

exportJRE_HOME=${JAVA_HOME}/jre

exportCLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

exportPATH=${JAVA_HOME}/bin:$PATH

#reboot

myhdp@myhdp:~$sudo update-alternatives --install /usr/bin/java java /usr/lib/java/bin/java300

update-alternatives:using /usr/lib/java/bin/java to provide /usr/bin/java (java) in auto mode

myhdp@myhdp:~$sudo update-alternatives --install /usr/bin/javac javac /usr/lib/java/bin/javac300

update-alternatives:using /usr/lib/java/bin/javac to provide /usr/bin/javac (javac) in auto mode

myhaspl@aaayun:~$  java -version

javaversion "1.7.0_71"

Java(TM)SE Runtime Environment (build 1.7.0_71-b14)

JavaHotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

myhaspl@aaayun:~$

myhaspl@aaayun:~$vi HelloWorldjava

publicclass HelloWorld{

publicstatic void main (String[] args){

System.out.println("HelloWorld");

}

}

myhaspl@aaayun:~$javac HelloWorld.java

myhaspl@aaayun:~$java HelloWorld

HelloWorld

myhdp@myhdp:~$sudo apt-get install ssh

myhdp@myhdp:~$sudo apt-get install rsync

2、hadoop2.6

下载解压hadoop

myhaspl@aaayun:~$wget http://mirrors.cnnic.cn/apache/hadoop/common/stable/hadoop-2.6.0.tar.gz

myhaspl@aaayun:~$tar -xzvf hadoop-2.6.0.tar.gz

将hadoop移动到系统文件夹，并授予相关权限。

myhaspl@aaayun:~$sudo mv hadoop-2.6.0 /usr/local/

myhaspl@aaayun:~$cd /usr/local/

myhaspl@aaayun:~$sudo chown -R root:root hadoop-2.6.0/

root@aaayun:/home/myhaspl#vi  /etc/profile

exportHADOOP_PREFIX=/usr/local/hadoop-2.6.0

myhdp@myhdp:/usr/local/hadoop-2.6.0$  export HADOOP_PREFIX=/usr/local/hadoop

myhdp@myhdp:/usr/local/hadoop-2.6.0$cd ~

显示hadoop文档

myhdp@myhdp:~$cd/usr/local/hadoop-2.6.0

myhdp@myhdp:~$bin/hadoop

Usage:hadoop [--config confdir] COMMAND

       where COMMAND is one of:

  fs                   run a generic filesystemuser client

  version              print the version

  jar <jar>            run a jar file

  checknative [-a|-h]  check native hadoop and compression librariesavailability

  distcp <srcurl> <desturl> copyfile or directories recursively

  archive -archiveName NAME -p <parentpath> <src>* <dest> create a hadoop archive

  classpath            prints the class path needed to get the

  credential           interact with credential providers

                       Hadoop jar and therequired libraries

  daemonlog            get/set the log level for eachdaemon

  trace                view and modify Hadoop tracingsettings

 or

  CLASSNAME            run the class named CLASSNAME

Mostcommands print help when invoked w/o parameters.

伪分布式

hdfs的地址及端口号

myhdp@myhdp:~$sudo vi /usr/local/hadoop-2.6.0/etc/hadoop/core-site.xml <configuration>

    <property>

        <name>fs.defaultFS</name>

       <value>hdfs://localhost:9000</value>

    </property>

</configuration>

~        

hdfs备份数量,改为1

etc/hadoop/hdfs-site.xml:

<configuration>

    <property>

       <name>dfs.replication</name>

        <value>1</value>

    </property>

</configuration>