RHEL6.5上使用Ambari快速搭建Hadoop集群

文章由LinuxBoy分享于2019-03-29 06:03:03热评（502）

RHEL6.5上使用Ambari快速搭建Hadoop集群

ip hostname
10.7.1.1 master.coke
10.7.1.2 dn1.coke
10.7.1.3 dn2.coke
10.7.1.4 dn3.coke
10.7.1.5 dn4.coke

1,ssh

master.coke 生成pub key 作为中控机，添加到每台机器包括master.coke 的 /root/.ssh/authorized_keys

2,hostname

对照表格更改hadoop集群的每台机器的hostname 和/etc/sysconfig/network HOSTNAME=XXXX

3,ntp

启动ntp 服务保证各台机器时钟同步

4,repo

https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+1.7.0+from+Public+Repositories

cd /etc/yum.repos.d/

wget http://public-repo-1.hortonworks.com/ambari/CentOS6/1.x/updates/1.7.0/ambari.repo

yum install ambari-server

yum install mysql-server mysql

提前建立库和授权

mysql>grant all on ambari.* to ‘ambari’@’localhost‘ identified by “175an.com”;

mysql> flush privileges;

[@master.coke yum.repos.d]# yum install ambari-server
Loaded plugins: product-id, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
6ASU5-updates | 1.1 kB 00:00
HighAvailability | 3.9 kB 00:00
LoadBalancer | 3.9 kB 00:00
ResilientStorage | 3.9 kB 00:00
ScalableFileSystem | 3.9 kB 00:00
Server | 3.9 kB 00:00
Updates-ambari-1.7.0 | 2.9 kB 00:00
ambari-1.x | 1.3 kB 00:00
epel | 1.1 kB 00:00
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package ambari-server.noarch 0:1.7.0-169 will be installed
–> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.7.0-169.noarch
–> Running transaction check
—> Package postgresql-server.x86_64 0:8.4.20-1.el6_5 will be installed
–> Processing Dependency: postgresql(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: postgresql-libs(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.20-1.el6_5.x86_64
–> Running transaction check
—> Package postgresql.x86_64 0:8.4.20-1.el6_5 will be installed
—> Package postgresql-libs.x86_64 0:8.4.20-1.el6_5 will be installed
–> Finished Dependency ResolutionDependencies Resolved

=================================================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================================================
Installing:
ambari-server noarch 1.7.0-169 Updates-ambari-1.7.0 96 M
Installing for dependencies:
postgresql x86_64 8.4.20-1.el6_5 6ASU5-updates 2.6 M
postgresql-libs x86_64 8.4.20-1.el6_5 6ASU5-updates 201 k
postgresql-server x86_64 8.4.20-1.el6_5 6ASU5-updates 3.4 M

Transaction Summary
=================================================================================================================================================================================
Install 4 Package(s)

Total download size: 102 M
Installed size: 152 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): ambari-server-1.7.0-169.noarch.rpm | 96 MB 00:14
(2/4): postgresql-8.4.20-1.el6_5.x86_64.rpm | 2.6 MB 00:00
(3/4): postgresql-libs-8.4.20-1.el6_5.x86_64.rpm | 201 kB 00:00
(4/4): postgresql-server-8.4.20-1.el6_5.x86_64.rpm | 3.4 MB 00:00
———————————————————————————————————————————————————————————
Total 6.7 MB/s | 102 MB 00:15
warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid: “Jenkins (HDP Builds) <jenkin@hortonworks.com>”
From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : postgresql-libs-8.4.20-1.el6_5.x86_64 1/4
Installing : postgresql-8.4.20-1.el6_5.x86_64 2/4
Installing : postgresql-server-8.4.20-1.el6_5.x86_64 3/4
Installing : ambari-server-1.7.0-169.noarch 4/4
Verifying : postgresql-server-8.4.20-1.el6_5.x86_64 1/4
Verifying : postgresql-libs-8.4.20-1.el6_5.x86_64 2/4
Verifying : postgresql-8.4.20-1.el6_5.x86_64 3/4
Verifying : ambari-server-1.7.0-169.noarch 4/4

Installed:
ambari-server.noarch 0:1.7.0-169

Dependency Installed:
postgresql.x86_64 0:8.4.20-1.el6_5 postgresql-libs.x86_64 0:8.4.20-1.el6_5 postgresql-server.x86_64 0:8.4.20-1.el6_5

Complete!

ambari-server setup

Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux…
SELinux status is ‘disabled’
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership…
Checking firewall…
Checking JDK…
[1] – Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[2] – Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
[3] – Custom JDK
==============================================================================
Enter choice (1):
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmland not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
JDK distribution size is 142376665 bytes
jdk-7u67-linux-x64.tar.gz… 100% (135.8 MB of 135.8 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.7.0_67
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] – PostgreSQL (Embedded)
[2] – Oracle
[3] – MySQL
[4] – PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database Name (ambari):
Username (ambari):
Enter Database Password (bigdata):
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
ERROR: Exiting with exit code -1.
REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.

yum install -y mysql-connector-java # resolve the error

重新来过

[@master.coke 6Server]# ambari-server setup

Using python /usr/bin/python2.6

Setup ambari-server

Checking SELinux…

SELinux status is ‘disabled’

Ambari-server daemon is configured to run under user ‘root’. Change this setting [y/n] (n)?

Adjusting ambari-server permissions and ownership…

Checking firewall…

Checking JDK…

Do you want to change Oracle JDK [y/n] (n)?

Completing setup…

Configuring database…

Enter advanced database configuration [y/n] (n)? y

==============================================================================

Choose one of the following options:

[1] – PostgreSQL (Embedded)

[2] – Oracle

[3] – MySQL

[4] – PostgreSQL

==============================================================================

Enter choice (1): 3

Hostname (localhost):

Port (3306):

Database Name (ambari):

Username (ambari): amb

Enter Database Password (bigdata):

Re-enter password:

Copying JDBC drivers to server resources…

Configuring remote database connection properties…

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

Proceed with configuring remote database connection properties [y/n] (y)?

Copying JDBC drivers to server resources…

Extracting system views…

…

Adjusting ambari-server permissions and ownership…

Ambari Server ‘setup’ completed successfully.

导入数据库

mysql ambari -u175an -p < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

grant all on ambari.* to “amb@localhost” identified by “amb123″;

Admin Name : admin

Cluster Name : coke

Total Hosts : 6 (6 new)

Repositories:

RedHat5 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos5/2.x/GA/2.2.0.0

redhat5 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos5

redhat6 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0

redhat6 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6

SUSE11 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/GA/2.2.0.0

suse11 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3

Ubuntu12 (HDP-2.2):
http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/GA/2.2.0.0

ubuntu12 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12

Services:
HDFS DataNode : 6 hosts
NameNode : master.coke
SNameNode : dn1.coke

YARN + MapReduce2 App Timeline Server : master.coke
NodeManager : 6 hosts
ResourceManager : master.coke

Tez Clients : 6 hosts

Nagios Server : master.coke
Administrator : nagiosadmin / (qilinli@sohu-inc.com)

Ganglia Server : master.coke

Hive Metastore : master.coke
HiveServer2 : master.coke
WebHCat Server : master.coke
Database : MySQL (Existing MySQL Database)

HBase Master : dn1.coke
RegionServer : 6 hosts

Pig Clients : 6 hosts

Sqoop Clients : 6 hosts

Oozie Server : dn1.coke
Database : MySQL (Existing MySQL Database)

ZooKeeper Server : 3 hosts

Falcon Server : dn3.coke

Storm DRPC Server : dn1.coke
Nimbus : dn1.coke
UI Server : dn1.coke
Supervisor : 6 hosts

Flume Flume : 2 hosts

Slider Clients : 6 hosts

Knox Gateway : dn1.coke

Kafka Broker : 3 hosts

CentOS安装和配置Hadoop2.2.0

Ubuntu 13.04上搭建Hadoop环境

Ubuntu 12.10 +Hadoop 1.2.1版本集群配置

Ubuntu上搭建Hadoop环境（单机模式+伪分布模式）

Ubuntu下Hadoop环境的配置

单机版搭建Hadoop环境图文教程详解

搭建Hadoop环境（在Winodws环境下用虚拟机虚拟两个Ubuntu系统进行搭建）

推荐文章：

RHEL6.5上使用Ambari快速搭建Hadoop集群