hdp

hdp

关注
发信
关注(28)粉丝(399)

HDP 大数据平台搭建

一、概述

二、安装部署

2.1 主机规划

| 序号 | IP地址 | 主机名 |系统版本|
| -------- | -------- | -------- |
| 1 | 172.20.2.222 | ambari-server |centos7.3
| 2 | 172.20.2.203 | hadoop-1 |centos7.3
| 3 | 172.20.2.204 | hadoop-2 |centos7.3
| 4 | 172.20.2.205 | hadoop-3 |centos7.3

2.2 部署

2.2.1 基础环境部署

a.修改主机名配置hosts

systemctl stop firewalld

hostnamectl set-hostname  ambari-server   #更改个主机名

sed -i 's/SELINUX=enforcing/SELINUX=disable/g' /etc/selinux/config

setenforce 0

cat >>/etc/hosts<<EOF    #添加各个节点hosts解析

172.20.2.222    ambari-server

172.20.2.203      hadoop-1

172.20.2.204      hadoop-2

172.20.2.205      hadoop-3

EOF

b.配置ambari-server免密钥登录其他节点

ssh-keygen -t rsa     #在ambari-server生成密钥对

for num in `seq 1 3`;do ssh-copy-id -i /root/.ssh/id_rsa.pub root@hadoop-$num;done

c.在ambari-server安装数据库
在ambari的setup中我们可以选择使用默认的postgresql，也可以自定义使用其他数据库，此处选用mariadb，便于后期管理维护

yum -y install mariadb-server mysql-connector-java

systemctl start mariadb

mysql -uroot password "mysqladmin"

登录数据库后我们采用root登录

d.配置本地yum源
由于ambari的各个组件包很大,HDP大约快6G，我们采取离线安装方式，将ambari-server配置为yum源，事先在网上下载HDP、HDP-UTILS和Ambari
软件版本：

参考url：ambari各版本参考链接
HDP和HDP UTILS参考链接

2.4.1.02.5.0.01.1.0.21
yum install httpd -y && systemctl start httpd

mkdir /software && cd /software

#下载软件

wget -c http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.1.0/ambari-2.4.1.0-centos7.tar.gz

wget -c http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.0.0/HDP-2.5.0.0-centos7-rpm.tar.gz

wget -c http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz

#创建目录

mkdir -p /var/www/html/ambari

mkdir /var/www/html/hdp/{HDP,HDP-UTILS-1.1.0.21}

#解压软件

tar -zxvf ambari-2.4.1.0-centos7.tar.gz -C /var/www/html/ambari/

tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS-1.1.0.21/

tar -zxvf HDP-2.5.0.0-centos7-rpm.tar.gz -C /var/www/html/hdp/

配置ambari YUM源

cat >/etc/yum.repos.d/ambari.repo<<EOF

#VERSION_NUMBER=2.4.1.0-22

[Updates-ambari-2.4.1.0]

name=ambari-2.4.1.0 - Updates

baseurl=http://172.20.2.222/ambari/AMBARI-2.4.1.0/centos7/2.4.1.0-22/

gpgcheck=1

gpgkey=http://172.20.2.222/ambari/AMBARI-2.4.1.0/centos7/2.4.1.0-22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

EOF

#配置hdp  YUM源

cat >/etc/yum.repos.d/hdp.repo<<EOF

#VERSION_NUMBER=2.5.0.0-1245

[HDP-2.5.0.0]

name=HDP Version - HDP-2.5.0.0

baseurl=http://172.20.2.222/hdp/HDP/centos7/

gpgcheck=1

gpgkey=http://172.20.2.222/hdp/HDP/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

[HDP-UTILS-1.1.0.21]

name=HDP-UTILS Version - HDP-UTILS-1.1.0.21

baseurl=http://172.20.2.222/hdp/HDP-UTILS-1.1.0.21/

gpgcheck=1

gpgkey=http://172.20.2.222/hdp/HDP-UTILS-1.1.0.21/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

EOF

2.2.2 安装配置ambari

a.安装ambari server

yum clean all

yum makecache

yum -y install ambari-server

#进入数据库创建ambari库和导入数据

mysql>create database ambari DEFAULT CHARSET utf8;

mysql>use ambari

mysql>source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

b.配置ambari

[root@ambari-server hdp]# ambari-server setup

Using python  /usr/bin/python

Setup ambari-server

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)? y

Customize user account for ambari-server daemon [y/n] (n)? y

Enter user account for ambari-server daemon (root):root

Adjusting ambari-server permissions and ownership...

Checking firewall status...

Checking JDK...

Do you want to change Oracle JDK [y/n] (n)? y

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

==============================================================================

Enter choice (1): 1

To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.or

acle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y

Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u77-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u77

-linux-x64.tar.gzjdk-8u77-linux-x64.tar.gz... 100% (173.0 MB of 173.0 MB)

Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u77-linux-x64.tar.gz

Installing JDK to /usr/jdk64/

Successfully installed JDK to /usr/jdk64/

Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to /var/lib/ambari-server/resources/jc

e_policy-8.zip

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip

Installing JCE policy...

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)? y

Configuring database...

==============================================================================

Choose one of the following options:

[1] - PostgreSQL (Embedded)

[2] - Oracle

[3] - MySQL / MariaDB

[4] - PostgreSQL

[5] - Microsoft SQL Server (Tech Preview)

[6] - SQL Anywhere

[7] - BDB

==============================================================================

Enter choice (3): 3

Hostname (ambari-server): ambari-server

Port (3306):

Database name (ambari):

Username (root): root

Enter Database Password (mysqladmin):

Configuring ambari database...

Copying JDBC drivers to server resources...

Configuring remote database connection properties...

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/r

esources/Ambari-DDL-MySQL-CREATE.sqlProceed with configuring remote database connection properties [y/n] (y)? y

Extracting system views...

............

Adjusting ambari-server permissions and ownership...

Ambari Server 'setup' completed successfully.

#启动服务

ambari-server start

c.浏览器文本界面安装
浏览器访问：http://172.20.2.222:8080 默认用户名密码admin/admin

 
HDP 大数据平台搭建-LMLPHP
image

启动安装向导

 
HDP 大数据平台搭建-LMLPHP
image

添加将要创建的集群名称

 
HDP 大数据平台搭建-LMLPHP
image

下载hdp版本

 
HDP 大数据平台搭建-LMLPHP
image

选择本地hdp源

 
HDP 大数据平台搭建-LMLPHP
image

添加写需要安装的各个目标节点的主机名，填写ambari-server免密钥登录其他节点的私钥，确认注册

 
HDP 大数据平台搭建-LMLPHP
image
 
HDP 大数据平台搭建-LMLPHP
image

选择需要安装的服务

 
HDP 大数据平台搭建-LMLPHP
image

选择安装软件所在的那个节点上

 
HDP 大数据平台搭建-LMLPHP
image
 
HDP 大数据平台搭建-LMLPHP
image

自定义服务路径或密码

 
HDP 大数据平台搭建-LMLPHP
image

Review

 
HDP 大数据平台搭建-LMLPHP
image

进行安装部署

 
HDP 大数据平台搭建-LMLPHP
image
 
HDP 大数据平台搭建-LMLPHP
image

添加服务

 
HDP 大数据平台搭建-LMLPHP
image

查看各组件的概况和配置

 
HDP 大数据平台搭建-LMLPHP
image
 
HDP 大数据平台搭建-LMLPHP
image


小礼物走一走，来简书关注我

作者：kaliarch
链接：https://www.jianshu.com/p/3fa3df26816b
来源：简书
简书著作权归作者所有，任何形式的转载都请联系作者获得授权并注明出处。

05-11 20:19
Powered by LMLPHP ©2024 hdp 0.065363