1、安装主机介绍及规划

本文档以在3台Centos7.5物理服务器上安装、部署CDH6.2.0集群为例，描述CDH集群的安装部署。

注意：主机的root账号密码相同，主机系统相同。

安装规划：

2、安装包准备

1、CM安装包地址

https://pan.baidu.com/s/1m6mE0CIxAx_SKrMTV5j0hQ 提取码：1231

2、CDH安装包地址

https://pan.baidu.com/s/1I9JapNamnRI519pUSOZqrA 提取码：6666

3、mysql安装包地址

链接：https://pan.baidu.com/s/1oO_38Wj5Ut1xcy9_EW7dOA 

提取码：2bnw

4、mysql-connector

链接：https://pan.baidu.com/s/1kbZ04-Ts6npSpCLCwiqrFA           提取码：c4hk

3、安装初始化准备

# 关闭防火墙

systemctl stop firewalld

systemctl disable firewalld

# 关闭selinux

sed -i 's/enforcing/disabled/' /etc/selinux/config#永久

setenforce 0# 临时

# 关闭THP

echo never > /sys/kernel/mm/transparent_hugepage/enabled

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'>> /etc/rc.local

echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'>> /etc/rc.local

# 时间同步

yum install ntpdate -y

ntpdate time.windows.com

echo "*/5 * * * * /usr/sbin/ntpdate ntp.api.bz >/dev/null 2 >&1" >> /var/spool/cron/root

# 分别在各节点设置主机名称

hostnamectl set-hostname cdh1#如在192.168.52.18配置

hostnamectl set-hostname cdh2

hostnamectl set-hostname cdh3

# 在所有节点

cat >> /etc/hosts << EOF

192.168.52.18cdh1

192.168.52.19cdh2

192.168.52.20cdh3

EOF

# 在所有节点，配置免密登陆

ssh-keygen -t rsa

ssh-copy-id root@192.168.52.18#拷贝本地密钥到其他服务器

yum install expect -yauto_ssh_copy_id() {    expect -c "set timeout -1;        spawn ssh-copy-id $1;        expect {            *(yes/no)* {send -- yes\r;exp_continue;}            *assword:* {send -- $2\r;exp_continue;}            eof        {exit 0;}        }";}ssh_copy_id_to_all() {    for SERVER in $SERVERS    do        auto_ssh_copy_id $SERVER $PASSWORD    done}ssh_copy_id_to_allsh auto_ssh-key.sh

#安装jdk

tar -xzf jdk-8u201-linux-x64.tar.gz -C /usr/local/

启用环境变量:

# vi /etc/profile

exportJAVA_HOME=/usr/local/jdk1.8.0_201

exportPATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/sbin

然后source使其生效

source /etc/profile

java -version

mkdir -p /usr/java

ln -s /usr/local/jdk1.8.0_201/usr/java/default

#修改vm.swappiness值

echo 'vm.swappiness=0' >> /etc/sysctl.conf

sysctl -p

#修改limits值

cat >> /etc/security/limits.conf << EOF

* soft nofile 65535

* hard nofile 65535

* soft nproc 192027

* hard nproc 192027

EOF

4、安装http（CDH1）

yum -y install httpd

systemctl start httpd

systemctl status httpd

systemctl enable httpd

#所有节点安装依赖包

yum -y install bind-utils chkconfig cyrus-sasl-gssapi cyrus-sasl-plain fuse fuse-libs gcc libxslt mod_ssl MySQL-python openssl openssl-devel perl portmap 

yum -y install postgresql-server psmisc python-devel python-psycopg2 python-setuptools sed sqlite swig zlib

5、安装MYSQL（CDH3）

5.1、安装mysql

tar -xvf mysql-5.7.19-1.el7.x86_64.rpm-bundle.tar

yum remove mysql-libs -y

rpm -ivh mysql-community-common-5.7.19-1.el7.x86_64.rpm --nodeps --force

rpm -ivh mysql-community-libs-5.7.19-1.el7.x86_64.rpm --nodeps --force

rpm -ivh mysql-community-libs-compat-5.7.19-1.el7.x86_64.rpm --nodeps --force

rpm -ivh mysql-community-devel-5.7.19-1.el7.x86_64.rpm --nodeps --force

rpm -ivh mysql-community-client-5.7.19-1.el7.x86_64.rpm --nodeps --force

rpm -ivh mysql-community-server-5.7.19-1.el7.x86_64.rpm --nodeps --force

#启动mysql

systemctl start mysqld

systemctl status mysqld

systemctl enable mysqld

#查看密码

cat /var/log/mysqld.log | grep password

mysql -u root -p'<za:&dead1&A'

mysql> SET PASSWORD = PASSWORD('1qaz@WSX');

mysql> show databases;

mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '1qaz@WSX' WITH GRANT OPTION;

mysql> FLUSH PRIVILEGES;

mysql> exit

测试登陆mysql: mysql -u root -p'1qaz@WSX'

配置master节点的Mysql，查看mysql进程是否正常存在

ps -ef |grep mysql#启动服务

netstat -ntpl | grep :3306#侦听端口

5.2、创建数据库

create database cmf default character set = 'utf8';

create database amon default character set = 'utf8';

create database hue default character set = 'utf8';

create database oozie default character set = 'utf8';

create database lyz default character set = 'utf8';

create database hive default character set = 'utf8';

5.3、mysql-connector-java上传整理

#在所有节点安装

mkdir /usr/share/java

cd /usr/share/java

mv mysql-connector-java-5.1.39.jar mysql-connector-java.jar

6、CM组件和节点安装

cdh2 cdh3集群只上传agent和daemons rpm包,cdh1三个包全部上传

6.1、全部节点都安装daemons和agent 

yum localinstall -y cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm

yum localinstall -y cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm

6.2、cdh1节点安装server

yum localinstall -y cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm

6.3、安装CDH6.2.0

下载之后在CDH1节点上传:

mv CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel* /opt/cloudera/parcel-repo/

mv PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel* /opt/cloudera/parcel-repo/

mv manifest.json /opt/cloudera/parcel-repo/

mv PHOENIX-1.0.jar /opt/cloudera/csd/

修改server配置文件:

vi /etc/cloudera-scm-server/db.properties

com.cloudera.cmf.db.host=cdh3

com.cloudera.cmf.db.name=cmf

com.cloudera.cmf.db.user=root

com.cloudera.cmf.db.password=1qaz@WSX

com.cloudera.cmf.db.setupType=EXTERNAL

修改agent配置文件:

# 全部节点修改agent

vi /etc/cloudera-scm-agent/config.ini

server_host=cdh1

6.4、启动CM，开始安装CDH集群

在cdh1上启动server,和agent

systemctl start cloudera-scm-server

systemctl status cloudera-scm-server          systemctl start cloudera-scm-agent

systemctl status cloudera-scm-agent

6.5、启动所有CM Agent

在其他节点上启动agent：

systemctl start cloudera-scm-agent

systemctl status cloudera-scm-agent

netstat -ntpl |grep 7180

6.6、CM安装成功

Google Chrome浏览器启动Cloudera Manager 6控制台（默认端口号是7180），启动成功就会看到登陆页面，输入admin/admin进入cm控制页面。

http://192.168.52.18:7180/cmf/login

7、操作WEB 界面

选择自定义服务，先安装个zookeeper

Activity Monitor我选择的是cdh1节点，zookeeper我选择的是1-3节点

数据库是在cdh3上部署，然后主机名称为cdh3,数据库选择amon,用户名为root密码1qaz@WSX然后点击测试连接

点击继续

点击继续

8、报错解决

rm -f /var/lib/cloudera-scm-agent/cm_guid 

systemctl restart cloudera-scm-agent