CDH 6.0.1 集群搭建「Process」-白红宇

CDH 6.0.1 集群搭建「Process」

阅读量：5346 次

发布时间：2019-06-15

本文共 11232 字，大约阅读时间需要 37 分钟。

这次搭建我使用的机器 os 是 Centos7.4 RH 系的下面以流的方式纪录搭建过程以及注意事项

Step1:

配置域名相关，因为只有三台机器组集群，所以直接使用了 hosts 的方法：

修改主机名hostnamectl set-hostname ryze-1.bigdata.com然后在 /etc/hosts 文件中配置相关机器的域名 和 域名简写x.x.x.x ryze-1.bigdata.com ryze-1x.x.x.x zed-1.bigdata.com  zed-1x.x.x.x zed-2.bigdata.com  zed-2并且测试其能互通配置 /etc/sysconfig/networkHOSTNAME=foo-1.example.com验证配置uname -a 需要和 hostname 得到一致的域名

Step2:

关闭防火墙

# 防火墙关闭iptables-save > /root/firewall.rulessudo chkconfig iptables offsudo service iptables stop

Step3:

启动 NTP 服务

yum install ntp设置一个同步时间服务器/etc/ntp.conf# 笔者使用的 aliyun 这里应该是可以跳过的，我检查了安装 ntp 之后，里面配置了非常多 aliyun 的相关节点。server 0.pool.ntp.orgserver 1.pool.ntp.orgserver 2.pool.ntp.org# 开启 NTP 服务sudo systemctl start ntpd# 配置 NTP 服务自启动sudo systemctl enable ntpd# 向某个服务器同步时间ntpdate -u 
     
      # 同步系统时间 hwclock --systohc

所有机器完成上面配置之后，我们开始进入安装的步骤。

Step1:

首先 Cloudera 为用户已经准备好了专用的程序仓库，我们需要将其下载下来

wget https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/# Import the repository signing GPG keysudo rpm --import https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPM-GPG-KEY-cloudera

Step2: 安装为 CDH 6.0.x 安装 jdk1.8

# yum 安装sudo yum install oracle-j2sdk1.8# 自己下载包安装tar xvfz /path/to/jdk-8u
     
      -linux-x64.tar.gz -C /usr/java/注意不能更换路径 这里如果使用 yum 安装 cloudera 的源的话会安装 1.8u141 并且会补充一个文件

The RHEL-compatible and Ubuntu operating systems supported by Cloudera Enterprise 6 all use AES-256 encryption by default for tickets. To support AES-256 bit encryption in JDK versions lower than 1.8u161, you must install the on all cluster and Hadoop user machines. Cloudera Manager can automatically install the policy files, or you can install them manually. For JCE Policy File installation instructions, see the README.txt file included in the jce_policy-x.zip file. JDK 1.8u161 and higher enable unlimited strength encryption by default, and do not require policy files.

On SLES platforms, do not install or try to use the IBM Java version bundled with the SLES distribution. CDH does not run correctly with that version.

需要注意一下，Java1.8u161 以上的版本似乎不受影响。

Step3:

安装 cloudera manager 到目标机和配置 TLS

sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server 警告如果没有证书，就不要配这个否则后面会出现 agent 无法上报的问题。然后配置 auto TSLsudo JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera /opt/cloudera/cm-agent/bin/certmanager setup --configure-servicesThat's it! When you start Cloudera Manager Server, it will have TLS enabled, and all hosts that you add to the cluster, as well as any supported services, will automatically have TLS configured and enabled.

Step4: 安装 CM 使用的数据库

# 安装数据库大礼包wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpmsudo rpm -ivh mysql-community-release-el7-5.noarch.rpmsudo yum updatesudo yum install mysql-serversudo systemctl start mysqld# 然后开始数据库配置sudo systemctl stop mysqld# 拷贝日志文件Move old InnoDB log files /var/lib/mysql/ib_logfile0 and /var/lib/mysql/ib_logfile1 out of /var/lib/mysql/ to a backup location# 配置数据库文件 /etc/my.conf[mysqld]datadir=/var/lib/mysqlsocket=/var/lib/mysql/mysql.socktransaction-isolation = READ-COMMITTED# Disabling symbolic-links is recommended to prevent assorted security risks;# to do so, uncomment this line:symbolic-links = 0key_buffer_size = 32Mmax_allowed_packet = 32Mthread_stack = 256Kthread_cache_size = 64query_cache_limit = 8Mquery_cache_size = 64Mquery_cache_type = 1max_connections = 550#expire_logs_days = 10#max_binlog_size = 100M#log_bin should be on a disk with enough free space.#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your#system and chown the specified folder to the mysql user.log_bin=/var/lib/mysql/mysql_binary_log#In later versions of MySQL, if you enable the binary log and do not set#a server_id, MySQL will not start. The server_id must be unique within#the replicating group.server_id=1binlog_format = mixedread_buffer_size = 2Mread_rnd_buffer_size = 16Msort_buffer_size = 8Mjoin_buffer_size = 8M# InnoDB settingsinnodb_file_per_table = 1innodb_flush_log_at_trx_commit  = 2innodb_log_buffer_size = 64Minnodb_buffer_pool_size = 4Ginnodb_thread_concurrency = 8innodb_flush_method = O_DIRECTinnodb_log_file_size = 512M[mysqld_safe]log-error=/var/log/mysqld.logpid-file=/var/run/mysqld/mysqld.pidsql_mode=STRICT_ALL_TABLES# 添加到自启动项sudo systemctl enable mysqld# 重新将 MySQL 启动起来sudo systemctl start mysqld

安装 JDBC 驱动

# Download the MySQL JDBC driver from http://www.mysql.com/downloads/connector/j/5.1.html (in .tar.gz format). As of the time of writing, you can download version 5.1.46 using wget as follows:wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz# Extract the JDBC driver JAR file from the downloaded file. For example:tar zxvf mysql-connector-java-5.1.46.tar.gz# Copy the JDBC driver, renamed, to /usr/share/java/. If the target directory does not yet exist, create it. For example:sudo mkdir -p /usr/share/java/cd mysql-connector-java-5.1.46sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

使用 utf8 编码规则创建新的数据库服务于需要使用到数据库的服务：

Databases for Cloudera Software
Service	Database	User
Cloudera Manager Server	scm	scm
Activity Monitor	amon	amon
Reports Manager	rman	rman
Hue	hue	hue
Hive Metastore Server	metastore	hive
Sentry Server	sentry	sentry
Cloudera Navigator Audit Server	nav	nav
Cloudera Navigator Metadata Server	navms	navms
Oozie	oozie	oozie

这里我们会使用的服务：

Cloudera Manager Server

Reports Manager

Hue

Hive Metastore Server

Cloudera Navigator Audit Server

Cloudera Navigator Metadata Server

但是我打算全部都建起来，后面再来看

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon';GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman';GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore';GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry';GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav';GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms';GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';

Step5: setup database

由于我们的数据库搭建在远端所以我们使用：

Example 2: Running the script when MySQL or MariaDB is installed on another hostThis example demonstrates how to run the script on the Cloudera Manager Server host (cm01.example.com) and connect to a remote MySQL or MariaDB host (db01.example.com):sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h db01.example.com --scm-host cm01.example.com scm scmEnter database password:JAVA_HOME=/usr/java/jdk1.8.0_141-clouderaVerifying that we can write to /etc/cloudera-scm-serverCreating SCM configuration file in /etc/cloudera-scm-serverExecuting:  /usr/java/jdk1.8.0_141-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.[                          main] DbCommandExecutor              INFO  Successfully connected to database.All done, your SCM database is configured correctly!

Step6: 安装 CDH 和其他的组件

# start cloudera-scm-serversudo systemctl start cloudera-scm-server# 通过该命令观察启动流程sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log当看到INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.the Cloudera Manager Admin Console is ready然后就可以通过 ip:8070 上去了

之后就是选择版本，选择完了之后进入到集群安装界面：

扫描 hosts 地址使用 FQDN 来标识，填写之后进行扫描。

Range Definition    Matching Hosts10.1.1.[1-4]    10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4 host[1-3].example.com    host1.example.com, host2.example.com, host3.example.comhost[07-10].company.com    host07.example.com, host08.example.com, host09.example.com, host10.example.comNote: Unqualified hostnames (short names) must be unique in a Cloudera Manager instance. For example, you cannot have both host01.example.com and host01.standby.example.com managed by the same Cloudera Manager Server.You can specify multiple addresses and address ranges by separating them with commas, semicolons, tabs, or blank spaces, or by placing them on separate lines. Use this technique to make more specific searches instead of searching overly wide ranges. Only scans that reach hosts running SSH will be selected for inclusion in your cluster by default. You can enter an address range that spans over unused addresses and then clear the nonexistent hosts later in the procedure, but wider ranges require more time to scan.

单机 serach 即可扫描，扫描到节点之后进行下一步。

Select Repository 使用默认选项直接选择下一步。

Accept JDK License 阅读协议☑️选择下一步。

ssh 设置，选择合适的选项下一步。

然后开始安装 agents 下载速度慢可以直接用 proxychain 或者找快的地方在 https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/RPMS/x86_64/ 下好之后上传。

然后开始安装 parcels 因为这个在国内安装下载速度也是超级慢所以推荐直接用网速快的地方去 https://archive.cloudera.com/cdh6/6.0.1/parcels/ 下载对应的包。

然后上传到服务器放到 cm 机器的 /opt/cloudera/parcel-repo/ 目录下。要下包 sha1 文件和 manifest 文件。

按照提示解决 warning

已启用透明大页面压缩，可能会导致重大性能问题。请运行“echo never > /sys/kernel/mm/transparent_hugepage/defrag”和 “echo never > /sys/kernel/mm/transparent_hugepage/enabled”以禁用此设置，然后将同一命令添加到 /etc/rc.local 等初始化脚本中，以便在系统重启时予以设置。以下主机将受到影 Starting with CDH 6, PostgreSQL-backed Hue requires the Psycopg2 version to be at least 2.5.4, see the  for more information. This warning can be ignored if hosts will not run CDH 6, or will not run Hue with PostgreSQL. The following hosts have an incompatible Psycopg2 version of '2.5.1':

Step7: Set Up a Cluster Using the Wizard

选择想要安装的服务

进行角色配置，master 集中在一台机器的话尽量让该机器的配置比较强。