dolphinScheduler(海豚调度器)分布式机群安装

1、安装包准备

下载好安装包 apache-dolphinscheduler-3.0.0-bin.tar.gz，上传至 /opt

2、解压，重命名

cd /opt
tar -zxvf apache-dolphinscheduler-3.0.0-bin.tar.gz
mv apache-dolphinscheduler-3.0.0-bin/ dolphin_install

3、在MySQL8中创建dolphinscheduler数据库

mysql -uroot -p
​
mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
​
mysql> GRANT ALL ON dolphinscheduler.* TO 'dolphinscheduler'@'%' IDENTIFIED BY 'Pw#123456';
​
mysql> flush privileges;

4、 修改application.yaml配置文件

vim /opt/dolphin_install/alert-server/conf/application.yaml
vim /opt/dolphin_install/api-server/conf/application.yaml
vim /opt/dolphin_install/master-server/conf/application.yaml
vim /opt/dolphin_install/worker-server/conf/application.yaml

找到on-profile: mysql

      on-profile: mysql
  datasource:
    driver-class-name: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://{ip}:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true
    username: dolphinscheduler
    password: Pw#123456

{ip}为mysql8数据库ip

vim /opt/dolphin_install/api-server/conf/application.yaml
vim /opt/dolphin_install/master-server/conf/application.yaml
vim /opt/dolphin_install/worker-server/conf/application.yaml

找到type: zookeeper

  type: zookeeper
  zookeeper:
    namespace: dolphinscheduler
    connect-string: bigdata1:2181,bigdata2:2181,bigdata3:2181

bigdata1:2181,bigdata2:2181,bigdata3:2181 为大数据zookeeper集群

5、 修改common.properties配置文件

vim /opt/dolphin_install/alert-server/conf/common.properties
vim /opt/dolphin_install/api-server/conf/common.properties
vim /opt/dolphin_install/master-server/conf/common.properties
vim /opt/dolphin_install/worker-server/conf/common.properties

修改以下配置

data.basedir.path=/opt/dolphinscheduler
resource.storage.type=HDFS
fs.defaultFS=hdfs://ns:8020
yarn.resourcemanager.ha.rm.ids=bigdata1,bigdata2

ns为HDFS的命名空间

bigdata1和bigdata2为安装了resource manager的主机名

5、 修改运行时的参数

vim /opt/dolph_install/bin/env/dolphinscheduler_env.sh

修改以下配置

export JAVA_HOME=${JAVA_HOME:-/usr/local/java/jdk1.8.0_231}
export DATABASE=${DATABASE:-mysql}
export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-GMT+8}
export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-bigdata1:2181,bigdata2:2181,bigdata3:2181}
export HADOOP_HOME=${HADOOP_HOME:-/opt/cloudera/parcels/CDH/lib/hadoop}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop}
export HIVE_HOME=${HIVE_HOME:-/opt/cloudera/parcels/CDH/lib/hive}

6、修改一键部署的相关配置文件(具体内容根据自己的实际规划进行修改)

vim /opt/dolph_install/bin/env/install_env.sh

修改以下配置

ips=${ips:-"bigdata10,bigdata11,bigdata12"}
sshPort=${sshPort:-"10022"}
masters=${masters:-"bigdata10,bigdata11"}
workers=${workers:-"bigdata10:default,bigdata11:default,bigdata12:default"}
alertServer=${alertServer:-"bigdata10"}
apiServers=${apiServers:-"bigdata11"}
installPath=${installPath:-"/opt/dolphinscheduler"}
​

7、将core-site.xml和hdfs-site.xml文件拷贝到api-server/conf目录下

cp /opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop/core-site.xml /opt/dolph_install/api-server/conf
cp /opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop/hdfs-site.xml /opt/dolph_install/api-server/conf

8、将mysql-connector-java-8.0.17.jar放入api-server/libs目录下

cp mysql-connector-java-8.0.17.jar /opt/dolph_install/api-server/libs

9、初始化数据库

mysql -uroot -pPw#123456 < /opt/dolph_install/tools/sql/sql/dolphinscheduler_mysql.sql

10、执行安装命令

sh /opt/dolph_install/bin/install.sh

FAQ：

1.解决客户端无法访问mysql8问题

use mysql;
update user set host='%' where user='root';
FLUSH PRIVILEGES;