一、双NameNode高可用Hadoop集群
1.1、双NameNode高可用Hadoop集群架构
双NameNode高可用Hadoop集群其实是主备模式（当主节点故障时，就自动将备节点启用作为主节点，这个监控切换是由ZooKeeper集群控制的），控制原理如下：
《1》主备NameNode节点启动时会连接到ZooKeeper集群并创建一个自身的状态文件，并且会每隔几秒(如6秒)就在ZooKeeper集群重写自身的状态文件，表示自己正常工作；
《2》当主NameNode节点故障时，就不会重写ZooKeeper集群中自身的状态文件了（如经过18秒还没有更新自身状态文件）则ZooKeeper就会通知备NameNode节点的FailOverController接管切换【目前完成了状态的监控与角色的切换】。
《3》JournalNode集群是用于同步主备节点的元数据，保证主备节点的元数据一致（即：NameNode主节点写元数据的时候不仅在自己上写，同时还给JounalNode集群写一份；然后NameNode的备用节点会定期从JounalNode拉取元数据）。
1.2、双NameNode高可用Hadoop集群规划
本文部署Hadoop集群的服务器统一采用AlmaLinux9.3版本。
| 角色/主机名 | namenodemaster 192.168.1.120 | yarnserver 192.168.1.6 | slave001 192.168.1.70 | slave002 192.168.1.151 | slave003 192.168.1.169 |
|---|---|---|---|---|---|
| NameNode | ✔ | ✔ | ❌ | ❌ | ❌ |
| DataNode | ❌ | ❌ | ✔ | ✔ | ✔ |
| JournalNode | ❌ | ❌ | ✔ | ✔ | ✔ |
| ZooKeeper | ❌ | ❌ | ✔ | ✔ | ✔ |
| ZKFC | ✔ | ✔ | ❌ | ❌ | ❌ |
| ResourceManager | ❌ | ✔ | ❌ | ❌ | ❌ |
| NodeManager | ❌ | ❌ | ✔ | ✔ | ✔ |
| JobHistoryServer | ❌ | ✔ | ❌ | ❌ | ❌ |
| 软件名称 | 软件版本 |
|---|---|
| JDK | openjdk2109010【建议使用openjdk11】 |
| ZooKeeper | zookeeper3.9.5 |
| Hadoop | hadoop-3.5.0 |
二、双NameNode高可用Hadoop集群环境准备
DevOps自动化运维实践_ansible-playbook的使用
https://blog.csdn.net/xiaochenxihua/article/details/159888140
2.1、给Hadoop集群的所有服务器做免密信任
实现Linux的ssh免密登录实操保姆级教程https://blog.csdn.net/xiaochenxihua/article/details/152375722
bash
#一、在Hadoop集群外的服务器(如：192.168.1.119)安装Ansible
#1-先安装第三方的epel源
yum install epel-release -y
#2-直接安装Ansible（由于Ansible依赖Pyhont环境，且是红帽系系统的一个组成部分，因此推荐用yum命令安装）
yum install ansible -y
#二、在Ansible中配置需要操作的Hadoop集群服务器
vi /etc/ansible
cat>>/etc/ansible/hosts<<'EOF'
[hadoop]
192.168.1.120 hostname=namenodemaster
192.168.1.6 hostname=yarnserver
192.168.1.70 hostname=slave001
192.168.1.151 hostname=slave002
192.168.1.169 hostname=slave003
EOF
# 查看文件末尾是否新增内容
tail /etc/ansible/hosts
bash
#编写ssh免密登录脚本实现本机免密登录批量操作的服务器
#1-创建ssh免密登录批量部署脚本
mkdir -p /data/ansible-script
cd /data/ansible-script
vi ssh.yml
#2-ssh.yml（ssh免密登录）的完整内容
- hosts: hadoop
gather_facts: no
vars:
ssh_key_type: ed25519
ssh_key_path: /root/.ssh/id_ed25519
tasks:
- name: 关闭 SSH 主机密钥检查（避免交互 yes/no）
lineinfile:
path: /etc/ssh/ssh_config
regexp: '.*StrictHostKeyChecking.*'
line: 'StrictHostKeyChecking no'
- name: 确保远程 .ssh 目录存在（权限 0700）
file:
path: /root/.ssh
state: directory
mode: '0700'
- name: 本地生成 ed25519 密钥（已存在则跳过）
delegate_to: localhost
become: true
run_once: true
openssh_keypair:
path: "{{ ssh_key_path }}"
type: "{{ ssh_key_type }}"
passphrase: ""
state: present
- name: 读取本地公钥内容
local_action:
module: slurp
src: "{{ ssh_key_path }}.pub"
register: ssh_key_slurp
- name: 解码公钥
set_fact:
ssh_pub_key: "{{ ssh_key_slurp.content | b64decode | trim }}"
- name: 推送公钥到目标主机（官方模块：安全、追加、不覆盖）
authorized_key:
user: root
key: "{{ ssh_pub_key }}"
state: present
exclusive: no # 不覆盖原有公钥（多密钥共存）
tags:
- install_ssh
#3-执行脚本
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ssh.yml -k
#4-验证本机免密登录Hadoop集群是否全部成功
ansible hadoop -m ping
#或者直接单独访问Hadoop集群的每台服务器（不用输入密码则成功）
ssh 192.168.1.120
ssh 192.168.1.6
ssh 192.168.1.70
ssh 192.168.1.151
ssh 192.168.1.169
| 配置项 | 控制主体 | 生效范围 |
|---|---|---|
/etc/ssh/ssh_config StrictHostKeyChecking no |
系统 ssh 命令 | 所有手动 ssh、scp、sftp，不控制 ansible 内置校验 |
ANSIBLE_HOST_KEY_CHECKING=False |
Ansible 程序 | Ansible 所有 ssh 连接（playbook、ad-hoc）专属开关【Ansible 在建立 SSH 通道前，自己会做主机指纹校验，这套逻辑独立于系统 ssh 配置，不受ssh_config里StrictHostKeyChecking控制】 |
为什么已经修改了/etc/ssh/ssh_config的配置，连接还是需要验证 |
|---|
Ansible 底层调用 ssh，但有两层校验： 1. Ansible 自身先读取known_hosts做指纹校验（由ANSIBLE_HOST_KEY_CHECKING控制） 2. 再调用系统 ssh 客户端，此时才会读取/etc/ssh/ssh_config /root/.ssh/config |
| 三种免交互方案： 方案 1：临时环境变量 ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ssh.yml -k ✅方案 2：写入 ansible.cfg 永久生效（推荐，不用每次输变量） #在 ansible.cfg 添加 [defaults] host_key_checking = False 方案 3：playbook 全局关闭 ansible 主机校验（无需外部变量） #在 play 顶部加environment，不用手动传环境变量 - hosts: hadoop gather_facts: no environment: ANSIBLE_HOST_KEY_CHECKING: "False" # ansible自身关闭指纹校验 vars: ssh_key_type: ed25519 ssh_key_path: /root/.ssh/id_ed25519 tasks: # 你的原有任务不变 |
2.2、给Hadoop集群的服务器修改为对应的主机名称
bash
#给Hadoop集群的服务器修改为对应的主机名称
cd /data/ansible-script/
vi set_hostname.yml
#set_hostname.yml文件的完整内容
- hosts: hadoop
remote_user: root
gather_facts: false
tasks:
# 1. 永久修改系统主机名（写入/etc/hostname）
- name: 永久设置主机名
hostname:
name: "{{ hostname }}"
# 2. 立即生效当前会话主机名
- name: 临时刷新主机名无需重启
shell: hostname {{ hostname | quote }}
# 3. 更新/etc/hosts本地解析，避免主机名无法解析
- name: 更新/etc/hosts本地回环解析
lineinfile:
path: /etc/hosts
regexp: '^127\.0\.0\.1'
line: "127.0.0.1 {{ hostname }} localhost localhost.localdomain"
state: present
#执行修改主机名称的命令【由于已经设置了免密登录，因此不用添加参数-k进行手动输入密码】
ansible-playbook set_hostname.yml
2.3、给Hadoop集群中服务器配置主机名与IP的映射
要想实现主机名称与IP的映射有两种方法**【内部构建DNS服务器】（适合大型集群【几百上千个】）、【本地解析】** （直接在本地的hosts文件中添加主机名称与IP的映射关系）
bash
#给Hadoop集群中服务器配置主机名与IP的映射【把Hadoop集群所有主机 IP + 主机名写入每台机器】
#1-创建hosts.j2模板
mkdir -p /etc/ansible/templates
cd /etc/ansible/templates
vi hosts.j2
#hosts.j2模板的完整内容
# 标准本地回环解析
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# Hadoop集群节点IP与主机名映射
{% for host in groups['hadoop'] %}
{{ host }} {{ hostvars[host].hostname }}
{% endfor %}
#2-实现服务器配置本地主机名与IP映射的脚本
cd /data/ansible-script/
vi hosts.yml
#hosts.yml文件完整内容
- name: 批量更新所有节点/etc/hosts（含本地回环+集群域名）
hosts: hadoop
remote_user: root
tasks:
- name: 推送hosts模板覆盖/etc/hosts
template:
src: /etc/ansible/templates/hosts.j2
dest: /etc/hosts
owner: root
group: root
mode: '0644'
backup: yes # 自动备份原有hosts文件，防止误操作
#3-执行脚本
ansible-playbook hosts.yml
2.4、给Hadoop集群中服务器做系统优化配置
bash
#给Hadoop集群中服务器做系统优化配置
cd /data/ansible-script/
vi optimize_os_config.yml
#optimize_os-config.yml文件的完整内容
- name: 系统内核、安全、时间同步标准化优化（Hadoop集群专用）
hosts: hadoop
gather_facts: true
become: true
handlers:
- name: restart_sshd
ansible.builtin.service:
name: sshd
state: restarted
tasks:
# ===================== 1 SELinux 关闭 =====================
- name: 临时关闭SELinux
ansible.builtin.command: setenforce 0
changed_when: true
failed_when: false
- name: 永久禁用SELinux配置
ansible.builtin.lineinfile:
path: /etc/selinux/config
regexp: '^SELINUX=.*'
line: 'SELINUX=disabled'
backrefs: true
backup: true
- name: 校验SELinux状态
command: getenforce
register: selinux_stat
failed_when: selinux_stat.stdout != 'Permissive' and selinux_stat.stdout != 'Disabled'
# ===================== 2 全局文件/进程限制（统一单文件，消除冲突） =====================
- name: 清理冲突limits主配置，统一由limits.d托管
lineinfile:
path: /etc/security/limits.conf
regexp: '^\*.*nofile|^\*.*nproc'
state: absent
backup: true
- name: limits.d 统一软硬限制（最终生效配置）
copy:
dest: /etc/security/limits.d/99-hadoop.conf
mode: '0644'
backup: true
content: |
* soft nproc 655360
* hard nproc 655360
* soft nofile 1048576
* hard nofile 1048576
root soft nproc unlimited
root hard nproc unlimited
# ===================== 3 防火墙完整清理 =====================
- name: 停止并禁用firewalld
service:
name: firewalld
state: stopped
enabled: false
ignore_errors: true
- name: 清空所有iptables表规则
shell: |
iptables -t filter -F && iptables -t nat -F && iptables -t mangle -F
iptables -X
changed_when: true
# ===================== 4 Chrony 时间同步 =====================
- name: 安装chrony
yum:
name: chrony
state: present
- name: 替换chrony时间源，删除原有pool/server
lineinfile:
path: /etc/chrony.conf
regexp: '^(pool|server) .*'
state: absent
backup: true
- name: 写入阿里云NTP源
lineinfile:
path: /etc/chrony.conf
line: 'pool ntp.aliyun.com iburst'
state: present
- name: 设置时区上海
timezone:
name: Asia/Shanghai
- name: 启动chronyd开机自启
systemd:
name: chronyd
state: started
enabled: true
- name: 强制同步阿里云NTP时间并写入硬件时钟
shell: |
chronyd -q 'pool ntp.aliyun.com iburst'
hwclock -w
register: time_sync_res
changed_when: "'System clock wrong by' in time_sync_res.stdout"
# ===================== 5 关闭透明大页THP =====================
- name: 临时关闭THP
shell: echo never > /sys/kernel/mm/transparent_hugepage/enabled && echo never > /sys/kernel/mm/transparent_hugepage/defrag
changed_when: false
ignore_errors: true
- name: 永久禁用THP systemd服务
copy:
dest: /etc/systemd/system/disable-thp.service
mode: '0644'
content: |
[Unit]
Description=Disable Transparent Huge Pages Hadoop Java
After=sysfs.target
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled && echo never > /sys/kernel/mm/transparent_hugepage/defrag'
[Install]
WantedBy=multi-user.target
- name: 重载systemd并启动THP禁用服务
systemd:
name: disable-thp
daemon_reload: true
enabled: true
state: started
- name: 校验THP已关闭
command: cat /sys/kernel/mm/transparent_hugepage/enabled
register: thp_check
failed_when: "'never' not in thp_check.stdout"
# ===================== 6 内核sysctl调优 =====================
- name: Hadoop专用sysctl内核参数
sysctl:
name: "{{ item.name }}"
value: "{{ item.value }}"
sysctl_file: /etc/sysctl.d/99-hadoop.conf
reload: true
loop:
- { name: vm.swappiness, value: 10 }
- { name: vm.vfs_cache_pressure, value: 50 }
- { name: vm.overcommit_memory, value: 1 }
- { name: net.core.somaxconn, value: 65535 }
- { name: net.core.netdev_max_backlog, value: 65535 }
- { name: net.ipv4.tcp_syncookies, value: 1 }
- { name: net.ipv4.tcp_tw_reuse, value: 1 }
- { name: net.ipv4.tcp_fin_timeout, value: 30 }
# ===================== 7 fstab 兼容UUID/LVM追加noatime =====================
- name: fstab挂载自动添加noatime，兼容UUID/LVM/sd设备
replace:
path: /etc/fstab
regexp: '(\s+)(defaults|relatime)(\s+0\s+0)$'
replace: '\1\2,noatime\3'
backup: true
- name: 重载挂载配置
command: mount -a
changed_when: false
# ===================== 8 IO调度自动适配所有磁盘 + 版本兼容 =====================
- name: 自动获取所有数据块设备（排除sr0光驱）
shell: ls /sys/block/ | grep -E 'sd|vd|nvme' | grep -v sr0
register: disk_list
changed_when: false
- name: 临时设置mq-deadline调度
shell: echo mq-deadline > /sys/block/{{ item }}/queue/scheduler
loop: "{{ disk_list.stdout_lines }}"
ignore_errors: true
changed_when: false
- name: RHEL8+ 使用grubby永久设置IO调度
shell: grubby --update-kernel=ALL --args="elevator=mq-deadline"
when: ansible_distribution_major_version | int >= 8
ignore_errors: true
changed_when: false
# ===================== 9 SSH高并发优化 =====================
- name: SSHD并发参数调优
lineinfile:
path: /etc/ssh/sshd_config
regexp: '^#?{{ item.key }}'
line: '{{ item.key }} {{ item.val }}'
backrefs: true
loop:
- { key: 'UseDNS', val: 'no' }
- { key: 'MaxSessions', val: '100' }
- { key: 'MaxStartups', val: '1000:80:10000' }
notify: restart_sshd
# 前置校验sshd语法，避免重启失败断连
- name: 校验sshd_config语法，防止重启断连
command: sshd -t
#执行脚本
ansible-playbook optimize_os_config.yml
#验证（直接脚本批量检验）
cd /data/ansible-script
vi check_os_optimize.yml
#【check_os_optimize.yml】校验文件的完整内容
- name: Hadoop系统优化结果批量校验
hosts: hadoop
gather_facts: true
become: true
tasks:
- name: 1.校验SELinux状态
command: getenforce
register: res_selinux
failed_when: false
- name: SELINUX配置文件
command: grep SELINUX= /etc/selinux/config
register: res_selinux_cfg
failed_when: false
- name: 2.校验limits配置
command: cat /etc/security/limits.d/99-hadoop.conf
register: res_limits
failed_when: false
- name: 3.防火墙运行状态
command: systemctl is-active firewalld
register: res_fw_run
failed_when: false
- name: 3.防火墙开机自启状态
command: systemctl is-enabled firewalld
register: res_fw_boot
failed_when: false
- name: 4.Chrony时区校验
command: timedatectl show -p Timezone
register: res_tz
failed_when: false
- name: 4.Chrony NTP源配置
command: grep pool /etc/chrony.conf
register: res_ntp_pool
failed_when: false
- name: 4.Chrony服务状态
command: systemctl is-active chronyd
register: res_chrony
failed_when: false
- name: 5.校验THP状态
command: cat /sys/kernel/mm/transparent_hugepage/enabled
register: res_thp
failed_when: false
- name: 5.校验THP禁用服务开机自启
command: systemctl is-enabled disable-thp
register: res_thp_svc
failed_when: false
- name: 6.内核sysctl参数
command: sysctl vm.swappiness vm.overcommit_memory net.core.somaxconn net.ipv4.tcp_tw_reuse
register: res_sysctl
failed_when: false
- name: 7.fstab挂载参数
command: cat /etc/fstab
register: res_fstab
failed_when: false
- name: 8.IO调度
shell: cat /sys/block/sda/queue/scheduler
register: res_io_sched
args:
warn: false
failed_when: false
- name: 9.SSHD配置校验
shell: grep -E 'UseDNS|MaxSessions|MaxStartups' /etc/ssh/sshd_config; sshd -t
register: res_sshd
args:
warn: false
failed_when: false
- name: ========== 输出当前节点全部校验结果 ==========
debug:
msg:
- "===== SELinux ====="
- "运行状态: {{ res_selinux.stdout | default('无输出') }}"
- "配置文件: {{ res_selinux_cfg.stdout | default('无输出') }}"
- "===== Limits文件 ====="
- "{{ res_limits.stdout_lines | default(['无输出']) }}"
- "===== Firewalld ====="
- "运行状态: {{ res_fw_run.stdout | default('无输出') }} 开机自启: {{ res_fw_boot.stdout | default('无输出') }}"
- "===== 时间同步Chrony ====="
- "时区: {{ res_tz.stdout | default('无输出') }}"
- "NTP源: {{ res_ntp_pool.stdout | default('无输出') }}"
- "chronyd运行状态: {{ res_chrony.stdout | default('无输出') }}"
- "===== THP透明大页 ====="
- "THP当前状态: {{ res_thp.stdout | default('无输出') }}"
- "disable-thp开机自启: {{ res_thp_svc.stdout | default('无输出') }}"
- "===== 内核sysctl ====="
- "{{ res_sysctl.stdout_lines | default(['无输出']) }}"
- "===== /etc/fstab挂载 ====="
- "{{ res_fstab.stdout_lines | default(['无输出']) }}"
- "===== 磁盘IO调度器 ====="
- "{{ res_io_sched.stdout | default('无输出') }}"
- "===== SSHD配置 ====="
- "{{ res_sshd.stdout_lines | default(['无输出']) }}"
#执行校验脚本
ansible-playbook check_os_optimize.yml
2.5、给Haddop集群中服务器新增hadoop用户
bash
#给Haddop集群中服务器新增hadoop用户
cd /data/ansible-script
vi add_user.yml
#【add_user.yml】文件的完整内容
- name: 创建hadoop集群运行用户
hosts: hadoop
remote_user: root
gather_facts: true
vars:
# 集群运行用户统一配置
run_user: hadoop
run_uid: 10000
run_gid: 10000
user_home: "/home/{{ run_user }}"
user_shell: /bin/bash
# 适配不同系统sudo用户组
sudo_group: "{{ 'wheel' if ansible_os_family == 'RedHat' else 'sudo' }}"
tasks:
# 1. 创建固定GID用户组
- name: 创建{{ run_user }}用户组 gid={{ run_gid }}
group:
name: "{{ run_user }}"
gid: "{{ run_gid }}"
state: present
# 2. 创建统一UID运行用户（删除不支持的参数）
- name: 创建{{ run_user }}集群用户 uid={{ run_uid }}
user:
name: "{{ run_user }}"
uid: "{{ run_uid }}"
group: "{{ run_user }}"
home: "{{ user_home }}"
shell: "{{ user_shell }}"
create_home: true
groups: "{{ sudo_group }}"
state: present
# 单独设置家目录权限 0700（替代home_mode）
- name: 设置hadoop家目录权限0700
file:
path: "{{ user_home }}"
mode: '0700'
owner: "{{ run_user }}"
group: "{{ run_user }}"
state: directory
# 3. chage 设置密码永不过期（替代password_expire/password_min_age/password_max_age）
# -m 0 最小修改间隔0天；-M -1 永不过期；-I -1 密码过期后不锁定
- name: 设置hadoop用户密码永不过期
shell: chage -m 0 -M -1 -I -1 {{ run_user }}
args:
executable: /bin/bash
# 4. sudo免密配置
- name: 新增sudoers分片文件，配置hadoop免密sudo
copy:
dest: "/etc/sudoers.d/{{ run_user }}"
content: |
{{ run_user }} ALL=(ALL) NOPASSWD: ALL
mode: '0440'
owner: root
group: root
validate: visudo -cf %s
#执行脚本
ansible-playbook add_user.yml
#验证
vi check_add_user.sh
#check_add_user.sh文件的完整内容
#!/bin/bash
echo "===== 1.用户UID/GID ====="
ansible hadoop -m shell -a "id hadoop"
echo -e "\n===== 2.家目录权限 ====="
ansible hadoop -m shell -a "ls -ld /home/hadoop"
echo -e "\n===== 3.密码过期策略 ====="
ansible hadoop -m shell -a "chage -l hadoop"
echo -e "\n===== 4./etc/passwd shell ====="
ansible hadoop -m shell -a "grep hadoop /etc/passwd"
echo -e "\n===== 5.sudoers配置 ====="
ansible hadoop -m shell -a "cat /etc/sudoers.d/hadoop; ls -l /etc/sudoers.d/hadoop"
echo -e "\n===== 6.免密sudo测试 ====="
ansible hadoop -m shell -a "su - hadoop -c 'sudo hostname'"
#执行验证文件验证
chmod +x check_add_user.sh
./check_add_user.sh2.6、给Hadoop集群中服务器配置JDK环境
bash
#给Hadoop集群中服务器配置JDK环境【推荐使用openjdk11】
#1-存放openjdk的压缩包
mkdir -p /etc/ansible/roles/files
cd /etc/ansible/roles/files/
#2-创建给服务器配置jdk环境的脚本
cd /data/ansible-script
vi set_openjdk.yml
#【set_openjdk.yml】文件的完整内容
- name: "给hadoop集群服务器配置OpenJDK21环境"
hosts: hadoop
remote_user: root
gather_facts: true
vars:
username: hadoop
openjdkname: openjdk2109010
jdk_local_file: "/etc/ansible/roles/files/{{ openjdkname }}.tar.xz"
jdkdir: /opt/openjdk
jdk_real_path: "{{ jdkdir }}/{{ openjdkname }}"
jdk_link_path: "{{ jdkdir }}/default"
jdk_global_env_file: /etc/profile.d/jdk21.sh
pre_tasks:
# 前置校验：本地JDK压缩包必须存在
- name: 校验本地JDK压缩包文件存在
delegate_to: localhost
stat:
path: "{{ jdk_local_file }}"
checksum_algorithm: sha256
register: jdk_local_stat
failed_when: not jdk_local_stat.stat.exists
tags: [always]
# 前置安装rsync、xz 同步和解压依赖
- name: 安装rsync & xz依赖(RedHat系)
yum:
name:
- rsync
- xz
state: present
when: ansible_os_family == "RedHat"
- name: 安装rsync & xz依赖(Debian/Ubuntu系)
apt:
name:
- rsync
- xz-utils
state: present
update_cache: yes
when: ansible_os_family == "Debian"
tasks:
# 1. 创建openjdk根目录
- name: 创建openjdk根目录 {{jdkdir}}
file:
path: "{{ jdkdir }}"
state: directory
mode: '0755'
owner: root
group: root
# 2. rsync同步JDK压缩包到远程
- name: rsync同步hadoop压缩包到远程
ansible.posix.synchronize:
src: "{{ jdk_local_file }}"
dest: "{{ jdkdir }}/"
archive: yes
compress: yes
rsync_opts:
- "--progress"
- "--stats"
become: no
# 3. 解压tar.xz，已解压则跳过
- name: 解压OpenJDK压缩包
unarchive:
src: "{{ jdkdir }}/{{ openjdkname }}.tar.xz"
dest: "{{ jdkdir }}"
remote_src: yes
creates: "{{ jdk_real_path }}"
mode: '0755'
owner: root
group: root
# 4. 重建JDK软链接（强制覆盖旧链接）
- name: 创建全局JDK软链接 default
file:
src: "{{ jdk_real_path }}"
dest: "{{ jdk_link_path }}"
state: link
force: yes
# 5. 全局系统环境变量 /etc/profile.d/jdk21.sh 所有用户生效
- name: 写入全局JDK21环境变量
copy:
content: |
export JAVA_HOME={{ jdk_link_path }}
export PATH=$JAVA_HOME/bin:$PATH
dest: "{{ jdk_global_env_file }}"
mode: '0644'
owner: root
group: root
# 6. hadoop用户家目录环境变量 .bashrc .bash_profile
- name: 写入hadoop用户专属JAVA环境
lineinfile:
path: /home/{{ username }}/{{ shell_file }}
regexp: '^export JAVA_HOME='
line: export JAVA_HOME={{ jdk_link_path }}
state: present
create: yes
mode: '0644'
owner: "{{ username }}"
group: "{{ username }}"
loop: ['.bash_profile', '.bashrc']
loop_control:
loop_var: shell_file
- name: 追加hadoop用户PATH环境
lineinfile:
path: /home/{{ username }}/{{ shell_file }}
regexp: '^export PATH=\$JAVA_HOME/bin:'
line: export PATH=$JAVA_HOME/bin:$PATH
state: present
create: yes
mode: '0644'
owner: "{{ username }}"
group: "{{ username }}"
loop: ['.bash_profile', '.bashrc']
loop_control:
loop_var: shell_file
# 7. 清理远程压缩包（可选，注释掉则保留）
- name: 删除远程JDK压缩包释放磁盘
file:
path: "{{ jdkdir }}/{{ openjdkname }}.tar.xz"
state: absent
#执行【试运行测试：ansible-playbook set_openjdk.yml --check】
ansible-playbook set_openjdk.yml
#验证【可在每台服务器上执行】
# 1. 全局环境验证
source /etc/profile
echo $JAVA_HOME
java -version
javac -version
# 2. 切换hadoop用户验证
su - hadoop
echo $JAVA_HOME
java -version
2.7、给Hadoop集群中服务器配置ZooKeeper
bash
#给Hadoop集群中服务器配置ZooKeeper
#1-先配置需要部署ZooKeeper的组【只能执行一次】
cd /etc/ansible/
cat>>hosts<<'EOF'
[zookeeper]
192.168.1.70 hostname=slave001 myid=1
192.168.1.151 hostname=slave002 myid=2
192.168.1.169 hostname=slave003 myid=3
EOF
#2-创建zookeeper的配置模板
mkdir -p /etc/ansible/roles/templates
cd /etc/ansible/roles/templates
cat>zoo.cfg.j2<<EOF
#zookeeper的核心心跳基准时间（单位是：毫秒ms）
tickTime=2000
#Follower 节点初始化连接、同步全量快照的最大允许时间(20 * tickTime = 20 * 2000 = 40秒)
initLimit=20
#集群正常运行时，Follower 和 Leader 之间数据同步最大延迟容忍时间(10 * tickTime = 10 * 2000 = 20秒)
syncLimit=10
dataDir={{ bigdatadir }}/zookeeper/current/data
dataLogDir={{ bigdatadir }}/zookeeper/current/dataLogDir
clientPort=2181
#设置 true：集群选举、同步端口监听本机所有网卡 IP（0.0.0.0）【默认 false：ZK 集群内部通信（2888/3888）只监听主机 hostname 绑定的 IP】
quorumListenOnAllIPs=false
# 自动循环zookeeper主机组生成集群配置，无需手动定义zk1_hostname等变量
{% for ip in groups['zookeeper'] %}
{% set zk_host = hostvars[ip] %}
server.{{ zk_host.myid }}={{ zk_host.hostname }}:2888:3888
{% endfor %}
#动清理周期，单位小时，每 12 小时执行一次清理；0 代表关闭自动清理
autopurge.snapRetainCount=3
#每次清理保留最近 3 份快照 + 对应事务日志，更早的数据自动删除，防止磁盘占满
autopurge.purgeInterval=12
EOF
#3-下载zookeeper的二进制文件到指定目录下
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.9.5/apache-zookeeper-3.9.5-bin.tar.gz -c -P /etc/ansible/roles/files/
#4-给Hadoop集群中的服务器都部署zookeeper
cd /data/ansible-script/
vi set_zookeeper.yml
#【vi set_zookeeper.yml】文件的完整内容
- name: "给hadoop集群的DataDode服务器安装zookeeper"
hosts: zookeeper
remote_user: root
gather_facts: true
vars:
zookeeper_tar: apache-zookeeper-3.9.5-bin.tar.gz
zk_version_dir: apache-zookeeper-3.9.5-bin
bigdatadir: /data/bigdata
role_path: /etc/ansible/roles
tasks:
- name: 1-创建大数据根目录
file:
path: "{{ bigdatadir }}"
mode: '0755'
state: directory
owner: root
group: root
- name: 1-1 创建zookeeper目录
file:
path: "{{ bigdatadir }}/zookeeper"
state: directory
owner: hadoop
group: hadoop
mode: '0755'
- name: 2-解压zookeeper的二进制安装包，且归属于hadoop用户
unarchive:
src: "{{ role_path }}/files/{{ zookeeper_tar }}"
dest: "{{ bigdatadir }}"
owner: hadoop
group: hadoop
creates: "{{ bigdatadir }}/{{ zk_version_dir }}"
- name: 3-创建软链接 current 统一访问入口
file:
src: "{{ bigdatadir }}/{{ zk_version_dir }}"
path: "{{ bigdatadir }}/zookeeper/current"
state: link
owner: hadoop
group: hadoop
- name: 4-推送zoo.cfg配置模板
template:
src: "{{ role_path }}/templates/zoo.cfg.j2"
dest: "{{ bigdatadir }}/zookeeper/current/conf/zoo.cfg"
owner: hadoop
group: hadoop
mode: '0644'
- name: 5-循环创建data、dataLogDir目录
file:
path: "{{ bigdatadir }}/zookeeper/current/{{ item }}"
mode: '0755'
state: directory
owner: hadoop
group: hadoop
loop:
- dataLogDir
- data
- name: 6-写入myid文件
copy:
content: "{{ myid }}\n"
dest: "{{ bigdatadir }}/zookeeper/current/data/myid"
owner: hadoop
group: hadoop
mode: '0644'
- name: 7-全局目录授权给hadoop用户
file:
path: "{{ bigdatadir }}/zookeeper"
owner: hadoop
group: hadoop
recurse: yes
- name: 8-启动zookeeper服务
shell: su -l hadoop -c "{{ bigdatadir }}/zookeeper/current/bin/zkServer.sh start"
args:
executable: /bin/bash
tags: start_zk
register: zk_start_result
changed_when: "'STARTED' in zk_start_result.stdout"
#执行
ansible-playbook set_zookeeper.yml
#验证【在DataNode节点（slave001、slave002、slave003）执行】
ps -ef | grep zookeeper
netstat -antlp
