大数据Hadoop运维应用实践——双NameNode高可用Hadoop集群架构（上）

大数据Hadoop运维应用实践------Hadoop大数据平台基础与架构（hadoop伪分布式的安装部署）文章浏览阅读88次。本文主要是关于hadoop的介绍与伪分布式安装部署操作实践教程；《1》hadoop的介绍包含（hadoop是什么？有啥用？优缺点及其适用与不适用场景；还有hadoop的生态圈解析）；《2》hadoop伪分布式的安装部署则包含（hadoop发行版的选型与建议、伪分布式的安装规划与准备、正式安装部署并启动）内容；最后对安装部署好的伪分布式hadoop进行了存储操作测试与分布式计算测试；是个人进行构建简单的开发环境或用于简单的功能测试的指导教程。https://coffeemilk.blog.csdn.net/article/details/162756251

Linux中安装部署Hadoop集群的保姆级安装配置教程文章浏览阅读331次。本文详细介绍了Hadoop 3.4.2集群的安装部署保姆级教程。主要内容包括：1) 规划3节点集群架构(1主2从)；2) 系统配置(主机名修改、免密登录、防火墙设置)；3) Java环境安装(OpenJDK21)；4) Hadoop安装与核心配置文件修改(core-site.xml等)；5) 集群启动与管理。文章提供了完整的操作命令和配置示例，特别强调了JDK版本兼容性、SSH端口设置等关键注意事项，并附带了常见问题解决方法。该指南适合想要搭建Hadoop生产环境的技术人员参考。_linux安装hadoophttps://blog.csdn.net/xiaochenxihua/article/details/154075870

一、双NameNode高可用Hadoop集群

1.1、双NameNode高可用Hadoop集群架构

双NameNode高可用Hadoop集群其实是主备模式（当主节点故障时，就自动将备节点启用作为主节点，这个监控切换是由ZooKeeper集群控制的），控制原理如下：

《1》主备NameNode节点启动时会连接到ZooKeeper集群并创建一个自身的状态文件，并且会每隔几秒(如6秒)就在ZooKeeper集群重写自身的状态文件，表示自己正常工作；

《2》当主NameNode节点故障时，就不会重写ZooKeeper集群中自身的状态文件了（如经过18秒还没有更新自身状态文件）则ZooKeeper就会通知备NameNode节点的FailOverController接管切换【目前完成了状态的监控与角色的切换】。

《3》JournalNode集群是用于同步主备节点的元数据，保证主备节点的元数据一致（即：NameNode主节点写元数据的时候不仅在自己上写，同时还给JounalNode集群写一份；然后NameNode的备用节点会定期从JounalNode拉取元数据）。

1.2、双NameNode高可用Hadoop集群规划

本文部署Hadoop集群的服务器统一采用AlmaLinux9.3版本。

角色/主机名	namenodemaster 192.168.1.120	yarnserver 192.168.1.6	slave001 192.168.1.70	slave002 192.168.1.151	slave003 192.168.1.169
NameNode	✔	✔	❌	❌	❌
DataNode	❌	❌	✔	✔	✔
JournalNode	❌	❌	✔	✔	✔
ZooKeeper	❌	❌	✔	✔	✔
ZKFC	✔	✔	❌	❌	❌
ResourceManager	❌	✔	❌	❌	❌
NodeManager	❌	❌	✔	✔	✔
JobHistoryServer	❌	✔	❌	❌	❌

软件名称	软件版本
JDK	openjdk2109010【建议使用openjdk11】
ZooKeeper	zookeeper3.9.5
Hadoop	hadoop-3.5.0

二、双NameNode高可用Hadoop集群环境准备

DevOps自动化运维实践_ansible-playbook的使用https://blog.csdn.net/xiaochenxihua/article/details/159888140

2.1、给Hadoop集群的所有服务器做免密信任

实现Linux的ssh免密登录实操保姆级教程https://blog.csdn.net/xiaochenxihua/article/details/152375722

#一、在Hadoop集群外的服务器(如：192.168.1.119)安装Ansible
#1-先安装第三方的epel源
yum install epel-release -y
#2-直接安装Ansible（由于Ansible依赖Pyhont环境，且是红帽系系统的一个组成部分，因此推荐用yum命令安装）
yum install ansible -y

#二、在Ansible中配置需要操作的Hadoop集群服务器
vi /etc/ansible
cat>>/etc/ansible/hosts<<'EOF'
[hadoop]
192.168.1.120 hostname=namenodemaster
192.168.1.6   hostname=yarnserver
192.168.1.70  hostname=slave001
192.168.1.151 hostname=slave002
192.168.1.169 hostname=slave003
EOF

# 查看文件末尾是否新增内容
tail /etc/ansible/hosts

#编写ssh免密登录脚本实现本机免密登录批量操作的服务器
#1-创建ssh免密登录批量部署脚本
mkdir -p /data/ansible-script
cd /data/ansible-script
vi ssh.yml

#2-ssh.yml（ssh免密登录）的完整内容
- hosts: hadoop
  gather_facts: no
  vars:
    ssh_key_type: ed25519
    ssh_key_path: /root/.ssh/id_ed25519

  tasks:
    - name: 关闭 SSH 主机密钥检查（避免交互 yes/no）
      lineinfile:
        path: /etc/ssh/ssh_config
        regexp: '.*StrictHostKeyChecking.*'
        line: 'StrictHostKeyChecking no'

    - name: 确保远程 .ssh 目录存在（权限 0700）
      file:
        path: /root/.ssh
        state: directory
        mode: '0700'

    - name: 本地生成 ed25519 密钥（已存在则跳过）
      delegate_to: localhost
      become: true
      run_once: true
      openssh_keypair:
        path: "{{ ssh_key_path }}"
        type: "{{ ssh_key_type }}"
        passphrase: ""
        state: present

    - name: 读取本地公钥内容
      local_action:
        module: slurp
        src: "{{ ssh_key_path }}.pub"
      register: ssh_key_slurp

    - name: 解码公钥
      set_fact:
        ssh_pub_key: "{{ ssh_key_slurp.content | b64decode | trim }}"

    - name: 推送公钥到目标主机（官方模块：安全、追加、不覆盖）
      authorized_key:
        user: root
        key: "{{ ssh_pub_key }}"
        state: present
        exclusive: no   # 不覆盖原有公钥（多密钥共存）
      tags:
        - install_ssh
        
#3-执行脚本
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ssh.yml -k

#4-验证本机免密登录Hadoop集群是否全部成功
ansible hadoop -m ping
#或者直接单独访问Hadoop集群的每台服务器（不用输入密码则成功）
ssh 192.168.1.120
ssh 192.168.1.6
ssh 192.168.1.70
ssh 192.168.1.151
ssh 192.168.1.169

配置项	控制主体	生效范围
/etc/ssh/ssh_config StrictHostKeyChecking no	系统 ssh 命令	所有手动 ssh、scp、sftp，不控制 ansible 内置校验
ANSIBLE_HOST_KEY_CHECKING=False	Ansible 程序	Ansible 所有 ssh 连接（playbook、ad-hoc）专属开关【Ansible 在建立 SSH 通道前，自己会做主机指纹校验，这套逻辑独立于系统 ssh 配置，不受ssh_config里StrictHostKeyChecking控制】

为什么已经修改了/etc/ssh/ssh_config的配置，连接还是需要验证
Ansible 底层调用 ssh，但有两层校验： 1. Ansible 自身先读取known_hosts做指纹校验（由ANSIBLE_HOST_KEY_CHECKING控制） 2. 再调用系统 ssh 客户端，此时才会读取/etc/ssh/ssh_config /root/.ssh/config
三种免交互方案： 方案 1：临时环境变量 ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ssh.yml -k ✅方案 2：写入 ansible.cfg 永久生效（推荐，不用每次输变量） #在 ansible.cfg 添加 [defaults] host_key_checking = False 方案 3：playbook 全局关闭 ansible 主机校验（无需外部变量） #在 play 顶部加environment，不用手动传环境变量 - hosts: hadoop gather_facts: no environment: ANSIBLE_HOST_KEY_CHECKING: "False" # ansible自身关闭指纹校验 vars: ssh_key_type: ed25519 ssh_key_path: /root/.ssh/id_ed25519 tasks: # 你的原有任务不变

2.2、给Hadoop集群的服务器修改为对应的主机名称

#给Hadoop集群的服务器修改为对应的主机名称
cd /data/ansible-script/
vi set_hostname.yml
#set_hostname.yml文件的完整内容
- hosts: hadoop
  remote_user: root
  gather_facts: false
  tasks:
    # 1. 永久修改系统主机名（写入/etc/hostname）
    - name: 永久设置主机名
      hostname:
        name: "{{ hostname }}"

    # 2. 立即生效当前会话主机名
    - name: 临时刷新主机名无需重启
      shell: hostname {{ hostname | quote }}

    # 3. 更新/etc/hosts本地解析，避免主机名无法解析
    - name: 更新/etc/hosts本地回环解析
      lineinfile:
        path: /etc/hosts
        regexp: '^127\.0\.0\.1'
        line: "127.0.0.1   {{ hostname }} localhost localhost.localdomain"
        state: present


#执行修改主机名称的命令【由于已经设置了免密登录，因此不用添加参数-k进行手动输入密码】
ansible-playbook set_hostname.yml

2.3、给Hadoop集群中服务器配置主机名与IP的映射

要想实现主机名称与IP的映射有两种方法**【内部构建DNS服务器】（适合大型集群【几百上千个】）、【本地解析】** （直接在本地的hosts文件中添加主机名称与IP的映射关系）

#给Hadoop集群中服务器配置主机名与IP的映射【把Hadoop集群所有主机 IP + 主机名写入每台机器】
#1-创建hosts.j2模板
mkdir -p /etc/ansible/templates
cd /etc/ansible/templates
vi hosts.j2
#hosts.j2模板的完整内容
# 标准本地回环解析
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

# Hadoop集群节点IP与主机名映射
{% for host in groups['hadoop'] %}
{{ host }}    {{ hostvars[host].hostname }}
{% endfor %}



#2-实现服务器配置本地主机名与IP映射的脚本
cd /data/ansible-script/
vi hosts.yml
#hosts.yml文件完整内容
- name: 批量更新所有节点/etc/hosts（含本地回环+集群域名）
  hosts: hadoop
  remote_user: root
  tasks:
    - name: 推送hosts模板覆盖/etc/hosts
      template:
        src: /etc/ansible/templates/hosts.j2
        dest: /etc/hosts
        owner: root
        group: root
        mode: '0644'
        backup: yes  # 自动备份原有hosts文件，防止误操作



#3-执行脚本
ansible-playbook hosts.yml

2.4、给Hadoop集群中服务器做系统优化配置

#给Hadoop集群中服务器做系统优化配置
cd /data/ansible-script/
vi optimize_os_config.yml

#optimize_os-config.yml文件的完整内容
- name: 系统内核、安全、时间同步标准化优化（Hadoop集群专用）
  hosts: hadoop
  gather_facts: true
  become: true
  handlers:
    - name: restart_sshd
      ansible.builtin.service:
        name: sshd
        state: restarted

  tasks:
    # ===================== 1 SELinux 关闭 =====================
    - name: 临时关闭SELinux
      ansible.builtin.command: setenforce 0
      changed_when: true
      failed_when: false

    - name: 永久禁用SELinux配置
      ansible.builtin.lineinfile:
        path: /etc/selinux/config
        regexp: '^SELINUX=.*'
        line: 'SELINUX=disabled'
        backrefs: true
        backup: true

    - name: 校验SELinux状态
      command: getenforce
      register: selinux_stat
      failed_when: selinux_stat.stdout != 'Permissive' and selinux_stat.stdout != 'Disabled'

    # ===================== 2 全局文件/进程限制（统一单文件，消除冲突） =====================
    - name: 清理冲突limits主配置，统一由limits.d托管
      lineinfile:
        path: /etc/security/limits.conf
        regexp: '^\*.*nofile|^\*.*nproc'
        state: absent
        backup: true

    - name: limits.d 统一软硬限制（最终生效配置）
      copy:
        dest: /etc/security/limits.d/99-hadoop.conf
        mode: '0644'
        backup: true
        content: |
          * soft nproc 655360
          * hard nproc 655360
          * soft nofile 1048576
          * hard nofile 1048576
          root soft nproc unlimited
          root hard nproc unlimited

    # ===================== 3 防火墙完整清理 =====================
    - name: 停止并禁用firewalld
      service:
        name: firewalld
        state: stopped
        enabled: false
      ignore_errors: true

    - name: 清空所有iptables表规则
      shell: |
        iptables -t filter -F && iptables -t nat -F && iptables -t mangle -F
        iptables -X
      changed_when: true

    # ===================== 4 Chrony 时间同步 =====================
    - name: 安装chrony
      yum:
        name: chrony
        state: present

    - name: 替换chrony时间源，删除原有pool/server
      lineinfile:
        path: /etc/chrony.conf
        regexp: '^(pool|server) .*'
        state: absent
        backup: true

    - name: 写入阿里云NTP源
      lineinfile:
        path: /etc/chrony.conf
        line: 'pool ntp.aliyun.com iburst'
        state: present

    - name: 设置时区上海
      timezone:
        name: Asia/Shanghai

    - name: 启动chronyd开机自启
      systemd:
        name: chronyd
        state: started
        enabled: true

    - name: 强制同步阿里云NTP时间并写入硬件时钟
      shell: |
        chronyd -q 'pool ntp.aliyun.com iburst'
        hwclock -w
      register: time_sync_res
      changed_when: "'System clock wrong by' in time_sync_res.stdout"

    # ===================== 5 关闭透明大页THP =====================
    - name: 临时关闭THP
      shell: echo never > /sys/kernel/mm/transparent_hugepage/enabled && echo never > /sys/kernel/mm/transparent_hugepage/defrag
      changed_when: false
      ignore_errors: true

    - name: 永久禁用THP systemd服务
      copy:
        dest: /etc/systemd/system/disable-thp.service
        mode: '0644'
        content: |
          [Unit]
          Description=Disable Transparent Huge Pages Hadoop Java
          After=sysfs.target
          [Service]
          Type=oneshot
          ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled && echo never > /sys/kernel/mm/transparent_hugepage/defrag'
          [Install]
          WantedBy=multi-user.target

    - name: 重载systemd并启动THP禁用服务
      systemd:
        name: disable-thp
        daemon_reload: true
        enabled: true
        state: started

    - name: 校验THP已关闭
      command: cat /sys/kernel/mm/transparent_hugepage/enabled
      register: thp_check
      failed_when: "'never' not in thp_check.stdout"

    # ===================== 6 内核sysctl调优 =====================
    - name: Hadoop专用sysctl内核参数
      sysctl:
        name: "{{ item.name }}"
        value: "{{ item.value }}"
        sysctl_file: /etc/sysctl.d/99-hadoop.conf
        reload: true
      loop:
        - { name: vm.swappiness, value: 10 }
        - { name: vm.vfs_cache_pressure, value: 50 }
        - { name: vm.overcommit_memory, value: 1 }
        - { name: net.core.somaxconn, value: 65535 }
        - { name: net.core.netdev_max_backlog, value: 65535 }
        - { name: net.ipv4.tcp_syncookies, value: 1 }
        - { name: net.ipv4.tcp_tw_reuse, value: 1 }
        - { name: net.ipv4.tcp_fin_timeout, value: 30 }

    # ===================== 7 fstab 兼容UUID/LVM追加noatime =====================
    - name: fstab挂载自动添加noatime，兼容UUID/LVM/sd设备
      replace:
        path: /etc/fstab
        regexp: '(\s+)(defaults|relatime)(\s+0\s+0)$'
        replace: '\1\2,noatime\3'
        backup: true

    - name: 重载挂载配置
      command: mount -a
      changed_when: false

    # ===================== 8 IO调度自动适配所有磁盘 + 版本兼容 =====================
    - name: 自动获取所有数据块设备（排除sr0光驱）
      shell: ls /sys/block/ | grep -E 'sd|vd|nvme' | grep -v sr0
      register: disk_list
      changed_when: false

    - name: 临时设置mq-deadline调度
      shell: echo mq-deadline > /sys/block/{{ item }}/queue/scheduler
      loop: "{{ disk_list.stdout_lines }}"
      ignore_errors: true
      changed_when: false

    - name: RHEL8+ 使用grubby永久设置IO调度
      shell: grubby --update-kernel=ALL --args="elevator=mq-deadline"
      when: ansible_distribution_major_version | int >= 8
      ignore_errors: true
      changed_when: false

    # ===================== 9 SSH高并发优化 =====================
    - name: SSHD并发参数调优
      lineinfile:
        path: /etc/ssh/sshd_config
        regexp: '^#?{{ item.key }}'
        line: '{{ item.key }} {{ item.val }}'
        backrefs: true
      loop:
        - { key: 'UseDNS', val: 'no' }
        - { key: 'MaxSessions', val: '100' }
        - { key: 'MaxStartups', val: '1000:80:10000' }
      notify: restart_sshd

    # 前置校验sshd语法，避免重启失败断连
    - name: 校验sshd_config语法，防止重启断连
      command: sshd -t



#执行脚本
ansible-playbook optimize_os_config.yml



#验证（直接脚本批量检验）
cd /data/ansible-script
vi check_os_optimize.yml
#【check_os_optimize.yml】校验文件的完整内容
- name: Hadoop系统优化结果批量校验
  hosts: hadoop
  gather_facts: true
  become: true
  tasks:
    - name: 1.校验SELinux状态
      command: getenforce
      register: res_selinux
      failed_when: false

    - name: SELINUX配置文件
      command: grep SELINUX= /etc/selinux/config
      register: res_selinux_cfg
      failed_when: false

    - name: 2.校验limits配置
      command: cat /etc/security/limits.d/99-hadoop.conf
      register: res_limits
      failed_when: false

    - name: 3.防火墙运行状态
      command: systemctl is-active firewalld
      register: res_fw_run
      failed_when: false

    - name: 3.防火墙开机自启状态
      command: systemctl is-enabled firewalld
      register: res_fw_boot
      failed_when: false

    - name: 4.Chrony时区校验
      command: timedatectl show -p Timezone
      register: res_tz
      failed_when: false

    - name: 4.Chrony NTP源配置
      command: grep pool /etc/chrony.conf
      register: res_ntp_pool
      failed_when: false

    - name: 4.Chrony服务状态
      command: systemctl is-active chronyd
      register: res_chrony
      failed_when: false

    - name: 5.校验THP状态
      command: cat /sys/kernel/mm/transparent_hugepage/enabled
      register: res_thp
      failed_when: false

    - name: 5.校验THP禁用服务开机自启
      command: systemctl is-enabled disable-thp
      register: res_thp_svc
      failed_when: false

    - name: 6.内核sysctl参数
      command: sysctl vm.swappiness vm.overcommit_memory net.core.somaxconn net.ipv4.tcp_tw_reuse
      register: res_sysctl
      failed_when: false

    - name: 7.fstab挂载参数
      command: cat /etc/fstab
      register: res_fstab
      failed_when: false

    - name: 8.IO调度
      shell: cat /sys/block/sda/queue/scheduler
      register: res_io_sched
      args:
        warn: false
      failed_when: false

    - name: 9.SSHD配置校验
      shell: grep -E 'UseDNS|MaxSessions|MaxStartups' /etc/ssh/sshd_config; sshd -t
      register: res_sshd
      args:
        warn: false
      failed_when: false

    - name: ========== 输出当前节点全部校验结果 ==========
      debug:
        msg:
          - "===== SELinux ====="
          - "运行状态: {{ res_selinux.stdout | default('无输出') }}"
          - "配置文件: {{ res_selinux_cfg.stdout | default('无输出') }}"
          - "===== Limits文件 ====="
          - "{{ res_limits.stdout_lines | default(['无输出']) }}"
          - "===== Firewalld ====="
          - "运行状态: {{ res_fw_run.stdout | default('无输出') }} 开机自启: {{ res_fw_boot.stdout | default('无输出') }}"
          - "===== 时间同步Chrony ====="
          - "时区: {{ res_tz.stdout | default('无输出') }}"
          - "NTP源: {{ res_ntp_pool.stdout | default('无输出') }}"
          - "chronyd运行状态: {{ res_chrony.stdout | default('无输出') }}"
          - "===== THP透明大页 ====="
          - "THP当前状态: {{ res_thp.stdout | default('无输出') }}"
          - "disable-thp开机自启: {{ res_thp_svc.stdout | default('无输出') }}"
          - "===== 内核sysctl ====="
          - "{{ res_sysctl.stdout_lines | default(['无输出']) }}"
          - "===== /etc/fstab挂载 ====="
          - "{{ res_fstab.stdout_lines | default(['无输出']) }}"
          - "===== 磁盘IO调度器 ====="
          - "{{ res_io_sched.stdout | default('无输出') }}"
          - "===== SSHD配置 ====="
          - "{{ res_sshd.stdout_lines | default(['无输出']) }}"


#执行校验脚本
ansible-playbook check_os_optimize.yml

2.5、给Haddop集群中服务器新增hadoop用户

#给Haddop集群中服务器新增hadoop用户
cd /data/ansible-script
vi add_user.yml

#【add_user.yml】文件的完整内容
- name: 创建hadoop集群运行用户
  hosts: hadoop
  remote_user: root
  gather_facts: true
  vars:
    # 集群运行用户统一配置
    run_user: hadoop
    run_uid: 10000
    run_gid: 10000
    user_home: "/home/{{ run_user }}"
    user_shell: /bin/bash
    # 适配不同系统sudo用户组
    sudo_group: "{{ 'wheel' if ansible_os_family == 'RedHat' else 'sudo' }}"

  tasks:
    # 1. 创建固定GID用户组
    - name: 创建{{ run_user }}用户组 gid={{ run_gid }}
      group:
        name: "{{ run_user }}"
        gid: "{{ run_gid }}"
        state: present

    # 2. 创建统一UID运行用户（删除不支持的参数）
    - name: 创建{{ run_user }}集群用户 uid={{ run_uid }}
      user:
        name: "{{ run_user }}"
        uid: "{{ run_uid }}"
        group: "{{ run_user }}"
        home: "{{ user_home }}"
        shell: "{{ user_shell }}"
        create_home: true
        groups: "{{ sudo_group }}"
        state: present

    # 单独设置家目录权限 0700（替代home_mode）
    - name: 设置hadoop家目录权限0700
      file:
        path: "{{ user_home }}"
        mode: '0700'
        owner: "{{ run_user }}"
        group: "{{ run_user }}"
        state: directory

    # 3. chage 设置密码永不过期（替代password_expire/password_min_age/password_max_age）
    # -m 0 最小修改间隔0天；-M -1 永不过期；-I -1 密码过期后不锁定
    - name: 设置hadoop用户密码永不过期
      shell: chage -m 0 -M -1 -I -1 {{ run_user }}
      args:
        executable: /bin/bash

    # 4. sudo免密配置
    - name: 新增sudoers分片文件，配置hadoop免密sudo
      copy:
        dest: "/etc/sudoers.d/{{ run_user }}"
        content: |
          {{ run_user }} ALL=(ALL) NOPASSWD: ALL
        mode: '0440'
        owner: root
        group: root
        validate: visudo -cf %s


#执行脚本
ansible-playbook add_user.yml



#验证
vi check_add_user.sh
#check_add_user.sh文件的完整内容
#!/bin/bash
echo "===== 1.用户UID/GID ====="
ansible hadoop -m shell -a "id hadoop"
echo -e "\n===== 2.家目录权限 ====="
ansible hadoop -m shell -a "ls -ld /home/hadoop"
echo -e "\n===== 3.密码过期策略 ====="
ansible hadoop -m shell -a "chage -l hadoop"
echo -e "\n===== 4./etc/passwd shell ====="
ansible hadoop -m shell -a "grep hadoop /etc/passwd"
echo -e "\n===== 5.sudoers配置 ====="
ansible hadoop -m shell -a "cat /etc/sudoers.d/hadoop; ls -l /etc/sudoers.d/hadoop"
echo -e "\n===== 6.免密sudo测试 ====="
ansible hadoop -m shell -a "su - hadoop -c 'sudo hostname'"

#执行验证文件验证
chmod +x check_add_user.sh
 ./check_add_user.sh

2.6、给Hadoop集群中服务器配置JDK环境

#给Hadoop集群中服务器配置JDK环境【推荐使用openjdk11】
#1-存放openjdk的压缩包
mkdir -p /etc/ansible/roles/files
cd /etc/ansible/roles/files/

#2-创建给服务器配置jdk环境的脚本
cd /data/ansible-script
vi set_openjdk.yml
#【set_openjdk.yml】文件的完整内容
- name: "给hadoop集群服务器配置OpenJDK21环境"
  hosts: hadoop
  remote_user: root
  gather_facts: true
  vars:
    username: hadoop
    openjdkname: openjdk2109010
    jdk_local_file: "/etc/ansible/roles/files/{{ openjdkname }}.tar.xz"
    jdkdir: /opt/openjdk
    jdk_real_path: "{{ jdkdir }}/{{ openjdkname }}"
    jdk_link_path: "{{ jdkdir }}/default"
    jdk_global_env_file: /etc/profile.d/jdk21.sh

  pre_tasks:
    # 前置校验：本地JDK压缩包必须存在
    - name: 校验本地JDK压缩包文件存在
      delegate_to: localhost
      stat:
        path: "{{ jdk_local_file }}"
        checksum_algorithm: sha256
      register: jdk_local_stat
      failed_when: not jdk_local_stat.stat.exists
      tags: [always]

    # 前置安装rsync、xz 同步和解压依赖
    - name: 安装rsync & xz依赖(RedHat系)
      yum:
        name:
          - rsync
          - xz
        state: present
      when: ansible_os_family == "RedHat"

    - name: 安装rsync & xz依赖(Debian/Ubuntu系)
      apt:
        name:
          - rsync
          - xz-utils
        state: present
        update_cache: yes
      when: ansible_os_family == "Debian"

  tasks:
    # 1. 创建openjdk根目录
    - name: 创建openjdk根目录 {{jdkdir}}
      file:
        path: "{{ jdkdir }}"
        state: directory
        mode: '0755'
        owner: root
        group: root

    # 2. rsync同步JDK压缩包到远程
    - name: rsync同步hadoop压缩包到远程
      ansible.posix.synchronize:
        src: "{{ jdk_local_file }}"
        dest: "{{ jdkdir }}/"
        archive: yes
        compress: yes
        rsync_opts:
          - "--progress"
          - "--stats"
      become: no

    # 3. 解压tar.xz，已解压则跳过
    - name: 解压OpenJDK压缩包
      unarchive:
        src: "{{ jdkdir }}/{{ openjdkname }}.tar.xz"
        dest: "{{ jdkdir }}"
        remote_src: yes
        creates: "{{ jdk_real_path }}"
        mode: '0755'
        owner: root
        group: root

    # 4. 重建JDK软链接（强制覆盖旧链接）
    - name: 创建全局JDK软链接 default
      file:
        src: "{{ jdk_real_path }}"
        dest: "{{ jdk_link_path }}"
        state: link
        force: yes

    # 5. 全局系统环境变量 /etc/profile.d/jdk21.sh 所有用户生效
    - name: 写入全局JDK21环境变量
      copy:
        content: |
          export JAVA_HOME={{ jdk_link_path }}
          export PATH=$JAVA_HOME/bin:$PATH
        dest: "{{ jdk_global_env_file }}"
        mode: '0644'
        owner: root
        group: root

    # 6. hadoop用户家目录环境变量 .bashrc .bash_profile
    - name: 写入hadoop用户专属JAVA环境
      lineinfile:
        path: /home/{{ username }}/{{ shell_file }}
        regexp: '^export JAVA_HOME='
        line: export JAVA_HOME={{ jdk_link_path }}
        state: present
        create: yes
        mode: '0644'
        owner: "{{ username }}"
        group: "{{ username }}"
      loop: ['.bash_profile', '.bashrc']
      loop_control:
        loop_var: shell_file

    - name: 追加hadoop用户PATH环境
      lineinfile:
        path: /home/{{ username }}/{{ shell_file }}
        regexp: '^export PATH=\$JAVA_HOME/bin:'
        line: export PATH=$JAVA_HOME/bin:$PATH
        state: present
        create: yes
        mode: '0644'
        owner: "{{ username }}"
        group: "{{ username }}"
      loop: ['.bash_profile', '.bashrc']
      loop_control:
        loop_var: shell_file

    # 7. 清理远程压缩包（可选，注释掉则保留）
    - name: 删除远程JDK压缩包释放磁盘
      file:
        path: "{{ jdkdir }}/{{ openjdkname }}.tar.xz"
        state: absent


#执行【试运行测试：ansible-playbook set_openjdk.yml --check】
ansible-playbook set_openjdk.yml


#验证【可在每台服务器上执行】
# 1. 全局环境验证
source /etc/profile
echo $JAVA_HOME
java -version
javac -version

# 2. 切换hadoop用户验证
su - hadoop
echo $JAVA_HOME
java -version

2.7、给Hadoop集群中服务器配置ZooKeeper

#给Hadoop集群中服务器配置ZooKeeper
#1-先配置需要部署ZooKeeper的组【只能执行一次】
cd /etc/ansible/
cat>>hosts<<'EOF'
[zookeeper]
192.168.1.70  hostname=slave001 myid=1
192.168.1.151 hostname=slave002 myid=2
192.168.1.169 hostname=slave003 myid=3
EOF



#2-创建zookeeper的配置模板
mkdir -p /etc/ansible/roles/templates
cd /etc/ansible/roles/templates
cat>zoo.cfg.j2<<EOF
#zookeeper的核心心跳基准时间（单位是：毫秒ms）
tickTime=2000
#Follower 节点初始化连接、同步全量快照的最大允许时间(20 * tickTime = 20 * 2000 = 40秒)
initLimit=20
#集群正常运行时，Follower 和 Leader 之间数据同步最大延迟容忍时间(10 * tickTime = 10 * 2000 = 20秒)
syncLimit=10
dataDir={{ bigdatadir }}/zookeeper/current/data
dataLogDir={{ bigdatadir }}/zookeeper/current/dataLogDir
clientPort=2181
#设置 true：集群选举、同步端口监听本机所有网卡 IP（0.0.0.0）【默认 false：ZK 集群内部通信（2888/3888）只监听主机 hostname 绑定的 IP】
quorumListenOnAllIPs=false
# 自动循环zookeeper主机组生成集群配置，无需手动定义zk1_hostname等变量
{% for ip in groups['zookeeper'] %}
{% set zk_host = hostvars[ip] %}
server.{{ zk_host.myid }}={{ zk_host.hostname }}:2888:3888
{% endfor %}
#动清理周期，单位小时，每 12 小时执行一次清理；0 代表关闭自动清理
autopurge.snapRetainCount=3
#每次清理保留最近 3 份快照 + 对应事务日志，更早的数据自动删除，防止磁盘占满
autopurge.purgeInterval=12
EOF



#3-下载zookeeper的二进制文件到指定目录下
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.9.5/apache-zookeeper-3.9.5-bin.tar.gz -c -P /etc/ansible/roles/files/


#4-给Hadoop集群中的服务器都部署zookeeper
cd /data/ansible-script/
vi set_zookeeper.yml
#【vi set_zookeeper.yml】文件的完整内容
- name: "给hadoop集群的DataDode服务器安装zookeeper"
  hosts: zookeeper
  remote_user: root
  gather_facts: true
  vars:
    zookeeper_tar: apache-zookeeper-3.9.5-bin.tar.gz
    zk_version_dir: apache-zookeeper-3.9.5-bin
    bigdatadir: /data/bigdata
    role_path: /etc/ansible/roles

  tasks:
    - name: 1-创建大数据根目录
      file:
        path: "{{ bigdatadir }}"
        mode: '0755'
        state: directory
        owner: root
        group: root
    - name: 1-1 创建zookeeper目录
      file:
        path: "{{ bigdatadir }}/zookeeper"
        state: directory
        owner: hadoop
        group: hadoop
        mode: '0755'

    - name: 2-解压zookeeper的二进制安装包，且归属于hadoop用户
      unarchive:
        src: "{{ role_path }}/files/{{ zookeeper_tar }}"
        dest: "{{ bigdatadir }}"
        owner: hadoop
        group: hadoop
        creates: "{{ bigdatadir }}/{{ zk_version_dir }}"

    - name: 3-创建软链接 current 统一访问入口
      file:
        src: "{{ bigdatadir }}/{{ zk_version_dir }}"
        path: "{{ bigdatadir }}/zookeeper/current"
        state: link
        owner: hadoop
        group: hadoop

    - name: 4-推送zoo.cfg配置模板
      template:
        src: "{{ role_path }}/templates/zoo.cfg.j2"
        dest: "{{ bigdatadir }}/zookeeper/current/conf/zoo.cfg"
        owner: hadoop
        group: hadoop
        mode: '0644'
        
    - name: 5-循环创建data、dataLogDir目录
      file:
        path: "{{ bigdatadir }}/zookeeper/current/{{ item }}"
        mode: '0755'
        state: directory
        owner: hadoop
        group: hadoop
      loop:
        - dataLogDir
        - data


    - name: 6-写入myid文件
      copy:
        content: "{{ myid }}\n"
        dest: "{{ bigdatadir }}/zookeeper/current/data/myid"
        owner: hadoop
        group: hadoop
        mode: '0644'

    - name: 7-全局目录授权给hadoop用户
      file:
        path: "{{ bigdatadir }}/zookeeper"
        owner: hadoop
        group: hadoop
        recurse: yes

    - name: 8-启动zookeeper服务
      shell: su -l hadoop -c "{{ bigdatadir }}/zookeeper/current/bin/zkServer.sh start"
      args:
        executable: /bin/bash
      tags: start_zk
      register: zk_start_result
      changed_when: "'STARTED' in zk_start_result.stdout"



#执行
ansible-playbook set_zookeeper.yml

#验证【在DataNode节点（slave001、slave002、slave003）执行】
ps -ef | grep zookeeper
netstat -antlp