使用Ansible对京东云服务器初始化

使用Ansible对京东云服务器初始化

概要：由于公司业务扩展，需要在线上新增30台服务器，系统为Ubuntu22.04 EFI版本，这么多要是人肉那着实有点蓝瘦，所以使用了Ansible进行部署，当然仅仅是对系统简单的初始化，配置相关环境(文章也算对工作做个记录)。

1. 准备相关目录结构

╭─    ~/Desktop/WolfWorkFile/DevOps/Ansible/ansible_ops ························································································· ✔  at 13:55:16  ─╮
╰─ tree                                                                                                                               
├── ansible.cfg
├── hosts  # 主机列表
├── inventory
├── playbooks 
│   ├── JD_os_ubuntu2204_init.yaml  # 指定运行role的路径
├── roles
│   ├── JD_os_ubuntu2204_init
│   │   ├── default
│   │   ├── files
│   │   ├── handlers
│   │   ├── meta
│   │   ├── tasks # 对系统进行初始化
│   │   │   ├── hostname_set.yml # 自动对系统进行命名
│   │   │   ├── main.yml # 主入口，就是你要执行哪些tasks中的yml文件
│   │   │   ├── os_env.yml # 系统环境初始化(环境变量，部分系统参数)
│   │   │   ├── service_config.yml # 安装自定义的一些服务(systemd)，对应服务文件在templates中
│   │   │   ├── service_enable_start.yml # 启动自定义服务
│   │   │   └── software_install.yml # 系统初始化安装一些软件，包括jdk node_export等
│   │   ├── templates # 这里模板文件 定义了2个系统服务(systemd)
│   │   │   ├── node-exporter.service
│   │   │   └── process-exporter.service
│   │   └── vars
│   │       └── os_name_var.yml # 在tasks中使用的变量
└── Shell
    ├── Centos79_init.sh
    └── Python399_install.sh

2. 编辑host(请注意我已经使用了ssh-key所以不用输入账号密码)

# vi hosts
[all:vars]
ansible_ssh_user=root
[jdcloud] # 主机列表
116.x.x.x
116.x.x.x
...

3. 编辑vars(ansible用到的环境变量)

# vi roles/vars/os_name_var.yml
cloud: JD
zone: CN
application: Service
#application: DBService

4.编辑tasks任务

4.1 系统命名

# vi roles/JD_os_ubuntu2204_init/tasks/hostname_set.yml

- name: Get public IP
  #shell: curl -s ifconfig.me
  shell: curl ip.sb
  register: public_ip
  changed_when: false

- name: Set IP variable
  set_fact:
    ip_last_two: "{{ public_ip.stdout.split('.')[-2:] | join('') }}"

- name: Set hostname
  ansible.builtin.command:
    cmd: "hostnamectl set-hostname --static {{ cloud }}-{{ zone }}-{{ application }}-{{ ip_last_two }}"
  become: yes

4.2 系统环境初始化

# vi roles/JD_os_ubuntu2204_init/tasks/os_env.yml

- name: Backup sysctl.conf
  ansible.builtin.copy:
    src: /etc/sysctl.conf
    dest: /etc/sysctl.conf_bak
    remote_src: yes

- name: Update sysctl.conf
  ansible.builtin.lineinfile:
    path: /etc/sysctl.conf
    line: 'vm.max_map_count=262144'
    create: yes

- name: Set nproc and memlock limits
  ansible.builtin.lineinfile:
    path: /etc/security/limits.conf
    line: "{{ item }}"
  loop:
    - "* soft nproc 102400"
    - "* soft memlock unlimited"
    - "* hard memlock unlimited"

- name: Set sysctl parameters
  ansible.builtin.sysctl:
    name: "{{ item.name }}"
    value: "{{ item.value }}"
    state: present
    sysctl_set: yes
    reload: yes
  loop:
    - { name: 'net.core.somaxconn', value: '65535' }
    - { name: 'net.core.netdev_max_backlog', value: '65535' }
    - { name: 'net.ipv4.tcp_max_syn_backlog', value: '65535' }
    - { name: 'net.ipv4.ip_local_port_range', value: '1024 65535' }
    - { name: 'fs.file-max', value: '500000000' }
    - { name: 'fs.nr_open', value: '10000000' }

- name: Set nofile soft limit
  ansible.builtin.pam_limits:
    domain: '*'
    limit_type: soft
    limit_item: nofile
    value: '65536'

- name: Set nofile hard limit
  ansible.builtin.pam_limits:
    domain: '*'
    limit_type: hard
    limit_item: nofile
    value: '65536'

- name: Set JAVA_HOME environment variable
  ansible.builtin.lineinfile:
    path: /etc/profile
    line: 'export JAVA_HOME=/usr/local/jdk17'
    create: yes

- name: Set CLASSPATH environment variable
  ansible.builtin.lineinfile:
    path: /etc/profile
    line: 'export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar'
    create: yes

- name: Set PATH environment variable
  ansible.builtin.lineinfile:
    path: /etc/profile
    line: 'export PATH=$PATH:$JAVA_HOME/bin'
    create: yes

4.3 cp自定义系统systemd服务

# vi roles/JD_os_ubuntu2204_init/tasks/service_config.yml
# 服务的模板在templates中

- name: Copy node-exporter.service
  ansible.builtin.template:
    src: node-exporter.service
    dest: /usr/lib/systemd/system/node-exporter.service
    mode: '0644'

- name: Copy process-exporter.service
  ansible.builtin.template:
    src: process-exporter.service
    dest: /usr/lib/systemd/system/process-exporter.service
    mode: '0644'

4.4 启动自定义systemd服务

# vi roles/JD_os_ubuntu2204_init/tasks/service_enable_start.yml

- name: Start and enable services at boot
  ansible.builtin.systemd:
    name: "{{ item }}"
    state: started
    enabled: yes
  loop:
    - node-exporter
    - process-exporter

- name: Reboot the system
  ansible.builtin.reboot:

- name: Wait for system to become reachable again
  ansible.builtin.wait_for_connection:
    delay: 60
    timeout: 300

4.5 服务器软件安装

# vi roles/JD_os_ubuntu2204_init/tasks/software_install.yml

- name: Update APT package cache
  ansible.builtin.apt:
    update_cache: yes

- name: Install required packages
  ansible.builtin.apt:
    name:
      - wget
      - unzip
      - tree
      - htop
      - btop
      - atop
      - nload
      - net-tools
      - bash-completion
      - tcpdump
      - mtr
      - iperf3
      - lrzsz
      - expect
      - nethogs
      - psmisc
      - lsof
      - iotop
      - iftop
    state: present

- name: Create directory
  ansible.builtin.file:
    path: /root/software
    state: directory

- name: Download software
  ansible.builtin.get_url:
    url: http://xxx:9999/work/os_init/os_ubuntu2204_init_software.tar.gz
    dest: /root/software/

- name: Extract software
  ansible.builtin.unarchive:
    src: /root/software/os_ubuntu2204_init_software.tar.gz
    dest: /usr/local/
    remote_src: yes

4.6 定义tasks执行顺序main.yml

# vi roles/JD_os_ubuntu2204_init/tasks/main.yml

- include_vars: vars/os_name_var.yml
# - include_vars: vars/os_env.yml
# - include_vars: vars/consul_env.yml

- include_tasks: tasks/hostname_set.yml
- include_tasks: tasks/software_install.yml
- include_tasks: tasks/os_env.yml
- include_tasks: tasks/service_config.yml
- include_tasks: tasks/service_enable_start.yml

5.定义systemd启动模板

# vi roles/JD_os_ubuntu2204_init/templates/node-exporter.service # prometheus 主机监控(linux)node-export启动
[Unit]
Description=Node Exporter
Documentation=https://github.com/prometheus/node_exporter
After=network.target

[Service]
User=root
ExecStart=/usr/local/node-exporter-1.8.2/node_exporter
Restart=on-failure

[Install]
WantedBy=multi-user.target

-----------------------------

# vi roles/JD_os_ubuntu2204_init/templates/process-exporter.service # prometheus 系统进程(linux)process-export启动

[Unit]
Description=Prometheus exporter for processors metrics, written in Go with pluggable metric collectors.
Documentation=https://github.com/ncabatoff/process-exporter
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/usr/local/process-exporter-0.8.4
ExecStart=/usr/local/process-exporter-0.8.4/process-exporter -config.path=/usr/local/process-exporter-0.8.4/process_name.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.target

6.软件包结构

╭─    ~/Dev/os_ubuntu2204_init_software ··········· ✔  at 14:24:36  ─╮
╰─ tree -L 1                                                                 ─╯
.
├── jdk17
├── node-exporter-1.8.2
└── process-exporter-0.8.4

7.编辑并执行Playbook

# 编辑Playbook
vi playbooks/JD_os_ubuntu2204_init.yaml

- hosts: jdcloud  # 这里是读取hosts文件中的jdcloud中的主机列表
  roles:
    - ../roles/JD_os_ubuntu2204_init # 执行角色为roles中JD_os_ubuntu2204_init角色

# 注意执行路径要在根文件夹下面
╭─    ~/Desktop/WolfWorkFile/DevOps/Ansible/ansible_ops ························································································· ✔  at 13:55:25  ─╮
╰─ ls                                                                                                                                                                      ─╯
ansible.cfg hosts       inventory   playbooks   roles       Shell

# 执行playbook
ansible-playbook playbooks/JD_os_ubuntu2204_init.yaml -i hosts

null

image-20250728172619057