Ansible自动化部署kubernetes集群

theo.wu2024-09-15 16:58

机器环境介绍

1.1. 机器信息介绍

|-----------------|--------------|------------------------------------------------------------------------------------------|---------|------------|
| IP | hostname | application | CPU | Memory |
| 192.168.204.129 | k8s-master01 | etcd，kube-apiserver，kube-controller-manager，kube-scheduler,kubelet,kube-proxy,containerd | 2C | 4G |
| 192.168.204.130 | k8s-worker01 | etcd，kubelet，kube-proxy，containerd | 2C | 4G |
| 192.168.204.131 | k8s-worker02 | etcd，kubelet，kube-proxy，containerd | 2C | 4G |

1.2. 规划IP地址介绍

在Kubernetes中CNI网络插件采用Calico,划分三个网段

|-----------|---------------|
| 网段信息 | 配置 |
| Pod网段 | 172.16.0.0/16 |
| Service网段 | 10.96.0.0/16 |

安装的kubernetets版本为1.28.5，Calico版本为3.26.4，容器运行环境为containerd

如果需要其他版本kuberneres，需要修改下面的脚本

  • 修改kubernetes源里面的版本
  • 修改安装master和worker节点里面定义的版本变量值

如下需要使用其他版本的CNI插件或者不同版本的calico插件，需要对网络插件部分脚本进行修改

安装配置ansible

2.1. ansible软件部署

  • 安装ansible软件

    apt update && apt install ansible -y

  • 配置ansible配置

    mkdir /etc/ansible/ && touch /etc/ansible/hosts

  • 配置/etc/ansible/hosts文件

    [master]
    192.168.204.129

    [worker]
    192.168.204.130
    192.168.204.131

  • 配置免密登录, 此过程中不要输入密码

    ssh-keygen -t rsa

  • 分发免密登录

    ssh-copy-id [email protected]
    ssh-copy-id [email protected]
    ssh-copy-id [email protected]

  • 配置hosts

    cat >> /etc/hosts <<EOF

2.2. 测试ansible连接性

  • 编写测试脚本

    cat >test_nodes.yml <<EOF

    • name: test nodes
      hosts:
      master
      worker
      tasks:
      • name: Ping nodes
        ping:
        EOF

  • 执行ansible测试

  • ansible-playbook test_node.yml

配置kubernetes脚本

3.1. 编写的kubernetes 脚本

  • 编写的install-kubernetes.yml文件内容如下
bash 复制代码 
---
- name: Performance Basic Config
  hosts: 
  	master
    worker
  become: yes
  tasks:
    - name: Check if fstab contains swap
      shell: grep -q "swap" /etc/fstab
      register: fstab_contains_swap

    - name: Temp Disable swap
      command: swapoff -a
      when: fstab_contains_swap.rc == 0

    - name: Permanent Disable swap
      shell: sed -i 's/.*swap.*/#&/g' /etc/fstab
      when: fstab_contains_swap.rc == 0

    - name: Disable Swap unit-files
      shell: |
        swap_units=$(systemctl list-unit-files | grep swap | awk '{print $1}')
        for unit in $swap_units; do
          systemctl mask $unit
        done

    - name: Stop UFW service
      service:
        name: ufw
        state: stopped

    - name: Disable UFW at boot
      service:
        name: ufw
        enabled: no

    - name: Set timezone
      shell: TZ='Asia/Shanghai'; export TZ

    - name: Set timezone permanently
      shell: |
        cat >> /etc/profile << EOF
        TZ='Asia/Shanghai'; export TZ
        EOF

    - name: Create .hushlogin file in $HOME
      file:
        path: "{{ ansible_env.HOME }}/.hushlogin"
        state: touch

    - name: Install required packages
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
          - apt-transport-https
          - ca-certificates
          - curl
          - gnupg
          - lsb-release

    - name: Add Aliyun Docker GPG key
      shell: curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add

    - name: Add Aliyun Docker repository
      shell: echo "deb [arch=amd64 signed-by=/etc/apt/trusted.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker-ce.list

    - name: Add Aliyun Kubernetes GPG key
      shell: curl -fsSL https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

    - name: Add Aliyun Kubernetes repository
      shell: echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list

    - name: Set apt sources to use USTC mirrors
      shell: sed -i 's#cn.archive.ubuntu.com#mirrors.aliyun.com#g' /etc/apt/sources.list

    - name: Update apt cache
      apt:
        update_cache: yes

    - name: Load br_netfilter on start
      shell: echo "modprobe br_netfilter" >> /etc/profile

    - name: Load br_netfilter
      shell: modprobe br_netfilter

    - name: Update sysctl settings
      sysctl:
        name: "{{ item.name }}"
        value: "{{ item.value }}"
        state: present
        reload: yes
      with_items:
        - { name: "net.bridge.bridge-nf-call-iptables", value: "1" }
        - { name: "net.bridge.bridge-nf-call-ip6tables", value: "1" }
        - { name: "net.ipv4.ip_forward", value: "1" }

    - name: Install IPVS
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
          - ipset
          - ipvsadm

    - name: Create ipvs modules
      file:
        name: /etc/modules-load.d/ipvs.modules
        mode: 0755
        state: touch

    - name: Write ipvs.modules file
      lineinfile:
        dest: /etc/modules-load.d/ipvs.modules
        line: "#!/bin/bash\nmodprobe -- ip_vs\nmodprobe -- ip_vs_rr\nmodprobe -- ip_vs_wrr\nmodprobe -- ip_vs_sh\nmodprobe -- nf_conntrack\nmodprobe -- overlay\nmodprobe -- br_netfilter"

    - name: Execute ipvs.modules script
      shell: sh /etc/modules-load.d/ipvs.modules

    - name: Install Containerd
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
          - containerd.io

    - name: Generate default containerd file
      shell: containerd config default > /etc/containerd/config.toml

    - name: Config sandbox image
      shell: sed -i 's#sandbox_image = "registry.k8s.io/pause:3.6"#sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"#g' /etc/containerd/config.toml

    - name: Modify Systemd Cgroup
      shell: sed -i 's#SystemdCgroup = false#SystemdCgroup = true#g' /etc/containerd/config.toml

    - name: Restart Containerd
      shell: systemctl restart containerd

    - name: Systemctl enable containerd
      shell: systemctl enable containerd

- name: Install Kubernetes Master
  hosts: master
  become: yes
  vars:
    kubernetes_version: "1.28.5"
    pod_network_cidr: "172.16.0.0/16"
    service_cidr: "10.96.0.0/16"
    image_repository: "registry.aliyuncs.com/google_containers"
    calico_version: "v3.26.4"
  tasks:
    - name: Install Master kubernetes packages
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
          - kubelet={{ kubernetes_version }}-1.1
          - kubeadm={{ kubernetes_version }}-1.1
          - kubectl={{ kubernetes_version }}-1.1

    - name: Initialize Kubernetes Master
      command: kubeadm init --kubernetes-version={{ kubernetes_version }} --pod-network-cidr={{ pod_network_cidr }} --service-cidr={{ service_cidr }} --image-repository={{ image_repository }}
      register: kubeadm_output
      changed_when: "'kubeadm join' in kubeadm_output.stdout"

    - name: Save join command
      copy:
        content: |
          {{ kubeadm_output.stdout_lines [-2] }}
          {{ kubeadm_output.stdout_lines [-1] }}
        dest: /root/kubeadm_join_master.sh
      when: kubeadm_output.changed

    - name: cope join master script
      shell: sed -i 's/"//g' /root/kubeadm_join_master.sh

    - name: copy kubernetes config
      shell: mkdir -p {{ ansible_env.HOME }}/.kube && cp -i /etc/kubernetes/admin.conf {{ ansible_env.HOME }}/.kube/config

    - name: enable kubectl
      command: systemctl enable kubelet

    - name: Create calico directory
      file:
        path: "{{ ansible_env.HOME }}/calico/{{ calico_version }}"
        state: directory

    - name: download calico tigera-operator.yaml
      command: wget https://ghproxy.net/https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version }}/manifests/tigera-operator.yaml -O {{ ansible_env.HOME }}/calico/{{ calico_version }}/tigera-operator.yaml

    - name: download calico custom-resources.yaml
      command: wget https://ghproxy.net/https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version }}/manifests/custom-resources.yaml -O {{ ansible_env.HOME }}/calico/{{ calico_version }}/custom-resources.yaml

    - name: set calico netwok range
      replace:
        path: "{{ ansible_env.HOME }}/calico/{{ calico_version }}/custom-resources.yaml"
        regexp: "blockSize: 26"
        replace: "blockSize: 24"

    - name: set calico ip pools
      replace:
        path: "{{ ansible_env.HOME }}/calico/{{ calico_version }}/custom-resources.yaml"
        regexp: "cidr: 192.168.0.0/16"
        replace: "cidr: {{ pod_network_cidr }}"

    - name: apply calico tigera-operator.yaml
      command: kubectl create -f {{ ansible_env.HOME }}/calico/{{ calico_version }}/tigera-operator.yaml

    - name: apply calico custom-resources.yaml
      command: kubectl create -f {{ ansible_env.HOME }}/calico/{{ calico_version }}/custom-resources.yaml

    - name: set crictl config
      command: crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock

- name: Install Kubernetes worker
  hosts: worker
  become: yes
  vars:
    kubernetes_version: "1.28.5"
  tasks:
    - name: Install worker kubernetes packages
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
          - kubelet={{ kubernetes_version }}-1.1
          - kubeadm={{ kubernetes_version }}-1.1

    - name: copy kubeadm join script to workers
      copy:
        src: /root/kubeadm_join_master.sh
        dest: /root/kubeadm_join_master.sh
        mode: 0755

    - name: worker join to cluster
      command: sh /root/kubeadm_join_master.sh

    - name: set crictl config
      command: crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock

    - name: enable kubectl
      command: systemctl enable kubelet

执行kubernetes脚本

ansible-playbook install-kubernetes.yml

  • 集群状态

kubectl get node -o wide

  • 集群pod状态

kubectl get pod -A

相关推荐
藥瓿亭3 小时前
K8S认证|CKS题库+答案| 6. 创建 Secret
运维·ubuntu·docker·云原生·容器·kubernetes·cks
孔令飞3 小时前
Kubernetes 节点自动伸缩（Cluster Autoscaler）原理与实践
ai·云原生·容器·golang·kubernetes
半导体守望者8 小时前
ADVANTEST R3764 66 R3765 67爱德万测试networki connection programming网络程序设计手册
经验分享·笔记·功能测试·自动化·制造
孔令飞10 小时前
Go 为何天生适合云原生？
ai·云原生·容器·golang·kubernetes
东方佑12 小时前
使用 Python 自动化 Word 文档样式复制与内容生成
python·自动化·word
藥瓿亭13 小时前
K8S认证|CKS题库+答案| 5.日志审计
linux·运维·docker·云原生·容器·kubernetes·cka
藥瓿锻14 小时前
2024 CKA题库+详尽解析| 15、备份还原Etcd
linux·运维·数据库·docker·容器·kubernetes·cka
会又不会14 小时前
JMeter-SSE响应数据自动化2.0
jmeter·自动化
新加坡内哥谈技术16 小时前
Meta计划借助AI实现广告创作全自动化
运维·人工智能·自动化
陈陈CHENCHEN1 天前
【Kubernetes】K8s 之 ETCD - 恢复备份
kubernetes
热门推荐
01【图像处理与机器视觉】XJTU期末考点02KGG转MP3工具|非KGM文件|解密音频03YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】04海康Visionmaster-常见问题排查方法-启动阶段05从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑06【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！07Coze扣子平台完整体验和实践（附国内和国际版对比）08DeepSeek各版本说明与优缺点分析09VMware虚拟机安装Win7专业版保姆级教程（附镜像包）1020分钟拿下字节Coze工作流搭建全部知识点，保姆级小白教学干货密集