建设一套企业 DevOps + 协同办公 + 可观测性 + 网络基础设施平台，并希望通过：
- Ansible Playbook：负责服务器初始化、软件部署、配置管理
- Docker Compose：负责应用容器编排
实现一键化部署
建议不要把所有服务堆在一台机器，而是拆成多个角色节点
最佳设计不是一个 site.yml 一次性全部安装，而是设计成：
模块化 Ansible + Docker Compose 平台
特点：
✅ 每个服务独立安装
✅ 每个服务独立卸载
✅ 每个服务独立升级
✅ 支持安装前依赖检查
✅ 支持多节点部署
✅ 支持生产环境 HA
✅ 支持 Docker Compose 服务
✅ 支持 Kubernetes/Helm 服务
✅ 支持统一配置管理
一、最终项目结构
enterprise-platform-ansible/
├── README.md
├── ansible.cfg
│
├── inventory/
│
│ ├── production/
│ │
│ │ ├── hosts.ini
│ │ │
│ │ └── group_vars/
│ │ ├── all.yml
│ │ ├── database.yml
│ │ ├── ldap.yml
│ │ ├── docker.yml
│ │ └── kubernetes.yml
│
│
├── playbooks/
│
│ ├── install.yml # 总入口
│ ├── uninstall.yml
│ ├── upgrade.yml
│
│
│ ├── common.yml
│ ├── docker.yml
│ ├── kubernetes.yml
│
│ ├── keepalived.yml
│ ├── haproxy.yml
│
│ ├── dns.yml
│ ├── proxy.yml
│
│ ├── ldap.yml
│ ├── keycloak.yml
│ ├── vault.yml
│
│ ├── gitlab.yml
│ ├── gerrit.yml
│ ├── jenkins.yml
│ ├── nexus.yml
│ ├── sonar.yml
│ ├── harbor.yml
│
│ ├── jira.yml
│ ├── confluence.yml
│ ├── nextcloud.yml
│ ├── collabora.yml
│ ├── onlyoffice.yml
│
│ ├── mail.yml
│ ├── ftp.yml
│ ├── openvpn.yml
│
│ ├── prometheus.yml
│ ├── alertmanager.yml
│ ├── grafana.yml
│ ├── loki.yml
│ ├── elk.yml
│ ├── thanos.yml
│
│ ├── argocd.yml
│ ├── velero.yml
│ └── restic.yml
│
│
├── roles/
│
│ ├── common/
│ ├── docker/
│ ├── kubernetes/
│ ├── keepalived/
│ ├── haproxy/
│ ├── nginx/
│
│ ├── dns/
│ ├── proxy/
│
│ ├── ldap/
│ ├── keycloak/
│ ├── vault/
│
│ ├── gitlab/
│ ├── gerrit/
│ ├── jenkins/
│ ├── nexus/
│ ├── sonar/
│ ├── harbor/
│
│ ├── jira/
│ ├── confluence/
│ ├── nextcloud/
│ ├── collabora/
│ ├── onlyoffice/
│
│ ├── mail/
│ ├── ftp/
│ ├── openvpn/
│
│ ├── prometheus/
│ ├── alertmanager/
│ ├── grafana/
│ ├── loki/
│ ├── elk/
│ ├── thanos/
│
│ ├── argocd/
│ ├── velero/
│ └── restic/
│
│
└── docker-compose/
├── gitlab/
│ └── docker-compose.yml
│
├── gerrit/
│
├── jenkins/
│
├── nexus/
│
├── sonar/
│
├── ldap/
│
├── jira/
│
├── confluence/
│
├── nextcloud/
│
├── mail/
│
├── ftp/
│
└── openvpn/二、安装方式设计
1. 单独安装 GitLab
执行：
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/gitlab.yml只安装：
GitLab
+
PostgreSQL
+
Redis
+
Nginx2. 单独安装 Jenkins
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/jenkins.yml结果：
Docker
└── Jenkins Container3. 单独安装 Nexus
ansible-playbook \
playbooks/nexus.yml部署：
Nexus Repository
docker-compose4. 单独安装 Kubernetes
ansible-playbook \
playbooks/kubernetes.yml安装：
containerd
kubeadm
kubelet
kubectl三、Playbook模板
例如：
gitlab.yml
---
- hosts: gitlab
become: yes
roles:
- gitlabroles/gitlab/tasks/main.yml
---
- name:
create directory
file:
path:
/opt/gitlab
state:
directory
- name:
copy compose file
template:
src:
docker-compose.yml.j2
dest:
/opt/gitlab/docker-compose.yml
- name:
start gitlab
community.docker.docker_compose_v2:
project_src:
/opt/gitlab四、支持 uninstall
例如：
ansible-playbook \
playbooks/uninstall.yml \
-e service=gitlab执行：
docker compose down
remove container
remove volume(optional)五、支持升级
例如：
升级 Jenkins：
ansible-playbook \
playbooks/upgrade.yml \
-e service=jenkins流程：
backup
pull new image
restart
health check六、服务依赖关系
自动检查：
例如：
GitLab
依赖：
docker
postgresql
redis
smtp
ldap(optional)Nextcloud
依赖：
docker
postgres
redis
ldap(optional)
collabora(optional)
onlyoffice(optional)Grafana
依赖：
prometheus
loki(optional)
ldap(optional)七、统一变量
group_vars/all.yml
domain:
example.com
timezone:
Asia/Shanghai
docker_root:
/data/docker
storage_root:
/mnt/allin_data
registry:
harbor.example.com
ldap:
dc=example,dc=com八、服务安装矩阵
| 服务 | 方式 |
|---|---|
| Gerrit | Docker Compose |
| GitLab | Docker Compose |
| Jira | Docker Compose |
| Confluence | Docker Compose |
| DNS | Ansible + Bind9 |
| Proxy | Squid/Nginx |
| Nexus | Docker Compose |
| Jenkins | Docker Compose |
| LDAP | Docker Compose |
| Nextcloud | Docker Compose |
| Collabora | Docker Compose |
| OnlyOffice | Docker Compose |
| Docker Compose | |
| Sonar | Docker Compose |
| ELK | Docker Compose/K8s |
| Loki | K8s Helm |
| Prometheus | K8s Helm |
| Alertmanager | K8s Helm |
| Grafana | K8s Helm |
| Thanos | K8s Helm |
| FTP | Docker Compose |
| OpenVPN | Docker Compose |
| Harbor | Helm |
| Vault | Helm |
| Keycloak | Helm |
| ArgoCD | Helm |
| Velero | Helm |
| Restic | Ansible |
| Keepalived | Ansible |
| HAProxy | Ansible |
九、最终使用方式
安装全部
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/install.yml只安装 DevOps
ansible-playbook \
playbooks/gitlab.yml
ansible-playbook \
playbooks/jenkins.yml
ansible-playbook \
playbooks/nexus.yml
ansible-playbook \
playbooks/sonar.yml只安装监控
ansible-playbook \
playbooks/prometheus.yml
ansible-playbook \
playbooks/grafana.yml
ansible-playbook \
playbooks/loki.yml十、建议下一步生成顺序
为了最终能直接落地，我建议按以下顺序生成代码：
Part 1 基础层
- ansible.cfg
- inventory
- common role
- Docker role
- Nginx
- Keepalived
- HAProxy
Part 2 身份认证
- LDAP
- Keycloak
- Vault
Part 3 DevOps
- GitLab
- Gerrit
- Jenkins
- Nexus
- Sonar
- Harbor
Part 4 Office
- Jira
- Confluence
- Nextcloud
- Collabora
- OnlyOffice
Part 5 Kubernetes
- kubeadm
- ArgoCD
- Harbor
- Velero
Part 6 Monitoring
- Prometheus
- Alertmanager
- Grafana
- Loki
- ELK
- Thanos
这样最终会得到一个可执行的：
enterprise-platform-ansible/每个服务均可：
install
uninstall
upgrade
backup
restore独立管理。
按照十的顺序生成代码
好的，我按照之前规划的 10个阶段 生成代码。
由于完整项目包含：
- 40+ Ansible Role
- 30+ Playbook
- 20+ Docker Compose
- Kubernetes Helm部署
- 配置模板
- 备份恢复脚本
一次消息无法完整放下，会分批输出
生成顺序如下：
Part 1 基础层
├── ansible.cfg
├── inventory
├── group_vars
├── common role
├── Docker role
├── Nginx role
├── Keepalived role
└── HAProxy role
Part 2 身份认证层
├── LDAP
├── Keycloak
└── Vault
Part 3 DevOps层
├── GitLab
├── Gerrit
├── Jenkins
├── Nexus
├── Sonar
└── Harbor
Part 4 Office层
├── Jira
├── Confluence
├── Nextcloud
├── Collabora
├── OnlyOffice
├── Mail
├── FTP
└── OpenVPN
Part 5 Kubernetes层
├── containerd
├── kubeadm
├── Kubernetes Cluster
└── ArgoCD
Part 6 Monitoring层
├── Prometheus
├── Alertmanager
├── Grafana
├── Loki
├── ELK
└── Thanos
Part 7 Backup层
├── Velero
└── Restic
Part 8 安全增强
├── TLS
├── Cert-manager
├── Trivy
└── OPA
Part 9 CI/CD集成
├── GitLab Runner
├── Jenkins Pipeline
├── Harbor CI
└── ArgoCD GitOps
Part 10 运维管理
├── install
├── uninstall
├── upgrade
├── backup
└── restorePart 1 基础层
目录
创建：
mkdir -p enterprise-platform-ansible
cd enterprise-platform-ansible
mkdir -p \
inventory/production/group_vars \
playbooks \
roles/{common,docker,nginx,keepalived,haproxy}最终：
enterprise-platform-ansible
├── ansible.cfg
├── inventory
├── playbooks
└── roles1. ansible.cfg
文件：
ansible.cfg内容：
[defaults]
inventory = inventory/production/hosts.ini
remote_user = root
host_key_checking = False
forks = 20
timeout = 30
roles_path = roles
[privilege_escalation]
become = true
become_method = sudo
[ssh_connection]
pipelining = True2. Inventory
文件：
inventory/production/hosts.ini内容：
################################
# Load Balancer
################################
[loadbalancer]
lb01 ansible_host=10.50.0.10
lb02 ansible_host=10.50.0.11
################################
# Proxy
################################
[proxy]
proxy01 ansible_host=10.50.0.20
################################
# Docker Host
################################
[docker_hosts]
docker01 ansible_host=10.50.0.30
################################
# Kubernetes
################################
[k8s_master]
master01 ansible_host=10.50.0.40
master02 ansible_host=10.50.0.41
master03 ansible_host=10.50.0.42
[k8s_worker]
worker01 ansible_host=10.50.0.50
worker02 ansible_host=10.50.0.51
################################
# Monitoring
################################
[monitor]
monitor01 ansible_host=10.50.0.603. 全局变量
文件：
inventory/production/group_vars/all.yml
################################
# System
################################
timezone: Asia/Shanghai
hostname_domain:
company.local
################################
# Storage
################################
data_root:
/data
docker_root:
/data/docker
################################
# Network
################################
vip:
10.50.0.100
interface:
eth0
################################
# Docker
################################
docker_version:
latest
################################
# Kubernetes
################################
kubernetes_version:
"1.31.0"4. Common Role
目录：
roles/common创建：
mkdir -p roles/common/tasksroles/common/tasks/main.yml
---
- name:
update system
apt:
update_cache: yes
- name:
install basic packages
apt:
name:
- vim
- curl
- wget
- net-tools
- jq
- unzip
- chrony
- lvm2
state:
present
- name:
set timezone
timezone:
name:
"{{ timezone }}"
- name:
enable chrony
service:
name:
chrony
state:
started
enabled:
yes5. Docker Role
目录：
roles/docker创建：
mkdir -p roles/docker/tasksroles/docker/tasks/main.yml
---
- name:
install docker
apt:
name:
- docker.io
- docker-compose-plugin
state:
present
- name:
enable docker
systemd:
name:
docker
state:
started
enabled:
yes6. 基础安装Playbook
文件：
playbooks/bootstrap.yml内容：
---
- hosts:
all
become:
yes
roles:
- common
- hosts:
docker_hosts
become:
yes
roles:
- docker执行：
ansible-playbook playbooks/bootstrap.yml完成：
- 系统初始化
- Docker安装
7. Keepalived Role
目录：
mkdir -p roles/keepalived/{tasks,templates}tasks/main.yml
---
- name:
install keepalived
apt:
name:
keepalived
state:
present
- name:
config keepalived
template:
src:
keepalived.conf.j2
dest:
/etc/keepalived/keepalived.conf
- name:
restart
systemd:
name:
keepalived
state:
restartedtemplates/keepalived.conf.j2
vrrp_instance VI_1 {
state MASTER
interface {{ interface }}
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
{{ vip }}
}
}8. HAProxy Role
目录：
roles/haproxytasks：
---
- name:
install haproxy
apt:
name:
haproxy
state:
present
- name:
config
template:
src:
haproxy.cfg.j2
dest:
/etc/haproxy/haproxy.cfg
- name:
restart
systemd:
name:
haproxy
state:
restartedhaproxy.cfg.j2
global
daemon
defaults
mode tcp
timeout connect 5s
timeout client 1m
timeout server 1m
frontend https
bind *:443
default_backend backend_apps
backend backend_apps
balance roundrobin
server app01 10.50.0.30:443 check
server app02 10.50.0.31:443 check9. HA Playbook
文件：
playbooks/ha.yml
---
- hosts:
loadbalancer
roles:
- keepalived
- haproxy执行：
ansible-playbook playbooks/ha.ymlPart 1 完成
当前已生成：
✅ Ansible基础框架
✅ Inventory
✅ 全局变量
✅ 系统初始化
✅ Docker安装
✅ Keepalived
✅ HAProxy
Part 2 身份认证层
本部分包含：
LDAP Server
├── OpenLDAP
├── phpLDAPadmin
└── LDAP Schema
Keycloak
├── LDAP Federation
├── OIDC
├── OAuth2
└── SSO
Vault
├── Standalone / HA
├── LDAP Authentication
├── Kubernetes Authentication
└── Secret Management目标：
Keycloak
|
|
OpenLDAP
|
+---------------+---------------+
| | |
GitLab Jenkins Grafana
Jira Nextcloud ArgoCD
Confluence HarborPart 2目录增加
最终：
enterprise-platform-ansible/
├── playbooks/
│
├── ldap.yml
├── keycloak.yml
└── vault.yml
├── roles/
├── ldap/
│ ├── tasks/
│ ├── templates/
│ └── defaults/
├── keycloak/
│ ├── tasks/
│ ├── templates/
│ └── defaults/
└── vault/
├── tasks/
├── templates/
└── defaults/
├── docker-compose/
├── ldap/
├── keycloak/
└── vault/一、LDAP Server
采用：
- OpenLDAP
- phpLDAPadmin
Docker Compose部署。
目录：
mkdir -p docker-compose/ldap1. LDAP docker-compose
文件：
docker-compose/ldap/docker-compose.yml
services:
openldap:
image: osixia/openldap:latest
container_name: openldap
restart: always
environment:
LDAP_ORGANISATION: "AllinTech"
LDAP_DOMAIN: "company.local"
LDAP_ADMIN_PASSWORD: "Admin@123456"
ports:
- "389:389"
- "636:636"
volumes:
- /data/ldap/database:/var/lib/ldap
- /data/ldap/config:/etc/ldap/slapd.d
phpldapadmin:
image: osixia/phpldapadmin:latest
container_name: phpldapadmin
restart: always
environment:
PHPLDAPADMIN_LDAP_HOSTS: openldap
ports:
- "8088:80"
depends_on:
- openldap2. LDAP目录规划
服务器：
/data/ldap
├── database
└── configAnsible创建：
- name:
create ldap directory
file:
path:
/data/ldap
state:
directory3. LDAP Role
创建：
mkdir -p roles/ldap/{tasks,templates,defaults}defaults/main.yml
ldap_domain:
company.local
ldap_password:
Admin@123456
ldap_org:
AllinTechtasks/main.yml
---
- name:
create ldap directory
file:
path:
/opt/ldap
state:
directory
- name:
copy compose
template:
src:
docker-compose.yml.j2
dest:
/opt/ldap/docker-compose.yml
- name:
start ldap
community.docker.docker_compose_v2:
project_src:
/opt/ldaptemplates/docker-compose.yml.j2
services:
openldap:
image:
osixia/openldap:latest
restart:
always
environment:
LDAP_DOMAIN:
"{{ ldap_domain }}"
LDAP_ADMIN_PASSWORD:
"{{ ldap_password }}"
ports:
- 389:389
- 636:636
phpldapadmin:
image:
osixia/phpldapadmin
environment:
PHPLDAPADMIN_LDAP_HOSTS:
openldap
ports:
- 8088:804. LDAP Playbook
文件：
playbooks/ldap.yml
---
- hosts:
ldap
become:
yes
roles:
- ldap执行：
ansible-playbook playbooks/ldap.yml二、Keycloak SSO
部署：
Docker Compose。
目录：
mkdir -p docker-compose/keycloakKeycloak Compose
文件：
docker-compose/keycloak/docker-compose.yml
services:
postgres:
image:
postgres:16
restart:
always
environment:
POSTGRES_DB:
keycloak
POSTGRES_USER:
keycloak
POSTGRES_PASSWORD:
password
volumes:
- /data/keycloak/postgres:/var/lib/postgresql/data
keycloak:
image:
quay.io/keycloak/keycloak:latest
restart:
always
command:
start-dev
environment:
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME:
keycloak
KC_DB_PASSWORD:
password
KEYCLOAK_ADMIN:
admin
KEYCLOAK_ADMIN_PASSWORD:
Admin@123456
ports:
- "8080:8080"
depends_on:
- postgresKeycloak配置
登录：
http://server:8080创建：
Realm:
companyLDAP Federation:
User Federation
Add provider
ldap配置：
Connection URL:
ldap://openldap:389
Bind DN:
cn=admin,dc=company,dc=local
Users DN:
ou=users,dc=company,dc=localKeycloak Role
目录：
mkdir -p roles/keycloak/{tasks,templates}tasks/main.yml
---
- name:
deploy keycloak
template:
src:
docker-compose.yml.j2
dest:
/opt/keycloak/docker-compose.yml
- name:
start keycloak
community.docker.docker_compose_v2:
project_src:
/opt/keycloakKeycloak Playbook
文件：
playbooks/keycloak.yml
---
- hosts:
keycloak
roles:
- keycloak三、Vault
用途：
管理：
- 数据库密码
- LDAP密码
- TLS证书
- API Token
- Kubernetes Secret
架构：
Application
|
Vault Agent
|
Vault ServerVault Compose
目录：
mkdir -p docker-compose/vault文件：
docker-compose/vault/docker-compose.yml
services:
vault:
image:
hashicorp/vault:latest
container_name:
vault
restart:
always
cap_add:
- IPC_LOCK
ports:
- "8200:8200"
volumes:
- /data/vault:/vault/data
- ./vault.hcl:/vault/config/vault.hcl
command:
vault server -config=/vault/config/vault.hclVault配置
文件：
vault.hcl
storage "file" {
path = "/vault/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = true
}
ui = trueVault初始化
启动：
docker compose up -d初始化：
vault operator init保存：
Unseal Key
Root Token解封：
vault operator unsealVault LDAP认证
启用：
vault auth enable ldap配置：
vault write auth/ldap/config \
url="ldap://openldap:389" \
userdn="ou=users,dc=company,dc=local" \
groupdn="ou=groups,dc=company,dc=local"Vault Role
目录：
mkdir -p roles/vault/{tasks,templates}tasks/main.yml
---
- name:
create vault dir
file:
path:
/opt/vault
state:
directory
- name:
deploy vault
template:
src:
docker-compose.yml.j2
dest:
/opt/vault/docker-compose.yml
- name:
start vault
community.docker.docker_compose_v2:
project_src:
/opt/vaultVault Playbook
文件：
playbooks/vault.yml
---
- hosts:
vault
roles:
- vault四、执行顺序
身份认证层部署：
# 1 LDAP
ansible-playbook playbooks/ldap.yml
# 2 Keycloak
ansible-playbook playbooks/keycloak.yml
# 3 Vault
ansible-playbook playbooks/vault.ymlPart 3：DevOps 层
本部分包含：
DevOps Platform
├── GitLab
│ ├── Git Repository
│ ├── CI/CD
│ ├── LDAP Login
│ ├── SMTP
│ └── Backup
├── Gerrit
│ ├── Code Review
│ ├── Git SSH
│ └── LDAP
├── Jenkins
│ ├── Pipeline
│ ├── Docker Agent
│ ├── LDAP
│ └── GitLab Integration
├── Nexus
│ ├── Maven Repository
│ ├── npm Repository
│ ├── Docker Registry
│ └── LDAP
├── SonarQube
│ ├── Code Scan
│ ├── Jenkins Integration
│ └── LDAP
└── Harbor
├── Docker Registry
├── LDAP
├── Scanner
└── Kubernetes Integration一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── gitlab.yml
├── gerrit.yml
├── jenkins.yml
├── nexus.yml
├── sonar.yml
└── harbor.yml
├── roles/
├── gitlab/
├── gerrit/
├── jenkins/
├── nexus/
├── sonar/
└── harbor/
└── docker-compose/
├── gitlab/
├── gerrit/
├── jenkins/
├── nexus/
├── sonar/
└── harbor/二、GitLab
1. 创建目录
mkdir -p docker-compose/gitlabGitLab Compose
文件：
docker-compose/gitlab/docker-compose.yml
services:
gitlab:
image: gitlab/gitlab-ee:latest
container_name: gitlab
restart: always
hostname:
gitlab.company.local
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.company.local'
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = {
'main' => {
'label' => 'LDAP',
'host' => 'openldap',
'port' => 389,
'uid' => 'uid',
'bind_dn' => 'cn=admin,dc=company,dc=local',
'password' => 'Admin@123456',
'base' => 'ou=users,dc=company,dc=local'
}
}
gitlab_rails['smtp_enable']=true
ports:
- "80:80"
- "443:443"
- "2222:22"
volumes:
- /data/gitlab/config:/etc/gitlab
- /data/gitlab/logs:/var/log/gitlab
- /data/gitlab/data:/var/opt/gitlabGitLab Role
目录：
mkdir -p roles/gitlab/tasksroles/gitlab/tasks/main.yml
---
- name:
create gitlab directory
file:
path:
/opt/gitlab
state:
directory
- name:
copy compose
copy:
src:
docker-compose.yml
dest:
/opt/gitlab/docker-compose.yml
- name:
start gitlab
community.docker.docker_compose_v2:
project_src:
/opt/gitlabGitLab Playbook
文件：
playbooks/gitlab.yml
---
- hosts:
gitlab
become:
yes
roles:
- gitlab执行：
ansible-playbook playbooks/gitlab.yml三、Gerrit
Compose
目录：
mkdir docker-compose/gerrit文件：
docker-compose/gerrit/docker-compose.yml
services:
gerrit:
image:
gerritcodereview/gerrit:latest
container_name:
gerrit
restart:
always
ports:
- "8080:8080"
- "29418:29418"
volumes:
- /data/gerrit:/var/gerrit/review_siteGerrit初始化
进入：
docker exec -it gerrit bash安装LDAP插件：
git clone \
https://gerrit.googlesource.com/plugins/ldapGerrit配置
文件：
gerrit.config
[ldap]
server = ldap://openldap:389
username = cn=admin,dc=company,dc=local
password = Admin@123456
accountBase = ou=users,dc=company,dc=localGerrit Role
---
- name:
deploy gerrit
community.docker.docker_compose_v2:
project_src:
/opt/gerritGerrit Playbook
---
- hosts:
gerrit
roles:
- gerrit四、Jenkins
架构：
Jenkins
+
Docker Agent
+
GitLab
+
Sonar
+
HarborCompose
目录：
mkdir docker-compose/jenkins文件：
docker-compose/jenkins/docker-compose.yml
services:
jenkins:
image:
jenkins/jenkins:lts
container_name:
jenkins
restart:
always
user:
root
ports:
- "8080:8080"
- "50000:50000"
volumes:
- /data/jenkins:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sockJenkins插件
自动安装：
git
gitlab-plugin
pipeline
docker-plugin
sonar
kubernetesJenkins LDAP
配置：
Manage Jenkins
Security
LDAP参数：
Server:
ldap://openldap:389
Root DN:
dc=company,dc=local五、Nexus
Compose
services:
nexus:
image:
sonatype/nexus3
container_name:
nexus
restart:
always
ports:
- "8081:8081"
volumes:
- /data/nexus:/nexus-dataNexus支持
仓库：
maven-releases
maven-snapshots
npm-private
docker-hosted
docker-proxy六、SonarQube
Compose
services:
postgres:
image:
postgres:15
sonar:
image:
sonarqube:lts-community
ports:
- "9000:9000"
environment:
SONAR_JDBC_URL:
jdbc:postgresql://postgres:5432/sonarJenkins集成
Jenkinsfile:
pipeline {
stages {
stage('Sonar Scan'){
steps{
sh '''
sonar-scanner \
-Dsonar.projectKey=test
'''
}
}
}
}七、Harbor
推荐：
官方离线安装包 + Ansible自动化。
目录：
roles/harbor安装：
wget \
https://github.com/goharbor/harbor/releases/download/v2.x.x/harbor-offline-installer.tgzHarbor配置
harbor.yml:
hostname:
harbor.company.local
https:
port:
443
database:
password:
Harbor@123456
data_volume:
/data/harborHarbor LDAP
配置：
Administration
Authentication
LDAP参数：
LDAP URL:
ldap://openldap:389
Base DN:
ou=users,dc=company,dc=local八、Playbook汇总
gitlab.yml
---
- hosts:
gitlab
roles:
- gitlabgerrit.yml
---
- hosts:
gerrit
roles:
- gerritjenkins.yml
---
- hosts:
jenkins
roles:
- jenkinsnexus.yml
---
- hosts:
nexus
roles:
- nexussonar.yml
---
- hosts:
sonar
roles:
- sonarharbor.yml
---
- hosts:
harbor
roles:
- harbor九、部署顺序
建议：
# 1 LDAP
ansible-playbook playbooks/ldap.yml
# 2 GitLab
ansible-playbook playbooks/gitlab.yml
# 3 Gerrit
ansible-playbook playbooks/gerrit.yml
# 4 Jenkins
ansible-playbook playbooks/jenkins.yml
# 5 Nexus
ansible-playbook playbooks/nexus.yml
# 6 Sonar
ansible-playbook playbooks/sonar.yml
# 7 Harbor
ansible-playbook playbooks/harbor.ymlPart 4 Office协作平台
本部分部署：
Office Platform
├── Jira
│ ├── Jira Software
│ ├── PostgreSQL
│ ├── LDAP Authentication
│ └── Backup
├── Confluence
│ ├── Confluence Wiki
│ ├── PostgreSQL
│ ├── LDAP
│ └── Attachment Storage
├── Nextcloud
│ ├── PostgreSQL
│ ├── Redis
│ ├── LDAP
│ ├── Collabora
│ └── OnlyOffice
├── Mail Server
│ ├── Postfix
│ ├── Dovecot
│ ├── Roundcube
│ └── LDAP
├── FTP Server
│ └── vsftpd
└── OpenVPN
├── Server
├── LDAP Authentication
└── Certificate Management一、目录结构增加
enterprise-platform-ansible/
├── playbooks/
│
├── jira.yml
├── confluence.yml
├── nextcloud.yml
├── collabora.yml
├── onlyoffice.yml
├── mail.yml
├── ftp.yml
└── openvpn.yml
roles/
├── jira/
├── confluence/
├── nextcloud/
├── collabora/
├── onlyoffice/
├── mail/
├── ftp/
└── openvpn/
docker-compose/
├── jira/
├── confluence/
├── nextcloud/
├── collabora/
├── onlyoffice/
├── mail/
├── ftp/
└── openvpn/二、Jira 部署
架构：
LDAP
|
|
Jira ---- PostgreSQL1. Jira Compose
创建：
mkdir -p docker-compose/jira文件：
docker-compose/jira/docker-compose.yml
services:
postgres:
image:
postgres:15
container_name:
jira-postgres
restart:
always
environment:
POSTGRES_DB:
jira
POSTGRES_USER:
jira
POSTGRES_PASSWORD:
Jira@123456
volumes:
- /data/jira/postgres:/var/lib/postgresql/data
jira:
image:
atlassian/jira-software:latest
container_name:
jira
restart:
always
depends_on:
- postgres
ports:
- "8080:8080"
environment:
ATL_JDBC_URL:
jdbc:postgresql://postgres:5432/jira
ATL_JDBC_USER:
jira
ATL_JDBC_PASSWORD:
Jira@123456
volumes:
- /data/jira/data:/var/atlassian/application-data/jiraJira LDAP配置
进入：
Jira
→
Administration
→
User Management
→
LDAP配置：
LDAP URL:
ldap://openldap:389
Base DN:
dc=company,dc=local
User DN:
ou=users,dc=company,dc=localJira Role
目录：
mkdir -p roles/jira/tasksroles/jira/tasks/main.yml
---
- name:
create jira dir
file:
path:
/opt/jira
state:
directory
- name:
deploy jira
community.docker.docker_compose_v2:
project_src:
/opt/jiraJira Playbook
playbooks/jira.yml
---
- hosts:
jira
roles:
- jira三、Confluence
架构：
LDAP
|
Confluence
|
PostgreSQL
|
NFS / MinIO Attachment StorageCompose
目录：
mkdir -p docker-compose/confluence文件：
docker-compose/confluence/docker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
confluence
POSTGRES_USER:
confluence
POSTGRES_PASSWORD:
Confluence@123456
volumes:
- /data/confluence/postgres:/var/lib/postgresql/data
confluence:
image:
atlassian/confluence-server:latest
restart:
always
ports:
- "8090:8090"
environment:
ATL_JDBC_URL:
jdbc:postgresql://postgres:5432/confluence
ATL_JDBC_USER:
confluence
ATL_JDBC_PASSWORD:
Confluence@123456
volumes:
- /data/confluence/data:/var/atlassian/application-data/confluence四、Nextcloud
架构：
LDAP
|
Client ---> Nextcloud
|
+--------+--------+
| |
PostgreSQL Redis
|
Collabora
|
OnlyOfficeNextcloud Compose
目录：
mkdir -p docker-compose/nextcloud文件：
docker-compose/nextcloud/docker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
nextcloud
POSTGRES_USER:
nextcloud
POSTGRES_PASSWORD:
Nextcloud@123456
volumes:
- /data/nextcloud/postgres:/var/lib/postgresql/data
redis:
image:
redis:7
restart:
always
nextcloud:
image:
nextcloud:latest
restart:
always
ports:
- "8081:80"
depends_on:
- postgres
- redis
volumes:
- /data/nextcloud/html:/var/www/htmlNextcloud LDAP
配置：
Administration
LDAP/AD integration参数：
Host:
openldap
Port:
389
Base DN:
dc=company,dc=local五、Collabora
用于：
- 在线编辑 Word
- Excel
- PPT
Compose：
services:
collabora:
image:
collabora/code
restart:
always
ports:
- "9980:9980"
environment:
domain:
nextcloud.company.local
username:
admin
password:
password六、OnlyOffice
Compose:
services:
onlyoffice:
image:
onlyoffice/documentserver
restart:
always
ports:
- "8082:80"
volumes:
- /data/onlyoffice:/var/www/onlyoffice/DataNextcloud集成
安装插件：
Apps
↓
Collabora Online
↓
OnlyOffice connector七、邮件系统
采用：
- Postfix
- Dovecot
- Roundcube
架构：
User
|
Roundcube
|
Dovecot
|
Postfix
|
LDAPMail Compose
目录：
mkdir -p docker-compose/mail文件：
docker-compose/mail/docker-compose.yml
services:
mailserver:
image:
mailserver/docker-mailserver
hostname:
mail.company.local
restart:
always
ports:
- "25:25"
- "143:143"
- "587:587"
- "993:993"
volumes:
- /data/mail/data:/var/mail
- /data/mail/config:/tmp/docker-mailserver/config
environment:
ENABLE_LDAP:
1
LDAP_SERVER_HOST:
openldap八、FTP
采用：
vsftpd
Compose:
services:
ftp:
image:
fauria/vsftpd
restart:
always
ports:
- "21:21"
- "21100-21110:21100-21110"
environment:
FTP_USER:
ftpuser
FTP_PASS:
password
volumes:
- /data/ftp:/home/vsftpd九、OpenVPN
架构：
Client
|
OpenVPN
|
LDAPCompose
services:
openvpn:
image:
kylemanna/openvpn
container_name:
openvpn
restart:
always
ports:
- "1194:1194/udp"
volumes:
- /data/openvpn:/etc/openvpnLDAP认证
安装插件：
openvpn-auth-ldap配置：
<LDAP>
URL ldap://openldap:389
BindDN cn=admin,dc=company,dc=local
Password Admin@123456
BaseDN ou=users,dc=company,dc=local
</LDAP>十、Playbook汇总
jira.yml
---
- hosts:
jira
roles:
- jiraconfluence.yml
---
- hosts:
confluence
roles:
- confluencenextcloud.yml
---
- hosts:
nextcloud
roles:
- nextcloudmail.yml
---
- hosts:
mail
roles:
- mailftp.yml
---
- hosts:
ftp
roles:
- ftpopenvpn.yml
---
- hosts:
vpn
roles:
- openvpn十一、部署顺序
建议：
# 1 PostgreSQL / LDAP基础
ansible-playbook playbooks/ldap.yml
# 2 Jira
ansible-playbook playbooks/jira.yml
# 3 Confluence
ansible-playbook playbooks/confluence.yml
# 4 Nextcloud
ansible-playbook playbooks/nextcloud.yml
# 5 Collabora
ansible-playbook playbooks/collabora.yml
# 6 OnlyOffice
ansible-playbook playbooks/onlyoffice.yml
# 7 Mail
ansible-playbook playbooks/mail.yml
# 8 FTP
ansible-playbook playbooks/ftp.yml
# 9 VPN
ansible-playbook playbooks/openvpn.ymlPart 5 Kubernetes层
本部分是整个企业平台的核心运行环境
目标：
- 自动化安装 Kubernetes 集群
- 支持 HA Master
- 支持 Keepalived + HAProxy
- 支持 containerd
- 支持 Helm
- 支持 Ingress
- 支持 ArgoCD GitOps
- 支持 Harbor 私有镜像仓库
- 支持 Velero / Restic 备份
Part 5 架构
VIP
10.50.0.100
|
+-------------+
| HAProxy |
+-------------+
| | |
| | |
+------+ +------+ +------+
| | | | | |
master1 master2 master3
Kubernetes API
|
+----------+-----------+
| |
Worker1 Worker2
Kubernetes Addons
|
+-- Nginx Ingress
|
+-- Cert-manager
|
+-- ArgoCD
|
+-- Harbor
|
+-- Velero
|
+-- Restic一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── kubernetes.yml
├── containerd.yml
├── kube-master.yml
├── kube-worker.yml
├── argocd.yml
├── ingress.yml
├── cert-manager.yml
├── velero.yml
└── restic.yml
roles/
├── containerd/
├── kubernetes/
├── kube-master/
├── kube-worker/
├── helm/
├── ingress-nginx/
├── cert-manager/
├── argocd/
├── velero/
└── restic/二、Inventory扩展
文件：
inventory/production/hosts.ini增加：
################################
# Kubernetes Cluster
################################
[k8s_master]
master01 ansible_host=10.50.0.40
master02 ansible_host=10.50.0.41
master03 ansible_host=10.50.0.42
[k8s_worker]
worker01 ansible_host=10.50.0.50
worker02 ansible_host=10.50.0.51
[k8s]
master01
master02
master03
worker01
worker02三、Containerd安装
Kubernetes 1.31推荐：
containerd + runc + cni创建：
mkdir -p roles/containerd/tasksroles/containerd/tasks/main.yml
---
- name:
install containerd
apt:
name:
- containerd
- runc
state:
present
- name:
create config
shell:
cmd:
containerd config default > /etc/containerd/config.toml
- name:
enable systemd cgroup
replace:
path:
/etc/containerd/config.toml
regexp:
'SystemdCgroup = false'
replace:
'SystemdCgroup = true'
- name:
restart containerd
systemd:
name:
containerd
enabled:
yes
state:
restartedContainerd Playbook
文件：
playbooks/containerd.yml
---
- hosts:
k8s
become:
yes
roles:
- containerd执行：
ansible-playbook playbooks/containerd.yml四、Kubernetes基础依赖
创建：
roles/kubernetes/tasks/main.yml
---
- name:
disable swap
command:
cmd:
swapoff -a
- name:
remove swap fstab
replace:
path:
/etc/fstab
regexp:
'^.*swap.*$'
replace:
''
- name:
install kube tools
apt:
name:
- kubelet
- kubeadm
- kubectl
state:
present
- name:
enable kubelet
systemd:
name:
kubelet
enabled:
yesKubernetes Role
roles/kubernetes/tasks/main.yml包含：
kubeadm
kubelet
kubectl
sysctl
network
iptables五、初始化Master
目录：
roles/kube-master/tasksmain.yml
---
- name:
initialize kubernetes
command:
cmd:
kubeadm init \
--control-plane-endpoint "{{ vip }}:6443" \
--upload-certs
args:
creates:
/etc/kubernetes/admin.confkubeadm配置
模板：
roles/kube-master/templates/kubeadm.yaml.j2
apiVersion:
kubeadm.k8s.io/v1beta4
kind:
ClusterConfiguration
kubernetesVersion:
v1.31.0
controlPlaneEndpoint:
"{{ vip }}:6443"
networking:
podSubnet:
10.244.0.0/16六、Worker加入
roles/kube-worker/tasks/main.yml
---
- name:
join cluster
command:
cmd:
kubeadm join {{ vip }}:6443 \
--token {{ kube_token }} \
--discovery-token-ca-cert-hash {{ discovery_hash }}七、Kubernetes安装总入口
文件：
playbooks/kubernetes.yml
---
- hosts:
k8s
become:
yes
roles:
- containerd
- kubernetes
- hosts:
k8s_master
roles:
- kube-master
- hosts:
k8s_worker
roles:
- kube-worker八、Helm安装
目录：
roles/helm/tasks/main.yml
---
- name:
install helm
shell:
cmd:
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash九、Ingress-Nginx
生产入口：
Internet
|
HAProxy
|
Ingress Controller
|
Service安装：
helm repo add ingress-nginx \
https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx \
ingress-nginx/ingress-nginx \
-n ingress-nginx \
--create-namespaceAnsible:
roles/ingress-nginx/tasks/main.yml
---
- name:
install ingress
command:
cmd:
helm install ingress-nginx \
ingress-nginx/ingress-nginx \
-n ingress-nginx \
--create-namespace十、Cert-manager
用于：
- Let's Encrypt
- 内部CA
- TLS自动签发
安装：
helm repo add jetstack \
https://charts.jetstack.io
helm install cert-manager \
jetstack/cert-manager \
-n cert-manager \
--create-namespace \
--set installCRDs=true十一、ArgoCD
GitOps核心。
架构：
GitLab
|
ArgoCD
|
Kubernetes安装：
kubectl create namespace argocd
helm repo add argo \
https://argoproj.github.io/argo-helm
helm install argocd \
argo/argo-cd \
-n argocdPlaybook：
playbooks/argocd.yml
---
- hosts:
k8s_master
roles:
- argocd十二、Harbor Kubernetes集成
使用：
imagePullSecrets创建：
kubectl create secret docker-registry harbor-secret \
--docker-server=harbor.company.local \
--docker-username=admin \
--docker-password=password应用：
imagePullSecrets:
- name:
harbor-secret十三、Velero备份
支持：
- Kubernetes资源
- PV
- S3
- MinIO
安装：
velero install \
--provider aws \
--bucket k8s-backup \
--backup-location-config \
region=minio,s3ForcePathStyle=true,s3Url=http://minio:9000备份：
velero backup create all-backup恢复：
velero restore create \
--from-backup all-backup十四、Restic
文件备份
安装：
apt install restic初始化：
restic init \
-r /backup备份：
restic backup /data恢复：
restic restore latest十五、Kubernetes完整部署顺序
执行：
1. 基础组件
ansible-playbook \
playbooks/containerd.yml2. Kubernetes
ansible-playbook \
playbooks/kubernetes.yml3. Helm
ansible-playbook \
playbooks/helm.yml4. Ingress
ansible-playbook \
playbooks/ingress.yml5. Cert Manager
ansible-playbook \
playbooks/cert-manager.yml6. ArgoCD
ansible-playbook \
playbooks/argocd.yml7. Backup
ansible-playbook \
playbooks/velero.yml
ansible-playbook \
playbooks/restic.ymlPart 6：Monitoring监控平台
本部分建设企业级可观测平台：
Monitoring Platform
Grafana
|
+-----------+------------+
| |
Prometheus Loki
| |
| Promtail
|
+-----+-------+
| |
Node Exporter App Metrics
|
|
Alertmanager
|
Email / Webhook / 企业微信
|
Thanos
|
Object Storage
(MinIO/S3)
ELK Stack:
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── prometheus.yml
├── alertmanager.yml
├── grafana.yml
├── loki.yml
├── elk.yml
└── thanos.yml
roles/
├── prometheus/
├── alertmanager/
├── grafana/
├── loki/
├── promtail/
├── elk/
└── thanos/
docker-compose/
├── prometheus/
├── grafana/
├── loki/
├── elk/
└── thanos/二、Prometheus
推荐：
生产环境：
Prometheus Operator
+
kube-prometheus-stack监控：
- Kubernetes
- Node
- Container
- Application
1. Helm安装
添加仓库：
helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts
helm repo update安装：
helm install monitoring \
prometheus-community/kube-prometheus-stack \
-n monitoring \
--create-namespace组件：
kube-prometheus-stack
├── Prometheus
├── Alertmanager
├── Grafana
├── Node Exporter
├── kube-state-metrics
└── Operator三、Prometheus Ansible Role
目录：
mkdir -p roles/prometheus/tasksroles/prometheus/tasks/main.yml
---
- name:
create namespace
command:
kubectl create namespace monitoring
ignore_errors:
yes
- name:
add repo
command:
helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts
- name:
install prometheus
command:
helm install monitoring \
prometheus-community/kube-prometheus-stack \
-n monitoringPrometheus Playbook
文件：
playbooks/prometheus.yml
---
- hosts:
k8s_master
roles:
- prometheus四、Alertmanager
功能：
Alert Rule
|
Alertmanager
|
+-----------+
|
Email
|
Webhook
|
DingTalk
|
WeChatAlertmanager配置
文件：
alertmanager.yml
global:
smtp_smarthost:
mail.company.com:587
smtp_from:
alert@company.com
route:
receiver:
email
receivers:
- name:
email
email_configs:
- to:
admin@company.comAnsible配置
roles:
roles/alertmanager/templates/模板：
alertmanager.yml.j2tasks:
---
- name:
apply alertmanager config
template:
src:
alertmanager.yml.j2
dest:
/tmp/alertmanager.ymlPlaybook
playbooks/alertmanager.yml
---
- hosts:
k8s_master
roles:
- alertmanager五、Grafana
功能：
- Dashboard
- LDAP登录
- Prometheus datasource
- Loki datasource
Helm安装
helm install grafana \
grafana/grafana \
-n monitoringGrafana LDAP
配置：
文件：
grafana.ini
[auth.ldap]
enabled = true
config_file = /etc/grafana/ldap.tomlldap.toml
[[servers]]
host = "openldap"
port = 389
bind_dn = "cn=admin,dc=company,dc=local"
search_base_dns = [
"ou=users,dc=company,dc=local"
]Grafana Role
---
- name:
deploy grafana
command:
helm install grafana \
grafana/grafana \
-n monitoring六、Loki日志平台
架构：
Container
|
Promtail
|
Loki
|
GrafanaHelm安装
添加：
helm repo add grafana \
https://grafana.github.io/helm-charts安装：
helm install loki \
grafana/loki-stack \
-n monitoringLoki配置
values.yaml:
loki:
persistence:
enabled:
true
size:
200Gi
promtail:
enabled:
truePromtail采集
采集：
/var/log/*
/var/lib/docker/containers/*七、ELK
架构：
Application
|
Filebeat
|
Logstash
|
Elasticsearch
|
KibanaDocker Compose
目录：
mkdir -p docker-compose/elkdocker-compose.yml
services:
elasticsearch:
image:
elasticsearch:8.15.0
environment:
discovery.type:
single-node
xpack.security.enabled:
false
volumes:
- /data/elasticsearch:/usr/share/elasticsearch/data
ports:
- "9200:9200"
logstash:
image:
logstash:8.15.0
ports:
- "5044:5044"
kibana:
image:
kibana:8.15.0
ports:
- "5601:5601"Filebeat配置
filebeat.yml
filebeat.inputs:
- type:
container
paths:
- /var/lib/docker/containers/*/*.log
output.logstash:
hosts:
- logstash:5044ELK Role
目录：
roles/elktasks:
---
- name:
deploy elk
community.docker.docker_compose_v2:
project_src:
/opt/elkELK Playbook
---
- hosts:
elk
roles:
- elk八、Thanos
作用：
解决：
Prometheus:
单节点
数据短期Thanos:
多Prometheus
+
长期存储
+
全局查询架构：
Prometheus-1
|
Thanos Sidecar
|
Object Storage
(MinIO)
|
Thanos Query
|
GrafanaThanos组件
thanos-sidecar
thanos-query
thanos-store
thanos-compactorHelm安装
helm repo add bitnami \
https://charts.bitnami.com/bitnami
helm install thanos \
bitnami/thanos \
-n monitoringThanos对象存储
配置：
objstore.yml
type:
s3
config:
bucket:
thanos
endpoint:
minio.company.local:9000
access_key:
admin
secret_key:
password九、统一监控拓扑
最终：
Grafana
|
+---------------+---------------+
| |
Prometheus Loki
| |
| Promtail
|
Alertmanager
|
|
Thanos
|
MinIO
ELK:
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana十、部署顺序
1. Prometheus
ansible-playbook \
playbooks/prometheus.yml2. Alertmanager
ansible-playbook \
playbooks/alertmanager.yml3. Grafana
ansible-playbook \
playbooks/grafana.yml4. Loki
ansible-playbook \
playbooks/loki.yml5. ELK
ansible-playbook \
playbooks/elk.yml6. Thanos
ansible-playbook \
playbooks/thanos.yml十一、监控覆盖范围
部署完成后：
| 对象 | 组件 |
|---|---|
| 服务器CPU | Node Exporter |
| 服务器内存 | Node Exporter |
| 磁盘 | Node Exporter |
| Docker | cAdvisor |
| Kubernetes | kube-state-metrics |
| Pod | Prometheus |
| 应用 | Exporter |
| 日志 | Loki |
| 审计日志 | ELK |
| 长期指标 | Thanos |
| 告警 | Alertmanager |
Part 7：Backup备份层
本部分建设企业级备份体系：
目标：
- Kubernetes 集群备份
- PVC 数据备份
- MySQL/PostgreSQL/MongoDB备份
- MinIO/S3对象存储备份
- 文件系统备份
- 加密备份
- 定时备份
- 灾难恢复
Part 7 架构
Backup Platform
|
+---------------+----------------+
| |
Velero Restic
| |
| |
Kubernetes Resources File System Backup
|
|
+-----+------+
| |
PV PVC
|
|
Object Storage
MinIO / S3
|
|
Long Term Storage一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── velero.yml
├── restic.yml
├── database-backup.yml
├── minio-backup.yml
└── restore.yml
roles/
├── velero/
├── restic/
├── database-backup/
├── minio-backup/
└── restore/二、备份规划
企业环境建议：
| 系统 | 备份方式 |
|---|---|
| Kubernetes资源 | Velero |
| PVC数据 | Velero + Restic |
| MySQL | mysqldump |
| PostgreSQL | pg_dump |
| MongoDB | mongodump |
| Redis | RDB |
| MinIO | mc mirror |
| 文件服务器 | Restic |
| GitLab | 官方backup |
| Jira | 数据库+附件 |
| Confluence | 数据库+附件 |
三、Velero Kubernetes备份
1. 安装客户端
wget \
https://github.com/vmware-tanzu/velero/releases/download/v1.15.0/velero-v1.15.0-linux-amd64.tar.gz
tar xf velero*.tar.gz
cp velero*/velero /usr/local/bin/四、MinIO作为备份仓库
架构：
Kubernetes
|
Velero
|
S3 API
|
MinIO
|
/backup/velero创建bucket：
mc alias set minio \
http://minio.company.local:9000 \
admin password
mc mb minio/k8s-backup五、Velero安装
安装：
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.10.0 \
--bucket k8s-backup \
--secret-file ./credentials \
--backup-location-config \
region=minio,\
s3ForcePathStyle=true,\
s3Url=http://minio.company.local:9000 \
--use-node-agentcredentials
文件：
credentials
[default]
aws_access_key_id=admin
aws_secret_access_key=password六、Velero Ansible Role
目录：
roles/velero/tasksmain.yml
---
- name:
create backup namespace
command:
kubectl create namespace velero
ignore_errors:
yes
- name:
install velero
shell:
cmd: |
velero install \
--provider aws \
--bucket k8s-backup \
--use-node-agent \
--secret-file /tmp/credentials \
--namespace veleroVelero Playbook
---
- hosts:
k8s_master
become:
yes
roles:
- velero七、创建备份任务
全量备份
velero backup create cluster-backup查看：
velero backup get结果：
NAME
cluster-backup
STATUS
Completed八、定时备份
每天凌晨2点：
velero schedule create daily-backup \
--schedule="0 2 * * *"查看：
velero schedule get九、恢复
模拟灾难：
删除namespace：
kubectl delete ns production恢复：
velero restore create \
--from-backup cluster-backup查看：
velero restore get十、Restic文件备份
适合：
/data
/home
/upload
附件
日志
配置文件安装
Ubuntu:
apt install resticCentOS:
yum install resticRestic仓库
创建：
mkdir /backup/restic初始化：
restic \
-r /backup/restic \
init设置密码：
export RESTIC_PASSWORD=password十一、Restic备份脚本
目录：
roles/restic/files/文件：
backup.sh内容：
#!/bin/bash
export RESTIC_PASSWORD=password
RESTIC_REPOSITORY=/backup/restic
restic backup \
/data \
--exclude=/data/tmp
restic forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 12 \
--prune授权：
chmod +x backup.sh十二、Systemd定时
文件：
/etc/systemd/system/restic-backup.service
[Unit]
Description=Restic Backup
[Service]
Type=oneshot
ExecStart=/usr/local/bin/restic-backup.shTimer:
restic-backup.timer
[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
[Install]
WantedBy=timers.target启动：
systemctl enable --now restic-backup.timer十三、数据库备份
PostgreSQL
脚本：
#!/bin/bash
pg_dumpall \
-h postgres \
-U postgres \
> /backup/postgres.sqlMySQL
mysqldump \
-u root \
-p \
--all-databases \
> mysql.sqlMongoDB
mongodump \
--out /backup/mongo十四、数据库Ansible
目录：
roles/database-backuptasks:
---
- name:
install backup tools
apt:
name:
- postgresql-client
- mysql-client
state:
present
- name:
deploy backup script
copy:
src:
backup.sh
dest:
/usr/local/bin/db-backup.sh
mode:
0755十五、MinIO备份
使用：
mc mirror脚本：
#!/bin/bash
mc alias set source \
http://minio01:9000 \
admin password
mc alias set backup \
http://minio02:9000 \
admin password
mc mirror \
--overwrite \
source/allin \
backup/allin十六、GitLab备份
GitLab自带：
gitlab-backup create产生：
/var/opt/gitlab/backups备份：
restic backup \
/var/opt/gitlab/backups十七、Jira / Confluence备份
数据：
PostgreSQL
+
Attachment目录备份：
restic backup \
/data/jira
/data/confluence十八、灾难恢复流程
完整恢复顺序：
1. 恢复基础服务器
|
2. 恢复Kubernetes
|
3. 安装ArgoCD
|
4. Velero恢复资源
|
5. Restic恢复数据
|
6. 恢复数据库
|
7. 恢复应用十九、Backup Playbook汇总
velero.yml
---
- hosts:
k8s_master
roles:
- velerorestic.yml
---
- hosts:
storage
roles:
- resticdatabase-backup.yml
---
- hosts:
database
roles:
- database-backup二十、企业备份策略
推荐：
日备份
每天02:00
Kubernetes
数据库
文件周备份
每周日
完整快照月归档
保留12个月Part 8：安全与身份认证层
本部分建设企业级安全体系：
目标：
- 统一身份认证（SSO）
- LDAP 用户中心
- OAuth2/OIDC
- Secret 密钥管理
- Kubernetes Secret 自动注入
- VPN 身份认证
- TLS 自动化
- 安全审计
- 主机入侵检测
Part 8 架构
Security Platform
|
+---------------------+---------------------+
| | |
OpenLDAP Keycloak Vault
| | |
| | |
User Center SSO/OIDC Secret Store
| | |
| | |
+----------+----------+---------------------+
|
Applications
GitLab
Jenkins
Grafana
Harbor
Jira
Confluence
Nextcloud
Kubernetes
|
Cert-manager
|
TLS
|
Wazuh
|
Security Audit一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── ldap.yml
├── keycloak.yml
├── vault.yml
├── cert-manager.yml
├── wazuh.yml
├── security-scan.yml
└── mfa.yml
roles/
├── openldap/
├── keycloak/
├── vault/
├── cert-manager/
├── wazuh/
├── trivy/
└── falco/二、OpenLDAP统一身份中心
架构
LDAP
|
+--------------+--------------+
| | |
GitLab Jenkins Harbor
|
Grafana
|
Jira
|
Confluence三、OpenLDAP Docker Compose
目录：
mkdir -p docker-compose/openldap文件：
docker-compose/openldap/docker-compose.yml
services:
openldap:
image:
osixia/openldap:latest
container_name:
openldap
restart:
always
environment:
LDAP_ORGANISATION:
Company
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
Admin@123456
ports:
- "389:389"
- "636:636"
volumes:
- /data/openldap/data:/var/lib/ldap
- /data/openldap/config:/etc/ldap/slapd.dLDAP目录设计
推荐：
dc=company,dc=local
├── ou=users
│
├── user001
├── user002
├── ou=groups
│
├── devops
├── developers
└── administratorsLDAP初始化LDIF
文件：
users.ldif
dn: ou=users,dc=company,dc=local
objectClass: organizationalUnit
ou: users
dn: ou=groups,dc=company,dc=local
objectClass: organizationalUnit
ou: groups导入：
ldapadd \
-x \
-D "cn=admin,dc=company,dc=local" \
-W \
-f users.ldif四、Keycloak SSO
作用：
一次登录
↓
所有系统认证
GitLab
Jenkins
Grafana
Harbor
KubernetesKeycloak架构
LDAP
|
Keycloak
|
+---------+---------+
| | |
GitLab Grafana JenkinsKeycloak Compose
目录：
mkdir docker-compose/keycloakdocker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
keycloak
POSTGRES_USER:
keycloak
POSTGRES_PASSWORD:
Keycloak@123456
keycloak:
image:
quay.io/keycloak/keycloak:latest
command:
start-dev
ports:
- "8080:8080"
environment:
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME:
keycloak
KC_DB_PASSWORD:
Keycloak@123456
KEYCLOAK_ADMIN:
admin
KEYCLOAK_ADMIN_PASSWORD:
admin123Keycloak LDAP Federation
配置：
Realm
↓
User Federation
↓
LDAP参数：
Connection URL:
ldap://openldap:389
Bind DN:
cn=admin,dc=company,dc=local
Users DN:
ou=users,dc=company,dc=local五、Vault 密钥管理
作用：
替代：
明文密码
yaml secret
.env文件管理：
数据库密码
API Token
SSH Key
TLS证书Vault架构
Application
|
Vault Agent
|
Vault Server
|
Encrypted StorageVault Compose
services:
vault:
image:
hashicorp/vault:latest
container_name:
vault
ports:
- "8200:8200"
cap_add:
- IPC_LOCK
environment:
VAULT_DEV_ROOT_TOKEN_ID:
root
command:
server -devVault初始化
进入：
vault login root开启KV：
vault secrets enable kv保存密码：
vault kv put secret/database \
password=Pass123读取：
vault kv get secret/database六、Kubernetes Vault Integration
安装：
helm repo add hashicorp \
https://helm.releases.hashicorp.com
helm install vault \
hashicorp/vault \
-n vault \
--create-namespaceSecret自动注入
Pod:
annotations:
vault.hashicorp.com/agent-inject:
"true"
vault.hashicorp.com/role:
app应用启动自动获得：
/vault/secrets/database七、Cert-manager TLS
用于：
HTTPS
Ingress TLS
内部CA
自动续期安装：
helm repo add jetstack \
https://charts.jetstack.io
helm install cert-manager \
jetstack/cert-manager \
-n cert-manager \
--create-namespace \
--set installCRDs=trueClusterIssuer
文件：
letsencrypt.yamlapiVersion:
cert-manager.io/v1
kind:
ClusterIssuer
metadata:
name:
letsencrypt
spec:
acme:
email:
admin@company.com
server:
https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name:
letsencrypt-key
solvers:
- http01:
ingress:八、Wazuh安全审计
功能：
服务器监控
文件完整性
漏洞检测
日志分析
入侵检测Wazuh架构
Agent
|
Manager
|
Indexer
|
DashboardDocker Compose
services:
wazuh-manager:
image:
wazuh/wazuh-manager
ports:
- "1514:1514"
- "55000:55000"
wazuh-indexer:
image:
wazuh/wazuh-indexer
wazuh-dashboard:
image:
wazuh/wazuh-dashboard
ports:
- "5601:5601"九、Trivy漏洞扫描
扫描：
Docker Image
Kubernetes
Filesystem安装：
apt install trivy扫描镜像：
trivy image nginx:latestHarbor集成
开启：
Harbor
↓
Scanner
↓
Trivy扫描：
Critical
High
Medium
Low十、Falco运行时安全
Kubernetes Runtime Security
安装：
helm repo add falcosecurity \
https://falcosecurity.github.io/charts
helm install falco \
falcosecurity/falco \
-n falco检测：
Shell进入容器
异常网络
异常文件修改
权限提升十一、MFA多因素认证
推荐：
Keycloak
+
Google Authenticator
+
Duo流程：
User Login
↓
Password
↓
OTP
↓
Access十二、安全Playbook
ldap.yml
---
- hosts:
ldap
roles:
- openldapkeycloak.yml
---
- hosts:
keycloak
roles:
- keycloakvault.yml
---
- hosts:
vault
roles:
- vaultwazuh.yml
---
- hosts:
security
roles:
- wazuh十三、安全部署顺序
# 1 LDAP
ansible-playbook ldap.yml
# 2 Keycloak
ansible-playbook keycloak.yml
# 3 Vault
ansible-playbook vault.yml
# 4 Cert-manager
ansible-playbook cert-manager.yml
# 5 Wazuh
ansible-playbook wazuh.yml
# 6 Trivy
ansible-playbook security-scan.yml十四、最终认证拓扑
完成后：
Keycloak
|
+-----------+------------+
| | |
GitLab Jenkins Grafana
|
Harbor
|
Kubernetes
LDAP
|
User Database
Vault
|
Secrets
Cert-manager
|
TLS
Wazuh
|
Security AuditPart 8：安全与身份认证层
Part 8 Security 安全层（Enterprise Production Edition）
目录扩展
enterprise-platform-ansible/
roles/
├── openldap-ha
├── keycloak-ha
├── vault-ha
├── kubernetes-security
├── rbac
├── network-policy
├── pod-security
├── image-security
├── os-hardening
├── audit
└── compliance
playbooks/
├── ldap-ha.yml
├── keycloak-ha.yml
├── vault-ha.yml
├── k8s-security.yml
├── os-hardening.yml
└── compliance.yml一、LDAP 高可用
生产环境：
Keepalived VIP
|
+----------------+
| LDAP VIP |
+----------------+
| |
LDAP01 LDAP02
| |
ReplicationOpenLDAP HA Compose
services:
ldap01:
image: osixia/openldap
hostname:
ldap01.company.local
environment:
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
password
ldap02:
image:
osixia/openldap
hostname:
ldap02.company.local
environment:
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
passwordLDAP同步
启用：
syncreplldap01:
olcSyncrepl:
rid=001
provider=ldap://ldap02.company.local
searchbase="dc=company,dc=local"
binddn="cn=admin,dc=company,dc=local"ldap02:
olcSyncrepl:
rid=002
provider=ldap://ldap01.company.local二、Keycloak生产HA
架构：
HAProxy
|
+---------+---------+
| |
Keycloak01 Keycloak02
| |
+---------+---------+
|
PostgreSQL HAKeycloak Compose
services:
keycloak:
image:
quay.io/keycloak/keycloak
replicas:
2
command:
start
environment:
KC_PROXY:
edge
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloakKeycloak安全配置
开启：
Realm Settings
↓
Security Defenses
Brute Force Detection
ON
Password Policy
ON
OTP Required
ON三、Vault HA
生产禁止：
-dev模式采用：
Vault Raft Cluster架构：
HAProxy
|
+----------+----------+
| | |
Vault01 Vault02 Vault03
RaftVault配置
文件：
vault.hcl
storage "raft" {
path="/vault/data"
}
listener "tcp" {
address="0.0.0.0:8200"
tls_disable=false
}
cluster_addr="https://vault01:8201"
api_addr="https://vault01:8200"
ui=trueVault初始化
第一台：
vault operator init生成：
Unseal Key
Root Token加入节点：
vault operator raft join \
https://vault01:8200查看集群：
vault operator raft list-peers输出：
vault01 leader
vault02 follower
vault03 follower四、Kubernetes安全加固
1. RBAC
目录：
roles/rbac创建管理员：
apiVersion:
rbac.authorization.k8s.io/v1
kind:
ClusterRoleBinding
metadata:
name:
platform-admin
subjects:
- kind:
User
name:
admin
roleRef:
kind:
ClusterRole
name:
cluster-admin
apiGroup:
rbac.authorization.k8s.io五、ServiceAccount安全
禁止默认账户：
automountServiceAccountToken:
false示例：
apiVersion:
v1
kind:
ServiceAccount
metadata:
name:
app
automountServiceAccountToken:
false六、NetworkPolicy
实现：
Pod隔离
东西向流量控制
微服务安全默认拒绝：
apiVersion:
networking.k8s.io/v1
kind:
NetworkPolicy
metadata:
name:
deny-all
spec:
podSelector:{}
policyTypes:
- Ingress
- Egress允许web访问api：
kind:
NetworkPolicy
spec:
podSelector:
matchLabels:
app:
api
ingress:
- from:
- podSelector:
matchLabels:
app:
web七、Pod Security Standard
启用：
Namespace:
kubectl label ns production \
pod-security.kubernetes.io/enforce=restricted级别：
privileged
baseline
restricted生产推荐：
restricted八、镜像安全
Harbor + Trivy
流程：
Developer
|
GitLab CI
|
Build Image
|
Harbor
|
Trivy Scan
|
PASS
|
Deploy KubernetesGitLab CI示例：
security_scan:
stage:
test
script:
- trivy image \
$IMAGE
allow_failure:
false九、Kubernetes Secret加密
默认：
Secret Base64生产：
EncryptionConfiguration文件：
encryption.yaml
apiVersion:
apiserver.config.k8s.io/v1
kind:
EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name:
key1
secret:
xxx
- identity:{}修改：
kube-apiserver增加：
--encryption-provider-config=/etc/kubernetes/encryption.yaml十、SSH安全加固
Ansible:
roles/os-hardening禁用密码登录：
- name:
disable ssh password
lineinfile:
path:
/etc/ssh/sshd_config
regexp:
PasswordAuthentication
line:
PasswordAuthentication no禁止root登录：
PermitRootLogin no十一、Linux安全基线
安装：
apt install auditd
apt install fail2ban
apt install apparmor十二、Auditd
监控：
/etc/passwd
/etc/shadow
/etc/sudoers
ssh配置规则：
/etc/audit/rules.d/security.rules
-w /etc/passwd -p wa
-w /etc/shadow -p wa
-w /etc/sudoers -p wa十三、Fail2ban
保护：
SSH
Nginx
VPN配置：
jail.local
[sshd]
enabled=true
maxretry=5
bantime=3600十四、漏洞扫描
Trivy
扫描：
镜像
trivy image nginx:latest文件系统
trivy fs /Kubernetes
trivy k8s cluster十五、CIS Kubernetes Benchmark
工具：
kube-bench安装：
curl -L \
https://github.com/aquasecurity/kube-bench/releases \运行：
kube-bench run检查：
PASS
WARN
FAIL十六、Falco运行时安全
检测：
容器逃逸
异常Shell
敏感文件读取
权限提升规则：
- rule:
Terminal shell in container
condition:
container.id != host
output:
Shell executed十七、安全监控接入
统一进入：
Grafana
+
Loki
+
Prometheus
+
Wazuh监控：
| 项目 | 组件 |
|---|---|
| 登录异常 | Wazuh |
| 漏洞 | Trivy |
| 容器异常 | Falco |
| 系统日志 | Loki |
| 指标 | Prometheus |
| 认证 | Keycloak |
| 密码 | Vault |
十八、安全部署总流程
# LDAP
ansible-playbook ldap-ha.yml
# SSO
ansible-playbook keycloak-ha.yml
# Secret
ansible-playbook vault-ha.yml
# Kubernetes安全
ansible-playbook k8s-security.yml
# OS加固
ansible-playbook os-hardening.yml
# 合规扫描
ansible-playbook compliance.ymlPart 9：Network & High Availability 网络高可用层
本部分建设企业生产级网络基础设施：
目标：
- 企业 DNS 服务
- 内外网代理
- 四层/七层负载均衡
- VIP 高可用
- VPN 接入
- 防火墙策略
- FTP/SFTP 文件交换
- Kubernetes Ingress 高可用
- 网络自动化部署
一、Network & HA 总体架构
Internet
|
Firewall
|
+---------------+
| HAProxy VIP |
+---------------+
/ \
/ \
HAProxy01 HAProxy02
| |
+----------------+
|
Kubernetes
|
+-------------+-------------+
| |
Ingress Services
|
Internal Network
DNS
|
Bind9
Proxy
|
Squid/Nginx
VPN
|
OpenVPN
File Transfer
|
SFTP/FTP二、目录增加
enterprise-platform-ansible/
roles/
├── bind9/
├── dnsmasq/
├── nginx-proxy/
├── squid/
├── haproxy/
├── keepalived/
├── openvpn/
├── wireguard/
├── vsftpd/
├── sftp/
├── firewall/
└── network-check/
playbooks/
├── dns.yml
├── proxy.yml
├── haproxy.yml
├── keepalived.yml
├── vpn.yml
├── ftp.yml
└── firewall.yml三、DNS 服务
企业推荐：
主DNS:
Bind9
备用DNS:
Bind9
内部DNS:
CoreDNSDNS 架构
Client
|
DNS VIP
|
+---------+---------+
| |
DNS01 DNS02
Zone Replication四、Bind9 Docker Compose
目录：
docker-compose/bind9docker-compose.yml
services:
bind9:
image:
internetsystemsconsortium/bind9:latest
container_name:
bind9
restart:
always
ports:
- "53:53/tcp"
- "53:53/udp"
volumes:
- /data/bind9/config:/etc/bind
- /data/bind9/cache:/var/cache/bind五、DNS Zone
文件：
company.local.zone
$TTL 86400
@ IN SOA dns01.company.local.
admin.company.local.
(
2026090901
3600
1800
604800
86400
)
@ IN NS dns01.company.local.
@ IN NS dns02.company.local.
dns01 IN A 10.50.0.10
dns02 IN A 10.50.0.11
gitlab IN A 10.50.0.20
jenkins IN A 10.50.0.21
harbor IN A 10.50.0.22
grafana IN A 10.50.0.23六、DNS Ansible Role
目录：
roles/bind9/tasks/main.yml
---
- name:
install bind9
package:
name:
- bind9
state:
present
- name:
start bind9
service:
name:
bind9
state:
started
enabled:
yes七、Proxy代理层
包含：
正向代理
Squid
反向代理
Nginx
八、Squid Proxy
用途：
- 软件包下载
- Docker镜像代理
- 内网出口控制
架构：
Client
|
Squid
|
InternetDocker Compose
services:
squid:
image:
sameersbn/squid
ports:
- "3128:3128"
volumes:
- /data/squid:/etc/squidSquid ACL
配置：
acl company src 10.50.0.0/16
http_access allow company
http_access deny all九、Nginx Reverse Proxy
用于：
https://gitlab.company.com
|
Nginx
|
GitLab Container配置：
server {
listen 443 ssl;
server_name gitlab.company.com;
location / {
proxy_pass http://gitlab:80;
proxy_set_header Host $host;
}
}十、HAProxy
用途：
- Kubernetes API
- TCP Load Balance
- HTTP Load Balance
架构：
VIP
|
HAProxy Cluster
/ \
node01 node02HAProxy Compose
services:
haproxy:
image:
haproxy:3
ports:
- "80:80"
- "443:443"
- "6443:6443"
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfgHAProxy配置
文件：
haproxy.cfgKubernetes API
frontend kubernetes-api
bind *:6443
mode tcp
default_backend k8s-master
backend k8s-master
mode tcp
balance roundrobin
server master01 10.50.0.101:6443 check
server master02 10.50.0.102:6443 check
server master03 10.50.0.103:6443 check十一、Keepalived VIP
作用：
提供：
HAProxy VIP
DNS VIP
Nginx VIP架构
VIP
|
+-------+-------+
| |
MASTER BACKUP
Keepalived KeepalivedKeepalived配置
master:
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass password
}
virtual_ipaddress {
10.50.0.100
}
}backup:
priority 90十二、OpenVPN
功能：
- 员工远程接入
- LDAP认证
- MFA
架构：
User
|
VPN
|
OpenVPN
|
LDAP
|
NetworkOpenVPN Compose
services:
openvpn:
image:
kylemanna/openvpn
ports:
- "1194:1194/udp"
cap_add:
- NET_ADMINLDAP认证
配置：
plugin /usr/lib/openvpn/openvpn-auth-ldap.so
<LDAP>
URL ldap://ldap.company.local
BindDN cn=admin,dc=company,dc=local
BaseDN ou=users,dc=company,dc=local
</LDAP>十三、WireGuard
轻量VPN：
Client
|
WireGuard
|
Internal Network配置：
server:
[Interface]
Address=10.8.0.1/24
ListenPort=51820
PrivateKey=xxxclient:
[Peer]
Endpoint=vpn.company.com:51820
AllowedIPs=10.50.0.0/16十四、FTP / SFTP
SFTP 推荐
替代FTP：
- 加密
- SSH协议
- 易审计
SFTP Docker
services:
sftp:
image:
atmoz/sftp
ports:
- "2222:22"
volumes:
- /data/upload:/home/user/upload
command:
user:password:::upload十五、vsftpd FTP
Compose：
services:
ftp:
image:
fauria/vsftpd
ports:
- "21:21"
environment:
FTP_USER:
ftpuser
FTP_PASS:
password十六、防火墙
推荐：
Ubuntu:
nftables
CentOS:
firewalldnftables规则
文件：
/etc/nftables.conf示例：
table inet filter {
chain input {
type filter hook input priority filter;
policy drop;
ct state established,related accept
iif lo accept
tcp dport 22 accept
tcp dport 443 accept
tcp dport 6443 accept
}
}十七、Kubernetes Ingress HA
生产：
Internet
|
HAProxy
|
Ingress Controller
|
ServiceIngress-nginx:
replicas: 2配置：
controller:
replicaCount:
2
service:
type:
LoadBalancer十八、网络检测
Ansible：
roles/network-checktasks:
---
- name:
check dns
command:
nslookup kubernetes.default
- name:
check port
wait_for:
host:
"{{ inventory_hostname }}"
port:
443十九、统一部署入口
DNS
ansible-playbook dns.ymlProxy
ansible-playbook proxy.ymlHAProxy
ansible-playbook haproxy.ymlKeepalived
ansible-playbook keepalived.ymlVPN
ansible-playbook vpn.ymlFirewall
ansible-playbook firewall.yml二十、生产推荐网络规划
示例：
| 用途 | 网段 |
|---|---|
| Management | 10.50.0.0/24 |
| Kubernetes | 10.50.10.0/24 |
| Storage | 10.50.20.0/24 |
| Database | 10.50.30.0/24 |
| VPN | 10.50.40.0/24 |
| Backup | 10.50.50.0/24 |
Part 10：Storage & Data Platform 存储数据层
本部分建设企业级存储与数据平台
目标：
- 分布式块存储
- 分布式文件存储
- 对象存储
- Kubernetes CSI存储
- 数据库高可用
- 消息队列
- 大数据存储
- 备份存储
- AI/LLM数据存储
Part 10 总体架构
Storage Platform
|
+----------------------+----------------------+
| |
Distributed Storage Database Platform
| |
| |
+------+-------+ +---------+---------+
| | | |
Ceph SeaweedFS PostgreSQL HA MySQL HA
| | |
| | |
RBD S3 Gateway Redis Cluster
|
|
CephFS
|
|
Kubernetes CSI
|
|
+------+---------+
| |
StatefulSet PVC一、目录增加
enterprise-platform-ansible/
roles/
├── ceph/
├── minio/
├── seaweedfs/
├── nfs/
├── iscsi/
├── longhorn/
├── postgresql-ha/
├── mysql-ha/
├── mongodb/
├── redis/
└── kafka/
playbooks/
├── ceph.yml
├── minio.yml
├── seaweedfs.yml
├── database.yml
├── kafka.yml
└── storage-class.yml二、存储规划
企业推荐：
| 类型 | 组件 | 用途 |
|---|---|---|
| Block Storage | Ceph RBD | 数据库/K8s PVC |
| File Storage | CephFS | 共享文件 |
| Object Storage | MinIO | AI/备份/S3 |
| Large File | SeaweedFS | 视频/LLM数据 |
| NAS | NFS | 简单共享 |
| Cloud Native | Longhorn | K8s存储 |
| Archive | Restic/MinIO | 备份 |
三、Ceph分布式存储
架构
Ceph Cluster
MON
+---------+---------+
| |
MON01 MON02
|
OSD
+--------+--------+
| | |
OSD01 OSD02 OSD03
|
RADOS
|
+--------+---------+
| |
RBD CephFS四、Ceph节点规划
示例：
ceph01
MON
MGR
OSD
ceph02
MON
OSD
ceph03
MON
OSD五、Ceph Ansible安装
目录：
roles/ceph/tasksmain.yml
---
- name:
install cephadm
shell:
curl --silent \
--remote-name \
https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm
- name:
install ceph
shell:
./cephadm add-repo \
--release reef
- name:
bootstrap ceph
shell:
cephadm bootstrap \
--mon-ip {{ ceph_mon_ip }}六、Ceph Pool
创建RBD：
ceph osd pool create kubernetes
rbd pool init kubernetes创建CephFS：
ceph fs volume create cephfs七、Kubernetes Ceph CSI
安装：
helm repo add ceph-csi \
https://ceph.github.io/csi-charts安装：
helm install ceph-csi \
ceph-csi/ceph-csi-rbd \
-n kube-systemStorageClass
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
ceph-rbd
provisioner:
rbd.csi.ceph.com
parameters:
clusterID:
ceph
pool:
kubernetes八、MinIO对象存储
用途：
- AI模型
- LLM知识库
- Backup
- Thanos
- Velero
- Harbor
MinIO架构
MinIO
+------+------+
| |
Node01 Node02
| |
Disk Pool
|
Erasure CodingMinIO Docker Compose
services:
minio:
image:
minio/minio
command:
server \
/data{1...4} \
--console-address ":9001"
ports:
- "9000:9000"
- "9001:9001"
volumes:
- /data/minio1:/data1
- /data/minio2:/data2
- /data/minio3:/data3
- /data/minio4:/data4
environment:
MINIO_ROOT_USER:
admin
MINIO_ROOT_PASSWORD:
passwordMinIO Bucket规划
minio
├── backup
│
├── velero
├── thanos
├── llm
├── model
├── file-storage
└── logsMinIO Ansible
---
- name:
deploy minio
community.docker.docker_compose_v2:
project_src:
/opt/minio九、SeaweedFS
适合：
- 海量小文件
- AI数据
- 图片
- 文档
SeaweedFS架构
SeaweedFS
|
Master
|
+---------+---------+
| |
Volume01 Volume02
|
Filer
|
S3 Gateway十、SeaweedFS部署
Master
weed master \
-ip=seaweed01 \
-port=9333Volume
weed volume \
-server=seaweed01:9333 \
-dir=/data/volume1 \
-port=8080Filer
weed filer \
-master=seaweed01:9333S3 Gateway
weed s3 \
-filer=seaweed01:8888SeaweedFS Systemd
[Service]
ExecStart=/usr/local/bin/weed filer \
-master=seaweed01:9333
Restart=always十一、NFS共享存储
适合：
- Jenkins workspace
- 软件包
- 小规模共享
服务端：
apt install nfs-serverexports:
/data/share
10.50.0.0/16(rw,sync,no_root_squash)客户端：
mount -t nfs \
10.50.0.10:/data/share \
/mnt/share十二、iSCSI块存储
用途：
- VMware
- Proxmox
- Database
安装：
apt install targetcli-fb创建Target：
targetcli十三、Longhorn Kubernetes Storage
适合：
- Kubernetes轻量存储
安装：
helm repo add longhorn \
https://charts.longhorn.io
helm install longhorn \
longhorn/longhorn \
-n longhorn-system十四、PostgreSQL HA
架构：
HAProxy
|
+---------+---------+
| |
PostgreSQL01 PostgreSQL02
|
Patroni
|
etcdPatroni部署
组件：
PostgreSQL
+
Patroni
+
etcd
+
HAProxyPostgreSQL Docker
services:
postgres:
image:
postgres:16
environment:
POSTGRES_PASSWORD:
password
volumes:
- /data/postgres:/var/lib/postgresql/data十五、MySQL HA
推荐：
MySQL InnoDB Cluster
+
MySQL Router架构：
Router
|
+-----------+-----------+
| | |
MySQL01 MySQL02 MySQL03
Group Replication十六、MongoDB ReplicaSet
架构：
MongoDB01
MongoDB02
MongoDB03
|
ReplicaSet启动：
mongod \
--replSet rs0初始化：
rs.initiate()十七、Redis Cluster
架构：
Redis01
Redis02
Redis03
+
Sentinel
+
ReplicationRedis Compose
services:
redis:
image:
redis:7
command:
redis-server \
--appendonly yes十八、Kafka消息平台
用途：
- 日志
- AI Pipeline
- 数据流
Kafka架构
Producer
|
Kafka Broker
|
ConsumerKafka KRaft模式
无需Zookeeper
Compose:
services:
kafka:
image:
bitnami/kafka
environment:
KAFKA_ENABLE_KRAFT:
yes
KAFKA_CFG_NODE_ID:
1十九、数据库备份集成
统一进入：
Restic
+
MinIO
+
Velero流程：
Database
|
Dump
|
Restic
|
MinIO
|
Archive二十、Storage部署入口
Ceph
ansible-playbook ceph.ymlMinIO
ansible-playbook minio.ymlSeaweedFS
ansible-playbook seaweedfs.ymlDatabase
ansible-playbook database.ymlKafka
ansible-playbook kafka.yml二十一、最终企业存储拓扑
Application
|
Kubernetes
|
+----------+----------+
| |
CSI Storage Object Storage
| |
Ceph MinIO
|
CephFS/RBD
|
Database
PostgreSQL
MySQL
MongoDB
Redis
Kafka
|
Backup
Velero
ResticPart 11：CI/CD & GitOps 持续交付层
本部分建设企业级 DevOps 自动化交付体系。
目标：
- Git代码管理
- 自动构建
- 自动测试
- 镜像构建
- 镜像安全扫描
- 制品管理
- Kubernetes自动发布
- GitOps持续部署
- 灰度发布
- 蓝绿发布
- DevSecOps安全流水线
Part 11 总体架构
Developer
|
Git Push
|
+----------------+
| Git Repository |
+----------------+
|
+------------------+------------------+
| |
GitLab CI Jenkins
| |
+------------------+------------------+
|
Build
|
Docker Image Build
|
Harbor
|
+--------------+--------------+
| |
Trivy SonarQube
Vulnerability Scan Code Quality
|
|
Security Gate
|
GitOps Repository
|
ArgoCD
|
Kubernetes Cluster
|
Argo Rollouts
|
Canary / Blue Green一、目录增加
enterprise-platform-ansible/
roles/
├── gitlab-runner/
├── jenkins-agent/
├── sonarqube/
├── harbor/
├── argocd/
├── argocd-image-updater/
├── argo-rollouts/
├── tekton/
├── helm/
├── kustomize/
├── trivy-ci/
└── sbom/
playbooks/
├── cicd.yml
├── gitlab-runner.yml
├── jenkins-agent.yml
├── sonarqube.yml
├── argocd.yml
├── tekton.yml
└── security-pipeline.yml二、DevOps平台组件
| 组件 | 用途 |
|---|---|
| GitLab | 代码仓库 |
| Jenkins | 流水线 |
| GitLab Runner | CI执行器 |
| Harbor | 镜像仓库 |
| SonarQube | 代码质量 |
| Trivy | 漏洞扫描 |
| Syft | SBOM |
| ArgoCD | GitOps |
| Argo Rollouts | 灰度发布 |
| Helm | 包管理 |
| Kustomize | 配置管理 |
| Tekton | 云原生CI |
三、GitLab CI/CD
Pipeline流程
.gitlab-ci.yml
|
|
Stage 1
Build
|
Stage 2
Test
|
Stage 3
Security
|
Stage 4
Docker Build
|
Stage 5
Push Harbor
|
Stage 6
Update GitOps四、GitLab Runner部署
Docker Compose
目录：
docker-compose/gitlab-runnerdocker-compose.yml
services:
gitlab-runner:
image:
gitlab/gitlab-runner:latest
container_name:
gitlab-runner
restart:
always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /data/gitlab-runner:/etc/gitlab-runner注册Runner
docker exec -it gitlab-runner \
gitlab-runner register输入：
GitLab URL:
https://gitlab.company.com
Token:
xxxxx
Executor:
docker
Image:
docker:latestRunner Ansible
roles/gitlab-runner/tasks/main.yml
---
- name:
deploy gitlab runner
community.docker.docker_compose_v2:
project_src:
/opt/gitlab-runner五、GitLab CI模板
项目：
.gitlab-ci.yml示例：
stages:
- build
- test
- security
- docker
build:
stage:
build
script:
- npm install
- npm run build
test:
stage:
test
script:
- npm test
security:
stage:
security
script:
- trivy fs .
docker:
stage:
docker
script:
- docker build -t harbor.company.com/app:v1 .
- docker push harbor.company.com/app:v1六、Jenkins Pipeline
适合：
- Java
- 大型项目
- 多阶段发布
Jenkinsfile
pipeline {
agent any
stages {
stage('Checkout'){
steps{
git 'https://git.company.com/app.git'
}
}
stage('Build'){
steps{
sh 'mvn clean package'
}
}
stage('Docker'){
steps{
sh '''
docker build \
-t harbor.company.com/app:${BUILD_NUMBER} .
'''
}
}
stage('Deploy'){
steps{
sh '''
kubectl apply -f deploy.yaml
'''
}
}
}
}七、Jenkins Kubernetes Agent
架构：
Jenkins
|
Kubernetes Plugin
|
Dynamic Pod
|
Build ContainerPod Template:
apiVersion:
v1
kind:
Pod
spec:
containers:
- name:
builder
image:
maven:3.9-jdk17八、Harbor镜像仓库
作用：
Docker Image
+
Security Scan
+
Replication
+
RBACHarbor架构
Developer
|
GitLab CI
|
Docker Build
|
Harbor
|
Trivy
|
KubernetesHarbor自动部署
helm repo add harbor \
https://helm.goharbor.io
helm install harbor \
harbor/harbor \
-n harbor九、SonarQube代码质量
检测：
- Bug
- Vulnerability
- Code Smell
- Coverage
Docker Compose
services:
sonarqube:
image:
sonarqube:lts
ports:
- "9000:9000"
environment:
SONAR_ES_BOOTSTRAP_CHECKS_DISABLE:
trueSonar Maven集成
<plugin>
<groupId>
org.sonarsource.scanner.maven
</groupId>
<artifactId>
sonar-maven-plugin
</artifactId>
</plugin>执行：
mvn sonar:sonar \
-Dsonar.host.url=http://sonar:9000十、Trivy DevSecOps
扫描阶段：
Code
|
Build
|
Image
|
Trivy
|
PASS
|
DeployGitLab CI：
trivy:
stage:
security
script:
- trivy image \
harbor.company.com/app:$CI_COMMIT_SHA十一、SBOM软件供应链安全
工具：
Syft
+
Grype生成SBOM：
syft nginx:latest \
-o cyclonedx-json输出：
sbom.json漏洞扫描：
grype sbom.json十二、ArgoCD GitOps
核心思想：
Git = Single Source Of Truth
Git修改
|
ArgoCD
|
KubernetesArgoCD安装
helm repo add argo \
https://argoproj.github.io/argo-helm
helm install argocd \
argo/argo-cd \
-n argocd \
--create-namespaceArgoCD目录结构
推荐：
gitops-repo/
├── applications/
├── charts/
├── environments/
│
├── dev/
├── test/
└── prod/十三、Application定义
文件：
app.yaml
apiVersion:
argoproj.io/v1alpha1
kind:
Application
metadata:
name:
app-prod
spec:
project:
default
source:
repoURL:
https://git.company.com/gitops.git
path:
apps/app
destination:
server:
https://kubernetes.default.svc
namespace:
production
syncPolicy:
automated:
prune:
true
selfHeal:
true十四、ArgoCD Image Updater
自动更新镜像：
Harbor
|
New Image
|
Image Updater
|
Git Commit
|
ArgoCD Sync安装：
helm install argocd-image-updater \
argo/argocd-image-updater \
-n argocd十五、Argo Rollouts 灰度发布
支持：
- Canary
- Blue Green
安装：
kubectl create namespace argo-rollouts
kubectl apply \
-f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yamlCanary示例
apiVersion:
argoproj.io/v1alpha1
kind:
Rollout
spec:
strategy:
canary:
steps:
- setWeight:
10
- pause:
duration:
5m
- setWeight:
50十六、Tekton Cloud Native CI
架构：
Git
|
Tekton Trigger
|
Pipeline
|
Task
|
Container Build安装：
kubectl apply \
-f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml十七、Helm规范
目录：
charts/
app/
├── Chart.yaml
├── values.yaml
└── templates/Chart.yaml
apiVersion:
v2
name:
app
version:
1.0.0values.yaml
image:
repository:
harbor.company.com/app
tag:
latest
replicas:
3十八、Kustomize环境管理
目录：
overlays/
├── dev/
├── test/
└── prod/prod/kustomization.yaml
resources:
- ../../base
patches:
- replica.yaml十九、完整企业流水线
最终：
Developer
|
GitLab
|
CI Pipeline
|
+----------------+
| Build |
| Test |
| Sonar |
| Trivy |
| SBOM |
+----------------+
|
|
Harbor Registry
|
|
GitOps Repo
|
|
ArgoCD
|
|
Kubernetes
|
|
Canary Release二十、CI/CD部署入口
GitLab Runner
ansible-playbook gitlab-runner.ymlJenkins Agent
ansible-playbook jenkins-agent.ymlSonarQube
ansible-playbook sonarqube.ymlArgoCD
ansible-playbook argocd.ymlSecurity Pipeline
ansible-playbook security-pipeline.ymlPart 12：AI Platform & MLOps 人工智能平台层
本部分建设企业级 AI 基础平台。
目标：
- GPU资源池化
- Kubernetes GPU调度
- 大模型推理服务
- 模型训练平台
- MLOps流程
- RAG知识库
- 向量数据库
- AI数据湖
- 模型生命周期管理
- AI应用发布体系
Part 12 总体架构
AI Platform
|
Kubernetes Cluster
|
+--------------------+--------------------+
| |
GPU Platform Data Platform
| |
NVIDIA GPU Operator MinIO / SeaweedFS
CUDA Toolkit PostgreSQL
TensorRT Kafka
|
|
+-------+---------+
| |
Inference Training
| |
vLLM Kubeflow
Ollama MLflow
TensorRT-LLM Airflow
|
|
AI Application
|
+-------+---------+
| |
RAG Platform Agent Platform
Milvus LangChain
ES LlamaIndex一、目录增加
enterprise-platform-ansible/
roles/
├── nvidia-driver/
├── cuda/
├── nvidia-container-toolkit/
├── gpu-operator/
├── vllm/
├── ollama/
├── tensorrt-llm/
├── mlflow/
├── kubeflow/
├── airflow/
├── milvus/
├── elasticsearch-ai/
├── langchain/
├── llamaindex/
└── ai-gateway/
playbooks/
├── gpu-node.yml
├── gpu-operator.yml
├── inference.yml
├── mlops.yml
├── rag.yml
└── ai-platform.yml二、GPU基础环境
GPU节点规划
示例：
gpu-node01
RTX PRO 5000
gpu-node02
RTX 4080S 32G
gpu-node03
A800/H800三、NVIDIA Driver自动安装
支持：
- Ubuntu 22.04
- Ubuntu 24.04
- Rocky Linux
Ansible:
roles/nvidia-driver/tasks/main.yml
---
- name:
install nvidia driver
apt:
name:
- nvidia-driver-595
state:
present
- name:
reboot gpu node
reboot:验证：
nvidia-smi输出：
GPU
Driver Version
CUDA Version
Memory四、CUDA Toolkit
安装：
apt install cuda-toolkit-12-8验证：
nvcc --version五、NVIDIA Container Toolkit
作用：
让Docker/Kubernetes使用GPU。
安装：
apt install nvidia-container-toolkit配置：
nvidia-ctk runtime configure \
--runtime=docker重启:
systemctl restart docker测试：
docker run \
--rm \
--gpus all \
nvidia/cuda:12.8.0-base \
nvidia-smi六、Kubernetes GPU支持
架构：
Kubernetes
|
NVIDIA Device Plugin
|
GPU Node
|
NVIDIA Driver七、NVIDIA GPU Operator
安装：
helm repo add nvidia \
https://helm.ngc.nvidia.com/nvidia
helm install gpu-operator \
nvidia/gpu-operator \
-n gpu-operator \
--create-namespace检查：
kubectl get pods \
-n gpu-operator输出：
gpu-operator
nvidia-device-plugin
nvidia-driver八、GPU节点标签
kubectl label node gpu01 \
accelerator=nvidia九、GPU资源测试
Pod:
apiVersion: v1
kind: Pod
metadata:
name: gpu-test
spec:
containers:
- name:
cuda
image:
nvidia/cuda:12.8-base
resources:
limits:
nvidia.com/gpu:
1验证：
kubectl exec gpu-test -- nvidia-smi十、大模型推理平台
包含：
| 组件 | 用途 |
|---|---|
| vLLM | 高性能LLM推理 |
| TensorRT-LLM | NVIDIA优化 |
| Ollama | 轻量模型运行 |
| TGI | HuggingFace推理 |
十一、vLLM部署
架构：
Client
|
API Gateway
|
vLLM Server
|
GPUDocker Compose:
services:
vllm:
image:
vllm/vllm-openai
runtime:
nvidia
volumes:
- /data/models:/models
command:
--model /models/Qwen2.5-72B
ports:
- 8000:8000启动：
docker compose up -dAPI测试：
curl http://localhost:8000/v1/modelsChat API:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '
{
"model":"Qwen2.5",
"messages":[
{"role":"user","content":"hello"}
]
}
'十二、Ollama平台
适合：
- 本地AI
- 开发测试
- 小模型
Compose:
services:
ollama:
image:
ollama/ollama
runtime:
nvidia
volumes:
- /data/ollama:/root/.ollama
ports:
- 11434:11434运行：
ollama run qwen2.5十三、TensorRT-LLM
用途：
NVIDIA高性能推理。
架构：
Model
|
TensorRT Build
|
Engine
|
GPU Runtime转换：
trtllm-build \
--checkpoint_dir model \
--output_dir engine十四、模型仓库
推荐：
MinIO
目录：
models/
├── qwen/
├── llama/
├── embedding/
├── reranker/
└── whisper/十五、MLflow模型管理
功能：
- 实验记录
- 模型版本
- 模型注册
架构：
Training
|
MLflow
|
Model Registry
|
ProductionMLflow Compose
services:
mlflow:
image:
ghcr.io/mlflow/mlflow
ports:
- 5000:5000
command:
mlflow server
--host 0.0.0.0访问：
http://mlflow.company.com十六、Kubeflow
用于：
- AI训练流水线
- Notebook
- Pipeline
架构：
Kubeflow
├── Notebook
├── Training Operator
├── Pipeline
├── Katib
└── Model Serving安装：
export PIPELINE_VERSION=2.3.0
kubectl apply \
-k https://github.com/kubeflow/manifests十七、Airflow
用途：
AI数据任务调度。
架构：
Scheduler
|
Worker
|
TaskCompose:
services:
airflow:
image:
apache/airflow
ports:
- 8080:8080DAG示例：
from airflow import DAG
from airflow.operators.python import PythonOperator
def train():
print("training")
with DAG(
"ai-training"
) as dag:
task=PythonOperator(
task_id="train",
python_callable=train
)十八、RAG平台
架构：
Document
|
Parser
|
Embedding Model
|
Vector Database
|
Retriever
|
LLM
|
Answer十九、Milvus向量数据库
用途：
- 文档检索
- 知识库
- Agent Memory
Compose:
services:
milvus:
image:
milvusdb/milvus
ports:
- 19530:19530Collection:
knowledge_base
fields:
id
embedding
text
metadata二十、Elasticsearch AI搜索
用途：
- 全文检索
- 混合搜索
架构：
Keyword Search
+
Vector Search
=
Hybrid Search索引：
{
"mappings":{
"properties":{
"text":{
"type":"text"
},
"vector":{
"type":"dense_vector"
}
}
}
}二十一、LangChain
用途：
AI应用开发框架。
架构：
Application
|
LangChain
|
Retriever
|
LLM
|
Tools安装：
pip install langchain示例：
from langchain_openai import ChatOpenAI
llm=ChatOpenAI(
base_url="http://vllm:8000"
)
response=llm.invoke(
"hello"
)二十二、LlamaIndex
用途：
数据连接层。
支持：
- MySQL
- Elasticsearch
- Milvus
- MinIO
安装：
pip install llama-index二十三、AI Gateway
统一入口：
User
|
API Gateway
|
+--------------+
| |
OpenAI API Local LLM
vLLM Ollama推荐：
- Higress
- Kong
- APISIX
二十四、AI安全
增加：
Prompt防护
Prompt Injection
Sensitive Data
Jailbreak模型权限
Keycloak:
User
|
OAuth2
|
AI API审计
记录：
user
model
prompt
response
token
cost进入：
Loki
+
Grafana二十五、AI流水线
完整流程：
Data
|
SeaweedFS
|
Airflow
|
Feature Engineering
|
Kubeflow
|
Training
|
MLflow
|
Model Registry
|
Harbor
|
vLLM
|
API Gateway
|
Application二十六、Ansible部署入口
GPU：
ansible-playbook gpu-node.ymlGPU Operator：
ansible-playbook gpu-operator.yml推理平台：
ansible-playbook inference.ymlMLOps：
ansible-playbook mlops.ymlRAG：
ansible-playbook rag.yml完整AI平台：
ansible-playbook ai-platform.ymlPart 13：Enterprise Automation & Operations 运维自动化层
本部分建设企业级 IT 自动化运维平台。
目标：
- 基础设施自动化
- 自动装机
- 配置管理
- 云资源编排
- CMDB资产管理
- ITSM流程管理
- 自动巡检
- 自动修复
- ChatOps运维机器人
- 企业级运维闭环
Part 13 总体架构
Enterprise Operations Platform
|
+-----------+-----------+
| |
Automation Engine Infrastructure
| |
+-----------+-----------+ |
| | |
Ansible AWX Terraform PXE
Rundeck OpenTofu Kickstart
Jenkins Packer Cloud-init
|
+------------+-------------+
| |
CMDB / ITSM Monitoring
| |
Assets Prometheus
Tickets Grafana
Workflow Alertmanager
|
ChatOps Robot
|
+-----------+-----------+
|
Auto Remediation一、目录增加
enterprise-platform-ansible/
roles/
├── awx/
├── ansible-runner/
├── rundeck/
├── terraform/
├── opentofu/
├── packer/
├── pxe/
├── kickstart/
├── cloud-init/
├── cmdb/
├── itsm/
├── chatops/
├── auto-remediation/
├── bastion/
└── operation-agent/
playbooks/
├── automation.yml
├── awx.yml
├── rundeck.yml
├── terraform.yml
├── pxe.yml
├── cmdb.yml
├── chatops.yml
└── remediation.yml二、自动化平台选型
| 功能 | 组件 |
|---|---|
| 配置管理 | Ansible |
| Web自动化 | AWX |
| 任务调度 | Rundeck |
| 基础设施编排 | Terraform |
| Terraform替代 | OpenTofu |
| 镜像制作 | Packer |
| 裸机安装 | PXE |
| 初始化 | Cloud-init |
| 资产管理 | CMDB |
| 流程管理 | ITSM |
| 机器人 | ChatOps |
| 自动修复 | Ansible + Alertmanager |
三、Ansible Automation Platform
架构
Developer
|
Git Repository
|
Ansible
|
+-------+-------+
| |
AWX UI CLI
|
Execution Node
|
Linux / Network / Cloud四、AWX部署
推荐 Kubernetes 部署。
架构：
Kubernetes
|
AWX Operator
|
+---------+---------+
| |
AWX PostgreSQL安装AWX Operator
kubectl create namespace awx
kubectl apply \
-f https://github.com/ansible/awx-operator/releases/latest/download/awx-operator.yamlAWX实例
文件：
awx.yaml
apiVersion:
awx.ansible.com/v1beta1
kind:
AWX
metadata:
name:
awx
spec:
service_type:
NodePort
postgres_storage_class:
ceph-rbd部署：
kubectl apply -f awx.yaml五、AWX组织结构
推荐：
AWX
├── Organizations
│
├── Production
│
├── Development
├── Projects
│
├── Network
│
├── Kubernetes
├── Inventories
│
├── Prod
│
├── Test
├── Job Templates
└── Credentials六、Ansible Inventory设计
目录:
inventory/
├── production
│
├── hosts.yml
├── group_vars/
└── host_vars/示例：
all:
children:
kubernetes:
hosts:
master01:
ansible_host:
10.50.0.101
worker01:
ansible_host:
10.50.0.111
database:
hosts:
pg01:
ansible_host:
10.50.0.50七、Rundeck任务平台
用途：
- 运维人员自助执行
- 定时任务
- 权限控制
架构：
User
|
Rundeck
|
Ansible
|
ServerDocker Compose
services:
rundeck:
image:
rundeck/rundeck
ports:
- "4440:4440"
volumes:
- /data/rundeck:/home/rundeck/server/dataRundeck Job示例
- defaultTab: nodes
description:
Restart nginx
sequence:
commands:
- exec:
sudo systemctl restart nginx八、Terraform基础设施自动化
用途：
- VMware
- OpenStack
- AWS
- Azure
- Proxmox
目录：
terraform/
├── main.tf
├── variables.tf
├── outputs.tf
└── modules/示例：创建VM
resource "proxmox_vm_qemu" "node01" {
name="k8s-node01"
cores=8
memory=32768
clone="ubuntu2404"
}执行：
terraform init
terraform plan
terraform apply九、OpenTofu
Terraform开源分支。
安装：
apt install opentofu执行：
tofu init
tofu apply十、Packer镜像自动构建
用途：
自动生成：
- Ubuntu镜像
- Rocky镜像
- Kubernetes Node镜像
流程：
ISO
|
Packer
|
VM Template
|
Terraform
|
Production VM示例：
source "vmware-iso" "ubuntu" {
iso_url="ubuntu.iso"
ssh_username="root"
}
build {
sources=[
"source.vmware-iso.ubuntu"
]
}十一、PXE自动装机平台
用途：
批量安装服务器。
架构：
Server
|
PXE Boot
|
DHCP
|
TFTP
|
HTTP
|
Kickstart/Autoinstall
|
OS组件：
DHCP
TFTP
HTTP
DNS
iPXE十二、PXE Docker Compose
services:
dnsmasq:
image:
jpillora/dnsmasq
ports:
- "67:67/udp"
- "69:69/udp"
volumes:
- ./conf:/etc/dnsmasq.conf十三、Ubuntu 24.04 Autoinstall
文件：
user-data示例：
autoinstall:
version:
1
identity:
hostname:
node01
username:
ubuntu
password:
encrypted-password
storage:
layout:
name:
lvm十四、Cloud-init
用途：
云初始化
支持：
- OpenStack
- VMware
- Proxmox
- Public Cloud
cloud-init:
#cloud-config
hostname:
app01
packages:
- docker.io
- git
runcmd:
- systemctl enable docker十五、CMDB资产管理
推荐：
- NetBox
- Snipe-IT
- GLPI
NetBox架构
Device
|
NetBox
|
API
|
Automation
|
AnsibleNetBox Docker
services:
netbox:
image:
netboxcommunity/netbox
ports:
- "8000:8080"NetBox数据模型
Tenant
|
Site
|
Rack
|
Device
|
Interface
|
IP Address十六、ITSM流程
推荐：
- Jira Service Management
- ServiceNow
- GLPI
流程：
User
|
Ticket
|
Approval
|
Automation
|
Change
|
Audit十七、ChatOps运维机器人
支持：
- 企业微信
- 钉钉
- Slack
- Teams
架构：
Engineer
|
Chat
|
Bot
|
Webhook
|
AWX/Rundeck
|
Execute Job十八、机器人命令示例
用户：
重启订单服务机器人：
检测权限
确认环境
调用AWX
执行:
kubectl rollout restart deployment order
返回结果十九、Alertmanager自动修复
流程：
Prometheus
|
Alertmanager
|
Webhook
|
Robot
|
Ansible
|
FixAlert规则
groups:
- name:
node
rules:
- alert:
DiskFull
expr:
node_filesystem_avail_bytes < 10%二十、自动修复Playbook
示例：
---
- name:
restart service
hosts:
all
tasks:
- name:
restart nginx
service:
name:
nginx
state:
restarted二十一、Bastion跳板机
架构：
Engineer
|
VPN
|
Bastion
|
Production安全：
- SSH审计
- MFA
- 命令记录
推荐：
- Teleport
- Boundary
- Jumpserver
二十二、运维Agent
部署：
Node
|
Node Exporter
|
OSQuery
|
Falco
|
Telegraf二十三、统一运维流程
完整闭环：
资产入库
|
CMDB
|
自动安装
|
PXE
|
配置管理
|
Ansible
|
部署应用
|
GitOps
|
监控
|
告警
|
自动修复
|
工单关闭二十四、自动化入口
AWX
ansible-playbook awx.ymlRundeck
ansible-playbook rundeck.ymlTerraform
ansible-playbook terraform.ymlPXE
ansible-playbook pxe.ymlCMDB
ansible-playbook cmdb.ymlChatOps
ansible-playbook chatops.ymlAuto Remediation
ansible-playbook remediation.ymlPart 14：Enterprise Architecture Integration & Production Deployment 企业架构集成与生产交付层
本部分将前面 Part 1 ～ Part 13 的所有组件整合成一个完整企业级生产平台
目标：
- 企业整体架构设计
- 多数据中心规划
- 双活/灾备体系
- 网络安全域规划
- Kubernetes生产架构
- 数据平台整合
- AI平台整合
- 自动化交付体系
- Day0-Day2 运维流程
- 一键化部署入口
- 企业交付文档模板
一、完整企业平台最终架构
Internet
|
Edge Firewall
|
+----------------+
| Load Balance |
| HAProxy |
+----------------+
|
+----------------+
| API Gateway |
| Higress/APISIX |
+----------------+
|
+-------------------------+-------------------------+
| |
Kubernetes Production Cluster Enterprise Services
| |
+-------+--------+ +----------+----------+
| | | |
Application AI Platform DevOps Platform Office
| | | |
Microservice vLLM GitLab Jira
Java/Python Ollama Jenkins Confluence
Go MLflow Harbor Nextcloud
|
|
Storage Layer
|
+-------+---------+
| |
Ceph MinIO
RBD S3
CephFS Backup
|
Database Layer
|
+-------+---------+
| | |
PG HA MySQL MongoDB
Redis Kafka
|
Monitoring
|
Prometheus
Grafana
Loki
Thanos
|
Security
|
Vault
Keycloak
LDAP
Falco
Trivy二、生产环境分层设计
Layer 0：基础设施层
包含：
服务器
网络
存储
机房
电源
IP
DNSLayer 1：OS层
组件：
Ubuntu 24.04
Rocky Linux 9
NVIDIA Driver
Kernel
NTP
SSH Hardening
AuditLayer 2：基础服务层
DNS
LDAP
Proxy
SMTP
FTP
VPN
NTPLayer 3：DevOps层
GitLab
Gerrit
Jenkins
Nexus
Harbor
SonarQubeLayer 4：Office协作
Jira
Confluence
Nextcloud
Collabora
OnlyOffice
MailLayer 5：Kubernetes
Control Plane
Worker
Ingress
CSI
CNI
RBAC
OperatorLayer 6：数据平台
Ceph
MinIO
SeaweedFS
PostgreSQL
Kafka
MongoDBLayer 7：AI平台
GPU
CUDA
vLLM
MLflow
Kubeflow
RAGLayer 8：运维安全
Prometheus
Grafana
Vault
Keycloak
AWX
Terraform三、多数据中心架构
生产推荐：
IDC-A IDC-B
Primary Disaster
| |
+---------------+ +---------------+
| Kubernetes | | Kubernetes |
| Storage | <------> | Storage |
| Database | | Database |
+---------------+ +---------------+四、双活架构
适合：
- 金融
- 制造
- SaaS
架构：
DNS
|
+---------+---------+
| |
IDC1 IDC2
| |
Kubernetes Kubernetes
| |
Database Sync Database Sync五、灾备架构
推荐：
生产
Primary灾备
Warm Standby数据同步：
| 数据 | 方案 |
|---|---|
| 对象存储 | MinIO Replication |
| 数据库 | Patroni/MySQL Replica |
| K8s资源 | Velero |
| 文件 | Restic |
| 配置 | GitOps |
六、服务器角色规划
示例：
Management节点
mgmt01
AWX
GitLab
Jenkins
GrafanaKubernetes Master
k8s-master01
k8s-master02
k8s-master03Worker
worker01
worker02
worker03GPU Node
gpu01
gpu02
gpu03Storage Node
storage01
storage02
storage03七、IP规划
示例：
管理网
10.50.0.0/24| 用途 | IP |
|---|---|
| DNS01 | 10.50.0.10 |
| DNS02 | 10.50.0.11 |
| HAProxy VIP | 10.50.0.100 |
| AWX | 10.50.0.20 |
Kubernetes
10.50.10.0/24| 节点 | IP |
|---|---|
| master01 | 10.50.10.11 |
| master02 | 10.50.10.12 |
| master03 | 10.50.10.13 |
| worker01 | 10.50.10.21 |
Storage
10.50.20.0/24| 服务 | IP |
|---|---|
| ceph01 | 10.50.20.11 |
| ceph02 | 10.50.20.12 |
| ceph03 | 10.50.20.13 |
八、域名规划
企业DNS：
company.com基础服务：
git.company.com
jenkins.company.com
harbor.company.com
nexus.company.com
sonar.company.com
grafana.company.com
argocd.company.comAI：
ai.company.com
llm.company.com
model.company.com九、安全域划分
网络：
Firewall
|
+-----------------+-----------------+
| | |
DMZ APP DATADMZ
允许：
443
80
VPNAPP
允许：
8080
8443DATA
禁止外部访问。
只允许：
APP --> DATA十、Kubernetes生产设计
Master HA
3 Master
etcd HA
HAProxy VIPContainer Runtime
推荐：
containerdCNI
推荐：
Calico
CiliumStorage
Ceph CSIIngress
Higress
Ingress-Nginx十一、GitOps生产流程
完整流程：
Developer
|
GitLab
|
CI
|
Build
|
Security Scan
|
Harbor
|
GitOps Repo
|
ArgoCD
|
Kubernetes
|
Production十二、数据库生产架构
PostgreSQL
HAProxy
|
Patroni
|
PostgreSQL
|
ReplicationMySQL
MySQL Router
|
InnoDB Cluster
|
Group ReplicationRedis
Redis Sentinel
+
ReplicationKafka
Kafka Broker x3
KRaft
Replication Factor=3十三、AI生产架构
User
|
AI Gateway
|
+---------+---------+
| |
RAG LLM
| |
Milvus vLLM
|
Embedding
|
MinIO十四、监控体系
四大黄金指标：
Metrics
PrometheusLogs
Loki
ELKTracing
Jaeger
TempoVisualization
Grafana十五、自动化部署总入口
最终目录：
enterprise-platform-ansible/
├── inventory/
├── group_vars/
├── roles/
├── docker-compose/
├── helm/
├── kubernetes/
├── terraform/
├── playbooks/
└── README.md十六、总安装入口
初始化服务器
ansible-playbook \
playbooks/bootstrap.yml网络层
ansible-playbook \
playbooks/network.ymlStorage
ansible-playbook \
playbooks/storage.ymlKubernetes
ansible-playbook \
playbooks/kubernetes.ymlDevOps
ansible-playbook \
playbooks/devops.ymlMonitoring
ansible-playbook \
playbooks/monitoring.ymlSecurity
ansible-playbook \
playbooks/security.ymlAI
ansible-playbook \
playbooks/ai-platform.yml十七、一键安装总Playbook
文件：
site.yml内容：
---
- import_playbook: bootstrap.yml
- import_playbook: network.yml
- import_playbook: storage.yml
- import_playbook: kubernetes.yml
- import_playbook: devops.yml
- import_playbook: office.yml
- import_playbook: monitoring.yml
- import_playbook: security.yml
- import_playbook: ai-platform.yml
- import_playbook: backup.yml执行：
ansible-playbook site.yml十八、Day0-Day2 运维流程
Day0 基础准备
服务器上架
网络配置
IP规划
DNS
PXE安装Day1 平台部署
OS
Docker
Kubernetes
Storage
Database
MiddlewareDay2 应用上线
GitLab
CI/CD
GitOps
Monitoring
Backup
Security十九、生产验收Checklist
OS
✓ NTP
✓ SSH安全
✓ Kernel参数
✓ AuditKubernetes
✓ HA
✓ RBAC
✓ NetworkPolicy
✓ CSI
✓ BackupStorage
✓ Ceph HEALTH_OK
✓ MinIO Replication
✓ Backup TestSecurity
✓ Vault
✓ Keycloak
✓ LDAP
✓ Vulnerability ScanMonitoring
✓ Metrics
✓ Logs
✓ Alert
✓ Dashboard二十、企业交付文档结构
最终输出：
Enterprise-Platform-Documentation/
├── Architecture/
│
├── Network Design/
│
├── Security Design/
│
├── Kubernetes Design/
│
├── Storage Design/
│
├── Operation Manual/
│
├── Backup Manual/
│
├── Disaster Recovery/
│
├── Deployment Guide/
│
└── Troubleshooting/至此完整企业平台蓝图：
Part 1 OS基础层
Part 2 基础服务层
Part 3 DevOps层
Part 4 Office层
Part 5 Kubernetes层
Part 6 Monitoring层
Part 7 Backup层
Part 8 Security层
Part 9 Network & HA层
Part 10 Storage & Data Platform层
Part 11 CI/CD & GitOps层
Part 12 AI Platform & MLOps层
Part 13 Enterprise Automation层
Part 14 Enterprise Architecture Integration层Part 15：完整 enterprise-platform-ansible 项目源码
由于整个项目代码量会非常大（预计 200+ 文件，1~3 万行 YAML/Shell/Compose/Helm/Terraform），会按照真实企业项目结构逐步编写
生成原则：
- ✅ 每个服务独立安装
- ✅ Ansible Role标准结构
- ✅ 支持 Ubuntu 22.04/24.04
- ✅ 支持 Rocky Linux 9（后续扩展）
- ✅ Docker Compose服务独立管理
- ✅ Kubernetes Helm独立管理
- ✅ 可单独执行
- ✅ 支持生产环境变量隔离
- ✅ 支持多节点Inventory
- ✅ 支持HA部署
Part 15.1 项目基础目录结构
最终目录：
enterprise-platform-ansible/
│
├── README.md
├── ansible.cfg
├── requirements.yml
├── site.yml
│
├── inventory/
│ │
│ ├── production/
│ │ ├── hosts.yml
│ │ ├── group_vars/
│ │ │ ├── all.yml
│ │ │ ├── kubernetes.yml
│ │ │ ├── database.yml
│ │ │ ├── gpu.yml
│ │ │ └── security.yml
│ │ │
│ │ └── host_vars/
│
│
├── playbooks/
│ │
│ ├── bootstrap.yml
│ ├── docker.yml
│ ├── storage.yml
│ ├── kubernetes.yml
│ ├── devops.yml
│ ├── office.yml
│ ├── monitoring.yml
│ ├── security.yml
│ ├── ai-platform.yml
│ ├── backup.yml
│ └── site.yml
│
│
├── roles/
│
│ ├── common/
│ ├── docker/
│ ├── containerd/
│ ├── kubernetes/
│ ├── nvidia-driver/
│ ├── gpu-operator/
│ │
│ ├── gitlab/
│ ├── gerrit/
│ ├── jenkins/
│ ├── nexus/
│ ├── harbor/
│ ├── sonarqube/
│ │
│ ├── jira/
│ ├── confluence/
│ ├── nextcloud/
│ ├── collabora/
│ ├── onlyoffice/
│ │
│ ├── prometheus/
│ ├── grafana/
│ ├── loki/
│ ├── elasticsearch/
│ ├── alertmanager/
│ │
│ ├── vault/
│ ├── keycloak/
│ ├── openldap/
│ │
│ ├── minio/
│ ├── seaweedfs/
│ ├── ceph/
│ │
│ ├── argocd/
│ ├── velero/
│ ├── restic/
│ │
│ ├── awx/
│ ├── rundeck/
│ ├── terraform/
│ │
│ └── ai-platform/
│
│
├── docker-compose/
│
│ ├── gitlab/
│ ├── jenkins/
│ ├── nexus/
│ ├── harbor/
│ ├── jira/
│ ├── confluence/
│ ├── monitoring/
│ ├── elk/
│ ├── minio/
│ ├── vault/
│ └── ai/
│
│
├── kubernetes/
│
│ ├── namespaces/
│ ├── ingress/
│ ├── storage/
│ ├── monitoring/
│ └── ai/
│
│
├── helm/
│
│ ├── values/
│ └── charts/
│
│
├── terraform/
│
│ ├── proxmox/
│ ├── vmware/
│ ├── openstack/
│ └── cloud/
│
│
└── scripts/
├── check.sh
├── backup.sh
└── healthcheck.shPart 15.2 基础Ansible配置
ansible.cfg
路径：
enterprise-platform-ansible/ansible.cfg内容：
[defaults]
inventory = inventory/production/hosts.yml
roles_path = roles
host_key_checking = False
forks = 20
timeout = 30
retry_files_enabled = False
stdout_callback = yaml
interpreter_python = auto_silent
[privilege_escalation]
become = True
become_method = sudo
become_user = root
[ssh_connection]
pipelining = TruePart 15.3 requirements.yml
安装Ansible依赖
---
collections:
- name: community.general
- name: ansible.posix
- name: kubernetes.core
- name: community.docker
- name: community.vmware
- name: community.libvirt安装：
ansible-galaxy collection install -r requirements.ymlPart 15.4 主Inventory
文件：
inventory/production/hosts.yml内容：
all:
children:
management:
hosts:
mgmt01:
ansible_host: 10.50.0.20
kubernetes_master:
hosts:
k8s-master01:
ansible_host: 10.50.10.11
k8s-master02:
ansible_host: 10.50.10.12
k8s-master03:
ansible_host: 10.50.10.13
kubernetes_worker:
hosts:
worker01:
ansible_host: 10.50.10.21
worker02:
ansible_host: 10.50.10.22
gpu_nodes:
hosts:
gpu01:
ansible_host:
10.50.30.11
storage:
hosts:
storage01:
ansible_host:
10.50.20.11
storage02:
ansible_host:
10.50.20.12
storage03:
ansible_host:
10.50.20.13Part 15.5 全局变量
文件：
inventory/production/group_vars/all.yml
---
company_domain:
company.com
timezone:
Asia/Shanghai
docker_version:
27.3.1
kubernetes_version:
1.31.0
storage_path:
/data
docker_registry:
harbor.company.com
ntp_servers:
- ntp.aliyun.com
- time.cloudflare.comPart 15.6 Bootstrap基础初始化Role
roles/common目录
roles/common/
├── tasks/
│ └── main.yml
│
├── handlers/
│ └── main.yml
│
├── templates/
│
└── defaults/
└── main.ymltasks/main.yml
---
- name:
update apt cache
apt:
update_cache:
yes
when:
ansible_os_family == "Debian"
- name:
install base packages
package:
name:
- vim
- curl
- wget
- net-tools
- lsof
- unzip
- chrony
- bash-completion
state:
present
- name:
set timezone
timezone:
name:
"{{ timezone }}"
- name:
enable chrony
service:
name:
chrony
state:
started
enabled:
yes
- name:
disable swap
command:
swapoff -a
when:
ansible_swaptotal_mb > 0Part 15.7 Bootstrap Playbook
文件：
playbooks/bootstrap.yml
---
- name:
Bootstrap servers
hosts:
all
become:
true
roles:
- common执行：
ansible-playbook playbooks/bootstrap.ymlPart 15.8 总入口 site.yml
文件：
site.yml
---
- import_playbook:
playbooks/bootstrap.yml
- import_playbook:
playbooks/docker.yml
- import_playbook:
playbooks/kubernetes.yml
- import_playbook:
playbooks/storage.yml
- import_playbook:
playbooks/devops.yml
- import_playbook:
playbooks/office.yml
- import_playbook:
playbooks/security.yml
- import_playbook:
playbooks/monitoring.yml
- import_playbook:
playbooks/backup.yml
- import_playbook:
playbooks/ai-platform.ymlPart 15.9 Docker安装模块
Playbook:
playbooks/docker.yml
---
- hosts:
all
become:
true
roles:
- dockerRole:
roles/docker/tasks/main.yml
---
- name:
install docker dependencies
apt:
name:
- ca-certificates
- curl
state:
present
- name:
add docker repo key
shell:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
gpg --dearmor -o /usr/share/keyrings/docker.gpg
- name:
install docker
apt:
name:
- docker-ce
- docker-ce-cli
- containerd.io
- docker-compose-plugin
state:
present
- name:
enable docker
service:
name:
docker
state:
started
enabled:
yes验证：
ansible all -m shell -a "docker version"当前完成
已生成：
✅ 项目目录
✅ ansible.cfg
✅ requirements.yml
✅ Production Inventory
✅ 全局变量
✅ Bootstrap Role
✅ Docker Role
✅ 主入口结构
Part 15.10 Kubernetes Role完整代码
本部分生成生产级 Kubernetes 自动化安装 Role。
目标：
- containerd Runtime
- kubeadm安装
- Kubernetes HA Cluster
- Master初始化
- Worker自动加入
- Calico CNI
- MetalLB
- CoreDNS
- kube-proxy
- Metrics Server
- StorageClass准备
- RBAC基础配置
支持：
Ubuntu 22.04
Ubuntu 24.04
Kubernetes:
v1.30+
v1.31+
Runtime:
containerd 2.x15.10.1 Kubernetes目录
创建：
roles/kubernetes/
├── defaults/
│ └── main.yml
│
├── tasks/
│ ├── main.yml
│ ├── repo.yml
│ ├── containerd.yml
│ ├── install.yml
│ ├── master.yml
│ ├── worker.yml
│ └── addons.yml
│
├── templates/
│ ├── containerd-config.toml.j2
│ └── kubeadm-config.yaml.j2
│
├── handlers/
│ └── main.yml
│
└── files/15.10.2 Kubernetes变量
文件：
roles/kubernetes/defaults/main.yml
---
kubernetes_version: "1.31.0"
kubernetes_packages_version:
"1.31.0-1.1"
pod_network_cidr:
192.168.0.0/16
service_cidr:
10.96.0.0/12
container_runtime:
containerd
kubernetes_control_plane_endpoint:
k8s-api.company.com:6443
calico_version:
v3.29.1
metallb_version:
v0.14.8
kube_user:
root15.10.3 Kubernetes Playbook
文件：
playbooks/kubernetes.yml
---
- name:
Install Kubernetes
hosts:
kubernetes_master:kubernetes_worker
become:
true
roles:
- kubernetes
- name:
Init Kubernetes Master
hosts:
kubernetes_master[0]
become:
true
roles:
- role:
kubernetes
tasks_from:
master.yml
- name:
Join Workers
hosts:
kubernetes_worker
become:
true
roles:
- role:
kubernetes
tasks_from:
worker.yml
- name:
Install Kubernetes addons
hosts:
kubernetes_master[0]
become:
true
roles:
- role:
kubernetes
tasks_from:
addons.yml15.10.4 安装Kubernetes仓库
文件：
roles/kubernetes/tasks/repo.yml
---
- name:
install dependencies
apt:
name:
- apt-transport-https
- ca-certificates
- curl
- gpg
state:
present
- name:
add kubernetes key
shell:
cmd:
|
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key |
gpg --dearmor -o /etc/apt/keyrings/kubernetes.gpg
- name:
add kubernetes repo
apt_repository:
repo:
deb [signed-by=/etc/apt/keyrings/kubernetes.gpg]
https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /
state:
present15.10.5 安装kubeadm
文件：
roles/kubernetes/tasks/install.yml
---
- name:
install kubernetes packages
apt:
name:
- kubelet
- kubeadm
- kubectl
state:
present
- name:
hold kubernetes packages
dpkg_selections:
name:
"{{ item }}"
selection:
hold
loop:
- kubelet
- kubeadm
- kubectl
- name:
enable kubelet
systemd:
name:
kubelet
enabled:
yes15.10.6 主任务入口
文件：
roles/kubernetes/tasks/main.yml
---
- import_tasks:
repo.yml
- import_tasks:
containerd.yml
- import_tasks:
install.yml15.10.7 containerd配置
文件：
roles/kubernetes/tasks/containerd.yml
---
- name:
install containerd
apt:
name:
containerd
state:
present
- name:
create containerd config
shell:
cmd:
|
containerd config default >
/etc/containerd/config.toml
- name:
enable systemd cgroup
replace:
path:
/etc/containerd/config.toml
regexp:
'SystemdCgroup = false'
replace:
'SystemdCgroup = true'
- name:
restart containerd
systemd:
name:
containerd
state:
restarted
enabled:
yes15.10.8 系统参数优化
新增：
roles/kubernetes/tasks/sysctl.yml
---
- name:
load kernel modules
modprobe:
name:
"{{ item }}"
state:
present
loop:
- overlay
- br_netfilter
- name:
kubernetes sysctl
copy:
dest:
/etc/sysctl.d/kubernetes.conf
content:
|
net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
- name:
reload sysctl
command:
sysctl --system然后main加入：
- import_tasks:
sysctl.yml15.10.9 Master初始化
文件：
roles/kubernetes/tasks/master.yml
---
- name:
check cluster
stat:
path:
/etc/kubernetes/admin.conf
register:
kube_exist
- name:
kubeadm init
command:
kubeadm init
--control-plane-endpoint={{ kubernetes_control_plane_endpoint }}
--pod-network-cidr={{ pod_network_cidr }}
--service-cidr={{ service_cidr }}
when:
not kube_exist.stat.exists
- name:
create kube config
shell:
|
mkdir -p ~/.kube
cp /etc/kubernetes/admin.conf ~/.kube/config
when:
not kube_exist.stat.exists15.10.10 Worker Join
文件：
roles/kubernetes/tasks/worker.yml
---
- name:
check node
shell:
kubectl get node {{ inventory_hostname }}
environment:
KUBECONFIG:
/etc/kubernetes/admin.conf
ignore_errors:
yes
- name:
generate join command
command:
kubeadm token create
--print-join-command
delegate_to:
"{{ groups['kubernetes_master'][0] }}"
register:
join_command
- name:
join cluster
command:
"{{ join_command.stdout }}"
when:
join_command.stdout is defined15.10.11 Calico CNI安装
文件：
roles/kubernetes/tasks/addons.yml
---
- name:
install calico
command:
kubectl apply -f
https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version }}/manifests/calico.yaml
environment:
KUBECONFIG:
/etc/kubernetes/admin.conf
- name:
install metrics server
command:
kubectl apply -f
https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
environment:
KUBECONFIG:
/etc/kubernetes/admin.conf15.10.12 MetalLB安装
继续 addons.yml：
- name:
install metallb
command:
kubectl apply -f
https://raw.githubusercontent.com/metallb/metallb/{{ metallb_version }}/config/manifests/metallb-native.yaml
environment:
KUBECONFIG:
/etc/kubernetes/admin.conf15.10.13 MetalLB地址池
目录：
kubernetes/network/文件：
metallb-pool.yaml
apiVersion:
metallb.io/v1beta1
kind:
IPAddressPool
metadata:
name:
production
namespace:
metallb-system
spec:
addresses:
- 10.50.10.200-10.50.10.220部署：
kubectl apply -f metallb-pool.yaml15.10.14 Kubernetes验证脚本
文件：
scripts/check-k8s.sh
#!/bin/bash
echo "Nodes"
kubectl get nodes
echo "Pods"
kubectl get pods -A
echo "Cluster"
kubectl cluster-info执行：
bash scripts/check-k8s.sh15.10.15 部署执行
安装基础：
ansible-playbook playbooks/bootstrap.yml安装Kubernetes：
ansible-playbook playbooks/kubernetes.yml部署完成验证
节点
kubectl get nodes示例：
NAME STATUS
master01 Ready
master02 Ready
master03 Ready
worker01 Ready
worker02 ReadyPod
kubectl get pods -A应该包含：
kube-system
calico-system
metallb-system
metrics-serverPart 15.10 完成
已生成：
✅ Kubernetes Role目录
✅ kubeadm安装
✅ containerd配置
✅ kernel优化
✅ Master初始化
✅ Worker Join
✅ Calico CNI
✅ MetalLB
✅ Metrics Server
✅ HA Endpoint设计
✅ Inventory关联
✅ 自动部署Playbook
Part 15.11 Storage Role完整代码
本部分生成企业级存储自动化 Role。
目标：
- Ceph生产存储集群
- MinIO对象存储
- SeaweedFS分布式文件系统
- NFS共享存储
- iSCSI块存储
- Kubernetes CSI
- StorageClass自动创建
- PVC动态供应
适配：
Ubuntu 22.04
Ubuntu 24.04
Kubernetes 1.30+
containerd
Ceph Quincy/Reef
MinIO RELEASE
SeaweedFS 4.x15.11.1 Storage总体架构
Kubernetes Cluster
|
+----------------+
| StorageClass |
+----------------+
|
+-------------------+-------------------+
| | |
Ceph CSI MinIO SeaweedFS
| | |
RBD/CephFS S3 API FUSE/S3
|
Persistent Volume15.11.2 Storage目录
创建：
roles/storage/
├── defaults/
│ └── main.yml
│
├── tasks/
│ ├── main.yml
│ ├── ceph.yml
│ ├── minio.yml
│ ├── seaweedfs.yml
│ ├── nfs.yml
│ ├── iscsi.yml
│ └── csi.yml
│
├── templates/
│ ├── docker-compose-minio.yml.j2
│ ├── seaweedfs.service.j2
│ └── ceph.conf.j2
│
└── files/15.11.3 Storage变量
文件：
roles/storage/defaults/main.yml
---
storage_root:
/mnt/allin_data
ceph_version:
reef
ceph_network:
10.50.20.0/24
ceph_cluster_name:
ceph
minio_version:
RELEASE.2025
minio_port:
9000
seaweedfs_version:
4.0
seaweedfs_master_port:
9333
seaweedfs_volume_port:
8080
nfs_export:
/data/nfs
iscsi_target_port:
326015.11.4 Storage Playbook
文件：
playbooks/storage.yml
---
- name:
Deploy Storage Platform
hosts:
storage
become:
true
roles:
- storage
- name:
Deploy Kubernetes CSI
hosts:
kubernetes_master[0]
become:
true
roles:
- role:
storage
tasks_from:
csi.yml15.11.5 Storage主入口
文件：
roles/storage/tasks/main.yml
---
- import_tasks:
ceph.yml
- import_tasks:
minio.yml
- import_tasks:
seaweedfs.yml
- import_tasks:
nfs.yml
- import_tasks:
iscsi.ymlPart 15.11.6 Ceph自动部署
推荐：
生产环境：
cephadm架构：
ceph01
|
MON
MGR
OSD
ceph02
|
MON
OSD
ceph03
|
MON
OSDCeph Role
文件：
roles/storage/tasks/ceph.yml
---
- name:
install ceph dependencies
apt:
name:
- curl
- lvm2
state:
present
- name:
download cephadm
shell:
cmd:
|
curl --silent --remote-name --location \
https://raw.githubusercontent.com/ceph/ceph/main/src/cephadm/cephadm
- name:
install cephadm
copy:
src:
cephadm
dest:
/usr/local/bin/cephadm
mode:
0755
- name:
install ceph
command:
cephadm add-repo --release {{ ceph_version }}
- name:
install packages
command:
cephadm install ceph-common15.11.7 初始化Ceph Cluster
只在第一节点执行
- name:
bootstrap ceph
command:
cephadm bootstrap
--mon-ip {{ ansible_host }}
--cluster-network {{ ceph_network }}
when:
inventory_hostname == groups['storage'][0]15.11.8 添加OSD
示例：
- name:
create osd
command:
ceph orch daemon add osd
{{ inventory_hostname }}:/dev/sdb生产：
/dev/sdb
/dev/sdc
/dev/sdd全部加入。
15.11.9 Ceph验证
ceph status输出：
cluster:
HEALTH_OK
services:
mon:3
mgr:3
osd:1215.11.10 MinIO部署
架构：
S3 Client
|
MinIO LB
|
+------------+------------+
| | |
node1 node2 node3MinIO Docker Compose
模板：
roles/storage/templates/docker-compose-minio.yml.j2
services:
minio:
image:
quay.io/minio/minio
container_name:
minio
restart:
always
ports:
- "9000:9000"
- "9001:9001"
volumes:
- /mnt/allin_data/minio:/data
environment:
MINIO_ROOT_USER:
admin
MINIO_ROOT_PASSWORD:
ChangeMe123!
command:
server /data
--console-address ":9001"MinIO任务
文件：
roles/storage/tasks/minio.yml
---
- name:
create minio dir
file:
path:
/mnt/allin_data/minio
state:
directory
- name:
deploy compose
template:
src:
docker-compose-minio.yml.j2
dest:
/opt/minio/docker-compose.yml
- name:
start minio
community.docker.docker_compose:
project_src:
/opt/minioMinIO验证
mc alias set local \
http://localhost:9000 \
admin \
ChangeMe123!查看：
mc admin info local15.11.11 SeaweedFS部署
架构：
Master
|
Filer
|
+--------+--------+
| |
Volume01 Volume02目录：
roles/storage/tasks/seaweedfs.yml安装weed
---
- name:
download seaweedfs
get_url:
url:
https://github.com/seaweedfs/seaweedfs/releases/latest/download/linux_amd64.tar.gz
dest:
/tmp/weed.tar.gz
- name:
extract
unarchive:
src:
/tmp/weed.tar.gz
dest:
/usr/local/bin/
remote_src:
yesSeaweedFS Master Service
模板：
templates/seaweedfs.service.j2
[Unit]
Description=SeaweedFS Master
[Service]
ExecStart=/usr/local/bin/weed master \
-ip={{ inventory_hostname }} \
-port=9333
Restart=always
[Install]
WantedBy=multi-user.target部署：
- name:
install master service
template:
src:
seaweedfs.service.j2
dest:
/etc/systemd/system/seaweedfs-master.service
- name:
start service
systemd:
daemon_reload:
yes
name:
seaweedfs-master
enabled:
yes
state:
started15.11.12 SeaweedFS Volume
示例：
weed volume \
-server=10.50.20.11:8080 \
-dir=/mnt/allin_data/seaweedfs/volume15.11.13 NFS服务
文件：
roles/storage/tasks/nfs.yml
---
- name:
install nfs
apt:
name:
- nfs-kernel-server
state:
present
- name:
create export
file:
path:
"{{ nfs_export }}"
state:
directory
- name:
configure export
lineinfile:
path:
/etc/exports
line:
"{{ nfs_export }} *(rw,sync,no_root_squash)"
- name:
restart nfs
systemd:
name:
nfs-server
state:
restarted15.11.14 iSCSI Target
安装：
---
- name:
install targetcli
apt:
name:
targetcli-fb
state:
present创建Target：
targetcli
/backstores/block create disk01 /dev/sdb
/iscsi create iqn.2026-09.com.company:storage15.11.15 Kubernetes CSI
文件：
roles/storage/tasks/csi.ymlCeph CSI
---
- name:
install ceph csi
command:
kubectl apply -f
https://raw.githubusercontent.com/ceph/ceph-csi/devel/deploy/rbd/kubernetes/csi-rbdplugin.yaml
environment:
KUBECONFIG:
/etc/kubernetes/admin.conf15.11.16 StorageClass
文件：
kubernetes/storage/ceph-storageclass.yaml
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
ceph-rbd
provisioner:
rbd.csi.ceph.com
parameters:
clusterID:
ceph
pool:
kubernetes
reclaimPolicy:
Delete
allowVolumeExpansion:
true部署：
kubectl apply -f ceph-storageclass.yaml15.11.17 PVC测试
apiVersion:
v1
kind:
PersistentVolumeClaim
metadata:
name:
test-pvc
spec:
storageClassName:
ceph-rbd
accessModes:
- ReadWriteOnce
resources:
requests:
storage:
10Gi测试：
kubectl get pvc结果：
NAME
test-pvc
STATUS
Bound15.11.18 Storage健康检查
文件：
scripts/check-storage.sh
#!/bin/bash
echo "Ceph"
ceph status
echo "MinIO"
curl http://localhost:9000/minio/health/live
echo "SeaweedFS"
curl http://localhost:9333/cluster/statusPart 15.11 完成
已生成：
✅ Storage Role结构
✅ Ceph cephadm部署
✅ MON/MGR/OSD设计
✅ MinIO Docker Compose
✅ SeaweedFS Master/Filer设计
✅ NFS服务
✅ iSCSI基础
✅ Ceph CSI
✅ Kubernetes StorageClass
✅ PVC测试
✅ 健康检查脚本
Part 15.12 DevOps Role完整代码
本部分生成企业级 DevOps 平台自动化部署代码
包含：
- GitLab CE
- Gerrit Code Review
- Jenkins CI/CD
- Nexus Repository
- Harbor Registry
- SonarQube代码质量扫描
- LDAP统一认证
- GitLab → Jenkins → Harbor → Kubernetes CI/CD链路
15.12.1 DevOps整体架构
Developer
|
|
GitLab
|
GitLab Webhook
|
v
Jenkins
|
+---------------+---------------+
| |
SonarQube Build
| |
+---------------+---------------+
|
v
Harbor
|
v
Kubernetes
|
v
ArgoCD15.12.2 DevOps目录结构
创建：
roles/devops/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── gitlab.yml
│ ├── gerrit.yml
│ ├── jenkins.yml
│ ├── nexus.yml
│ ├── harbor.yml
│ └── sonarqube.yml
├── templates/
│ ├── docker-compose-gitlab.yml.j2
│ ├── docker-compose-jenkins.yml.j2
│ ├── docker-compose-nexus.yml.j2
│ ├── docker-compose-harbor.yml.j2
│ └── docker-compose-sonar.yml.j2
└── handlers/
└── main.yml15.12.3 DevOps变量
文件：
roles/devops/defaults/main.yml
---
devops_root:
/opt/devops
domain:
company.com
gitlab_domain:
gitlab.company.com
jenkins_domain:
jenkins.company.com
nexus_domain:
nexus.company.com
harbor_domain:
harbor.company.com
sonar_domain:
sonar.company.com
gitlab_image:
gitlab/gitlab-ce:latest
jenkins_image:
jenkins/jenkins:lts
nexus_image:
sonatype/nexus3
harbor_version:
v2.11.0
sonar_image:
sonarqube:lts-community15.12.4 DevOps Playbook
文件：
playbooks/devops.yml
---
- name:
Deploy DevOps Platform
hosts:
management
become:
true
roles:
- devops15.12.5 DevOps入口
文件：
roles/devops/tasks/main.yml
---
- import_tasks:
gitlab.yml
- import_tasks:
gerrit.yml
- import_tasks:
jenkins.yml
- import_tasks:
nexus.yml
- import_tasks:
harbor.yml
- import_tasks:
sonarqube.ymlPart 15.12.6 Docker基础目录创建
所有服务统一：
/opt/devops/
├── gitlab
├── jenkins
├── nexus
├── harbor
└── sonar任务：
- name:
create devops dirs
file:
path:
"{{ devops_root }}/{{ item }}"
state:
directory
mode:
"0755"
loop:
- gitlab
- jenkins
- nexus
- harbor
- sonarPart 15.12.7 GitLab CE部署
Compose模板
文件：
roles/devops/templates/docker-compose-gitlab.yml.j2
services:
gitlab:
image:
{{ gitlab_image }}
container_name:
gitlab
restart:
always
hostname:
"{{ gitlab_domain }}"
ports:
- "80:80"
- "443:443"
- "2222:22"
volumes:
- /opt/devops/gitlab/config:/etc/gitlab
- /opt/devops/gitlab/logs:/var/log/gitlab
- /opt/devops/gitlab/data:/var/opt/gitlab
environment:
GITLAB_OMNIBUS_CONFIG:
|
external_url 'https://{{ gitlab_domain }}'
gitlab_rails['ldap_enabled'] = trueGitLab部署任务
文件：
roles/devops/tasks/gitlab.yml
---
- name:
deploy gitlab compose
template:
src:
docker-compose-gitlab.yml.j2
dest:
/opt/devops/gitlab/docker-compose.yml
- name:
start gitlab
community.docker.docker_compose:
project_src:
/opt/devops/gitlabGitLab验证
docker ps | grep gitlab访问：
https://gitlab.company.com15.12.8 Gerrit部署
架构：
Developer
|
Gerrit
|
Git Repository
|
Jenkins TriggerGerrit Compose
services:
gerrit:
image:
gerritcodereview/gerrit
container_name:
gerrit
ports:
- "8081:8080"
- "29418:29418"
volumes:
- /opt/devops/gerrit:/var/gerrit/review_siteTask
- name:
deploy gerrit
copy:
content:
"{{ lookup('template',
'docker-compose-gerrit.yml.j2') }}"
dest:
/opt/devops/gerrit/docker-compose.yml15.12.9 Jenkins部署
Jenkins Compose
文件：
docker-compose-jenkins.yml.j2
services:
jenkins:
image:
{{ jenkins_image }}
container_name:
jenkins
restart:
always
ports:
- "8080:8080"
- "50000:50000"
volumes:
- /opt/devops/jenkins:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sockJenkins任务
---
- name:
deploy jenkins
template:
src:
docker-compose-jenkins.yml.j2
dest:
/opt/devops/jenkins/docker-compose.yml
- name:
start jenkins
community.docker.docker_compose:
project_src:
/opt/devops/jenkinsJenkins Pipeline示例
项目：
Jenkinsfile
pipeline {
agent any
stages {
stage('Checkout'){
steps{
git 'https://gitlab.company.com/app.git'
}
}
stage('Build'){
steps{
sh 'docker build -t app:${BUILD_NUMBER} .'
}
}
stage('Push Harbor'){
steps{
sh '''
docker login harbor.company.com
docker push harbor.company.com/app:${BUILD_NUMBER}
'''
}
}
stage('Deploy'){
steps{
sh '''
kubectl rollout restart deployment app
'''
}
}
}
}15.12.10 Nexus Repository
用途：
- Maven
- npm
- Python
- Docker
Compose：
services:
nexus:
image:
sonatype/nexus3
ports:
- "8081:8081"
volumes:
- /opt/devops/nexus:/nexus-dataTask：
- name:
deploy nexus
community.docker.docker_compose:
project_src:
/opt/devops/nexus15.12.11 Harbor Registry
生产建议：
Harbor组件：
Harbor
├── Core
├── Portal
├── Registry
├── Trivy
├── Database
└── Redis安装方式：
wget https://github.com/goharbor/harbor/releases/download/v2.11.0/harbor-offline-installer.tgzAnsible：
- name:
install harbor
unarchive:
src:
harbor-offline-installer.tgz
dest:
/opt生成配置：
hostname:
harbor.company.com
https:
port:
443启动：
./install.sh15.12.12 SonarQube
Compose：
services:
sonarqube:
image:
sonarqube:lts-community
ports:
- "9000:9000"
environment:
SONAR_JDBC_URL:
jdbc:postgresql://postgres:5432/sonar部署：
- name:
start sonar
community.docker.docker_compose:
project_src:
/opt/devops/sonar15.12.13 LDAP统一认证
所有平台接入：
LDAP
|
+---------+---------+---------+
| | | |
GitLab Jenkins Harbor GrafanaGitLab LDAP配置：
gitlab_rails['ldap_servers'] = {
'main' => {
'label'=>'LDAP',
'host'=>'ldap.company.com',
'port'=>389,
'base'=>'dc=company,dc=com'
}
}Jenkins:
安装插件：
LDAP Plugin
Role Strategy Plugin15.12.14 CI/CD完整链路
代码提交
↓
GitLab
↓ webhook
Jenkins
↓
SonarQube
↓
Docker Build
↓
Harbor
↓
ArgoCD
↓
Kubernetes
↓
Production15.12.15 DevOps健康检查
文件：
scripts/check-devops.sh
#!/bin/bash
services=(
gitlab
jenkins
nexus
harbor
sonarqube
)
for s in ${services[@]}
do
docker ps | grep $s
done15.12完成
✅ GitLab CE Role
✅ Gerrit Role
✅ Jenkins Role
✅ Nexus Role
✅ Harbor部署框架
✅ SonarQube Role
✅ Docker Compose模板
✅ LDAP统一认证设计
✅ Jenkins Pipeline模板
✅ GitLab→Jenkins→Harbor→K8s流程
✅ 健康检查脚本
Part 15.13 Office Role完整代码
本部分完成企业协同办公平台自动化部署代码
包含：
- Jira Software
- Confluence
- Nextcloud
- Collabora Online
- ONLYOFFICE Docs
- Mail Server
- OpenLDAP
- LDAP统一认证
- PostgreSQL数据库
- Redis缓存
目标：
企业员工
|
Keycloak / LDAP
|
+------------+------------+
| | |
Jira Confluence Nextcloud
|
Collabora / OnlyOffice
|
PostgreSQL
|
Backup15.13.1 Office目录结构
创建：
enterprise-platform-ansible/
roles/
└── office/
│
├── defaults/
│ └── main.yml
│
├── tasks/
│ ├── main.yml
│ ├── postgres.yml
│ ├── openldap.yml
│ ├── jira.yml
│ ├── confluence.yml
│ ├── nextcloud.yml
│ ├── collabora.yml
│ ├── onlyoffice.yml
│ └── mail.yml
│
├── templates/
│
│ ├── docker-compose-jira.yml.j2
│ ├── docker-compose-confluence.yml.j2
│ ├── docker-compose-nextcloud.yml.j2
│ ├── docker-compose-collabora.yml.j2
│ ├── docker-compose-onlyoffice.yml.j2
│ ├── docker-compose-openldap.yml.j2
│ └── docker-compose-mail.yml.j2
│
└── handlers/
└── main.yml15.13.2 Office变量
文件：
roles/office/defaults/main.yml
---
office_root:
/opt/office
domain:
company.com
ldap_domain:
dc=company,dc=com
ldap_admin_password:
ChangeMeLDAP123!
postgres_password:
ChangeMeDB123!
jira_domain:
jira.company.com
confluence_domain:
confluence.company.com
nextcloud_domain:
cloud.company.com
onlyoffice_domain:
office.company.com
collabora_domain:
collabora.company.com
mail_domain:
company.com
postgres_image:
postgres:16
redis_image:
redis:715.13.3 Office Playbook
文件：
playbooks/office.yml
---
- name:
Deploy Office Platform
hosts:
management
become:
true
roles:
- office15.13.4 Office入口
文件：
roles/office/tasks/main.yml
---
- import_tasks:
postgres.yml
- import_tasks:
openldap.yml
- import_tasks:
jira.yml
- import_tasks:
confluence.yml
- import_tasks:
nextcloud.yml
- import_tasks:
collabora.yml
- import_tasks:
onlyoffice.yml
- import_tasks:
mail.yml15.13.5 创建目录
增加：
---
- name:
create office directories
file:
path:
"{{ office_root }}/{{ item }}"
state:
directory
loop:
- postgres
- ldap
- jira
- confluence
- nextcloud
- collabora
- onlyoffice
- mailPart 15.13.6 PostgreSQL部署
Jira / Confluence / Nextcloud数据库统一：
PostgreSQL
├── jira
├── confluence
└── nextcloudCompose
文件：
templates/docker-compose-postgres.yml.j2
services:
postgres:
image:
postgres:16
container_name:
office-postgres
restart:
always
environment:
POSTGRES_USER:
postgres
POSTGRES_PASSWORD:
{{ postgres_password }}
volumes:
- /opt/office/postgres:/var/lib/postgresql/data
ports:
- "5432:5432"Task
---
- name:
deploy postgres
template:
src:
docker-compose-postgres.yml.j2
dest:
/opt/office/postgres/docker-compose.yml
- name:
start postgres
community.docker.docker_compose:
project_src:
/opt/office/postgres15.13.7 OpenLDAP部署
架构：
LDAP
|
+---------+---------+
| | |
Jira GitLab JenkinsCompose
文件：
docker-compose-openldap.yml.j2
services:
ldap:
image:
osixia/openldap:latest
container_name:
openldap
restart:
always
environment:
LDAP_ORGANISATION:
Company
LDAP_DOMAIN:
company.com
LDAP_ADMIN_PASSWORD:
{{ ldap_admin_password }}
ports:
- "389:389"
- "636:636"
volumes:
- /opt/office/ldap/data:/var/lib/ldap
- /opt/office/ldap/config:/etc/ldap/slapd.dLDAP部署
---
- name:
deploy ldap
template:
src:
docker-compose-openldap.yml.j2
dest:
/opt/office/ldap/docker-compose.yml
- name:
start ldap
community.docker.docker_compose:
project_src:
/opt/office/ldapLDAP组织结构
文件：
base.ldif
dn: dc=company,dc=com
objectClass: top
objectClass: domain
dc: company
dn: ou=users,dc=company,dc=com
objectClass: organizationalUnit
ou: users
dn: ou=groups,dc=company,dc=com
objectClass: organizationalUnit
ou: groups导入：
ldapadd \
-x \
-D "cn=admin,dc=company,dc=com" \
-W \
-f base.ldif15.13.8 Jira部署
架构：
User
|
Jira
|
PostgreSQL
|
LDAPCompose
services:
jira:
image:
atlassian/jira-software
container_name:
jira
restart:
always
ports:
- "8080:8080"
volumes:
- /opt/office/jira:/var/atlassian/application-data/jiraTask
---
- name:
deploy jira
template:
src:
docker-compose-jira.yml.j2
dest:
/opt/office/jira/docker-compose.yml
- name:
start jira
community.docker.docker_compose:
project_src:
/opt/office/jiraJira LDAP配置
路径：
Jira Administration
-> User Management
-> LDAP参数：
Host:
openldap
Port:
389
Base DN:
dc=company,dc=com15.13.9 Confluence部署
Compose
services:
confluence:
image:
atlassian/confluence
container_name:
confluence
restart:
always
ports:
- "8090:8090"
volumes:
- /opt/office/confluence:/var/atlassian/application-data/confluenceTask
---
- name:
deploy confluence
community.docker.docker_compose:
project_src:
/opt/office/confluence15.13.10 Nextcloud部署
架构：
Browser
|
Nextcloud
|
Redis
|
PostgreSQL
|
LDAPCompose
services:
nextcloud:
image:
nextcloud:latest
container_name:
nextcloud
restart:
always
ports:
- "8088:80"
volumes:
- /opt/office/nextcloud:/var/www/html
redis:
image:
redis:7
restart:
alwaysLDAP配置
Nextcloud：
Administration
>
LDAP user and group backend
LDAP Host:
openldap
Base:
dc=company,dc=com15.13.11 Collabora Online
用于：
- 在线编辑Word
- Excel
- PPT
Compose：
services:
collabora:
image:
collabora/code
container_name:
collabora
restart:
always
ports:
- "9980:9980"
environment:
domain:
cloud.company.com
username:
admin
password:
ChangeMe15.13.12 ONLYOFFICE
Compose：
services:
onlyoffice:
image:
onlyoffice/documentserver
container_name:
onlyoffice
restart:
always
ports:
- "8082:80"
volumes:
- /opt/office/onlyoffice:/var/www/onlyoffice/DataNextcloud集成OnlyOffice
配置：
Apps
|
ONLYOFFICE connector
Document Server:
http://onlyoffice15.13.13 Mail Server
推荐：
- Mailcow
- Poste.io
- iRedMail
生产推荐：
Mailcow
↓
Postfix
Dovecot
Rspamd
ClamAVCompose结构
mail/
├── postfix
├── dovecot
├── rspamd
├── clamav
└── nginxMailcow部署
Ansible：
---
- name:
install mailcow
git:
repo:
https://github.com/mailcow/mailcow-dockerized
dest:
/opt/office/mailcow
- name:
start mailcow
shell:
cmd:
./generate_config.sh && docker compose up -d
chdir:
/opt/office/mailcow15.13.14 SMTP DNS配置
生产必须：
MX
company.com
MX
mail.company.comSPF
v=spf1 mx ~allDKIM
由Rspamd生成
DMARC
_dmarc.company.com
v=DMARC1;p=quarantine15.13.15 Office统一认证
最终：
LDAP
|
+---------+------+-------+
| | |
Jira Confluence Nextcloud
|
GitLab
|
Jenkins15.13.16 Office健康检查
文件：
scripts/check-office.sh
#!/bin/bash
services=(
postgres
openldap
jira
confluence
nextcloud
collabora
onlyoffice
)
for s in ${services[@]}
do
echo $s
docker ps | grep $s
done15.13完成
✅ PostgreSQL办公数据库
✅ OpenLDAP目录服务
✅ Jira部署
✅ Confluence部署
✅ Nextcloud部署
✅ Collabora Online
✅ ONLYOFFICE Docs
✅ Mail Server框架
✅ LDAP统一认证
✅ SMTP规划
✅ 健康检查脚本
Part 15.14 Monitoring Role完整代码
本部分生成企业级监控平台自动化部署代码。
包含：
- Prometheus
- Alertmanager
- Grafana
- Loki
- Promtail
- ELK Stack
- Elasticsearch
- Logstash
- Kibana
- Thanos长期存储
- Node Exporter
- Kubernetes监控
- Docker监控
- 告警规则
- Dashboard自动导入
适用于：
Ubuntu 22.04
Ubuntu 24.04
Docker
Kubernetes 1.30+
Prometheus 3.x
Grafana 11+
Loki 3.x
Thanos 0.37+15.14.1 Monitoring整体架构
Kubernetes
|
|
kube-state-metrics
|
|
Node Exporter -------- Prometheus -------- Alertmanager
|
|
Thanos Sidecar
|
|
Object Storage
(MinIO/S3)
|
Long Term Metrics
Logs:
Container Logs
|
Promtail
|
Loki
|
GrafanaELK：
Application
|
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana15.14.2 Monitoring目录结构
创建：
roles/monitoring/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── prometheus.yml
│ ├── alertmanager.yml
│ ├── grafana.yml
│ ├── loki.yml
│ ├── promtail.yml
│ ├── elasticsearch.yml
│ ├── logstash.yml
│ ├── kibana.yml
│ ├── thanos.yml
│ └── exporters.yml
├── templates/
│ ├── docker-compose-monitoring.yml.j2
│ ├── prometheus.yml.j2
│ ├── alertmanager.yml.j2
│ ├── loki-config.yml.j2
│ └── thanos.yml.j2
├── files/
│
└── dashboards/15.14.3 Monitoring变量
文件：
roles/monitoring/defaults/main.yml
---
monitoring_root:
/opt/monitoring
prometheus_version:
v3.1.0
grafana_image:
grafana/grafana:latest
alertmanager_version:
v0.28.0
loki_version:
3.3.0
thanos_version:
v0.37.2
elasticsearch_version:
8.15.0
prometheus_port:
9090
grafana_port:
3000
alertmanager_port:
9093
loki_port:
3100
thanos_port:
1090115.14.4 Monitoring Playbook
文件：
playbooks/monitoring.yml
---
- name:
Deploy Monitoring Platform
hosts:
monitoring
become:
true
roles:
- monitoring15.14.5 Monitoring入口
文件：
roles/monitoring/tasks/main.yml
---
- import_tasks:
prometheus.yml
- import_tasks:
alertmanager.yml
- import_tasks:
grafana.yml
- import_tasks:
loki.yml
- import_tasks:
promtail.yml
- import_tasks:
elasticsearch.yml
- import_tasks:
logstash.yml
- import_tasks:
kibana.yml
- import_tasks:
thanos.yml
- import_tasks:
exporters.yml15.14.6 创建目录
---
- name:
create monitoring directories
file:
path:
"{{ monitoring_root }}/{{ item }}"
state:
directory
loop:
- prometheus
- alertmanager
- grafana
- loki
- promtail
- elasticsearch
- logstash
- kibana
- thanosPart 15.14.7 Prometheus部署
Prometheus配置
文件：
templates/prometheus.yml.j2
global:
scrape_interval:
15s
rule_files:
- /etc/prometheus/rules/*.yml
scrape_configs:
- job_name:
prometheus
static_configs:
- targets:
- localhost:9090
- job_name:
node
static_configs:
- targets:
- node-exporter:9100
- job_name:
kubernetes
kubernetes_sd_configs:
- role:
nodeDocker Compose
services:
prometheus:
image:
prom/prometheus:latest
container_name:
prometheus
restart:
always
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
- /opt/monitoring/prometheus/data:/prometheus
command:
- "--storage.tsdb.retention.time=30d"部署
---
- name:
deploy prometheus
template:
src:
prometheus.yml.j2
dest:
/opt/monitoring/prometheus/prometheus.yml
- name:
start prometheus
community.docker.docker_compose:
project_src:
/opt/monitoring/prometheusPrometheus验证
访问：
http://monitoring.company.com:9090检查：
curl localhost:9090/-/healthy15.14.8 Alertmanager
架构：
Prometheus
|
Alert Rules
|
Alertmanager
|
+----+----+
| |
Email WeChat配置：
global:
smtp_smarthost:
mail.company.com:587
route:
receiver:
email
receivers:
- name:
email
email_configs:
- to:
ops@company.com部署：
---
- name:
deploy alertmanager
community.docker.docker_compose:
project_src:
/opt/monitoring/alertmanager15.14.9 Grafana部署
Compose：
services:
grafana:
image:
grafana/grafana:latest
ports:
- "3000:3000"
volumes:
- /opt/monitoring/grafana:/var/lib/grafana启动：
---
- name:
deploy grafana
community.docker.docker_compose:
project_src:
/opt/monitoring/grafana默认：
URL:
http://monitoring.company.com:3000
账号:
admin
密码:
admin自动添加Prometheus数据源
文件：
grafana-datasource.yml
apiVersion:
1
datasources:
- name:
Prometheus
type:
prometheus
url:
http://prometheus:9090
access:
proxy15.14.10 Loki日志平台
架构：
Container Log
|
Promtail
|
Loki
|
GrafanaLoki配置：
auth_enabled:
false
server:
http_listen_port:
3100
storage_config:
filesystem:
directory:
/loki/chunks部署：
---
- name:
deploy loki
community.docker.docker_compose:
project_src:
/opt/monitoring/loki15.14.11 Promtail
配置：
server:
http_listen_port:
9080
clients:
- url:
http://loki:3100/loki/api/v1/push
positions:
filename:
/tmp/positions.yaml
scrape_configs:
- job_name:
docker
static_configs:
- targets:
- localhost
labels:
job:
docker
__path__:
/var/lib/docker/containers/*/*.log15.14.12 ELK Stack
Elasticsearch
Compose：
services:
elasticsearch:
image:
docker.elastic.co/elasticsearch/elasticsearch:8.15.0
environment:
discovery.type:
single-node
xpack.security.enabled:
false
ports:
- "9200:9200"Logstash
配置：
input {
beats {
port => 5044
}
}
filter {
json {
}
}
output {
elasticsearch {
hosts => [
"elasticsearch:9200"
]
}
}Kibana
Compose：
services:
kibana:
image:
docker.elastic.co/kibana/kibana:8.15.0
ports:
- "5601:5601"15.14.13 Thanos长期存储
架构：
Prometheus
|
Thanos Sidecar
|
MinIO/S3
|
Thanos Store
|
GrafanaThanos启动：
---
- name:
install thanos
docker_container:
name:
thanos-sidecar
image:
quay.io/thanos/thanos:v0.37.2
command:
- sidecar
- --prometheus.url=http://prometheus:909015.14.14 Node Exporter
所有服务器安装
---
- name:
deploy node exporter
docker_container:
name:
node-exporter
image:
prom/node-exporter
ports:
- "9100:9100"
restart_policy:
always15.14.15 Kubernetes监控组件
安装：
kube-state-metrics
kubectl apply -f \
https://github.com/kubernetes/kube-state-metrics/releases/latest/download/standard-install.yamlMetrics Server
kubectl apply -f \
https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml15.14.16 Prometheus告警规则
目录：
rules/
├── node.yml
├── kubernetes.yml
└── application.yml示例：
groups:
- name:
node
rules:
- alert:
HighCPU
expr:
100-avg(rate(node_cpu_seconds_total{mode="idle"}[5m]))*100 > 90
for:
5m
labels:
severity:
critical15.14.17 Grafana Dashboard
自动导入：
dashboards:
node-exporter:
id:
1860
kubernetes:
id:
31515.14.18 监控健康检查
文件：
scripts/check-monitoring.sh
#!/bin/bash
services=(
prometheus
grafana
alertmanager
loki
elasticsearch
kibana
)
for s in ${services[@]}
do
docker ps | grep $s
done15.14完成
✅ Prometheus Role
✅ Alertmanager Role
✅ Grafana Role
✅ Loki日志平台
✅ Promtail日志采集
✅ ELK Stack
✅ Elasticsearch
✅ Logstash
✅ Kibana
✅ Thanos长期存储
✅ Node Exporter
✅ Kubernetes监控
✅ Grafana Dashboard自动导入
✅ 企业告警规则
Part 15.15 Backup & Disaster Recovery Role完整代码
本部分完成企业级备份与灾难恢复自动化部署代码
包含：
- Kubernetes Velero
- Restic
- BorgBackup
- Rsync增量备份
- PostgreSQL备份
- MySQL备份
- MongoDB备份
- Elasticsearch Snapshot
- MinIO/S3备份
- 定时备份
- 自动恢复
- 灾难恢复演练
适用于：
Ubuntu 22.04/24.04
Kubernetes 1.28+
Docker
MinIO/S3
PostgreSQL
MongoDB
MySQL
ElasticSearch15.15.1 Backup整体架构
Production
|
+---------------+----------------+
| |
Kubernetes Standalone
| |
Velero Restic
| |
+---------------+----------------+
|
Object Storage
MinIO / S3
|
Disaster Recovery15.15.2 Backup目录结构
创建：
roles/backup/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── velero.yml
│ ├── restic.yml
│ ├── borg.yml
│ ├── rsync.yml
│ ├── database.yml
│ ├── elasticsearch.yml
│ ├── schedule.yml
│ └── recovery.yml
├── templates/
│ ├── velero-values.yml.j2
│ ├── restic-backup.sh.j2
│ ├── postgres-backup.sh.j2
│ ├── mysql-backup.sh.j2
│ ├── mongodb-backup.sh.j2
│ └── crontab.j2
└── handlers/
└── main.yml15.15.3 Backup变量
文件：
roles/backup/defaults/main.yml
---
backup_root:
/backup
backup_storage:
minio
backup_bucket:
backup
backup_endpoint:
https://minio.company.com
backup_retention:
30
velero_namespace:
velero
restic_repository:
s3:https://minio.company.com/backup
restic_password:
ChangeMeRestic123
postgres_host:
postgres
postgres_user:
postgres
mysql_host:
mysql
mongodb_host:
mongodb15.15.4 Backup Playbook
文件：
playbooks/backup.yml
---
- name:
Deploy Backup System
hosts:
backup
become:
true
roles:
- backup15.15.5 Backup入口
文件：
roles/backup/tasks/main.yml
---
- import_tasks:
velero.yml
- import_tasks:
restic.yml
- import_tasks:
borg.yml
- import_tasks:
rsync.yml
- import_tasks:
database.yml
- import_tasks:
elasticsearch.yml
- import_tasks:
schedule.yml
- import_tasks:
recovery.yml15.15.6 创建备份目录
---
- name:
create backup directories
file:
path:
"{{ backup_root }}/{{ item }}"
state:
directory
loop:
- database
- kubernetes
- files
- scripts
- logsPart 15.15.7 Velero Kubernetes备份
架构
Kubernetes
|
Velero
|
Restic
|
MinIO安装CLI
任务：
---
- name:
download velero
get_url:
url:
https://github.com/vmware-tanzu/velero/releases/latest/download/velero-linux-amd64.tar.gz
dest:
/tmp/velero.tar.gz安装：
- name:
install velero
unarchive:
src:
/tmp/velero.tar.gz
dest:
/usr/local/bin
remote_src:
yesVelero部署
使用helm：
---
- name:
add velero repo
kubernetes.core.helm_repository:
name:
vmware
repo_url:
https://vmware-tanzu.github.io/helm-charts安装：
- name:
install velero
kubernetes.core.helm:
name:
velero
chart_ref:
vmware-tanzu/velero
namespace:
velero
create_namespace:
yesVelero配置
values：
configuration:
provider:
aws
backupStorageLocation:
- name:
default
provider:
aws
bucket:
backup
config:
s3Url:
http://minio:9000
s3ForcePathStyle:
true创建备份
velero backup create \
cluster-backup \
--include-namespaces "*"查看：
velero backup get恢复
velero restore create \
--from-backup cluster-backup15.15.8 Restic文件备份
用途：
- 配置文件
- 数据目录
- 上传文件
安装
---
- name:
install restic
get_url:
url:
https://github.com/restic/restic/releases/latest/download/restic_linux_amd64.bz2
dest:
/tmp/restic.bz2安装：
- name:
install binary
shell:
cmd:
bunzip2 /tmp/restic.bz2 && mv /tmp/restic /usr/local/bin/resticRestic初始化
export RESTIC_PASSWORD=xxxx
restic \
-r s3:https://minio.company.com/backup \
initRestic备份脚本
文件：
templates/restic-backup.sh.j2
#!/bin/bash
export RESTIC_PASSWORD={{ restic_password }}
restic \
-r {{ restic_repository }} \
backup \
/etc \
/opt \
/data
restic \
-r {{ restic_repository }} \
forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 12 \
--prune15.15.9 BorgBackup
适用于：
- 本地备份
- SSH备份服务器
安装：
- name:
install borg
apt:
name:
borgbackup
state:
present备份：
borg create \
backup@server:/repo::{now} \
/etc \
/data15.15.10 Rsync增量备份
文件：
templates/rsync-backup.sh.j2
#!/bin/bash
SOURCE=/data
TARGET=backup@backup-server:/backup/data
rsync \
-avz \
--delete \
--progress \
$SOURCE \
$TARGET15.15.11 PostgreSQL备份
脚本：
postgres-backup.sh
#!/bin/bash
DATE=$(date +%F)
pg_dump \
-h {{ postgres_host }} \
-U {{ postgres_user }} \
-F c \
all_db \
>
{{ backup_root }}/database/postgres-$DATE.dump定时：
0 2 * * * /backup/scripts/postgres-backup.sh15.15.12 MySQL备份
脚本：
#!/bin/bash
mysqldump \
-h mysql \
-u root \
-pPASSWORD \
--all-databases \
>
mysql.sql15.15.13 MongoDB备份
#!/bin/bash
mongodump \
--host mongodb \
--out /backup/mongodb/$(date +%F)15.15.14 Elasticsearch Snapshot
创建仓库：
PUT _snapshot/backup
{
"type":"s3",
"settings":{
"bucket":"backup"
}
}执行：
PUT _snapshot/backup/snapshot-00115.15.15 自动备份计划
文件：
templates/crontab.j2
# Kubernetes
0 1 * * *
velero backup create daily
# Restic
0 2 * * *
/backup/scripts/restic-backup.sh
# Database
0 3 * * *
/backup/scripts/postgres-backup.sh
# Rsync
0 4 * * *
/backup/scripts/rsync-backup.sh部署：
---
- name:
install backup cron
cron:
name:
"{{ item.name }}"
minute:
"{{ item.minute }}"
job:
"{{ item.job }}"15.15.16 灾难恢复流程
场景1：Kubernetes集群损坏
恢复：
velero restore create \
--from-backup daily恢复：
- Namespace
- Deployment
- Service
- ConfigMap
- Secret
- PVC
场景2：数据库损坏
PostgreSQL：
pg_restore \
-d database \
backup.dump场景3：文件恢复
Restic：
查看：
restic snapshots恢复：
restic restore latest --target /15.15.17 Backup健康检查
文件：
scripts/check-backup.sh
#!/bin/bash
echo "Velero"
velero backup get
echo "Restic"
restic snapshots
echo "Disk"
df -h /backup15.15.18 生产备份策略
推荐：
| 类型 | 周期 | 保留 |
|---|---|---|
| Kubernetes | 每天 | 30天 |
| PostgreSQL | 每天 | 30天 |
| MongoDB | 每天 | 30天 |
| 文件 | 每小时 | 7天 |
| 全量 | 每周 | 6个月 |
| 月备份 | 每月 | 1年 |
Part 15.15完成
✅ Velero Kubernetes备份
✅ Restic对象存储备份
✅ BorgBackup
✅ Rsync增量备份
✅ PostgreSQL备份
✅ MySQL备份
✅ MongoDB备份
✅ Elasticsearch Snapshot
✅ MinIO/S3备份
✅ 自动Cron计划
✅ 恢复流程
✅ 灾备演练流程
Part 15.16 Security Role完整代码
本部分生成企业级安全平台自动化部署代码。
包含：
- Hashicorp Vault 密钥管理
- Keycloak IAM身份认证
- OAuth2 Proxy统一登录
- Cert-Manager自动证书
- Sealed Secrets Kubernetes Secret加密
- Trivy安全扫描
- Fail2ban主机防护
- Auditd审计
- CIS安全加固
- Container Security
- Kubernetes Security
适用于：
Ubuntu 22.04 / 24.04
Docker
Kubernetes 1.28+
Helm 3+
Ingress-Nginx
MetalLB / HAProxy15.16.1 Security整体架构
User
|
|
Keycloak IAM
|
+-----------------+-----------------+
| | |
GitLab Jenkins Grafana
| | |
+-----------------+-----------------+
|
OAuth2 Proxy
|
Kubernetes
|
+-----------------+----------------+
| | |
Vault Cert-manager Sealed Secret
| | |
Secrets TLS证书 加密配置15.16.2 Security目录结构
创建：
roles/security/
├── defaults/
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── vault.yml
│ ├── keycloak.yml
│ ├── oauth2-proxy.yml
│ ├── cert-manager.yml
│ ├── sealed-secret.yml
│ ├── trivy.yml
│ ├── fail2ban.yml
│ ├── auditd.yml
│ └── hardening.yml
├── templates/
│ ├── docker-compose-vault.yml.j2
│ ├── docker-compose-keycloak.yml.j2
│ ├── oauth2-values.yml.j2
│ ├── vault-config.hcl.j2
│ └── security-cron.j2
└── files/
└── policies/15.16.3 Security变量
文件：
roles/security/defaults/main.yml
---
security_root:
/opt/security
vault_version:
1.17
vault_domain:
vault.company.com
keycloak_domain:
auth.company.com
oauth2_domain:
login.company.com
keycloak_admin:
admin
keycloak_password:
ChangeMeKeycloak123
vault_password:
ChangeMeVault123
trivy_version:
latest
cert_manager_version:
v1.16.215.16.4 Security Playbook
文件：
playbooks/security.yml
---
- name:
Deploy Security Platform
hosts:
security
become:
true
roles:
- security15.16.5 Security入口
文件：
roles/security/tasks/main.yml
---
- import_tasks:
vault.yml
- import_tasks:
keycloak.yml
- import_tasks:
oauth2-proxy.yml
- import_tasks:
cert-manager.yml
- import_tasks:
sealed-secret.yml
- import_tasks:
trivy.yml
- import_tasks:
fail2ban.yml
- import_tasks:
auditd.yml
- import_tasks:
hardening.yml15.16.6 创建目录
---
- name:
create security directories
file:
path:
"{{ security_root }}/{{ item }}"
state:
directory
loop:
- vault
- keycloak
- trivy
- policiesPart 15.16.7 Hashicorp Vault部署
架构
Application
|
|
Vault Agent
|
|
Vault Server
|
|
Encrypted StorageDocker Compose
文件：
templates/docker-compose-vault.yml.j2
services:
vault:
image:
hashicorp/vault:1.17
container_name:
vault
restart:
always
cap_add:
- IPC_LOCK
ports:
- "8200:8200"
volumes:
- /opt/security/vault:/vault/config
command:
vault server -config=/vault/config/vault.hclVault配置
文件：
vault-config.hcl.j2
ui = true
storage "file" {
path="/vault/data"
}
listener "tcp" {
address="0.0.0.0:8200"
tls_disable=true
}部署：
---
- name:
deploy vault
template:
src:
docker-compose-vault.yml.j2
dest:
/opt/security/vault/docker-compose.yml
- name:
start vault
community.docker.docker_compose:
project_src:
/opt/security/vaultVault初始化
docker exec -it vault vault operator init输出：
Unseal Key 1
Unseal Key 2
Unseal Key 3
Root TokenVault启用KV
vault secrets enable kv创建：
vault kv put secret/database \
password=xxxx15.16.8 Keycloak IAM部署
功能：
- SSO
- OAuth2
- LDAP Federation
- MFA
- RBAC
架构：
LDAP
|
Keycloak
|
SSO
|
GitLab/Jenkins/JiraCompose
services:
keycloak:
image:
quay.io/keycloak/keycloak:latest
container_name:
keycloak
restart:
always
ports:
- "8080:8080"
environment:
KEYCLOAK_ADMIN:
admin
KEYCLOAK_ADMIN_PASSWORD:
{{ keycloak_password }}
command:
start-dev部署：
---
- name:
deploy keycloak
community.docker.docker_compose:
project_src:
/opt/security/keycloakKeycloak LDAP连接
路径：
Realm
|
User Federation
|
LDAP参数：
Connection URL:
ldap://openldap:389
Base DN:
dc=company,dc=com
Bind DN:
cn=admin,dc=company,dc=com15.16.9 OAuth2 Proxy
用途：
保护：
- Grafana
- Kibana
- Jenkins
- Harbor
Helm Values：
config:
clientID:
oauth2
clientSecret:
secret
provider:
oidc
oidcIssuerURL:
https://auth.company.com/realms/company
cookieSecret:
xxxx部署：
---
- name:
install oauth2 proxy
kubernetes.core.helm:
name:
oauth2-proxy
chart_ref:
oauth2-proxy/oauth2-proxy15.16.10 Cert-Manager
功能：
自动申请：
- Let's Encrypt
- 企业CA
安装：
---
- name:
install cert-manager
kubernetes.core.helm:
name:
cert-manager
chart_ref:
jetstack/cert-manager
namespace:
cert-manager
create_namespace:
yes
values:
installCRDs:
trueClusterIssuer
文件：
apiVersion:
cert-manager.io/v1
kind:
ClusterIssuer
metadata:
name:
letsencrypt
spec:
acme:
server:
https://acme-v02.api.letsencrypt.org/directory
email:
admin@company.com
solvers:
- http01:
ingress:
class:
nginx15.16.11 Sealed Secrets
用途：
Git保存加密Secret。
安装：
---
- name:
install sealed secret
kubernetes.core.helm:
name:
sealed-secrets
chart_ref:
bitnami-labs/sealed-secrets创建：
kubeseal \
--format yaml \
<secret.yaml \
>sealed-secret.yaml15.16.12 Trivy漏洞扫描
扫描：
- Image
- Filesystem
- Kubernetes
安装：
---
- name:
install trivy
apt:
name:
trivy
state:
present扫描镜像：
trivy image nginx:latest扫描K8s：
trivy k8s cluster15.16.13 Fail2ban
保护：
- SSH
- Nginx
- VPN
安装：
---
- name:
install fail2ban
apt:
name:
fail2ban
state:
present配置：
[sshd]
enabled=true
maxretry=5
bantime=3600启动：
systemd:
name:
fail2ban
state:
started
enabled:
yes15.16.14 Auditd审计
安装：
---
- name:
install auditd
apt:
name:
auditd
state:
present规则：
-w /etc/passwd -p wa
-w /etc/sudoers -p wa
-w /var/log -p wa15.16.15 Linux安全加固
SSH加固
模板：
/etc/ssh/sshd_config配置：
PermitRootLogin no
PasswordAuthentication no
MaxAuthTries 3Kernel安全参数
---
- name:
enable security sysctl
sysctl:
name:
"{{ item.key }}"
value:
"{{ item.value }}"
loop:
- key:
net.ipv4.ip_forward
value:
0
- key:
kernel.randomize_va_space
value:
215.16.16 Container安全策略
Docker:
docker run \
--read-only \
--cap-drop ALL \
--security-opt=no-new-privilegesKubernetes安全策略
启用：
PodSecurity:
restricted示例：
apiVersion:
v1
kind:
Namespace
metadata:
labels:
pod-security.kubernetes.io/enforce:
restricted15.16.17 Security扫描计划
Cron：
# Image Scan
0 1 * * *
trivy image company/app
# Kubernetes Scan
0 2 * * *
trivy k8s cluster
# Audit
0 3 * * *
ausearch -ts today15.16.18 Security健康检查
文件：
scripts/check-security.sh
#!/bin/bash
echo "Vault"
curl localhost:8200/v1/sys/health
echo "Keycloak"
curl localhost:8080
echo "Cert Manager"
kubectl get certificate -A
echo "Secrets"
kubectl get sealedsecret -APart 15.16完成
✅ Vault密钥管理
✅ Keycloak IAM
✅ LDAP Federation
✅ OAuth2 Proxy
✅ Cert-manager TLS自动化
✅ Sealed Secrets
✅ Trivy漏洞扫描
✅ Fail2ban防护
✅ Auditd审计
✅ Linux Hardening
✅ Kubernetes Security
✅ Container Security
Part 15.17 Network & HA Role完整代码
本部分生成企业级网络、高可用、负载均衡自动化部署代码。
包含：
- Keepalived VIP高可用
- HAProxy四层/七层负载均衡
- Nginx反向代理
- Bind9 DNS服务器
- Squid Proxy企业代理
- OpenVPN远程接入
- WireGuard VPN
- MetalLB Kubernetes LoadBalancer
- Firewall安全策略
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Debian 12
Kubernetes 1.28+
Docker
HAProxy 3.x
Keepalived 2.x
Bind9
OpenVPN 2.6
WireGuard15.17.1 Network & HA整体架构
Internet
|
|
Firewall
|
|
Keepalived VIP
10.10.10.100
+----------+----------+
| |
HAProxy1 HAProxy2
MASTER BACKUP
| |
+----------+----------+
|
Kubernetes / Apps
|
Internal DNS
|
Bind9 DNS Cluster
VPN:
Remote User
|
OpenVPN/WireGuard
|
Corporate Network15.17.2 Network目录结构
创建：
roles/network-ha/
├── defaults/
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── keepalived.yml
│ ├── haproxy.yml
│ ├── nginx.yml
│ ├── bind9.yml
│ ├── squid.yml
│ ├── openvpn.yml
│ ├── wireguard.yml
│ ├── metallb.yml
│ └── firewall.yml
├── templates/
│ ├── keepalived.conf.j2
│ ├── haproxy.cfg.j2
│ ├── nginx.conf.j2
│ ├── named.conf.j2
│ ├── db.company.com.j2
│ ├── squid.conf.j2
│ ├── openvpn-server.conf.j2
│ └── wg0.conf.j2
└── handlers/
└── main.yml15.17.3 Network变量
文件：
roles/network-ha/defaults/main.yml
---
network_domain:
company.com
vip_address:
10.10.10.100
interface:
ens192
keepalived_router_id:
51
haproxy_port:
80
https_port:
443
dns_domain:
company.com
dns_server:
10.10.10.53
proxy_port:
3128
vpn_network:
10.20.0.0/24
openvpn_port:
1194
wireguard_port:
5182015.17.4 Network Playbook
文件：
playbooks/network-ha.yml
---
- name:
Deploy Network HA
hosts:
network
become:
true
roles:
- network-ha15.17.5 主入口
文件：
roles/network-ha/tasks/main.yml
---
- import_tasks:
keepalived.yml
- import_tasks:
haproxy.yml
- import_tasks:
nginx.yml
- import_tasks:
bind9.yml
- import_tasks:
squid.yml
- import_tasks:
openvpn.yml
- import_tasks:
wireguard.yml
- import_tasks:
metallb.yml
- import_tasks:
firewall.ymlPart 15.17.6 Keepalived VIP
架构
VIP
10.10.10.100
|
+-------+-------+
| |
HAProxy1 HAProxy2
MASTER BACKUP安装：
---
- name:
install keepalived
apt:
name:
keepalived
state:
present配置模板：
文件：
templates/keepalived.conf.j2
vrrp_instance VI_1 {
state {{ keepalived_state }}
interface {{ interface }}
virtual_router_id {{ keepalived_router_id }}
priority {{ keepalived_priority }}
advert_int 1
authentication {
auth_type PASS
auth_pass password
}
virtual_ipaddress {
{{ vip_address }}
}
}部署：
---
- name:
configure keepalived
template:
src:
keepalived.conf.j2
dest:
/etc/keepalived/keepalived.conf
notify:
restart keepalivedHandler:
---
- name:
restart keepalived
service:
name:
keepalived
state:
restarted验证：
ip addr show输出：
10.10.10.10015.17.7 HAProxy部署
用途：
- Kubernetes API HA
- Ingress HA
- Web服务
架构：
Client
|
VIP
|
HAProxy
|
+-----+-----+
| |
K8S-M1 K8S-M2安装：
---
- name:
install haproxy
apt:
name:
haproxy
state:
present配置：
文件：
templates/haproxy.cfg.j2
global
log /dev/log local0
maxconn 50000
defaults
mode tcp
timeout connect 10s
timeout client 1h
timeout server 1h
frontend kubernetes_api
bind *:6443
default_backend kube_masters
backend kube_masters
balance roundrobin
server master1 10.10.10.11:6443 check
server master2 10.10.10.12:6443 check
frontend http
bind *:80
mode http
default_backend ingress
backend ingress
mode http
balance roundrobin
server ingress1 10.10.10.21:80 check
server ingress2 10.10.10.22:80 check15.17.8 Nginx反向代理
用途：
- Web入口
- SSL终止
- API Gateway
安装：
---
- name:
install nginx
apt:
name:
nginx
state:
present配置：
server {
listen 443 ssl;
server_name gitlab.company.com;
location / {
proxy_pass http://gitlab;
proxy_set_header Host $host;
}
}15.17.9 Bind9 DNS
架构：
DNS-MASTER
10.10.10.53
DNS-SLAVE
10.10.10.54安装：
---
- name:
install bind9
apt:
name:
- bind9
- bind9utils
state:
presentnamed.conf：
zone "company.com" {
type master;
file "/etc/bind/db.company.com";
};区域文件：
db.company.com.j2
$TTL 86400
@ IN SOA dns.company.com. admin.company.com.
@ IN NS dns.company.com.
gitlab IN A 10.10.10.20
jenkins IN A 10.10.10.30
harbor IN A 10.10.10.40
grafana IN A 10.10.10.50
vault IN A 10.10.10.6015.17.10 Squid Proxy
用途：
- 软件仓库代理
- Docker代理
- 出口控制
安装：
---
- name:
install squid
apt:
name:
squid
state:
present配置：
http_port 3128
acl localnet src 10.0.0.0/8
http_access allow localnet
cache_dir aufs /var/spool/squid 10000 16 25615.17.11 OpenVPN
架构：
Laptop
|
VPN
|
OpenVPN Server
|
Internal Network安装：
---
- name:
install openvpn
apt:
name:
- openvpn
- easy-rsa
state:
present配置：
port 1194
proto udp
dev tun
server 10.20.0.0 255.255.255.0
push "route 10.10.0.0 255.255.0.0"
keepalive 10 120启动：
systemctl enable openvpn-server@server
systemctl start openvpn-server@server15.17.12 WireGuard
优势：
- 高性能
- 移动端友好
安装：
---
- name:
install wireguard
apt:
name:
wireguard配置：
wg0.conf
[Interface]
Address = 10.30.0.1/24
ListenPort = 51820
PrivateKey = SERVER_KEY
[Peer]
PublicKey = CLIENT_KEY
AllowedIPs = 10.30.0.2/32启动：
systemctl enable wg-quick@wg0
systemctl start wg-quick@wg015.17.13 MetalLB Kubernetes
用途：
裸机Kubernetes LoadBalancer。
安装：
---
- name:
install metallb
kubernetes.core.k8s:
src:
https://raw.githubusercontent.com/metallb/metallb/v0.14.9/config/manifests/metallb-native.yaml地址池：
apiVersion:
metallb.io/v1beta1
kind:
IPAddressPool
metadata:
name:
default
spec:
addresses:
- 10.10.10.200-10.10.10.25015.17.14 Firewall
使用：
- UFW
- Firewalld
Ubuntu：
---
- name:
enable firewall
ufw:
state:
enabled开放端口：
- name:
allow ports
ufw:
rule:
allow
port:
"{{ item }}"
loop:
- 22
- 80
- 443
- 6443
- 1194
- 5182015.17.15 Network健康检查
文件：
scripts/check-network.sh
#!/bin/bash
echo "VIP"
ip addr |grep 10.10.10.100
echo "HAProxy"
systemctl status haproxy
echo "DNS"
dig company.com
echo "VPN"
ss -tunlp |grep -E "1194|51820"15.17完成
✅ Keepalived VIP高可用
✅ HAProxy负载均衡
✅ Kubernetes API HA
✅ Nginx反向代理
✅ Bind9 DNS
✅ Squid企业代理
✅ OpenVPN
✅ WireGuard
✅ MetalLB
✅ Firewall策略
✅ 网络健康检查
Part 15.18 Storage & Data Platform Role完整代码
本部分完成企业级 Storage & Data Platform 自动化部署 Ansible Role
包含：
- SeaweedFS 分布式文件系统
- MinIO 对象存储
- Ceph 集群
- Rook-Ceph Kubernetes存储
- NFS共享存储
- MongoDB ReplicaSet
- PostgreSQL HA
- MySQL InnoDB Cluster
- Redis Cluster
- Kafka Cluster
- Elasticsearch
- TDengine
- 数据备份集成
- StorageClass自动化
适用于：
Ubuntu 22.04 / 24.04
Debian 12
CentOS Stream 9
Kubernetes 1.28+
Docker Compose
Ansible >=2.1515.18.1 Storage整体架构
Application Layer
|
|
Kubernetes PVC / S3 API
|
+--------------+---------------+
| | |
Rook-Ceph SeaweedFS MinIO
| | |
| | |
Block/RBD File Object
|
|
Ceph Cluster
Data Services:
+-------------+-------------+-------------+
| | |
MongoDB PostgreSQL MySQL
| | |
Redis Kafka TDengine15.18.2 Role目录结构
创建：
roles/storage-data/
├── defaults/
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── seaweedfs.yml
│ ├── minio.yml
│ ├── ceph.yml
│ ├── rook-ceph.yml
│ ├── nfs.yml
│ ├── mongodb.yml
│ ├── postgresql.yml
│ ├── mysql.yml
│ ├── redis.yml
│ ├── kafka.yml
│ ├── elasticsearch.yml
│ ├── tdengine.yml
│ └── storageclass.yml
├── templates/
│ ├── docker-compose-minio.yml.j2
│ ├── seaweedfs-master.service.j2
│ ├── seaweedfs-volume.service.j2
│ ├── mongodb-compose.yml.j2
│ ├── postgres-compose.yml.j2
│ ├── redis-compose.yml.j2
│ ├── kafka-compose.yml.j2
│ └── elasticsearch.yml.j2
└── handlers/
└── main.yml15.18.3 默认变量
文件：
roles/storage-data/defaults/main.yml---
storage_root:
/mnt/storage
storage_network:
10.10.10.0/24
################
# SeaweedFS
################
seaweedfs_version:
latest
seaweedfs_master_port:
9333
seaweedfs_volume_port:
8080
################
# MinIO
################
minio_root:
/mnt/storage/minio
minio_user:
minio
minio_password:
ChangeMeMinio123
################
# Ceph
################
ceph_version:
reef
################
# MongoDB
################
mongodb_version:
7.0
mongodb_root:
/mnt/storage/mongodb
################
# PostgreSQL
################
postgres_version:
16
################
# Redis
################
redis_version:
7.2
################
# Kafka
################
kafka_version:
3.8
################
# Elasticsearch
################
elastic_version:
8.15
################
# TDengine
################
tdengine_version:
3.315.18.4 Playbook
文件：
playbooks/storage-data.yml---
- name:
Deploy Storage Data Platform
hosts:
storage
become:
true
roles:
- storage-data15.18.5 Role入口
文件：
tasks/main.yml---
- import_tasks:
nfs.yml
- import_tasks:
seaweedfs.yml
- import_tasks:
minio.yml
- import_tasks:
ceph.yml
- import_tasks:
rook-ceph.yml
- import_tasks:
mongodb.yml
- import_tasks:
postgresql.yml
- import_tasks:
mysql.yml
- import_tasks:
redis.yml
- import_tasks:
kafka.yml
- import_tasks:
elasticsearch.yml
- import_tasks:
tdengine.yml
- import_tasks:
storageclass.ymlPart 15.18.6 NFS共享存储
用途：
- Jenkins Workspace
- Kubernetes RWX PVC
- 文件共享
安装：
---
- name:
install nfs
apt:
name:
- nfs-server
state:
present创建目录：
- name:
create nfs directory
file:
path:
/mnt/storage/nfs
state:
directoryexports：
/mnt/storage/nfs
10.10.10.0/24
(rw,sync,no_root_squash)加载：
exportfs -ra验证：
showmount -e localhost15.18.7 SeaweedFS
架构
Filer
|
+-------+-------+
| |
Master1 Master2
|
|
Volume ServerMaster Service
模板：
templates/seaweedfs-master.service.j2[Unit]
Description=SeaweedFS Master
[Service]
ExecStart=/usr/local/bin/weed master \
-ip={{ inventory_hostname }} \
-port=9333
Restart=always
[Install]
WantedBy=multi-user.target安装：
---
- name:
download weed
get_url:
url:
https://github.com/seaweedfs/seaweedfs/releases/latest/download/linux_amd64.tar.gz
dest:
/tmp/weed.tar.gz解压：
- unarchive:
src:
/tmp/weed.tar.gz
dest:
/usr/local/bin
remote_src:
yes启动：
- systemd:
name:
seaweedfs-master
enabled:
yes
state:
startedVolume Server
配置：
weed volume \
-server.port=8080 \
-dir=/mnt/storage/volume验证：
curl http://localhost:9333/cluster/status15.18.8 MinIO对象存储
Docker Compose
模板：
docker-compose-minio.yml.j2services:
minio:
image:
minio/minio:latest
container_name:
minio
restart:
always
ports:
- 9000:9000
- 9001:9001
environment:
MINIO_ROOT_USER:
{{ minio_user }}
MINIO_ROOT_PASSWORD:
{{ minio_password }}
volumes:
- {{ minio_root }}:/data
command:
server /data --console-address ":9001"启动：
docker compose up -d验证：
mc alias set minio http://localhost:900015.18.9 Ceph集群
推荐：
生产：
cephadm组件：
MON
MGR
OSD
MDS
RGW安装：
---
- name:
install cephadm
shell:
cmd:
curl --silent --remote-name \
https://download.ceph.com/rpm-reef/el9/noarch/cephadm创建：
cephadm bootstrap \
--mon-ip 10.10.10.11验证：
ceph -s15.18.10 Rook-Ceph Kubernetes
安装：
---
- name:
install rook
kubernetes.core.k8s:
src:
https://raw.githubusercontent.com/rook/rook/master/deploy/examples/crds.yaml部署：
kubectl apply \
-f rook-ceph-operator.yamlStorageClass：
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
ceph-rbd
provisioner:
rook-ceph.rbd.csi.ceph.com15.18.11 MongoDB ReplicaSet
Compose:
services:
mongo:
image:
mongo:7
command:
mongod --replSet rs0
volumes:
- /mnt/storage/mongo:/data/db初始化：
rs.initiate(
{
_id:"rs0",
members:[
{
_id:0,
host:"mongo:27017"
}
]
}
)15.18.12 PostgreSQL
Compose:
services:
postgres:
image:
postgres:16
environment:
POSTGRES_PASSWORD:
password
volumes:
- /mnt/storage/postgres:/var/lib/postgresql/data备份：
pg_dumpall > backup.sql15.18.13 MySQL
部署：
services:
mysql:
image:
mysql:8.4
environment:
MYSQL_ROOT_PASSWORD:
password
volumes:
- /mnt/storage/mysql:/var/lib/mysql15.18.14 Redis Cluster
配置：
services:
redis:
image:
redis:7.2
command:
redis-server --cluster-enabled yes创建：
redis-cli \
--cluster create \
10.10.10.1:6379 \
10.10.10.2:637915.18.15 Kafka
架构：
Kafka
|
Zookeeper/KRaft
|
Broker1
Broker2
Broker3Compose:
services:
kafka:
image:
bitnami/kafka:latest
environment:
KAFKA_ENABLE_KRAFT:
yes
KAFKA_CFG_NODE_ID:
1测试：
kafka-topics.sh --list15.18.16 Elasticsearch
配置：
services:
elasticsearch:
image:
elasticsearch:8.15
environment:
discovery.type:
single-node
volumes:
- /mnt/storage/es:/usr/share/elasticsearch/data验证：
curl localhost:920015.18.17 TDengine
部署：
services:
tdengine:
image:
tdengine/tdengine:latest
ports:
- 6041:6041
volumes:
- /mnt/storage/tdengine:/var/lib/taos验证：
taos15.18.18 StorageClass统一管理
目录：
storageclass.yml支持：
ceph-rbd
cephfs
nfs
seaweedfs
minio-s3示例：
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
nfs-storage
provisioner:
k8s-sigs.io/nfs-subdir-external-provisioner15.18.19 健康检查
文件：
scripts/check-storage.sh#!/bin/bash
echo "SeaweedFS"
curl localhost:9333/cluster/status
echo "MinIO"
mc admin info minio
echo "Ceph"
ceph -s
echo "MongoDB"
mongosh --eval "rs.status()"
echo "PostgreSQL"
pg_isready
echo "Redis"
redis-cli ping
echo "Kafka"
kafka-topics.sh --listPart 15.18完成
已生成：
✅ SeaweedFS分布式文件系统
✅ MinIO对象存储
✅ Ceph存储集群
✅ Rook-Ceph Kubernetes存储
✅ NFS共享存储
✅ MongoDB ReplicaSet
✅ PostgreSQL
✅ MySQL
✅ Redis Cluster
✅ Kafka KRaft
✅ Elasticsearch
✅ TDengine
✅ Kubernetes StorageClass
✅ 存储健康检查
Part 15.19 Enterprise Data Service & Middleware Role完整代码
本部分生成企业级 Middleware 中间件平台自动化部署 Ansible Role
包含：
- RabbitMQ 消息队列
- Nacos 服务注册配置中心
- ZooKeeper 集群
- etcd 集群
- Consul 服务发现
- Apache Pulsar 消息平台
- Apache Flink 流计算
- Apache Spark 大数据计算
- Apache Airflow 工作流调度
- Debezium CDC数据同步
- Data Pipeline 数据流水线
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Docker Compose
Kubernetes 1.28+
Helm 3
Ansible >=2.1515.19.1 Middleware整体架构
Application
|
+-----------------+-----------------+
| | |
Nacos Consul etcd
(Service Discovery) (Service Mesh) (Kubernetes)
|
|
+----+-----+
| |
RabbitMQ Pulsar
MQ Streaming
|
|
+--------+---------+
| |
Flink Spark
Stream Batch
Compute Compute
|
|
Data Lake
|
Airflow
ETL Workflow
|
Debezium
CDC15.19.2 Middleware目录结构
创建：
roles/middleware/
├── defaults/
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── rabbitmq.yml
│ ├── nacos.yml
│ ├── zookeeper.yml
│ ├── etcd.yml
│ ├── consul.yml
│ ├── pulsar.yml
│ ├── flink.yml
│ ├── spark.yml
│ ├── airflow.yml
│ ├── debezium.yml
│ └── datapipeline.yml
├── templates/
│
│ ├── rabbitmq-compose.yml.j2
│ ├── nacos-compose.yml.j2
│ ├── zookeeper-compose.yml.j2
│ ├── etcd-compose.yml.j2
│ ├── consul-compose.yml.j2
│ ├── pulsar-compose.yml.j2
│ ├── flink-compose.yml.j2
│ ├── spark-compose.yml.j2
│ ├── airflow-compose.yml.j2
│ └── debezium-compose.yml.j2
└── handlers/
└── main.yml15.19.3 Middleware变量
文件：
roles/middleware/defaults/main.yml
---
middleware_root:
/opt/middleware
################
# RabbitMQ
################
rabbitmq_version:
3.13
rabbitmq_user:
admin
rabbitmq_password:
ChangeMeRabbit123
################
# Nacos
################
nacos_version:
2.4
################
# ZooKeeper
################
zookeeper_version:
3.9
################
# etcd
################
etcd_version:
3.5
################
# Consul
################
consul_version:
1.19
################
# Pulsar
################
pulsar_version:
3.3
################
# Flink
################
flink_version:
1.20
################
# Spark
################
spark_version:
3.5
################
# Airflow
################
airflow_version:
2.10
################
# Debezium
################
debezium_version:
3.015.19.4 Playbook
文件：
playbooks/middleware.yml
---
- name:
Deploy Enterprise Middleware
hosts:
middleware
become:
true
roles:
- middleware15.19.5 Role入口
文件：
tasks/main.yml
---
- import_tasks:
rabbitmq.yml
- import_tasks:
nacos.yml
- import_tasks:
zookeeper.yml
- import_tasks:
etcd.yml
- import_tasks:
consul.yml
- import_tasks:
pulsar.yml
- import_tasks:
flink.yml
- import_tasks:
spark.yml
- import_tasks:
airflow.yml
- import_tasks:
debezium.yml
- import_tasks:
datapipeline.ymlPart 15.19.6 RabbitMQ
用途：
- 微服务消息
- 异步任务
- 事件驱动
架构：
Application
|
RabbitMQ Cluster
|
Exchange
|
QueueDocker Compose
模板：
services:
rabbitmq:
image:
rabbitmq:3.13-management
container_name:
rabbitmq
restart:
always
ports:
- 5672:5672
- 15672:15672
environment:
RABBITMQ_DEFAULT_USER:
{{ rabbitmq_user }}
RABBITMQ_DEFAULT_PASS:
{{ rabbitmq_password }}
volumes:
- {{ middleware_root }}/rabbitmq:/var/lib/rabbitmq部署：
- name:
deploy rabbitmq
community.docker.docker_compose:
project_src:
"{{ middleware_root }}/rabbitmq"验证：
rabbitmqctl status15.19.7 Nacos服务注册配置中心
用途：
- Spring Cloud
- 配置管理
- 服务发现
架构：
Service
|
Nacos
|
MySQLCompose:
services:
nacos:
image:
nacos/nacos-server:v2.4
environment:
MODE:
standalone
NACOS_AUTH_ENABLE:
true
ports:
- 8848:8848
volumes:
- ./data:/home/nacos/data验证：
curl http://localhost:8848/nacos15.19.8 ZooKeeper
用途：
- Kafka旧架构
- 分布式协调
Compose：
services:
zookeeper:
image:
zookeeper:3.9
ports:
- 2181:2181
volumes:
- zk:/data验证：
echo ruok | nc localhost 2181返回：
imok15.19.9 etcd
用途：
- Kubernetes核心存储
- 分布式KV
Compose:
services:
etcd:
image:
quay.io/coreos/etcd:v3.5
command:
etcd
--name=node1
--data-dir=/etcd-data
volumes:
- ./data:/etcd-data验证：
etcdctl member list15.19.10 Consul
用途：
- Service Mesh
- 服务发现
Compose:
services:
consul:
image:
hashicorp/consul:1.19
ports:
- 8500:8500
command:
agent -server -ui \
-bootstrap-expect=1 \
-client=0.0.0.0验证：
curl localhost:8500/v1/catalog/services15.19.11 Apache Pulsar
用途：
- 高吞吐消息
- Streaming
架构：
Producer
|
Broker
|
BookKeeper
|
StorageCompose：
services:
pulsar:
image:
apachepulsar/pulsar:3.3
command:
bin/pulsar standalone
ports:
- 6650:6650
- 8080:8080测试：
pulsar-client produce test-topic \
-m "hello"15.19.12 Apache Flink
用途：
- 实时计算
- AI Pipeline
架构：
JobManager
|
TaskManagerCompose：
services:
jobmanager:
image:
flink:1.20
command:
jobmanager
ports:
- 8081:8081
taskmanager:
image:
flink:1.20
command:
taskmanager验证：
curl localhost:808115.19.13 Apache Spark
用途：
- ETL
- ML
- Batch计算
Compose：
services:
spark:
image:
bitnami/spark:3.5
environment:
SPARK_MODE:
master
ports:
- 8080:8080测试：
spark-submit app.py15.19.14 Apache Airflow
用途：
- 数据编排
- 自动任务
架构：
Scheduler
|
Executor
|
WorkerCompose：
services:
airflow:
image:
apache/airflow:2.10
ports:
- 8080:8080
environment:
AIRFLOW__CORE__EXECUTOR:
LocalExecutor初始化：
airflow db init15.19.15 Debezium CDC
用途：
数据库实时同步。
支持：
MySQL
PostgreSQL
MongoDB
Oracle
SQLServer架构：
Database
|
Debezium Connector
|
Kafka
|
ConsumerCompose：
services:
debezium:
image:
debezium/connect:3.0
ports:
- 8083:8083
environment:
BOOTSTRAP_SERVERS:
kafka:9092创建Connector：
POST /connectors
{
"name":"mysql-sync",
"config":
{
"connector.class":
"io.debezium.connector.mysql.MySqlConnector"
}
}15.19.16 Data Pipeline
目录：
datapipeline/
├── ingest
├── transform
├── load
└── scheduler示例：
pipeline:
source:
mysql
transform:
flink
sink:
kafka15.19.17 Middleware健康检查
文件：
scripts/check-middleware.sh
#!/bin/bash
echo RabbitMQ
rabbitmqctl status
echo Nacos
curl localhost:8848/nacos
echo Consul
curl localhost:8500/v1/status/leader
echo Pulsar
pulsar-admin brokers list
echo Flink
curl localhost:8081
echo Airflow
curl localhost:808015.19完成
✅ RabbitMQ消息队列
✅ Nacos注册中心
✅ ZooKeeper
✅ etcd集群
✅ Consul服务发现
✅ Pulsar消息平台
✅ Flink流计算
✅ Spark计算平台
✅ Airflow数据编排
✅ Debezium CDC同步
✅ Data Pipeline框架
✅ Ansible自动部署结构
✅ Docker Compose模板
Part 15.20 AI Platform & MLOps Role完整代码
本部分生成企业级 AI Platform & MLOps 自动化部署 Ansible Role。
覆盖企业 AI 基础平台：
- NVIDIA GPU Driver 自动安装
- CUDA Toolkit
- NVIDIA Container Toolkit
- Kubernetes GPU Node
- GPU Operator
- Kubeflow ML Platform
- KServe Model Serving
- MLflow 实验管理
- JupyterHub AI开发环境
- Ray 分布式训练平台
- vLLM LLM推理服务
- Ollama本地模型服务
- LangChain应用框架
- LlamaIndex知识库框架
- Feast Feature Store
- Model Registry
- AI Pipeline
适用于：
Ubuntu 22.04 / 24.04
NVIDIA RTX 4080/4090
RTX PRO 5000 Blackwell
A100/H100
Kubernetes 1.28+
Docker
containerd
Ansible >=2.1515.20.1 AI Platform整体架构
User
|
|
API Gateway
|
Model Serving
|
+-----------------+------------------+
| | |
vLLM KServe Ollama
LLM API Kubernetes ML Local LLM
|
|
Kubernetes GPU Cluster
+-----------------+----------------+
| |
GPU Operator Kubeflow
| |
NVIDIA Runtime Training Pipeline
|
|
Data Platform
+-----------------+----------------+
| | |
MLflow Feast MinIO
Model Registry Feature Store Model Data15.20.2 Role目录结构
创建：
roles/ai-platform/
├── defaults/
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── gpu-driver.yml
│ ├── cuda.yml
│ ├── container-toolkit.yml
│ ├── gpu-operator.yml
│ ├── kubeflow.yml
│ ├── kserve.yml
│ ├── mlflow.yml
│ ├── jupyterhub.yml
│ ├── ray.yml
│ ├── vllm.yml
│ ├── ollama.yml
│ ├── langchain.yml
│ ├── llamaindex.yml
│ ├── feast.yml
│ └── model-registry.yml
├── templates/
│
│ ├── mlflow-compose.yml.j2
│ ├── vllm-compose.yml.j2
│ ├── ollama-compose.yml.j2
│ ├── ray-cluster.yaml.j2
│ └── jupyterhub-config.py.j2
└── handlers/
└── main.yml15.20.3 默认变量
文件：
roles/ai-platform/defaults/main.yml
---
ai_root:
/opt/ai
#################
# NVIDIA
#################
nvidia_driver_version:
550
cuda_version:
12.6
#################
# Kubernetes GPU
#################
gpu_operator_version:
v24.6
#################
# MLflow
#################
mlflow_version:
latest
mlflow_port:
5000
#################
# Jupyter
#################
jupyterhub_port:
8000
#################
# Ray
#################
ray_version:
2.35
#################
# vLLM
#################
vllm_model:
Qwen/Qwen2.5-7B-Instruct
vllm_port:
8000
#################
# Ollama
#################
ollama_port:
1143415.20.4 Playbook
文件：
playbooks/ai-platform.yml
---
- name:
Deploy AI Platform
hosts:
gpu_nodes
become:
true
roles:
- ai-platform15.20.5 Role入口
文件：
tasks/main.yml
---
- import_tasks:
gpu-driver.yml
- import_tasks:
cuda.yml
- import_tasks:
container-toolkit.yml
- import_tasks:
gpu-operator.yml
- import_tasks:
kubeflow.yml
- import_tasks:
kserve.yml
- import_tasks:
mlflow.yml
- import_tasks:
jupyterhub.yml
- import_tasks:
ray.yml
- import_tasks:
vllm.yml
- import_tasks:
ollama.yml
- import_tasks:
langchain.yml
- import_tasks:
llamaindex.yml
- import_tasks:
feast.yml
- import_tasks:
model-registry.yml15.20.6 NVIDIA GPU Driver自动安装
安装依赖
---
- name:
install gpu dependency
apt:
name:
- gcc
- make
- dkms
- linux-headers-{{ ansible_kernel }}
state:
presentNVIDIA Driver
- name:
install nvidia driver
apt:
name:
- nvidia-driver-{{ nvidia_driver_version }}
state:
present重启：
- name:
reboot gpu node
reboot:验证：
nvidia-smi输出：
GPU Name
RTX PRO 5000
CUDA Version: 12.615.20.7 CUDA Toolkit
安装：
---
- name:
install cuda
apt:
name:
- nvidia-cuda-toolkit
state:
present验证：
nvcc --version15.20.8 NVIDIA Container Toolkit
用途：
Docker/Kubernetes调用GPU。
安装：
---
- name:
install nvidia container toolkit
apt:
name:
- nvidia-container-toolkit
state:
present配置 Docker：
nvidia-ctk runtime configure \
--runtime=docker重启：
systemctl restart docker测试：
docker run \
--rm \
--gpus all \
nvidia/cuda:12.6-base \
nvidia-smi15.20.9 Kubernetes GPU Operator
安装：
---
- name:
install gpu operator
kubernetes.core.helm:
name:
gpu-operator
chart_ref:
nvidia/gpu-operator
namespace:
gpu-operator验证：
kubectl get pods -n gpu-operator输出：
nvidia-device-plugin
nvidia-container-toolkit
gpu-feature-discovery15.20.10 Kubeflow
用途：
- AI训练平台
- Notebook
- Pipeline
安装：
---
- name:
deploy kubeflow
kubernetes.core.k8s:
src:
https://github.com/kubeflow/manifests/releases/latest/download/kustomization.yaml组件：
Kubeflow Dashboard
Notebook
Katib
Training Operator
Pipeline
Model Registry访问：
https://kubeflow.company.com15.20.11 KServe模型服务
用途：
生产模型部署
架构：
Model
|
KServe
|
InferenceService
|
GPU Pod部署：
apiVersion:
serving.kserve.io/v1beta1
kind:
InferenceService
metadata:
name:
qwen-model
spec:
predictor:
model:
modelFormat:
name:
pytorch
storageUri:
s3://models/qwen验证：
kubectl get inferenceservice15.20.12 MLflow
用途：
- 实验管理
- 模型注册
Compose：
services:
mlflow:
image:
ghcr.io/mlflow/mlflow
ports:
- 5000:5000
command:
mlflow server \
--host 0.0.0.0 \
--backend-store-uri sqlite:///mlflow.db启动：
docker compose up -d访问：
http://host:500015.20.13 JupyterHub
用途：
AI开发环境。
安装：
---
- name:
install jupyterhub
pip:
name:
- jupyterhub
- notebook配置：
模板：
c.JupyterHub.bind_url=":8000"
c.Spawner.default_url="/lab"启动：
jupyterhub15.20.14 Ray Cluster
用途：
- 分布式训练
- LLM Fine-tuning
Kubernetes：
apiVersion:
ray.io/v1
kind:
RayCluster
metadata:
name:
ai-cluster
spec:
headGroupSpec:
template:
spec:
containers:
- name:
ray-head
image:
rayproject/ray:2.35查看：
kubectl get raycluster15.20.15 vLLM LLM推理平台
用途：
高性能大模型API。
Compose：
services:
vllm:
image:
vllm/vllm-openai
runtime:
nvidia
command:
--model {{ vllm_model }}
ports:
- 8000:8000启动：
docker compose up -d测试：
curl http://localhost:8000/v1/models15.20.16 Ollama
用途：
本地模型服务。
Compose：
services:
ollama:
image:
ollama/ollama
runtime:
nvidia
ports:
- 11434:11434
volumes:
- /opt/ai/models:/root/.ollama下载模型：
ollama pull qwen2.5测试：
curl localhost:11434/api/generate15.20.17 LangChain
安装：
---
- name:
install langchain
pip:
name:
- langchain
- langchain-community示例：
from langchain.llms import Ollama
llm=Ollama(
model="qwen2.5"
)
print(
llm.invoke(
"hello"
)
)15.20.18 LlamaIndex
安装：
pip:
name:
- llama-index用途：
Document
|
Embedding
|
Vector DB
|
LLM15.20.19 Feast Feature Store
用途：
机器学习特征管理。
安装：
pip:
name:
- feast初始化：
feast init fraud_project15.20.20 Model Registry
统一模型生命周期：
Training
|
MLflow
|
Registry
|
KServe
|
Production配置：
model_registry:
backend:
mlflow
storage:
s3://models15.20.21 AI健康检查
文件：
scripts/check-ai.sh
#!/bin/bash
echo GPU
nvidia-smi
echo Docker GPU
docker run \
--rm \
--gpus all \
nvidia/cuda \
nvidia-smi
echo MLflow
curl localhost:5000
echo vLLM
curl localhost:8000/v1/models
echo Ollama
curl localhost:11434/api/tags
echo Kubernetes GPU
kubectl get nodes -o json |grep nvidiaPart 15.20完成
✅ NVIDIA Driver自动化
✅ CUDA Toolkit
✅ NVIDIA Container Toolkit
✅ GPU Operator
✅ Kubernetes GPU节点
✅ Kubeflow
✅ KServe
✅ MLflow
✅ JupyterHub
✅ Ray Cluster
✅ vLLM推理平台
✅ Ollama
✅ LangChain
✅ LlamaIndex
✅ Feast Feature Store
✅ Model Registry
✅ AI健康检查
Part 15.21 Enterprise Application Platform Role完整代码
本部分生成企业级 Application Platform 应用运行平台自动化部署 Ansible Role。
覆盖企业应用生命周期：
- Spring Boot 微服务运行环境
- Node.js 前端运行环境
- Python FastAPI运行环境
- Golang服务运行环境
- .NET运行环境
- PHP运行环境
- Tomcat应用服务器
- Apache HTTP Server
- Nginx Gateway
- API Gateway
- Service Mesh
- Istio
- Envoy Proxy
- 应用可观测性集成
- OpenTelemetry链路追踪
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Docker
Kubernetes
containerd
Ansible >=2.15
Spring Boot 3.x
Node.js 22+
Python 3.12+
Go 1.23+
.NET 815.21.1 Application Platform整体架构
User
|
|
Load Balancer
|
|
API Gateway
|
+------------+-------------+
| |
Nginx Ingress Envoy
|
|
Kubernetes Service Mesh
|
+------------+-------------+-------------+
| | | |
Java Python Node.js Go
Spring FastAPI Vue/React Gin
| | | |
+------------+-------------+-------------+
Application Runtime
|
|
Database / MQ / Storage
|
|
Observability Platform
Prometheus + Grafana + Loki + Tempo15.21.2 Role目录结构
创建：
roles/application-platform/
├── defaults/
│
│ └── main.yml
├── tasks/
│
│ ├── main.yml
│ ├── docker.yml
│ ├── nginx.yml
│ ├── apache.yml
│ ├── tomcat.yml
│ ├── java.yml
│ ├── nodejs.yml
│ ├── python.yml
│ ├── golang.yml
│ ├── dotnet.yml
│ ├── php.yml
│ ├── api-gateway.yml
│ ├── istio.yml
│ ├── envoy.yml
│ ├── servicemesh.yml
│ └── observability.yml
├── templates/
│
│ ├── nginx.conf.j2
│ ├── springboot.service.j2
│ ├── tomcat.service.j2
│ ├── fastapi.service.j2
│ ├── node.service.j2
│ ├── envoy.yaml.j2
│ └── otel-config.yaml.j2
└── handlers/
└── main.yml15.21.3 默认变量
文件：
roles/application-platform/defaults/main.yml
---
app_root:
/opt/apps
##################
# Java
##################
java_version:
21
##################
# Node
##################
node_version:
22
##################
# Python
##################
python_version:
3.12
##################
# Golang
##################
go_version:
1.23
##################
# .NET
##################
dotnet_version:
8
##################
# Tomcat
##################
tomcat_version:
10.1
##################
# Nginx
##################
nginx_port:
80
https_port:
443
##################
# Gateway
##################
gateway_type:
nginx
##################
# Istio
##################
istio_version:
1.23
##################
# Observability
##################
otel_endpoint:
http://otel-collector:431715.21.4 Playbook
文件：
playbooks/application-platform.yml
---
- name:
Deploy Application Platform
hosts:
app_nodes
become:
true
roles:
- application-platform15.21.5 Role入口
文件：
tasks/main.yml
---
- import_tasks:
docker.yml
- import_tasks:
java.yml
- import_tasks:
nodejs.yml
- import_tasks:
python.yml
- import_tasks:
golang.yml
- import_tasks:
dotnet.yml
- import_tasks:
php.yml
- import_tasks:
nginx.yml
- import_tasks:
apache.yml
- import_tasks:
tomcat.yml
- import_tasks:
api-gateway.yml
- import_tasks:
envoy.yml
- import_tasks:
istio.yml
- import_tasks:
servicemesh.yml
- import_tasks:
observability.yml15.21.6 Docker Runtime
应用统一运行基础。
安装：
---
- name:
install docker
apt:
name:
- docker.io
- docker-compose-plugin
state:
present启动：
- name:
enable docker
service:
name:
docker
enabled:
yes
state:
started验证：
docker version15.21.7 Java Spring Boot Runtime
安装JDK
---
- name:
install java
apt:
name:
- openjdk-21-jdk
state:
present验证：
java -versionSpring Boot Service模板
文件：
templates/springboot.service.j2
[Unit]
Description=Spring Boot Application
After=network.target
[Service]
User=app
WorkingDirectory={{ app_root }}/{{ app_name }}
ExecStart=/usr/bin/java \
-jar {{ app_root }}/{{ app_name }}/app.jar
Restart=always
[Install]
WantedBy=multi-user.target部署：
- template:
src:
springboot.service.j2
dest:
/etc/systemd/system/{{app_name}}.service启动：
systemctl enable app
systemctl start app15.21.8 Node.js Runtime
安装：
---
- name:
install nodejs
shell:
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -安装：
- apt:
name:
- nodejs
state:
present验证：
node -v
npm -vNode应用Service
模板：
[Service]
WorkingDirectory=/opt/apps/frontend
ExecStart=/usr/bin/npm run start
Restart=always15.21.9 Python FastAPI Runtime
安装：
---
- name:
install python
apt:
name:
- python3.12
- python3-pip
state:
presentFastAPI服务：
[Service]
WorkingDirectory=/opt/apps/api
ExecStart=/opt/apps/api/.venv/bin/uvicorn \
main:app \
--host 0.0.0.0 \
--port 8080 \
--workers 4
Restart=always启动：
systemctl start fastapi15.21.10 Golang Runtime
安装：
---
- name:
install golang
apt:
name:
- golang-go
state:
present验证：
go version运行：
./app15.21.11 .NET Runtime
安装：
---
- name:
install dotnet
apt:
name:
- dotnet-sdk-8.0
state:
present验证：
dotnet --version15.21.12 PHP Runtime
安装：
---
- name:
install php
apt:
name:
- php8.3
- php8.3-fpm
state:
present15.21.13 Nginx Gateway
用途：
- SSL终止
- 反向代理
- 静态资源
安装：
---
- name:
install nginx
apt:
name:
nginx
state:
present配置：
模板：
templates/nginx.conf.j2
upstream backend {
server app-service:8080;
}
server {
listen 80;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
}
}15.21.14 Apache
安装：
---
- name:
install apache
apt:
name:
apache2
state:
present15.21.15 Tomcat
安装：
---
- name:
install tomcat
apt:
name:
- tomcat10
state:
present部署：
cp app.war /var/lib/tomcat10/webapps/15.21.16 API Gateway
支持：
- Nginx
- Kong
- APISIX
- Spring Cloud Gateway
变量：
gateway_type:
apisix部署APISIX：
services:
apisix:
image:
apache/apisix
ports:
- 9080:908015.21.17 Envoy Proxy
用途：
Service Mesh数据面。
配置：
模板：
static_resources:
listeners:
- name:
http
address:
socket_address:
address:
0.0.0.0
port_value:
8080
clusters:
- name:
backend
connect_timeout:
1s启动：
envoy -c envoy.yaml15.21.18 Istio Service Mesh
安装：
---
- name:
install istio
shell:
cmd:
istioctl install --set profile=production -y启用Sidecar：
kubectl label namespace default \
istio-injection=enabled验证：
kubectl get pods显示：
app
istio-proxy15.21.19 OpenTelemetry
用途：
统一链路追踪。
组件：
Application
|
OTEL SDK
|
Collector
|
Tempo / Jaeger配置：
receivers:
otlp:
protocols:
grpc:
exporters:
otlp:
endpoint:
{{ otel_endpoint }}15.21.20 应用监控集成
Prometheus：
metrics:
enabled:
true
endpoint:
/metricsSpring Boot：
增加：
micrometer-registry-prometheus访问：
/actuator/prometheusFastAPI：
安装：
pip install prometheus-fastapi-instrumentator15.21.21 应用健康检查
文件：
scripts/check-app.sh
#!/bin/bash
echo Java
systemctl status springboot
echo Node
pm2 list
echo Python
curl localhost:8080/health
echo Nginx
curl localhost
echo Kubernetes
kubectl get podsPart 15.21完成
✅ Docker运行环境
✅ Java Spring Boot平台
✅ Node.js平台
✅ Python FastAPI平台
✅ Golang平台
✅ .NET平台
✅ PHP平台
✅ Nginx Gateway
✅ Apache
✅ Tomcat
✅ API Gateway
✅ Envoy Proxy
✅ Istio Service Mesh
✅ OpenTelemetry
✅ Prometheus应用监控集成
✅ 企业应用健康检查
Part 15.22 Enterprise Database Platform Role完整代码
本部分生成企业级 Database Platform 数据库平台自动化部署 Ansible Role
覆盖企业生产数据库体系：
- MySQL 8.4 HA
- PostgreSQL 16 HA
- Patroni + Etcd 高可用
- MongoDB ReplicaSet
- Redis Cluster
- ProxySQL 数据库代理
- PgBouncer连接池
- SQL Server
- Oracle Client
- ClickHouse OLAP分析数据库
- TDengine时序数据库
- InfluxDB
- TimescaleDB
- 数据库备份恢复
- 自动化巡检
适用于：
Ubuntu 22.04 / 24.04
Debian 12
CentOS Stream 9
Docker
Kubernetes
Ansible >=2.15
生产HA环境15.22.1 Database Platform整体架构
Application
|
|
Database Proxy
+------------+-------------+
| |
ProxySQL PgBouncer
| |
+--------+---------+ +-------+-------+
| | | |
MySQL Cluster Redis PostgreSQL Patroni
|
|
Replication
|
|
+------+-------+
| |
Backup Monitoring
| |
S3 Prometheus
Analytics:
ClickHouse
TDengine
InfluxDB15.22.2 Role目录结构
创建：
roles/database-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── mysql.yml
│ ├── postgresql.yml
│ ├── patroni.yml
│ ├── mongodb.yml
│ ├── redis.yml
│ ├── proxysql.yml
│ ├── pgbouncer.yml
│ ├── sqlserver.yml
│ ├── oracle-client.yml
│ ├── clickhouse.yml
│ ├── tdengine.yml
│ ├── influxdb.yml
│ ├── timescaledb.yml
│ └── backup.yml
├── templates/
│
│ ├── mysql-compose.yml.j2
│ ├── postgres-compose.yml.j2
│ ├── patroni.yml.j2
│ ├── redis-compose.yml.j2
│ ├── proxysql.cnf.j2
│ ├── pgbouncer.ini.j2
│ ├── clickhouse.yml.j2
│ └── backup.sh.j2
└── handlers/
└── main.yml15.22.3 默认变量
文件：
roles/database-platform/defaults/main.yml
---
database_root:
/mnt/database
#################
# MySQL
#################
mysql_version:
8.4
mysql_root_password:
ChangeMeMysql123
#################
# PostgreSQL
#################
postgres_version:
16
postgres_password:
ChangeMePostgres123
#################
# MongoDB
#################
mongodb_version:
7.0
#################
# Redis
#################
redis_version:
7.2
#################
# Patroni
#################
patroni_version:
3.3
#################
# ProxySQL
#################
proxysql_version:
2.6
#################
# ClickHouse
#################
clickhouse_version:
24.8
#################
# TDengine
#################
tdengine_version:
3.3
#################
# InfluxDB
#################
influxdb_version:
2.715.22.4 Playbook
文件：
playbooks/database-platform.yml
---
- name:
Deploy Database Platform
hosts:
database
become:
true
roles:
- database-platform15.22.5 Role入口
文件：
tasks/main.yml
---
- import_tasks:
mysql.yml
- import_tasks:
postgresql.yml
- import_tasks:
patroni.yml
- import_tasks:
mongodb.yml
- import_tasks:
redis.yml
- import_tasks:
proxysql.yml
- import_tasks:
pgbouncer.yml
- import_tasks:
sqlserver.yml
- import_tasks:
oracle-client.yml
- import_tasks:
clickhouse.yml
- import_tasks:
tdengine.yml
- import_tasks:
influxdb.yml
- import_tasks:
timescaledb.yml
- import_tasks:
backup.yml15.22.6 MySQL 8.4部署
架构
Application
|
ProxySQL
|
+-----------+-----------+
| |
MySQL-Master MySQL-Slave
Replication安装：
---
- name:
install mysql
apt:
name:
- mysql-server
state:
present初始化：
mysql_secure_installation创建复制用户：
CREATE USER 'repl'
@'%'
IDENTIFIED BY 'password';
GRANT REPLICATION SLAVE
ON *.*
TO 'repl'@'%';查看状态：
SHOW MASTER STATUS;15.22.7 PostgreSQL
安装：
---
- name:
install postgresql
apt:
name:
- postgresql-16
state:
present配置：
listen_addresses='*'
max_connections=500
wal_level=replica
archive_mode=on创建用户：
CREATE USER app
WITH PASSWORD 'password';
CREATE DATABASE appdb;15.22.8 Patroni PostgreSQL HA
架构：
HAProxy
|
+--------+--------+
| |
PostgreSQL1 PostgreSQL2
| |
+--------+--------+
|
etcd安装：
---
- name:
install patroni
pip:
name:
- patroni
- psycopg2模板：
templates/patroni.yml.j2
scope:
postgres-cluster
name:
{{ inventory_hostname }}
restapi:
listen:
0.0.0.0:8008
postgresql:
listen:
0.0.0.0:5432
parameters:
wal_level:
replica
etcd:
hosts:
etcd1:2379启动：
patroni /etc/patroni.yml查看：
patronictl list15.22.9 MongoDB ReplicaSet
架构：
MongoDB
+---------+---------+
| | |
Node1 Node2 Node3Docker：
services:
mongodb:
image:
mongo:7
command:
mongod --replSet rs0
volumes:
- /mnt/database/mongo:/data/db初始化：
rs.initiate({
_id:"rs0",
members:[
{
_id:0,
host:"mongo1:27017"
}
]
})状态：
rs.status()15.22.10 Redis Cluster
架构：
Redis Cluster
+------+------+
6379 6380 6381Compose：
services:
redis:
image:
redis:7.2
command:
redis-server \
--cluster-enabled yes创建：
redis-cli \
--cluster create \
10.0.0.1:6379 \
10.0.0.2:6379 \
10.0.0.3:6379检查：
redis-cli cluster info15.22.11 ProxySQL
用途：
- MySQL读写分离
- 自动故障切换
安装：
---
- name:
install proxysql
apt:
name:
proxysql
state:
present配置：
mysql_servers:
10.10.10.11:3306
10.10.10.12:3306登录：
mysql \
-h127.0.0.1 \
-P603215.22.12 PgBouncer
用途：
PostgreSQL连接池。
安装：
---
- name:
install pgbouncer
apt:
name:
pgbouncer配置：
[databases]
appdb=
host=postgres
port=5432
[pgbouncer]
pool_mode=session
max_client_conn=100015.22.13 SQL Server
Docker：
services:
sqlserver:
image:
mcr.microsoft.com/mssql/server:2022-latest
environment:
ACCEPT_EULA:
Y
SA_PASSWORD:
Password123!验证：
sqlcmd -S localhost15.22.14 Oracle Client
安装：
---
- name:
install oracle client
apt:
name:
- libaio1环境：
export ORACLE_HOME=/opt/oracle
export LD_LIBRARY_PATH=$ORACLE_HOME/lib测试：
sqlplus user/password@db15.22.15 ClickHouse
用途：
- OLAP
- 日志分析
- BI
Docker：
services:
clickhouse:
image:
clickhouse/clickhouse-server:24
ports:
- 8123:8123
volumes:
- /mnt/database/clickhouse:/var/lib/clickhouse测试：
clickhouse-client创建表：
CREATE TABLE logs
(
time DateTime,
message String
)
ENGINE=MergeTree
ORDER BY time;15.22.16 TDengine
用途：
工业时序数据。
部署：
services:
tdengine:
image:
tdengine/tdengine:3.3
ports:
- 6041:6041验证：
taos15.22.17 InfluxDB
部署：
services:
influxdb:
image:
influxdb:2.7
ports:
- 8086:8086创建：
influx setup查询：
from(bucket:"metrics")15.22.18 TimescaleDB
PostgreSQL时序扩展。
安装：
---
- name:
install timescaledb
apt:
name:
- timescaledb-postgresql-16启用：
CREATE EXTENSION timescaledb;15.22.19 数据库备份
支持：
MySQL
PostgreSQL
MongoDB
Redis
ClickHouse模板：
#!/bin/bash
DATE=$(date +%F)
mkdir -p /backup/$DATE
mysqldump \
-u root \
-p password \
--all-databases \
> /backup/$DATE/mysql.sql
pg_dumpall \
> /backup/$DATE/postgres.sql定时：
- cron:
name:
database backup
minute:
0
hour:
2
job:
/opt/scripts/db-backup.sh15.22.20 数据库健康检查
文件：
scripts/check-database.sh
#!/bin/bash
echo MySQL
mysqladmin ping
echo PostgreSQL
pg_isready
echo MongoDB
mongosh --eval "db.serverStatus()"
echo Redis
redis-cli ping
echo ClickHouse
curl localhost:8123/ping
echo TDengine
taos -s "show databases"Part 15.22完成
✅ MySQL 8.4
✅ PostgreSQL 16
✅ Patroni PostgreSQL HA
✅ MongoDB ReplicaSet
✅ Redis Cluster
✅ ProxySQL
✅ PgBouncer
✅ SQL Server
✅ Oracle Client
✅ ClickHouse OLAP
✅ TDengine
✅ InfluxDB
✅ TimescaleDB
✅ 数据库备份恢复
✅ 自动化健康检查
Part 15.23 Enterprise Messaging & Event Platform Role完整代码
本部分生成企业级 Messaging & Event Streaming Platform 消息事件平台自动化部署 Ansible Role
覆盖企业生产消息中间件：
- Apache Kafka Cluster
- KRaft Kafka HA
- Apache RocketMQ
- RabbitMQ Cluster
- Apache Pulsar
- NATS JetStream
- MQTT Broker
- EMQX IoT消息平台
- ActiveMQ Artemis
- Debezium CDC
- Kafka Connect
- Schema Registry
- Event Monitoring
- 消息平台健康检查
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Docker
Kubernetes
Ansible >=2.15
生产HA环境15.23.1 Messaging Platform整体架构
Application
|
|
Event Gateway Layer
|
+-------------------+-------------------+
| | |
Kafka RocketMQ Pulsar
| | |
+-------------------+-------------------+
Event Streaming
|
+-------------------+-------------------+
| | |
Consumer CDC Pipeline Analytics
|
Data Platform
+-----------+------------+------------+
| | | |
MySQL MongoDB PostgreSQL Oracle
|
Monitoring
Prometheus + Grafana + Loki15.23.2 Role目录结构
创建：
roles/messaging-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── kafka.yml
│ ├── kafka-connect.yml
│ ├── schema-registry.yml
│ ├── rabbitmq.yml
│ ├── rocketmq.yml
│ ├── pulsar.yml
│ ├── nats.yml
│ ├── mqtt.yml
│ ├── emqx.yml
│ ├── activemq.yml
│ ├── debezium.yml
│ └── monitoring.yml
├── templates/
│
│ ├── kafka-compose.yml.j2
│ ├── kafka-server.properties.j2
│ ├── kafka-connect.yml.j2
│ ├── rabbitmq.conf.j2
│ ├── rocketmq-compose.yml.j2
│ ├── pulsar-compose.yml.j2
│ ├── nats.conf.j2
│ ├── emqx.conf.j2
│ └── debezium.json.j2
└── handlers/
└── main.yml15.23.3 默认变量
文件：
roles/messaging-platform/defaults/main.yml
---
message_root:
/mnt/messaging
################
# Kafka
################
kafka_version:
3.8
kafka_cluster_id:
kafka-prod
kafka_port:
9092
################
# RabbitMQ
################
rabbitmq_version:
3.13
rabbitmq_port:
5672
################
# RocketMQ
################
rocketmq_version:
5.3
rocketmq_port:
9876
################
# Pulsar
################
pulsar_version:
3.3
################
# NATS
################
nats_version:
2.10
################
# EMQX
################
emqx_version:
5.8
################
# Debezium
################
debezium_version:
2.715.23.4 Inventory
新增：
inventory/production/hosts.yml
---
kafka_nodes:
children:
kafka:
hosts:
kafka01:
ansible_host:
10.10.10.31
kafka02:
ansible_host:
10.10.10.32
kafka03:
ansible_host:
10.10.10.33
rabbitmq_nodes:
hosts:
rabbit01:
rabbit02:
rabbit03:15.23.5 Playbook
文件：
playbooks/messaging-platform.yml
---
- name:
Deploy Messaging Platform
hosts:
messaging
become:
true
roles:
- messaging-platform15.23.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
kafka.yml
- import_tasks:
kafka-connect.yml
- import_tasks:
schema-registry.yml
- import_tasks:
rabbitmq.yml
- import_tasks:
rocketmq.yml
- import_tasks:
pulsar.yml
- import_tasks:
nats.yml
- import_tasks:
mqtt.yml
- import_tasks:
emqx.yml
- import_tasks:
activemq.yml
- import_tasks:
debezium.yml
- import_tasks:
monitoring.yml15.23.7 Kafka KRaft Cluster
架构
Producer
|
|
+---------+---------+
| | |
Kafka1 Kafka2 Kafka3
| | |
+---------+---------+
KRaft
Controller Quorum安装Docker
---
- name:
install docker
apt:
name:
- docker.io
- docker-compose-plugin
state:
presentKafka Compose
模板：
templates/kafka-compose.yml.j2
services:
kafka:
image:
bitnami/kafka:{{kafka_version}}
container_name:
kafka
ports:
- "{{kafka_port}}:9092"
environment:
KAFKA_ENABLE_KRAFT:
yes
KAFKA_CFG_NODE_ID:
{{node_id}}
KAFKA_CFG_PROCESS_ROLES:
broker,controller
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS:
1@kafka01:9093,
2@kafka02:9093,
3@kafka03:9093
KAFKA_CFG_LISTENERS:
PLAINTEXT://:9092,
CONTROLLER://:9093
volumes:
- {{message_root}}/kafka:/bitnami/kafka启动：
docker compose up -d验证：
kafka-topics.sh \
--bootstrap-server localhost:9092 \
--list15.23.8 Kafka Topic自动创建
任务：
---
- name:
create kafka topic
shell:
cmd:
kafka-topics.sh \
--create \
--topic application-event \
--partitions 12 \
--replication-factor 315.23.9 Kafka Connect
用途：
- CDC
- ETL
- 数据同步
部署：
services:
connect:
image:
confluentinc/cp-kafka-connect
ports:
- 8083:8083验证：
curl localhost:8083/connectors15.23.10 Debezium CDC
架构：
MySQL
|
Binlog
|
Debezium
|
Kafka
|
ConsumerConnector：
模板：
templates/debezium.json.j2
{
"name":
"mysql-cdc",
"config":{
"connector.class":
"io.debezium.connector.mysql.MySqlConnector",
"database.hostname":
"mysql",
"database.user":
"root",
"database.password":
"password",
"topic.prefix":
"mysql"
}
}注册：
curl \
-X POST \
localhost:8083/connectors \
-d @debezium.json15.23.11 Schema Registry
用途：
消息Schema管理。
部署：
services:
schema-registry:
image:
confluentinc/cp-schema-registry
ports:
- 8081:8081测试：
curl localhost:8081/subjects15.23.12 RabbitMQ HA
架构：
HAProxy
|
+---------+---------+
| | |
Rabbit1 Rabbit2 Rabbit3
Cluster部署：
services:
rabbitmq:
image:
rabbitmq:3.13-management
hostname:
rabbit01
environment:
RABBITMQ_ERLANG_COOKIE:
secret
ports:
- 5672:5672
- 15672:15672集群：
rabbitmqctl join_cluster rabbit@rabbit01状态：
rabbitmqctl cluster_status15.23.13 RocketMQ
架构：
Producer
|
NameServer
|
Broker Cluster
|
ConsumerCompose：
services:
namesrv:
image:
apache/rocketmq
command:
sh mqnamesrv
broker:
image:
apache/rocketmq
command:
sh mqbroker检查：
mqadmin clusterList15.23.14 Apache Pulsar
部署：
services:
pulsar:
image:
apachepulsar/pulsar:3.3
command:
bin/pulsar standalone
ports:
- 6650:6650
- 8080:8080创建Topic：
pulsar-admin topics create \
persistent://public/default/test15.23.15 NATS JetStream
用途：
- 轻量消息
- 云原生事件
部署：
services:
nats:
image:
nats:2.10
command:
-js
ports:
-4222:4222测试：
nats server check15.23.16 MQTT Broker
支持：
- IoT
- 工业设备
部署：
services:
mosquitto:
image:
eclipse-mosquitto
ports:
-1883:1883测试：
发布：
mosquitto_pub \
-t test \
-m hello订阅：
mosquitto_sub \
-t test15.23.17 EMQX
企业MQTT平台。
部署：
services:
emqx:
image:
emqx/emqx:5.8
ports:
-1883:1883
-18083:18083访问：
http://host:1808315.23.18 ActiveMQ Artemis
部署：
services:
activemq:
image:
apache/activemq-artemis
ports:
-61616:6161615.23.19 Event Monitoring
Prometheus采集：
Kafka：
scrape_configs:
- job_name:
kafka
static_configs:
- targets:
- kafka-exporter:9308RabbitMQ：
- job_name:
rabbitmq
targets:
- rabbitmq-exporter:941915.23.20 Grafana Dashboard
导入：
Kafka Overview
RabbitMQ Overview
RocketMQ Overview
EMQX Dashboard
Pulsar Dashboard15.23.21 消息平台健康检查
文件：
scripts/check-message.sh#!/bin/bash
echo Kafka
kafka-topics.sh \
--bootstrap-server localhost:9092 \
--list
echo RabbitMQ
rabbitmqctl status
echo RocketMQ
mqadmin clusterList
echo Pulsar
pulsar-admin brokers list
echo NATS
nats server check
echo MQTT
mosquitto_sub \
-t healthPart 15.23完成
✅ Kafka KRaft Cluster
✅ Kafka Connect
✅ Schema Registry
✅ Debezium CDC
✅ RabbitMQ HA
✅ RocketMQ Cluster
✅ Pulsar
✅ NATS JetStream
✅ MQTT
✅ EMQX IoT平台
✅ ActiveMQ Artemis
✅ Prometheus监控
✅ Grafana Dashboard
✅ 自动健康检查
Part 15.24 Enterprise API Gateway & Service Mesh Role完整代码
本部分生成企业级 API Gateway & Service Mesh Platform 自动化部署 Ansible Role
覆盖企业生产入口平台：
- Nginx Ingress Controller
- Kong API Gateway
- Apache APISIX Gateway
- Traefik Gateway
- Istio Gateway
- Envoy Gateway
- OAuth2 Proxy
- Keycloak OIDC集成
- API Authentication
- JWT验证
- Rate Limit限流
- WAF安全防护
- API Security策略
- 多租户API管理
- Gateway Monitoring
适用于：
Ubuntu 22.04 / 24.04
Kubernetes 1.28+
Docker
containerd
Istio 1.23+
Helm 3.x
Ansible >=2.15
生产环境15.24.1 API Platform整体架构
Internet
|
|
Cloud LoadBalancer
|
API Gateway Layer
+--------------------------+--------------------------+
| | |
Nginx Ingress Kong APISIX
| | |
+--------------------------+--------------------------+
Service Mesh
|
Istio Gateway
|
Envoy
|
+-------------------+-------------------+
| | |
Service A Service B Service C
| | |
Kubernetes Cluster
|
Observability
Prometheus + Grafana + Loki15.24.2 Role目录结构
创建：
roles/api-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── nginx-ingress.yml
│ ├── kong.yml
│ ├── apisix.yml
│ ├── traefik.yml
│ ├── istio.yml
│ ├── envoy-gateway.yml
│ ├── oauth2-proxy.yml
│ ├── keycloak.yml
│ ├── jwt.yml
│ ├── rate-limit.yml
│ ├── waf.yml
│ ├── security.yml
│ └── monitoring.yml
├── templates/
│
│ ├── kong-values.yaml.j2
│ ├── apisix-values.yaml.j2
│ ├── ingress.yaml.j2
│ ├── envoy.yaml.j2
│ ├── oauth2-proxy.yaml.j2
│ ├── gateway.yaml.j2
│ └── security-policy.yaml.j2
└── handlers/
└── main.yml15.24.3 默认变量
文件：
roles/api-platform/defaults/main.yml
---
api_platform_root:
/opt/api-platform
#################
# Gateway
#################
gateway_type:
nginx
#################
# Nginx
#################
nginx_ingress_version:
4.11
#################
# Kong
#################
kong_version:
3.8
kong_port:
8000
#################
# APISIX
#################
apisix_version:
3.10
#################
# Traefik
#################
traefik_version:
3.1
#################
# Istio
#################
istio_version:
1.23
#################
# OAuth
#################
oauth2_proxy_version:
7.7
#################
# Security
#################
enable_waf:
true
enable_rate_limit:
true
#################
# Monitoring
#################
enable_metrics:
true15.24.4 Inventory
新增：
inventory/production/hosts.yml
---
gateway_nodes:
children:
gateway:
hosts:
gateway01:
ansible_host:
10.10.20.10
gateway02:
ansible_host:
10.10.20.1115.24.5 Playbook
文件：
playbooks/api-platform.yml
---
- name:
Deploy API Gateway Platform
hosts:
gateway
become:
true
roles:
- api-platform15.24.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
nginx-ingress.yml
- import_tasks:
kong.yml
- import_tasks:
apisix.yml
- import_tasks:
traefik.yml
- import_tasks:
istio.yml
- import_tasks:
envoy-gateway.yml
- import_tasks:
oauth2-proxy.yml
- import_tasks:
keycloak.yml
- import_tasks:
jwt.yml
- import_tasks:
rate-limit.yml
- import_tasks:
waf.yml
- import_tasks:
security.yml
- import_tasks:
monitoring.yml15.24.7 Nginx Ingress Controller
用途：
- Kubernetes入口
- TLS终止
- 七层代理
架构：
Client
|
LoadBalancer
|
Nginx Ingress
|
Kubernetes Service
|
Pod安装：
---
- name:
install nginx ingress
kubernetes.core.helm:
name:
ingress-nginx
chart_ref:
ingress-nginx
release_namespace:
ingress-nginx
create_namespace:
true验证：
kubectl get pods -n ingress-nginx输出：
ingress-nginx-controller
Running15.24.8 Ingress规则模板
文件：
templates/ingress.yaml.j2
apiVersion:
networking.k8s.io/v1
kind:
Ingress
metadata:
name:
{{app_name}}
annotations:
nginx.ingress.kubernetes.io/rewrite-target:
/
spec:
ingressClassName:
nginx
rules:
- host:
{{domain}}
http:
paths:
- path:
/
pathType:
Prefix
backend:
service:
name:
{{service_name}}
port:
number:
8015.24.9 Kong API Gateway
架构：
Client
|
Kong Gateway
|
Plugin
|
Service部署：
---
- name:
deploy kong
kubernetes.core.helm:
name:
kong
chart_ref:
kong
release_namespace:
kong
create_namespace:
yes验证：
kubectl get pods -n kongKong Route配置
apiVersion:
configuration.konghq.com/v1
kind:
KongIngress
metadata:
name:
api-route
route:
methods:
- GET
- POST15.24.10 Apache APISIX Gateway
特点：
- 高性能
- 动态路由
- 插件丰富
部署：
---
- name:
deploy apisix
kubernetes.core.helm:
name:
apisix
chart_ref:
apisix
release_namespace:
apisix
create_namespace:
yes查看：
kubectl get pods -n apisixAPISIX Route
apiVersion:
apisix.apache.org/v2
kind:
ApisixRoute
metadata:
name:
app-route
spec:
http:
- name:
backend
match:
paths:
- /api/*
backends:
- serviceName:
backend
servicePort:
808015.24.11 Traefik Gateway
部署：
---
- name:
install traefik
kubernetes.core.helm:
name:
traefik
chart_ref:
traefik
release_namespace:
traefik
create_namespace:
true验证：
kubectl get svc -n traefik15.24.12 Istio Service Mesh
安装：
---
- name:
install istio
shell:
cmd:
istioctl install \
--set profile=production \
-y启用：
kubectl label namespace default \
istio-injection=enabled验证：
kubectl get pods显示：
app-container
istio-proxy15.24.13 Istio Gateway
模板：
templates/gateway.yaml.j2
apiVersion:
networking.istio.io/v1
kind:
Gateway
metadata:
name:
app-gateway
spec:
selector:
istio:
ingressgateway
servers:
- port:
number:
80
name:
http
hosts:
- "*"VirtualService
apiVersion:
networking.istio.io/v1
kind:
VirtualService
spec:
hosts:
- "*"
gateways:
- app-gateway
http:
- match:
- uri:
prefix:
/api
route:
- destination:
host:
backend15.24.14 Envoy Gateway
部署：
---
- name:
install envoy gateway
kubernetes.core.helm:
name:
envoy-gateway
chart_ref:
envoy-gateway
release_namespace:
envoy-gateway-system
create_namespace:
yesGatewayClass：
apiVersion:
gateway.networking.k8s.io/v1
kind:
GatewayClass
metadata:
name:
envoy
spec:
controllerName:
gateway.envoyproxy.io/gatewayclass-controller15.24.15 OAuth2 Proxy
用途：
企业统一登录。
支持：
- Keycloak
- LDAP
- GitHub OAuth
- OIDC
部署：
---
- name:
deploy oauth2 proxy
kubernetes.core.helm:
name:
oauth2-proxy
chart_ref:
oauth2-proxy配置：
provider:
oidc
oidc_issuer_url:
https://keycloak.company.com/realms/main
email_domains:
- "*"15.24.16 Keycloak OIDC集成
认证流程：
User
|
Gateway
|
OAuth2 Proxy
|
Keycloak
|
JWT Token
|
Backend配置：
oidc:
issuer:
https://keycloak.company.com
client_id:
api-gateway
client_secret:
xxx15.24.17 JWT验证
Istio策略：
apiVersion:
security.istio.io/v1
kind:
RequestAuthentication
metadata:
name:
jwt-auth
spec:
jwtRules:
- issuer:
https://keycloak.company.com
jwksUri:
https://keycloak.company.com/protocol/openid-connect/certs授权：
apiVersion:
security.istio.io/v1
kind:
AuthorizationPolicy
spec:
rules:
- from:
- source:
requestPrincipals:
- "*"15.24.18 API Rate Limit
支持：
- Kong Plugin
- APISIX Plugin
- Istio EnvoyFilter
Kong示例：
apiVersion:
configuration.konghq.com/v1
kind:
KongPlugin
metadata:
name:
rate-limit
plugin:
rate-limiting
config:
minute:
1000
policy:
local15.24.19 WAF安全
集成：
- ModSecurity
- Coraza WAF
安装：
---
- name:
enable waf
apt:
name:
- libapache2-mod-security2
state:
present规则：
REQUEST-942-APPLICATION-ATTACK-SQLI
REQUEST-941-XSS
REQUEST-932-RCE15.24.20 API Security策略
包含：
TLS
JWT
OAuth2
OIDC
Rate Limit
IP Blacklist
API Key
WAF
Audit Log15.24.21 Gateway Monitoring
Prometheus：
scrape_configs:
- job_name:
gateway
static_configs:
- targets:
- nginx-ingress-controller-metrics:10254
- kong-metrics:9542
- apisix-metrics:9091Grafana Dashboard：
导入：
Nginx Ingress Dashboard
Kong Dashboard
APISIX Dashboard
Istio Mesh Dashboard
Envoy Dashboard15.24.22 Gateway健康检查
文件：
scripts/check-gateway.sh#!/bin/bash
echo Nginx
kubectl get pods \
-n ingress-nginx
echo Kong
kubectl get pods \
-n kong
echo APISIX
kubectl get pods \
-n apisix
echo Istio
istioctl proxy-status
echo Gateway Test
curl -I https://api.company.comPart 15.24完成
✅ Nginx Ingress Controller
✅ Kong API Gateway
✅ APISIX Gateway
✅ Traefik Gateway
✅ Istio Service Mesh
✅ Istio Gateway
✅ Envoy Gateway
✅ OAuth2 Proxy
✅ Keycloak OIDC集成
✅ JWT认证
✅ Rate Limit限流
✅ WAF安全
✅ API Security策略
✅ Prometheus监控
✅ Grafana Dashboard
✅ Gateway健康检查
Part 15.25 Enterprise Identity & Access Management Role完整代码
本部分生成企业级 Identity & Access Management（IAM）平台自动化部署 Ansible Role。
覆盖企业生产统一身份认证体系：
- OpenLDAP 企业目录服务
- FreeIPA 企业身份管理
- Microsoft AD集成
- Keycloak SSO
- OAuth2
- OpenID Connect(OIDC)
- SAML 2.0
- Kerberos认证
- RADIUS认证
- Vault Secrets Management
- LDAP用户同步
- RBAC权限管理
- MFA多因素认证
- Access Audit审计
- 用户生命周期管理
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Docker
Kubernetes
Ansible >=2.15
生产企业环境15.25.1 IAM整体架构
User
|
|
Identity Layer
|
+-----------------+-----------------+
| | |
LDAP Keycloak Vault
| | |
| OIDC/SAML |
| | |
+-----------------+-----------------+
|
Application
+---------+---------+---------+
| | | |
GitLab Jenkins Grafana Kubernetes
|
Authorization
|
RBAC / Policy15.25.2 Role目录结构
创建：
roles/iam-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── openldap.yml
│ ├── freeipa.yml
│ ├── keycloak.yml
│ ├── oauth2.yml
│ ├── oidc.yml
│ ├── saml.yml
│ ├── kerberos.yml
│ ├── radius.yml
│ ├── vault.yml
│ ├── ldap-sync.yml
│ ├── rbac.yml
│ ├── mfa.yml
│ └── audit.yml
├── templates/
│
│ ├── ldap-compose.yml.j2
│ ├── keycloak-values.yaml.j2
│ ├── vault-config.hcl.j2
│ ├── krb5.conf.j2
│ ├── radius.conf.j2
│ ├── ldap-sync.yaml.j2
│ └── audit-policy.yaml.j2
└── handlers/
└── main.yml15.25.3 默认变量
文件：
roles/iam-platform/defaults/main.yml
---
iam_root:
/mnt/iam
#################
# LDAP
#################
ldap_domain:
company.com
ldap_org:
Company
ldap_admin_password:
ChangeMeLDAP123
#################
# Keycloak
#################
keycloak_version:
26.0
keycloak_admin:
admin
keycloak_password:
ChangeMeKeycloak123
#################
# Vault
#################
vault_version:
1.17
vault_storage:
raft
#################
# FreeIPA
#################
freeipa_version:
4.11
#################
# Radius
#################
radius_version:
3.2
#################
# MFA
#################
enable_mfa:
true
#################
# Audit
#################
enable_audit:
true15.25.4 Inventory
新增：
inventory/production/hosts.yml
---
iam_nodes:
children:
identity:
hosts:
iam01:
ansible_host:
10.10.30.10
iam02:
ansible_host:
10.10.30.1115.25.5 Playbook
文件：
playbooks/iam-platform.yml
---
- name:
Deploy IAM Platform
hosts:
identity
become:
true
roles:
- iam-platform15.25.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
openldap.yml
- import_tasks:
freeipa.yml
- import_tasks:
keycloak.yml
- import_tasks:
oauth2.yml
- import_tasks:
oidc.yml
- import_tasks:
saml.yml
- import_tasks:
kerberos.yml
- import_tasks:
radius.yml
- import_tasks:
vault.yml
- import_tasks:
ldap-sync.yml
- import_tasks:
rbac.yml
- import_tasks:
mfa.yml
- import_tasks:
audit.yml15.25.7 OpenLDAP企业目录服务
架构：
Applications
|
LDAP Bind
|
+---------------+
| OpenLDAP |
+---------------+
|
LDAP DatabaseDocker部署
模板：
templates/ldap-compose.yml.j2
services:
ldap:
image:
osixia/openldap:1.5.0
container_name:
openldap
environment:
LDAP_ORGANISATION:
"{{ldap_org}}"
LDAP_DOMAIN:
"{{ldap_domain}}"
LDAP_ADMIN_PASSWORD:
"{{ldap_admin_password}}"
ports:
- 389:389
- 636:636
volumes:
- {{iam_root}}/ldap:/var/lib/ldap启动：
docker compose up -d验证：
ldapsearch \
-x \
-H ldap://localhost \
-D "cn=admin,dc=company,dc=com"15.25.8 LDAP组织结构
创建：
base.ldif
dn: dc=company,dc=com
objectClass: domain
dc: company
dn: ou=users,dc=company,dc=com
objectClass: organizationalUnit
ou: users
dn: ou=groups,dc=company,dc=com
objectClass: organizationalUnit
ou: groups导入：
ldapadd \
-x \
-D cn=admin,dc=company,dc=com \
-f base.ldif15.25.9 FreeIPA身份管理
用途：
- LDAP
- Kerberos
- DNS
- Certificate
部署：
services:
freeipa:
image:
freeipa/freeipa-server
hostname:
ipa.company.com
environment:
PASSWORD:
Password123!验证：
ipa user-find15.25.10 Keycloak SSO
架构：
User
|
Keycloak
|
OIDC Token
|
ApplicationHelm安装：
---
- name:
install keycloak
kubernetes.core.helm:
name:
keycloak
chart_ref:
bitnami/keycloak
namespace:
keycloak
create_namespace:
true访问：
https://sso.company.com创建Realm：
kcadm.sh create realms \
-s realm=company创建Client：
kcadm.sh create clients \
-r company \
-s clientId=gitlab15.25.11 OAuth2
支持：
- GitLab
- Jenkins
- Grafana
- Harbor
- Kubernetes
配置：
oauth:
provider:
keycloak
client_id:
gitlab
issuer:
https://sso.company.com/realms/company15.25.12 OpenID Connect
认证流程：
User
|
Application
|
OIDC Redirect
|
Keycloak
|
JWT
|
ApplicationOIDC Discovery:
curl:
https://sso.company.com/.well-known/openid-configuration15.25.13 SAML 2.0
用途：
企业传统系统SSO。
配置：
saml:
entity_id:
company-app
idp:
https://sso.company.com支持：
- Jira
- Confluence
- GitLab Enterprise
- Office365
15.25.14 Kerberos
用途：
企业域认证。
安装：
- name:
install kerberos
apt:
name:
- krb5-user
state:
present配置：
模板：
templates/krb5.conf.j2
[libdefaults]
default_realm = COMPANY.COM
[realms]
COMPANY.COM = {
kdc = ipa.company.com
}测试：
kinit admin查看：
klist15.25.15 RADIUS认证
用途：
- VPN
- WiFi
- Network设备
安装：
---
- name:
install freeradius
apt:
name:
- freeradius配置：
clients.conf
client vpn {
ipaddr = 10.0.0.0/8
secret = radius-secret
}测试：
radtest user password localhost 0 secret15.25.16 Vault Secrets Management
架构：
Application
|
Vault Agent
|
Vault Cluster
|
Encrypted StorageHelm：
---
- name:
install vault
kubernetes.core.helm:
name:
vault
chart_ref:
hashicorp/vault
namespace:
vaultVault初始化：
vault operator init启用LDAP：
vault auth enable ldap配置：
vault write auth/ldap/config
url=ldap://ldap.company.com
userdn="ou=users,dc=company,dc=com"15.25.17 LDAP同步
用途：
同步：
LDAP
|
Keycloak
|
GitLab
|
Jenkins模板：
ldap-sync.yaml.j2
source:
ldap
target:
keycloak
schedule:
"*/10 * * * *"15.25.18 RBAC权限管理
统一模型：
User
|
Group
|
Role
|
Permission
|
ResourceKubernetes RBAC：
apiVersion:
rbac.authorization.k8s.io/v1
kind:
Role
metadata:
name:
developer
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list15.25.19 MFA多因素认证
支持：
- TOTP
- WebAuthn
- FIDO2
Keycloak开启：
kcadm.sh update authentication/flows/browser用户绑定：
OTP Required15.25.20 Access Audit审计
记录：
- 登录
- Token
- API调用
- 权限变更
模板：
audit-policy.yaml.j2
apiVersion:
audit.k8s.io/v1
kind:
Policy
rules:
- level:
Metadata
resources:
- group:
authentication.k8s.io15.25.21 IAM Monitoring
Prometheus：
scrape_configs:
- job_name:
keycloak
static_configs:
- targets:
- keycloak:8080
- job_name:
vault
targets:
- vault:8200Grafana：
导入：
Keycloak Dashboard
Vault Dashboard
LDAP Monitoring
OAuth Login Metrics15.25.22 IAM健康检查
文件：
scripts/check-iam.sh
#!/bin/bash
echo LDAP
ldapsearch \
-x \
-H ldap://localhost
echo Keycloak
curl \
-k \
https://sso.company.com
echo Vault
vault status
echo Kerberos
klist
echo Radius
systemctl status freeradiusPart 15.25完成
✅ OpenLDAP
✅ FreeIPA
✅ Keycloak SSO
✅ OAuth2
✅ OpenID Connect
✅ SAML 2.0
✅ Kerberos
✅ RADIUS
✅ Vault Secrets Management
✅ LDAP同步
✅ RBAC权限
✅ MFA认证
✅ Access Audit
✅ Prometheus监控
✅ Grafana Dashboard
✅ IAM健康检查
Part 15.26 Enterprise Logging & Observability Platform Role完整代码
本部分生成企业级 Logging & Observability Platform 自动化部署 Ansible Role
覆盖企业生产可观测平台：
- Elasticsearch Cluster
- Logstash
- Kibana
- Filebeat
- Fluent Bit
- Loki
- Promtail
- Grafana
- Prometheus
- AlertManager
- Thanos
- OpenTelemetry Collector
- Jaeger Distributed Tracing
- Grafana Tempo
- Uptime Monitor
- Audit Log Platform
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Docker
Kubernetes
Helm
Ansible >=2.15
企业生产环境15.26.1 Observability整体架构
Application
|
+----------------+----------------+
| | |
Logs Metrics Trace
| | |
Fluent-bit Prometheus OpenTelemetry
| | |
+-------+-------+ | +------+------+
| | | | |
Elasticsearch Loki | Jaeger Tempo
| | | | |
+---------------+--------+--------+-------------+
|
Grafana
|
Dashboard / Alert / Query
|
AlertManager
|
Email / Slack / Webhook15.26.2 Role目录结构
创建：
roles/observability-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── elasticsearch.yml
│ ├── logstash.yml
│ ├── kibana.yml
│ ├── filebeat.yml
│ ├── fluent-bit.yml
│ ├── loki.yml
│ ├── promtail.yml
│ ├── prometheus.yml
│ ├── alertmanager.yml
│ ├── grafana.yml
│ ├── thanos.yml
│ ├── opentelemetry.yml
│ ├── jaeger.yml
│ ├── tempo.yml
│ ├── uptime.yml
│ └── audit.yml
├── templates/
│
│ ├── elasticsearch.yml.j2
│ ├── logstash.conf.j2
│ ├── kibana.yml.j2
│ ├── prometheus.yml.j2
│ ├── alertmanager.yml.j2
│ ├── loki.yaml.j2
│ ├── promtail.yaml.j2
│ ├── grafana.ini.j2
│ ├── thanos.yml.j2
│ └── otel-config.yaml.j2
└── handlers/
└── main.yml15.26.3 默认变量
文件：
roles/observability-platform/defaults/main.yml
---
observability_root:
/mnt/observability
##################
# Elasticsearch
##################
elasticsearch_version:
8.15
elastic_heap:
4g
##################
# Loki
##################
loki_version:
3.2
##################
# Prometheus
##################
prometheus_version:
v2.55
##################
# Grafana
##################
grafana_version:
11.2
##################
# AlertManager
##################
alertmanager_version:
v0.27
##################
# Thanos
##################
thanos_version:
v0.37
##################
# OpenTelemetry
##################
otel_version:
0.110
##################
# Jaeger
##################
jaeger_version:
1.62
##################
# Tempo
##################
tempo_version:
2.615.26.4 Inventory
新增：
inventory/production/hosts.yml
---
observability:
children:
monitoring:
hosts:
monitor01:
ansible_host:
10.10.40.10
monitor02:
ansible_host:
10.10.40.1115.26.5 Playbook
文件：
playbooks/observability.yml
---
- name:
Deploy Observability Platform
hosts:
monitoring
become:
true
roles:
- observability-platform15.26.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
elasticsearch.yml
- import_tasks:
logstash.yml
- import_tasks:
kibana.yml
- import_tasks:
filebeat.yml
- import_tasks:
fluent-bit.yml
- import_tasks:
loki.yml
- import_tasks:
promtail.yml
- import_tasks:
prometheus.yml
- import_tasks:
alertmanager.yml
- import_tasks:
grafana.yml
- import_tasks:
thanos.yml
- import_tasks:
opentelemetry.yml
- import_tasks:
jaeger.yml
- import_tasks:
tempo.yml
- import_tasks:
uptime.yml
- import_tasks:
audit.yml15.26.7 Elasticsearch Cluster
架构
Application Logs
|
Filebeat
|
+--------+--------+
| | |
ES01 ES02 ES03
Elasticsearch Cluster
|
KibanaDocker Compose:
services:
elasticsearch:
image:
docker.elastic.co/elasticsearch/elasticsearch:8.15.0
environment:
discovery.type:
multi-node
cluster.name:
production-es
ES_JAVA_OPTS:
"-Xms4g -Xmx4g"
volumes:
- /mnt/observability/es:/usr/share/elasticsearch/data
ports:
- 9200:9200检查：
curl http://localhost:9200输出：
{
"cluster_name":"production-es"
}15.26.8 Logstash
作用：
Input
|
Filter
|
Output模板：
templates/logstash.conf.j2
input {
beats {
port => 5044
}
}
filter {
json {
source => "message"
}
}
output {
elasticsearch {
hosts =>
["http://elasticsearch:9200"]
index =>
"application-%{+YYYY.MM.dd}"
}
}启动：
docker compose up -d logstash15.26.9 Kibana
部署：
services:
kibana:
image:
docker.elastic.co/kibana/kibana:8.15.0
ports:
- 5601:5601
environment:
ELASTICSEARCH_HOSTS:
http://elasticsearch:9200访问：
http://monitor:560115.26.10 Filebeat
用途：
采集：
- Linux日志
- Docker日志
- Kubernetes日志
配置：
filebeat.inputs:
- type:
container
paths:
- /var/lib/docker/containers/*/*.log
output.elasticsearch:
hosts:
- elasticsearch:9200启动：
systemctl restart filebeat15.26.11 Fluent Bit
Kubernetes推荐。
DaemonSet：
apiVersion:
apps/v1
kind:
DaemonSet
metadata:
name:
fluent-bit
spec:
template:
spec:
containers:
- name:
fluent-bit
image:
fluent/fluent-bit:3.1输出：
Pod Log
|
Fluent Bit
|
Loki15.26.12 Loki
架构：
Logs
|
Promtail
|
Loki
|
Grafana部署：
services:
loki:
image:
grafana/loki:3.2.0
ports:
- 3100:3100
volumes:
- /mnt/observability/loki:/loki测试：
curl localhost:3100/ready15.26.13 Promtail
配置：
server:
port:
9080
clients:
- url:
http://loki:3100/loki/api/v1/push
positions:
filename:
/tmp/positions.yaml
scrape_configs:
- job_name:
kubernetes15.26.14 Prometheus
架构：
Exporter
|
Prometheus
|
Grafana配置：
模板：
global:
scrape_interval:
15s
scrape_configs:
- job_name:
node
static_configs:
- targets:
- node-exporter:9100
- job_name:
kubernetes
kubernetes_sd_configs:
- role:
pod部署：
services:
prometheus:
image:
prom/prometheus:v2.55
ports:
- 9090:9090检查：
curl localhost:9090/-/healthy15.26.15 Node Exporter
安装：
- name:
install node exporter
docker_container:
name:
node-exporter
image:
prom/node-exporter指标：
CPU
Memory
Disk
Network
Filesystem15.26.16 AlertManager
配置：
模板：
global:
smtp_smarthost:
smtp.company.com:25
route:
receiver:
email
receivers:
- name:
email
email_configs:
- to:
admin@company.com启动：
docker compose up -d alertmanager15.26.17 Grafana
部署：
services:
grafana:
image:
grafana/grafana:11.2
ports:
- 3000:3000
volumes:
- /mnt/observability/grafana:/var/lib/grafana自动添加Datasource：
datasources:
- name:
Prometheus
type:
prometheus
url:
http://prometheus:9090
- name:
Loki
type:
loki
url:
http://loki:310015.26.18 Thanos
用途：
Prometheus长期存储。
架构：
Prometheus
|
Sidecar
|
Object Storage
|
Thanos Query
|
Grafana部署：
services:
thanos-query:
image:
quay.io/thanos/thanos:v0.37
command:
query
ports:
- 10902:10902支持：
- MinIO
- S3
- OSS
- Ceph RGW
15.26.19 OpenTelemetry Collector
统一采集：
Trace
Metric
Log配置：
receivers:
otlp:
protocols:
grpc:
http:
exporters:
prometheus:
jaeger:
service:
pipelines:
traces:
metrics:
logs:15.26.20 Jaeger
部署：
services:
jaeger:
image:
jaegertracing/all-in-one:1.62
ports:
- 16686:16686
- 4317:4317访问：
http://monitor:1668615.26.21 Grafana Tempo
部署：
services:
tempo:
image:
grafana/tempo:2.6
ports:
- 3200:3200Trace查询：
Grafana
|
Tempo15.26.22 Uptime Monitor
部署：
services:
uptime-kuma:
image:
louislam/uptime-kuma
ports:
- 3001:3001监控：
HTTP
TCP
Ping
SSL
Docker15.26.23 Audit Log Platform
统一审计：
采集：
Linux auditd
Kubernetes Audit
IAM Audit
Application Audit配置：
audit:
sources:
- kubernetes
- linux
- keycloak
- vault
retention:
365d15.26.24 Monitoring健康检查
文件：
scripts/check-observability.sh
#!/bin/bash
echo Elasticsearch
curl localhost:9200
echo Loki
curl localhost:3100/ready
echo Prometheus
curl localhost:9090/-/healthy
echo Grafana
curl localhost:3000/api/health
echo AlertManager
curl localhost:9093/-/healthy
echo Thanos
curl localhost:10902/-/healthy
echo Jaeger
curl localhost:16686Part 15.26完成
✅ Elasticsearch Cluster
✅ Logstash
✅ Kibana
✅ Filebeat
✅ Fluent Bit
✅ Loki
✅ Promtail
✅ Prometheus
✅ AlertManager
✅ Grafana
✅ Thanos长期存储
✅ OpenTelemetry Collector
✅ Jaeger
✅ Grafana Tempo
✅ Uptime Monitor
✅ Audit Platform
✅ Monitoring Health Check
Part 15.27 Enterprise Backup & Disaster Recovery Platform Role完整代码
本部分生成企业级 Backup & Disaster Recovery Platform 自动化部署 Ansible Role。
覆盖企业生产备份体系：
- Kubernetes Backup
- Velero
- Restic
- BorgBackup
- Rclone
- MinIO Backup
- S3 / OSS / Ceph Backup
- MySQL Backup
- PostgreSQL Backup
- MongoDB Backup
- Redis Backup
- Elasticsearch Backup
- Etcd Backup
- PersistentVolume Snapshot
- Disaster Recovery
- Backup Verification
- Restore Testing
- Backup Monitoring
适用于：
Ubuntu 22.04 / 24.04
CentOS Stream 9
Kubernetes 1.28+
Docker
Containerd
Ansible >=2.15
生产环境15.27.1 Backup Platform整体架构
Production System
|
+-----------------------+-----------------------+
| | |
Kubernetes Database Files
| | |
Velero Dump Restic
| | |
+-----------------------+-----------------------+
|
Backup Gateway
|
+-----------------------+-----------------------+
| | |
MinIO S3 OSS
| | |
+-----------------------+-----------------------+
|
Disaster Recovery Site15.27.2 Role目录结构
创建：
roles/backup-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── velero.yml
│ ├── restic.yml
│ ├── borg.yml
│ ├── rclone.yml
│ ├── minio-backup.yml
│ ├── mysql-backup.yml
│ ├── postgres-backup.yml
│ ├── mongodb-backup.yml
│ ├── redis-backup.yml
│ ├── elasticsearch-backup.yml
│ ├── etcd-backup.yml
│ ├── snapshot.yml
│ ├── disaster-recovery.yml
│ ├── restore-test.yml
│ └── monitoring.yml
├── templates/
│
│ ├── velero-values.yaml.j2
│ ├── restic.conf.j2
│ ├── rclone.conf.j2
│ ├── backup-cron.yaml.j2
│ ├── mysql-backup.sh.j2
│ ├── postgres-backup.sh.j2
│ ├── mongodb-backup.sh.j2
│ ├── restore-test.sh.j2
│ └── backup-alert.yaml.j2
└── handlers/
└── main.yml15.27.3 默认变量
文件：
roles/backup-platform/defaults/main.yml
---
backup_root:
/mnt/backup
####################
# Storage
####################
backup_backend:
minio
backup_bucket:
enterprise-backup
backup_retention:
180
####################
# Velero
####################
velero_version:
1.15
velero_namespace:
velero
####################
# Restic
####################
restic_version:
0.17
####################
# Databases
####################
mysql_backup:
true
postgres_backup:
true
mongodb_backup:
true
redis_backup:
true
####################
# Schedule
####################
backup_schedule:
"0 2 * * *"
####################
# Encryption
####################
enable_encryption:
true15.27.4 Inventory
新增：
inventory/production/hosts.yml
---
backup_nodes:
children:
backup:
hosts:
backup01:
ansible_host:
10.10.50.10
backup02:
ansible_host:
10.10.50.1115.27.5 Playbook
文件：
playbooks/backup-platform.yml
---
- name:
Deploy Backup Platform
hosts:
backup
become:
true
roles:
- backup-platform15.27.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
velero.yml
- import_tasks:
restic.yml
- import_tasks:
borg.yml
- import_tasks:
rclone.yml
- import_tasks:
minio-backup.yml
- import_tasks:
mysql-backup.yml
- import_tasks:
postgres-backup.yml
- import_tasks:
mongodb-backup.yml
- import_tasks:
redis-backup.yml
- import_tasks:
elasticsearch-backup.yml
- import_tasks:
etcd-backup.yml
- import_tasks:
snapshot.yml
- import_tasks:
disaster-recovery.yml
- import_tasks:
restore-test.yml
- import_tasks:
monitoring.yml15.27.7 Velero Kubernetes Backup
架构
Kubernetes Cluster
|
Velero
|
Object Storage
|
MinIO / S3安装：
---
- name:
install velero
kubernetes.core.helm:
name:
velero
chart_ref:
vmware-tanzu/velero
namespace:
velero
create_namespace:
true配置：
模板：
templates/velero-values.yaml.j2
configuration:
provider:
aws
backupStorageLocation:
name:
default
bucket:
{{backup_bucket}}
config:
s3Url:
http://minio:9000
region:
us-east-1检查：
kubectl get pods -n velero输出：
velero-server Running15.27.8 Kubernetes Backup任务
创建：
backup-cron.yaml.j2
apiVersion:
velero.io/v1
kind:
Schedule
metadata:
name:
daily-backup
spec:
schedule:
{{backup_schedule}}
template:
includedNamespaces:
- "*"
ttl:
720h执行：
kubectl apply -f backup-cron.yaml查看：
velero schedule get15.27.9 Restic备份
用途：
- 文件系统
- PV
- NAS
安装：
---
- name:
install restic
apt:
name:
restic
state:
present初始化仓库：
restic init \
-r /mnt/backup/restic备份：
restic backup \
/data \
-r /mnt/backup/restic恢复：
restic restore latest \
--target /15.27.10 BorgBackup
适合：
- Linux服务器
- 增量备份
安装：
---
- name:
install borg
apt:
name:
borgbackup初始化：
borg init \
/mnt/backup/borg备份：
borg create \
/mnt/backup/borg::daily \
/etc \
/home15.27.11 Rclone云备份
支持：
- AWS S3
- MinIO
- OSS
- COS
- Azure Blob
安装：
---
- name:
install rclone
apt:
name:
rclone配置：
模板：
templates/rclone.conf.j2
[minio]
type=s3
provider=Minio
endpoint=http://minio:9000
access_key_id={{minio_user}}
secret_access_key={{minio_password}}同步：
rclone sync \
/mnt/data \
minio:enterprise-backup15.27.12 MinIO Backup
架构：
MinIO
|
mc mirror
|
Backup MinIO脚本：
mc mirror \
--overwrite \
source/ \
backup/定时：
0 3 * * *
/usr/local/bin/minio-backup.sh15.27.13 MySQL Backup
模板：
mysql-backup.sh.j2
#!/bin/bash
DATE=$(date +%F)
mysqldump \
-u root \
-p${MYSQL_PASSWORD} \
--all-databases \
| gzip \
> /mnt/backup/mysql/mysql-${DATE}.sql.gz恢复：
gunzip mysql.sql.gz | mysql15.27.14 PostgreSQL Backup
脚本：
#!/bin/bash
DATE=$(date +%F)
pg_dumpall \
-U postgres \
| gzip \
> postgres-${DATE}.sql.gz恢复：
gunzip postgres.sql.gz \
| psql15.27.15 MongoDB Backup
脚本：
mongodump:
--archive=/backup/mongo.archive
--gzip恢复：
mongorestore \
--gzip \
--archive=mongo.archive15.27.16 Redis Backup
RDB：
redis-cli save复制：
cp dump.rdb \
/mnt/backup/redis/15.27.17 Elasticsearch Backup
Snapshot：
创建：
PUT _snapshot/backup执行：
PUT _snapshot/backup/daily恢复：
POST _snapshot/backup/daily/_restore15.27.18 Etcd Backup
Kubernetes核心数据。
备份：
ETCDCTL_API=3 \
etcdctl snapshot save \
etcd.db检查：
etcdctl snapshot status etcd.db恢复：
etcdctl snapshot restore etcd.db15.27.19 Volume Snapshot
支持：
- Ceph
- AWS EBS
- CSI
PVC Snapshot：
apiVersion:
snapshot.storage.k8s.io/v1
kind:
VolumeSnapshot
metadata:
name:
db-snapshot
spec:
source:
persistentVolumeClaimName:
mysql-data15.27.20 Disaster Recovery
DR架构：
Primary Site
|
Replication
|
Secondary Site
|
Restore支持：
- 同城双活
- 异地灾备
- 冷备
配置：
dr:
primary:
dc1
secondary:
dc2
rpo:
15m
rto:
2h15.27.21 Restore自动测试
脚本：
restore-test.sh.j2
#!/bin/bash
echo "Restore Test"
velero restore create \
--from-backup daily-backup
kubectl wait pod \
--for=condition=Ready \
--all15.27.22 Backup Monitoring
Prometheus指标：
backup_status:
type:
gauge
backup_last_success:
type:
timestampAlert规则：
模板：
backup-alert.yaml.j2
groups:
- name:
backup
rules:
- alert:
BackupFailed
expr:
backup_status == 0
for:
30m15.27.23 Backup健康检查
文件：
scripts/check-backup.sh
#!/bin/bash
echo Velero
velero backup get
echo Restic
restic snapshots
echo MinIO
mc admin info minio
echo Database Backup
ls -lh /mnt/backup15.27.24 自动化执行
安装：
ansible-playbook \
-i inventory/production \
playbooks/backup-platform.yml验证：
kubectl get backup -A
velero backup get
restic snapshots
ls /mnt/backupPart 15.27完成
✅ Velero Kubernetes Backup
✅ Restic Backup
✅ BorgBackup
✅ Rclone Cloud Backup
✅ MinIO Backup
✅ MySQL Backup
✅ PostgreSQL Backup
✅ MongoDB Backup
✅ Redis Backup
✅ Elasticsearch Snapshot
✅ Etcd Backup
✅ CSI Volume Snapshot
✅ Disaster Recovery
✅ Restore Test
✅ Backup Monitoring
✅ Prometheus Alert
Part 15.28 Enterprise Storage & File Service Platform Role完整代码
本部分生成企业级 Storage & File Service Platform 自动化部署 Ansible Role
覆盖企业生产存储平台：
- Ceph Distributed Storage
- Rook-Ceph Kubernetes Storage
- MinIO Object Storage
- SeaweedFS Distributed File/Object Storage
- NFS Server
- Samba SMB File Service
- FTP Service
- SFTP Service
- GlusterFS
- Longhorn Kubernetes Storage
- CSI Driver
- Kubernetes StorageClass
- Storage Quota Management
- Snapshot
- Replication
- Data Tiering
适用于：
Ubuntu 22.04 / 24.04
Rocky Linux 9
Kubernetes 1.28+
Docker
containerd
Ansible >=2.15
生产环境15.28.1 Storage Platform整体架构
Application
|
+------------+------------+
| |
Kubernetes Virtual Machine
| |
CSI Storage NFS/iSCSI
| |
+--------+---------+---------------+
| | |
Ceph RBD MinIO SeaweedFS
| | |
+------------------+---------------+
Storage Layer
+-----------------------------------+
| |
HDD Capacity SSD Performance
| |
+-----------------------------------+15.28.2 Role目录结构
创建：
roles/storage-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── ceph.yml
│ ├── rook-ceph.yml
│ ├── minio.yml
│ ├── seaweedfs.yml
│ ├── nfs.yml
│ ├── samba.yml
│ ├── ftp.yml
│ ├── sftp.yml
│ ├── glusterfs.yml
│ ├── longhorn.yml
│ ├── csi.yml
│ ├── storageclass.yml
│ ├── quota.yml
│ ├── snapshot.yml
│ └── replication.yml
├── templates/
│
│ ├── ceph.conf.j2
│ ├── minio-compose.yml.j2
│ ├── seaweedfs-master.yml.j2
│ ├── seaweedfs-volume.yml.j2
│ ├── nfs-export.j2
│ ├── smb.conf.j2
│ ├── vsftpd.conf.j2
│ ├── storageclass.yaml.j2
│ └── quota.yaml.j2
└── handlers/
└── main.yml15.28.3 默认变量
文件：
roles/storage-platform/defaults/main.yml
---
storage_root:
/mnt/storage
###################
# Storage Type
###################
storage_backend:
ceph
###################
# Ceph
###################
ceph_version:
reef
ceph_cluster_name:
storage
###################
# MinIO
###################
minio_version:
RELEASE.2025
minio_root:
/mnt/storage/minio
###################
# SeaweedFS
###################
seaweedfs_version:
3.80
seaweedfs_master_port:
9333
seaweedfs_volume_port:
8080
###################
# NFS
###################
nfs_export_root:
/data/share
###################
# Samba
###################
samba_workgroup:
WORKGROUP
###################
# Longhorn
###################
longhorn_version:
1.7
###################
# CSI
###################
enable_csi:
true
###################
# Snapshot
###################
enable_snapshot:
true15.28.4 Inventory
文件：
inventory/production/hosts.yml
---
storage_nodes:
children:
storage:
hosts:
storage01:
ansible_host:
10.10.60.10
storage02:
ansible_host:
10.10.60.11
storage03:
ansible_host:
10.10.60.1215.28.5 Playbook
文件：
playbooks/storage-platform.yml
---
- name:
Deploy Storage Platform
hosts:
storage
become:
true
roles:
- storage-platform15.28.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
ceph.yml
- import_tasks:
rook-ceph.yml
- import_tasks:
minio.yml
- import_tasks:
seaweedfs.yml
- import_tasks:
nfs.yml
- import_tasks:
samba.yml
- import_tasks:
ftp.yml
- import_tasks:
sftp.yml
- import_tasks:
glusterfs.yml
- import_tasks:
longhorn.yml
- import_tasks:
csi.yml
- import_tasks:
storageclass.yml
- import_tasks:
quota.yml
- import_tasks:
snapshot.yml
- import_tasks:
replication.yml15.28.7 Ceph Distributed Storage
架构
Client
|
RADOS
|
+-----------+-----------+
| | |
MON01 MON02 MON03
| | |
+-----------+-----------+
|
OSD Node
+-----------+-----------+
| | |
Disk1 Disk2 Disk3安装依赖
---
- name:
install ceph packages
apt:
name:
- ceph-common
- cephadm
state:
presentBootstrap
cephadm bootstrap \
--mon-ip 10.10.60.10添加节点
ceph orch host add storage02
ceph orch host add storage03创建OSD
ceph orch daemon add osd \
storage01:/dev/sdb查看状态
ceph -s输出：
HEALTH_OK
mon:3
osd:1215.28.8 Ceph RBD
创建Pool：
ceph osd pool create kubernetes创建户：
ceph auth get-or-create client.k8s用于：
- Kubernetes PV
- OpenStack
- VM
15.28.9 Rook-Ceph Kubernetes Storage
安装：
---
- name:
install rook ceph
kubernetes.core.helm:
name:
rook-ceph
chart_ref:
rook-ceph/rook-ceph
namespace:
rook-ceph
create_namespace:
trueCephCluster：
apiVersion:
ceph.rook.io/v1
kind:
CephCluster
metadata:
name:
rook-ceph
spec:
cephVersion:
image:
quay.io/ceph/ceph:v19
dataDirHostPath:
/var/lib/rook15.28.10 MinIO Object Storage
架构：
Application
|
S3 API
|
MinIO Cluster
|
DisksCompose模板：
templates/minio-compose.yml.j2
services:
minio:
image:
minio/minio
command:
server
/data
--console-address ":9001"
ports:
- 9000:9000
- 9001:9001
volumes:
- {{minio_root}}:/data
environment:
MINIO_ROOT_USER:
admin
MINIO_ROOT_PASSWORD:
password启动：
docker compose up -d检查：
mc admin info minio15.28.11 SeaweedFS
适合：
- 海量小文件
- AI数据集
- 图片
- 模型文件
架构：
Client
|
Filer
|
Master
|
Volume Server
|
DiskMaster
模板：
seaweedfs-master.yml.j2
services:
master:
image:
chrislusf/seaweedfs
command:
master
-ip=master
-port=9333
ports:
-9333:9333Volume
services:
volume:
image:
chrislusf/seaweedfs
command:
volume
-mserver=master:9333
-port=8080
volumes:
- /mnt/storage/volume:/data检查：
weed shell
> volume.list15.28.12 NFS Server
用途：
- Linux共享
- Kubernetes RWX
安装：
---
- name:
install nfs
apt:
name:
- nfs-kernel-serverExport模板：
templates/nfs-export.j2
/data/share
*(rw,sync,no_root_squash)生效：
exportfs -ra客户端：
mount -t nfs \
10.10.60.10:/data/share \
/mnt/share15.28.13 Samba SMB
用途：
Windows共享
配置：
templates/smb.conf.j2[global]
workgroup={{samba_workgroup}}
security=user
[data]
path=/data/share
browseable=yes
writable=yes启动：
systemctl restart smb访问：
\\storage01\data15.28.14 FTP
安装：
---
- name:
install ftp
apt:
name:
- vsftpd配置：
anonymous_enable=NO
write_enable=YES启动：
systemctl restart vsftpd15.28.15 SFTP
基于SSH：
配置：
Subsystem sftp internal-sftp
Match Group sftp
ChrootDirectory /data/sftp
ForceCommand internal-sftp测试：
sftp user@server15.28.16 GlusterFS
架构：
Node1
Node2
Node3
|
Volume安装：
apt:
name:
- glusterfs-server创建：
gluster volume create gv0 \
replica 3 \
node1:/data \
node2:/data \
node3:/data挂载：
mount -t glusterfs \
node1:/gv0 \
/mnt/data15.28.17 Longhorn Kubernetes Storage
安装：
---
- name:
install longhorn
kubernetes.core.helm:
name:
longhorn
chart_ref:
longhorn/longhorn
namespace:
longhorn-system
create_namespace:
true特点：
- 云原生块存储
- 快照
- Replica
15.28.18 CSI Driver
支持：
- Ceph CSI
- NFS CSI
- Longhorn CSI
安装：
---
- name:
install CSI
kubernetes.core.helm:
name:
csi
chart_ref:
csi-driver15.28.19 StorageClass
模板：
templates/storageclass.yaml.j2
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
enterprise-storage
provisioner:
{{csi_driver}}
reclaimPolicy:
Delete
allowVolumeExpansion:
true创建：
kubectl apply -f storageclass.yaml15.28.20 PVC示例
apiVersion:
v1
kind:
PersistentVolumeClaim
metadata:
name:
app-data
spec:
storageClassName:
enterprise-storage
accessModes:
- ReadWriteMany
resources:
requests:
storage:
100Gi15.28.21 Storage Quota
模板：
quota.yaml.j2
apiVersion:
v1
kind:
ResourceQuota
metadata:
name:
storage-quota
spec:
hard:
requests.storage:
10Ti15.28.22 Snapshot
VolumeSnapshot：
apiVersion:
snapshot.storage.k8s.io/v1
kind:
VolumeSnapshot
metadata:
name:
daily
spec:
source:
persistentVolumeClaimName:
app-data15.28.23 Replication
支持：
- Ceph Replication
- MinIO Bucket Replication
- SeaweedFS Replication
MinIO：
mc replicate add \
source/bucket \
--remote-bucket backup15.28.24 Storage Monitoring
Prometheus指标：
scrape_configs:
- job_name:
ceph
static_configs:
- targets:
- ceph-exporter:9283
- job_name:
minio
static_configs:
- targets:
- minio:9000Grafana Dashboard：
Ceph Dashboard
MinIO Dashboard
Longhorn Dashboard
NFS Metrics
Storage Capacity15.28.25 Storage健康检查
文件：
scripts/check-storage.sh
#!/bin/bash
echo "Ceph"
ceph -s
echo "MinIO"
mc admin info minio
echo "SeaweedFS"
curl localhost:9333/cluster/status
echo "NFS"
showmount -e
echo "Longhorn"
kubectl get pods -n longhorn-system15.28.26 自动执行
部署：
ansible-playbook \
-i inventory/production \
playbooks/storage-platform.yml验证：
ceph -s
kubectl get storageclass
kubectl get pvc -A
mc admin info minio
weed shellPart 15.28完成
✅ Ceph Distributed Storage
✅ Rook-Ceph Kubernetes Storage
✅ MinIO Object Storage
✅ SeaweedFS Distributed Storage
✅ NFS Server
✅ Samba SMB
✅ FTP
✅ SFTP
✅ GlusterFS
✅ Longhorn
✅ CSI Driver
✅ Kubernetes StorageClass
✅ PVC自动供给
✅ Quota管理
✅ Snapshot
✅ Replication
✅ Storage Monitoring
✅ Health Check
Part 15.29 Enterprise Network & Security Edge Platform Role完整代码
本部分生成企业级 Network & Security Edge Platform 自动化部署 Ansible Role
覆盖企业生产网络入口层：
- Keepalived 高可用 VIP
- HAProxy L4/L7 Load Balancer
- Nginx Reverse Proxy
- Traefik Gateway
- OpenVPN VPN Gateway
- WireGuard VPN
- Bastion Host 堡垒机
- JumpServer 运维审计
- Firewall
- iptables
- nftables
- WAF
- DDoS Protection
- Network Monitoring
- DNS Forwarder
- TCP/UDP Proxy
- SSL/TLS Certificate Management
适用于：
Ubuntu 22.04 / 24.04
Rocky Linux 9
Debian 12
Kubernetes
Docker
Ansible >=2.15
生产企业网络环境15.29.1 网络安全整体架构
Internet
|
ISP / BGP / CDN
|
+-------+-------+
| |
Firewall DDoS
|
Keepalived VIP
|
+---------+---------+
| |
HAProxy01 HAProxy02
| |
+---------+---------+
|
Nginx Gateway
|
+-----------+------------+
| |
Kubernetes Ingress VM Service
|
Service Mesh
|
Applications15.29.2 Role目录结构
创建：
roles/network-edge-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── keepalived.yml
│ ├── haproxy.yml
│ ├── nginx.yml
│ ├── traefik.yml
│ ├── firewall.yml
│ ├── iptables.yml
│ ├── nftables.yml
│ ├── openvpn.yml
│ ├── wireguard.yml
│ ├── bastion.yml
│ ├── jumpserver.yml
│ ├── waf.yml
│ ├── ddos.yml
│ ├── dns.yml
│ ├── ssl.yml
│ └── monitoring.yml
├── templates/
│
│ ├── keepalived.conf.j2
│ ├── haproxy.cfg.j2
│ ├── nginx.conf.j2
│ ├── traefik.yml.j2
│ ├── firewall.rules.j2
│ ├── nftables.conf.j2
│ ├── openvpn.conf.j2
│ ├── wireguard.conf.j2
│ ├── ssl-renew.sh.j2
│ └── alert.rules.j2
└── handlers/
└── main.yml15.29.3 默认变量
文件：
roles/network-edge-platform/defaults/main.yml
---
network_root:
/etc/network-platform
##################
# HA
##################
enable_keepalived:
true
vip_address:
10.10.1.100
interface:
eth0
##################
# HAProxy
##################
haproxy_version:
3.0
haproxy_frontend_port:
443
##################
# Nginx
##################
nginx_version:
stable
##################
# VPN
##################
enable_openvpn:
true
enable_wireguard:
true
vpn_network:
10.20.0.0/24
##################
# Firewall
##################
firewall_backend:
nftables
##################
# WAF
##################
enable_waf:
true
waf_engine:
modsecurity
##################
# SSL
##################
ssl_provider:
letsencrypt15.29.4 Inventory
文件：
inventory/production/hosts.yml
---
network_edge:
children:
load_balancer:
hosts:
edge01:
ansible_host:
10.10.1.10
edge02:
ansible_host:
10.10.1.11
vpn:
hosts:
vpn01:
ansible_host:
10.10.2.1015.29.5 Playbook
文件：
playbooks/network-edge.yml
---
- name:
Deploy Network Edge Platform
hosts:
network_edge
become:
true
roles:
- network-edge-platform15.29.6 Role入口
文件：
tasks/main.yml
---
- import_tasks:
keepalived.yml
- import_tasks:
haproxy.yml
- import_tasks:
nginx.yml
- import_tasks:
traefik.yml
- import_tasks:
firewall.yml
- import_tasks:
iptables.yml
- import_tasks:
nftables.yml
- import_tasks:
openvpn.yml
- import_tasks:
wireguard.yml
- import_tasks:
bastion.yml
- import_tasks:
jumpserver.yml
- import_tasks:
waf.yml
- import_tasks:
ddos.yml
- import_tasks:
dns.yml
- import_tasks:
ssl.yml
- import_tasks:
monitoring.yml15.29.7 Keepalived 高可用 VIP
架构
Client
|
10.10.1.100 VIP
|
+-------+-------+
| |
HAProxy01 HAProxy02
MASTER BACKUP安装：
- name:
install keepalived
package:
name:
keepalived
state:
present模板：
templates/keepalived.conf.j2
vrrp_instance VI_1 {
state MASTER
interface {{interface}}
virtual_router_id 51
priority 150
authentication {
auth_type PASS
auth_pass password
}
virtual_ipaddress {
{{vip_address}}
}
}启动：
systemctl enable keepalived
systemctl restart keepalived验证：
ip addr show15.29.8 HAProxy
用途：
- 四层TCP
- 七层HTTP
- Kubernetes入口
安装：
- name:
install haproxy
package:
name:
haproxy
state:
present配置模板：
templates/haproxy.cfg.j2
global
log /dev/log local0
maxconn 50000
defaults
mode http
timeout connect 5s
timeout client 60s
timeout server 60s
frontend https
bind *:443 ssl crt /etc/ssl/server.pem
default_backend backend_web
backend backend_web
balance roundrobin
server web01 10.10.10.10:443 check
server web02 10.10.10.11:443 check检查：
haproxy -c -f /etc/haproxy/haproxy.cfg15.29.9 Nginx Reverse Proxy
架构：
Client
|
Nginx
|
Backend Service安装：
package:
name:
nginx模板：
server {
listen 443 ssl;
server_name example.com;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}重载：
nginx -t
systemctl reload nginx15.29.10 Traefik Gateway
Kubernetes入口。
安装：
kubernetes.core.helm:
name:
traefik
chart_ref:
traefik/traefik
namespace:
traefik
create_namespace:
trueIngressRoute：
apiVersion:
traefik.io/v1alpha1
kind:
IngressRoute
spec:
routes:
- match:
Host(`app.company.com`)
services:
- name:
app
port:
8015.29.11 Firewall
支持：
- firewalld
- nftables
- iptables
默认策略：
Allow:
22
80
443
1194
51820
Deny:
Other安装：
package:
name:
firewalld规则：
firewall-cmd \
--add-service=https \
--permanent
firewall-cmd --reload15.29.12 nftables
模板：
templates/nftables.conf.j2table inet filter {
chain input {
type filter hook input priority 0;
policy drop;
tcp dport {22,80,443} accept;
icmp type echo-request accept;
}
}加载：
nft -f /etc/nftables.conf15.29.13 OpenVPN
架构：
Remote User
|
OpenVPN Server
|
Internal Network安装：
package:
name:
- openvpn
- easy-rsa配置：
port 1194
proto udp
dev tun
server 10.20.0.0 255.255.255.0
push "redirect-gateway def1"
push "dhcp-option DNS 10.10.0.10"启动：
systemctl enable openvpn-server@server15.29.14 WireGuard
配置：
templates/wireguard.conf.j2
[Interface]
Address = 10.30.0.1/24
ListenPort = 51820
PrivateKey = SERVER_KEY
[Peer]
PublicKey = CLIENT_KEY
AllowedIPs = 10.30.0.2/32启动：
wg-quick up wg0查看：
wg show15.29.15 Bastion Host
用途：
- SSH入口
- 运维跳板
SSH强化：
PermitRootLogin no
PasswordAuthentication no
AllowGroups ops安全策略：
User
|
Bastion
|
Internal Server15.29.16 JumpServer
部署：
services:
jumpserver:
image:
jumpserver/jms_all
ports:
- 80:80功能：
- SSH审计
- 命令记录
- 权限控制
- MFA
15.29.17 WAF
支持：
- ModSecurity
- OWASP CRS
安装：
package:
name:
- libapache2-mod-security2规则：
SQL Injection
XSS
Command Injection
Scanner Detection15.29.18 DDoS Protection
策略：
Layer 3
- Rate Limit
Layer 4
- SYN Flood Protection
Layer 7
- Request LimitNginx：
limit_req_zone
$binary_remote_addr
zone=req:10m
rate=10r/s;15.29.19 DNS Forwarder
部署：
package:
name:
dnsmasq配置：
server=8.8.8.8
cache-size=1000015.29.20 SSL自动化
Let's Encrypt：
安装：
certbot自动续期：
0 3 * * *
certbot renew15.29.21 Network Monitoring
采集：
- SNMP
- NetFlow
- Blackbox
Prometheus：
scrape_configs:
- job_name:
network
static_configs:
- targets:
- router:9100Grafana：
Dashboard：
HAProxy Metrics
Nginx Metrics
VPN Users
Bandwidth
Connection
Firewall Drop15.29.22 健康检查
脚本：
scripts/check-network.sh
#!/bin/bash
echo HAProxy
systemctl status haproxy
echo Keepalived
systemctl status keepalived
echo Firewall
nft list ruleset
echo VPN
wg show
echo Nginx
nginx -t15.29.23 自动部署
执行：
ansible-playbook \
-i inventory/production \
playbooks/network-edge.yml验证：
ip addr
systemctl status keepalived
systemctl status haproxy
curl https://vip
wg show
kubectl get ingressPart 15.29完成
✅ Keepalived HA VIP
✅ HAProxy L4/L7 Load Balancer
✅ Nginx Reverse Proxy
✅ Traefik Kubernetes Gateway
✅ Firewall
✅ iptables
✅ nftables
✅ OpenVPN
✅ WireGuard
✅ Bastion Host
✅ JumpServer
✅ ModSecurity WAF
✅ DDoS Protection
✅ DNS Forwarder
✅ SSL自动化
✅ Network Monitoring
✅ Health Check
Part 15.30 Enterprise Message & Integration Platform Role完整代码
本部分生成企业级 Message & Integration Platform 自动化部署 Ansible Role。
覆盖企业消息中间件体系：
- Apache Kafka Cluster
- Apache Kafka KRaft Mode
- ZooKeeper
- Kafka Connect
- Debezium CDC
- Schema Registry
- RabbitMQ Cluster
- RabbitMQ HA
- RocketMQ Cluster
- Apache Pulsar
- NATS JetStream
- MQTT Broker (EMQX)
- ActiveMQ Artemis
- Event Streaming Platform
- Message Monitoring
- Message Backup
- Disaster Recovery
适用于：
Ubuntu 22.04 / 24.04
Rocky Linux 9
Docker
Kubernetes
Ansible >=2.15
生产企业环境15.30.1 Message Platform整体架构
Application
|
+-------------------+-------------------+
| | |
Kafka RabbitMQ MQTT
| | |
Event Stream Business MQ IoT
| | |
+-------------------+-------------------+
|
Integration Layer
|
+-------------------+-------------------+
| | |
Debezium Kafka Connect Schema Registry
|
Database CDC
|
Data Platform15.30.2 Role目录结构
创建：
roles/message-platform/
├── defaults/
│ └── main.yml
├── tasks/
│ ├── main.yml
│ ├── kafka.yml
│ ├── kraft.yml
│ ├── zookeeper.yml
│ ├── kafka-connect.yml
│ ├── schema-registry.yml
│ ├── debezium.yml
│ ├── rabbitmq.yml
│ ├── rocketmq.yml
│ ├── pulsar.yml
│ ├── nats.yml
│ ├── mqtt.yml
│ ├── activemq.yml
│ ├── backup.yml
│ └── monitoring.yml
├── templates/
│
│ ├── kafka-server.properties.j2
│ ├── kraft.properties.j2
│ ├── docker-compose.yml.j2
│ ├── rabbitmq.conf.j2
│ ├── rocketmq.conf.j2
│ ├── pulsar.yaml.j2
│ ├── nats.conf.j2
│ ├── emqx.conf.j2
│ ├── connect-distributed.properties.j2
│ ├── debezium.json.j2
│ └── alert.rules.j2
└── handlers/
└── main.yml15.30.3 默认变量
文件：
roles/message-platform/defaults/main.yml---
message_root:
/mnt/message
##################
# Kafka
##################
kafka_version:
3.9
kafka_cluster_name:
enterprise-kafka
kafka_broker_count:
3
kafka_heap:
4G
##################
# ZooKeeper
##################
zookeeper_version:
3.9
##################
# RabbitMQ
##################
rabbitmq_version:
3.13
##################
# RocketMQ
##################
rocketmq_version:
5.3
##################
# Pulsar
##################
pulsar_version:
3.3
##################
# NATS
##################
nats_version:
2.10
##################
# MQTT
##################
mqtt_backend:
emqx
##################
# Retention
##################
message_retention_days:
715.30.4 Inventory
文件：
inventory/production/hosts.yml---
message_platform:
children:
kafka_cluster:
hosts:
kafka01:
ansible_host:
10.10.70.10
kafka02:
ansible_host:
10.10.70.11
kafka03:
ansible_host:
10.10.70.12
rabbitmq:
hosts:
mq01:
ansible_host:
10.10.71.1015.30.5 Playbook
文件：
playbooks/message-platform.yml---
- name:
Deploy Message Platform
hosts:
message_platform
become:
true
roles:
- message-platform15.30.6 Role入口
文件：
tasks/main.yml---
- import_tasks:
kafka.yml
- import_tasks:
kraft.yml
- import_tasks:
zookeeper.yml
- import_tasks:
kafka-connect.yml
- import_tasks:
schema-registry.yml
- import_tasks:
debezium.yml
- import_tasks:
rabbitmq.yml
- import_tasks:
rocketmq.yml
- import_tasks:
pulsar.yml
- import_tasks:
nats.yml
- import_tasks:
mqtt.yml
- import_tasks:
activemq.yml
- import_tasks:
backup.yml
- import_tasks:
monitoring.yml15.30.7 Apache Kafka Cluster
架构
Producer
|
Kafka Cluster
+-----------+-----------+
| | |
Broker1 Broker2 Broker3
| | |
+-----------+-----------+
ConsumerDocker Compose
模板：
templates/docker-compose.yml.j2services:
kafka01:
image:
apache/kafka:3.9.0
hostname:
kafka01
ports:
- 9092:9092
environment:
KAFKA_NODE_ID:
1
KAFKA_PROCESS_ROLES:
broker,controller
KAFKA_CONTROLLER_QUORUM_VOTERS:
1@kafka01:9093,
2@kafka02:9093,
3@kafka03:9093
KAFKA_LISTENERS:
PLAINTEXT://:9092,CONTROLLER://:9093启动：
docker compose up -d检查：
kafka-topics.sh \
--bootstrap-server localhost:9092 \
--list15.30.8 Kafka KRaft Mode
替代 ZooKeeper。
配置：
process.roles=
broker,controller
node.id=1
controller.quorum.voters=
1@kafka01:9093,
2@kafka02:9093,
3@kafka03:9093初始化：
kafka-storage.sh format \
-t UUID \
-c server.properties15.30.9 ZooKeeper
用于传统Kafka。
部署：
services:
zookeeper:
image:
zookeeper:3.9
ports:
- 2181:2181检查：
echo ruok | nc localhost 2181返回：
imok15.30.10 Kafka Connect
用途：
- 数据同步
- CDC
- ETL
配置：
模板：
connect-distributed.properties.j2bootstrap.servers=
kafka01:9092
group.id=
connect-cluster
key.converter=
org.apache.kafka.connect.json.JsonConverter
offset.storage.topic=
connect-offsets启动：
connect-distributed.sh connect.properties查看Connector：
curl localhost:8083/connectors15.30.11 Schema Registry
用途：
管理：
- Avro
- JSON Schema
- Protobuf
部署：
services:
schema-registry:
image:
confluentinc/cp-schema-registry
ports:
- 8081:8081检查：
curl localhost:8081/subjects15.30.12 Debezium CDC
用途：
数据库实时同步。
支持：
- MySQL
- PostgreSQL
- MongoDB
配置：
模板：
debezium.json.j2{
"name":
"mysql-cdc",
"config":{
"connector.class":
"io.debezium.connector.mysql.MySqlConnector",
"database.hostname":
"mysql",
"database.user":
"root",
"topic.prefix":
"db"
}
}注册：
curl -X POST \
localhost:8083/connectors \
-d @debezium.json15.30.13 RabbitMQ Cluster
架构：
Client
|
+------+------+
| |
Rabbit01 Rabbit02
| |
+------+------+
|
Queue部署：
services:
rabbitmq:
image:
rabbitmq:3.13-management
ports:
- 5672:5672
- 15672:15672集群：
rabbitmqctl join_cluster rabbit@mq01检查：
rabbitmqctl cluster_status15.30.14 RabbitMQ HA Queue
策略：
rabbitmqctl set_policy ha-all \
".*" \
'{"ha-mode":"all"}'15.30.15 RocketMQ
架构：
Producer
|
NameServer
|
Broker
|
Consumer部署：
services:
namesrv:
image:
apache/rocketmq
broker:
image:
apache/rocketmq检查：
mqadmin clusterList15.30.16 Apache Pulsar
架构：
Client
|
Broker
|
BookKeeper
|
ZooKeeper部署：
services:
pulsar:
image:
apachepulsar/pulsar
command:
standalone测试：
pulsar-client produce test \
-m "hello"15.30.17 NATS JetStream
用途：
- IoT
- 微服务事件
配置：
nats.confjetstream {
store_dir:
/data
}
port:
4222启动：
nats-server \
-c nats.conf检查：
nats server info15.30.18 MQTT EMQX
用途：
IoT消息。
部署：
services:
emqx:
image:
emqx/emqx
ports:
- 1883:1883
- 18083:18083访问：
http://mqtt-server:1808315.30.19 ActiveMQ Artemis
部署：
services:
activemq:
image:
apache/activemq-artemis支持：
- JMS
- AMQP
- MQTT
- STOMP
15.30.20 Message Backup
备份：
Kafka Topic：
kafka-replica-verification.shRabbitMQ：
rabbitmq-dump-queuesPulsar：
pulsar-admin topics list15.30.21 Message Monitoring
Prometheus指标：
Kafka：
scrape_configs:
- job_name:
kafka
static_configs:
- targets:
- kafka-exporter:9308RabbitMQ：
rabbitmq_exporterGrafana Dashboard：
Kafka:
Broker Throughput
Consumer Lag
Partition
RabbitMQ:
Queue Depth
Message Rate
RocketMQ:
TPS
Latency
MQTT:
Connection Count15.30.22 Alert规则
模板：
alert.rules.j2groups:
- name:
message
rules:
- alert:
KafkaConsumerLag
expr:
kafka_consumergroup_lag > 10000
for:
5m15.30.23 健康检查
脚本：
scripts/check-message.sh#!/bin/bash
echo Kafka
kafka-topics.sh \
--bootstrap-server kafka01:9092 \
--list
echo RabbitMQ
rabbitmqctl status
echo RocketMQ
mqadmin clusterList
echo Pulsar
pulsar-admin brokers list15.30.24 自动部署
执行：
ansible-playbook \
-i inventory/production \
playbooks/message-platform.yml验证：
# Kafka
kafka-topics.sh --list \
--bootstrap-server kafka01:9092
# RabbitMQ
rabbitmqctl cluster_status
# RocketMQ
mqadmin clusterList
# Pulsar
pulsar-admin brokers list
# MQTT
emqx ctl statusPart 15.30完成
已生成：
✅ Kafka Cluster
✅ Kafka KRaft Mode
✅ ZooKeeper
✅ Kafka Connect
✅ Schema Registry
✅ Debezium CDC
✅ RabbitMQ Cluster
✅ RabbitMQ HA Queue
✅ RocketMQ
✅ Apache Pulsar
✅ NATS JetStream
✅ MQTT EMQX
✅ ActiveMQ Artemis
✅ Message Backup
✅ Message Monitoring
✅ Prometheus Metrics
✅ Grafana Dashboard
✅ Alert Rules