企业 DevOps + 协同办公 + 可观测性 + 网络基础设施平台-Part 15.18 Storage & Data Platform Role

本部分完成企业级 Storage & Data Platform 自动化部署 Ansible Role

包含：

SeaweedFS 分布式文件系统
MinIO 对象存储
Ceph 集群
Rook-Ceph Kubernetes存储
NFS共享存储
MongoDB ReplicaSet
PostgreSQL HA
MySQL InnoDB Cluster
Redis Cluster
Kafka Cluster
Elasticsearch
TDengine
数据备份集成
StorageClass自动化

适用于：

复制代码

Ubuntu 22.04 / 24.04

Debian 12

CentOS Stream 9

Kubernetes 1.28+

Docker Compose

Ansible >=2.15

15.18.1 Storage整体架构

复制代码

                 Application Layer


                       |

                       |


              Kubernetes PVC / S3 API


                       |


        +--------------+---------------+

        |              |               |

     Rook-Ceph      SeaweedFS       MinIO

        |              |               |

        |              |               |

     Block/RBD       File          Object


        |

        |

   Ceph Cluster



Data Services:


+-------------+-------------+-------------+

|             |             |

MongoDB     PostgreSQL     MySQL


|             |             |

Redis       Kafka       TDengine

15.18.2 Role目录结构

创建：

复制代码

roles/storage-data/


├── defaults/

│   └── main.yml


├── tasks/

│
│   ├── main.yml
│   ├── seaweedfs.yml
│   ├── minio.yml
│   ├── ceph.yml
│   ├── rook-ceph.yml
│   ├── nfs.yml
│   ├── mongodb.yml
│   ├── postgresql.yml
│   ├── mysql.yml
│   ├── redis.yml
│   ├── kafka.yml
│   ├── elasticsearch.yml
│   ├── tdengine.yml
│   └── storageclass.yml


├── templates/


│   ├── docker-compose-minio.yml.j2
│   ├── seaweedfs-master.service.j2
│   ├── seaweedfs-volume.service.j2
│   ├── mongodb-compose.yml.j2
│   ├── postgres-compose.yml.j2
│   ├── redis-compose.yml.j2
│   ├── kafka-compose.yml.j2
│   └── elasticsearch.yml.j2


└── handlers/

    └── main.yml

15.18.3 默认变量

文件：

复制代码

roles/storage-data/defaults/main.yml

---

storage_root:

 /mnt/storage



storage_network:

 10.10.10.0/24



################
# SeaweedFS
################

seaweedfs_version:

 latest


seaweedfs_master_port:

 9333


seaweedfs_volume_port:

 8080



################
# MinIO
################


minio_root:

 /mnt/storage/minio


minio_user:

 minio


minio_password:

 ChangeMeMinio123



################
# Ceph
################


ceph_version:

 reef



################
# MongoDB
################


mongodb_version:

 7.0


mongodb_root:

 /mnt/storage/mongodb



################
# PostgreSQL
################


postgres_version:

 16



################
# Redis
################


redis_version:

 7.2



################
# Kafka
################


kafka_version:

 3.8



################
# Elasticsearch
################


elastic_version:

 8.15



################
# TDengine
################


tdengine_version:

 3.3

15.18.4 Playbook

文件：

复制代码

playbooks/storage-data.yml

---

- name:

    Deploy Storage Data Platform


  hosts:

    storage


  become:

    true


  roles:


    - storage-data

15.18.5 Role入口

文件：

复制代码

tasks/main.yml

---

- import_tasks:

    nfs.yml


- import_tasks:

    seaweedfs.yml


- import_tasks:

    minio.yml


- import_tasks:

    ceph.yml


- import_tasks:

    rook-ceph.yml


- import_tasks:

    mongodb.yml


- import_tasks:

    postgresql.yml


- import_tasks:

    mysql.yml


- import_tasks:

    redis.yml


- import_tasks:

    kafka.yml


- import_tasks:

    elasticsearch.yml


- import_tasks:

    tdengine.yml


- import_tasks:

    storageclass.yml

Part 15.18.6 NFS共享存储

用途：

Jenkins Workspace
Kubernetes RWX PVC
文件共享

安装：

复制代码

---

- name:

    install nfs


  apt:


    name:


      - nfs-server


    state:

      present

创建目录：

复制代码

- name:

    create nfs directory


  file:


    path:

      /mnt/storage/nfs


    state:

      directory

exports：

复制代码

/mnt/storage/nfs

10.10.10.0/24

(rw,sync,no_root_squash)

加载：

复制代码

exportfs -ra

验证：

复制代码

showmount -e localhost

15.18.7 SeaweedFS

架构

复制代码

                 Filer


                  |

          +-------+-------+

          |               |

       Master1         Master2


          |

          |

      Volume Server

Master Service

模板：

复制代码

templates/seaweedfs-master.service.j2

[Unit]

Description=SeaweedFS Master


[Service]

ExecStart=/usr/local/bin/weed master \
-ip={{ inventory_hostname }} \
-port=9333


Restart=always



[Install]

WantedBy=multi-user.target

安装：

复制代码

---

- name:

    download weed


  get_url:


    url:

      https://github.com/seaweedfs/seaweedfs/releases/latest/download/linux_amd64.tar.gz


    dest:

      /tmp/weed.tar.gz

解压：

复制代码

- unarchive:


    src:

      /tmp/weed.tar.gz


    dest:

      /usr/local/bin


    remote_src:

      yes

启动：

复制代码

- systemd:


    name:

      seaweedfs-master


    enabled:

      yes


    state:

      started

Volume Server

配置：

复制代码

weed volume \
-server.port=8080 \
-dir=/mnt/storage/volume

验证：

复制代码

curl http://localhost:9333/cluster/status

15.18.8 MinIO对象存储

Docker Compose

模板：

复制代码

docker-compose-minio.yml.j2

services:


 minio:


  image:

   minio/minio:latest


  container_name:

   minio


  restart:

   always


  ports:


   - 9000:9000

   - 9001:9001



  environment:


   MINIO_ROOT_USER:

    {{ minio_user }}


   MINIO_ROOT_PASSWORD:

    {{ minio_password }}



  volumes:


   - {{ minio_root }}:/data



  command:


   server /data --console-address ":9001"

启动：

复制代码

docker compose up -d

验证：

复制代码

mc alias set minio http://localhost:9000

15.18.9 Ceph集群

15.18.10 Rook-Ceph Kubernetes

安装：

复制代码

---

- name:

 install rook


 kubernetes.core.k8s:


  src:


   https://raw.githubusercontent.com/rook/rook/master/deploy/examples/crds.yaml

部署：

复制代码

kubectl apply \
-f rook-ceph-operator.yaml

StorageClass：

复制代码

apiVersion:

storage.k8s.io/v1


kind:

StorageClass


metadata:


 name:

 ceph-rbd


provisioner:

 rook-ceph.rbd.csi.ceph.com

15.18.11 MongoDB ReplicaSet

Compose:

复制代码

services:


 mongo:


  image:

   mongo:7


  command:


   mongod --replSet rs0


  volumes:


   - /mnt/storage/mongo:/data/db

初始化：

复制代码

rs.initiate(
{
_id:"rs0",
members:[
{
_id:0,
host:"mongo:27017"
}
]
}
)

15.18.12 PostgreSQL

Compose:

复制代码

services:


 postgres:


  image:

   postgres:16


  environment:


   POSTGRES_PASSWORD:

    password



  volumes:


   - /mnt/storage/postgres:/var/lib/postgresql/data

备份：

复制代码

pg_dumpall > backup.sql

15.18.13 MySQL

部署：

复制代码

services:


 mysql:


  image:

   mysql:8.4


  environment:


   MYSQL_ROOT_PASSWORD:

    password


  volumes:


   - /mnt/storage/mysql:/var/lib/mysql

15.18.14 Redis Cluster

配置：

复制代码

services:


 redis:


  image:

   redis:7.2


  command:


   redis-server --cluster-enabled yes

创建：

复制代码

redis-cli \
--cluster create \
10.10.10.1:6379 \
10.10.10.2:6379

15.18.15 Kafka

架构：

复制代码

Kafka


 |

Zookeeper/KRaft


 |

Broker1

Broker2

Broker3

Compose:

复制代码

services:


 kafka:


  image:

   bitnami/kafka:latest


  environment:


   KAFKA_ENABLE_KRAFT:

    yes


   KAFKA_CFG_NODE_ID:

    1

测试：

复制代码

kafka-topics.sh --list

15.18.16 Elasticsearch

配置：

复制代码

services:


 elasticsearch:


  image:

   elasticsearch:8.15


  environment:


   discovery.type:

    single-node



  volumes:


   - /mnt/storage/es:/usr/share/elasticsearch/data

验证：

复制代码

curl localhost:9200

15.18.17 TDengine

部署：

复制代码

services:


 tdengine:


  image:

   tdengine/tdengine:latest


  ports:


   - 6041:6041


  volumes:


   - /mnt/storage/tdengine:/var/lib/taos

验证：

复制代码

taos

15.18.18 StorageClass统一管理

复制代码

storageclass.yml

支持：

复制代码

ceph-rbd

cephfs

nfs

seaweedfs

minio-s3

示例：

复制代码

apiVersion:

storage.k8s.io/v1


kind:

StorageClass


metadata:


 name:

 nfs-storage


provisioner:

 k8s-sigs.io/nfs-subdir-external-provisioner

15.18.19 健康检查

文件：

复制代码

scripts/check-storage.sh

#!/bin/bash


echo "SeaweedFS"

curl localhost:9333/cluster/status



echo "MinIO"

mc admin info minio



echo "Ceph"

ceph -s



echo "MongoDB"

mongosh --eval "rs.status()"



echo "PostgreSQL"

pg_isready



echo "Redis"

redis-cli ping



echo "Kafka"

kafka-topics.sh --list

Part 15.18完成

已生成：

✅ SeaweedFS分布式文件系统

✅ MinIO对象存储

✅ Ceph存储集群

✅ Rook-Ceph Kubernetes存储

✅ NFS共享存储

✅ MongoDB ReplicaSet

✅ PostgreSQL

✅ MySQL

✅ Redis Cluster

✅ Kafka KRaft

✅ Elasticsearch

✅ TDengine

✅ Kubernetes StorageClass

✅ 存储健康检查