K8S集群实践之九: Ceph

Rook is an open source cloud-native storage orchestrator, providing the platform, framework, and support for Ceph storage to natively integrate with cloud-native environments.1. 说明

因香橙派和树莓派资源所限,转移到基于VirtualBox建立的VMs继续实践。

虚拟机环境列表:

|----------------|----------|-------------------------------------------------|-----|
| Host Name | IP | 配置 | 备注 |
| k8s-c0-master0 | 10.0.3.6 | Ubuntu 22.04.3 LTS 8核32G, 200G(sda) + 100G(sdb) | 虚拟机 |
| k8s-c0-node0 | 10.0.3.7 | Ubuntu 22.04.3 LTS 4核8G, 200G(sda) + 100G(sdb) | ... |
| k8s-c0-node1 | 10.0.3.8 | Ubuntu 22.04.3 LTS 4核8G, 200G(sda) + 100G(sdb) | ... |

2. 准备工作

  • 下载 ubuntu-22.04.2-live-server-amd64.iso,并映射到启动光盘

  • 虚拟机设置双网卡,1作桥接(同网访问),1作NAT网络(K8s内网)

  • 安装时选择镜像(重要)https://mirrors.aliyun.com/ubuntu

  • 初始化K8s环境

  • 为避免CTR拉取镜像失败,设置容器代理,编辑 /lib/systemd/system/containerd.service

    复制代码
    [Service]
    Environment="HTTP_PROXY=http://192.168.0.108:1081"
    Environment="HTTPS_PROXY=http://192.168.0.108:1081"
    Environment="NO_PROXY=aliyun.com,aliyuncs.com,huaweicloud.com,k8s-master-0,k8s-master-1,k8s-worker-0,localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
    
    systemctl daemon-reload && systemctl restart containerd

3. 方案及安装步骤

3.1 Rook

Rook is an open source cloud-native storage orchestrator, providing the platform, framework, and support for Ceph storage to natively integrate with cloud-native environments.

3.2 安装要求

  • Raw devices (no partitions or formatted filesystems)
  • Raw partitions (no formatted filesystem)
  • LVM Logical Volumes (no formatted filesystem)
  • Persistent Volumes available from a storage class in block mode

一句话:有个裸盘 (100G /dev/sdb)

3.3 安装步骤

  1. 获取rook仓库到安装路径,如:/k8s_apps/rook

    git clone --single-branch --branch v1.12.6 https://github.com/rook/rook.git

  2. 编写安装脚本,/k8s_apps/scripts/k8s-rook-ceph.sh

    #!/bin/bash

    kubectl apply -f /k8s_apps/rook/deploy/examples/crds.yaml
    kubectl apply -f /k8s_apps/rook/deploy/examples/common.yaml
    kubectl apply -f /k8s_apps/rook/deploy/examples/operator.yaml

    kubectl -n rook-ceph get pod

    kubectl apply -f /k8s_apps/rook/deploy/examples/cluster.yaml

  3. 编写删除脚本,/k8s_apps/scripts/rook-ceph-delete.sh

    #!/bin/bash

    kubectl delete -f /k8s_apps/rook/deploy/examples/wordpress.yaml
    kubectl delete -f /k8s_apps/rook/deploy/examples/mysql.yaml
    kubectl delete -n rook-ceph cephblockpool replicapool
    kubectl delete storageclass rook-ceph-block
    kubectl delete -f /k8s_apps/rook/deploy/examples/csi/cephfs/kube-registry.yaml
    kubectl delete storageclass csi-cephfs

    kubectl -n rook-ceph patch cephcluster rook-ceph --type merge -p '{"spec":{"cleanupPolicy":{"confirmation":"yes-really-destroy-data"}}}'

    kubectl -n rook-ceph delete cephcluster rook-ceph

    kubectl -n rook-ceph get cephcluster

    kubectl delete -f /k8s_apps/rook/deploy/examples/operator.yaml
    kubectl delete -f /k8s_apps/rook/deploy/examples/common.yaml
    kubectl delete -f /k8s_apps/rook/deploy/examples/crds.yaml

  4. 执行安装,一切正常的话,如下图,不正常的话,执行删除脚本,排除问题后重来。

4. 参考

相关推荐
IT成长日记2 小时前
【Docker基础】Docker数据卷管理:docker volume inspect及其参数详解
运维·docker·容器·volume·inspect
伤不起bb2 小时前
Kubernetes 服务发布基础
云原生·容器·kubernetes
ladymorgana2 小时前
【Docker】如何设置 `wiredTigerCacheSizeGB` 和 `resources.limits.memory`
运维·docker·容器
mcdx2 小时前
基于Docker构建OrangePi5 SDK环境
docker·容器
别骂我h5 小时前
Kubernetes服务发布基础
云原生·容器·kubernetes
weixin_399380697 小时前
k8s一键部署tongweb企业版7049m6(by why+lqw)
java·linux·运维·服务器·云原生·容器·kubernetes
IT成长日记7 小时前
【Docker基础】Docker数据卷管理:docker volume ls及其参数详解
运维·docker·容器·volume ls
偷萧逸苦茶7 小时前
docker常见命令
docker·容器·eureka
高山莫衣21 小时前
Docker Desktop导致存储空间不足时的解决方案
docker·容器·eureka
鹏大师运维21 小时前
在银河麒麟V10 SP1上手动安装与配置高版本Docker的完整指南
linux·运维·docker·容器·麒麟·统信uos·中科方德