ceph更换硬盘

ceph -s //检查故障

k8s70132:~$ ceph -s
  cluster:
    id:     d10b3028-b78d-4b23-bacb-ca26c0a35c99
    health: HEALTH_WARN
            1 daemons have recently crashed
 
  services:
    mon: 5 daemons, quorum k8sceph70201,k8sceph70202,k8sceph70203,k8sceph70204,k8sceph70205 (age 4d)
    mgr: k8sceph70204(active, since 11w), standbys: k8sceph70201, k8sceph70205
    mds: cephfs:1 {0=k8sceph70204=up:active} 2 up:standby
    osd: 23 osds: 22 up (since 4d), 22 in (since 4d)
    rgw: 3 daemons active (k8sceph70201, k8sceph70204, k8sceph70205)
 
  task status:
 
  data:
    pools:   11 pools, 281 pgs
    objects: 809.23k objects, 24 GiB
    usage:   159 GiB used, 38 TiB / 38 TiB avail
    pgs:     281 active+clean
 
  io:
    client:   47 KiB/s wr, 0 op/s rd, 2 op/s wr

这里health有告警,查看故障点:

ceph crush ls

k8s70132:~$ ceph crash ls
ID                                                                ENTITY  NEW  
2023-04-30T19:46:01.008208Z_26692ab3-ba90-4129-9929-2ad8f29f0acb  osd.1        
2023-09-03T07:42:42.451722Z_d0e7268f-0da6-4d59-b706-35c49ee8617b  osd.2        
2023-12-13T22:17:16.706091Z_8dbfc488-1309-4e9a-b4b9-c1eadeb3016e  osd.0    * 

ceph osd tree

k8s70132:~$ ceph osd tree
ID   CLASS  WEIGHT    TYPE NAME              STATUS  REWEIGHT  PRI-AFF
 -1         42.02631  root default                                    
 -3         10.91606      host k8sceph70201                           
  0    hdd   3.63869          osd.0            down         0  1.00000
  3    hdd   3.63869          osd.3              up   1.00000  1.00000
  6    hdd   3.63869          osd.6              up   1.00000  1.00000
 -5          2.72910      host k8sceph70202                           
  1    hdd   0.90970          osd.1              up   1.00000  1.00000
  4    hdd   0.90970          osd.4              up   1.00000  1.00000
  7    hdd   0.90970          osd.7              up   1.00000  1.00000
 -7         10.91606      host k8sceph70203                           
  2    hdd   3.63869          osd.2              up   1.00000  1.00000
  5    hdd   3.63869          osd.5              up   1.00000  1.00000
  8    hdd   3.63869          osd.8              up   1.00000  1.00000

到k8sceph70201查看

sudo systmectl |grep ceph-osd

k8sceph70201:~$ sudo systemctl |grep ceph-osd
  var-lib-ceph-osd-ceph\x2d0.mount                                                               loaded active mounted   /var/lib/ceph/osd/ceph-0                                                      
  var-lib-ceph-osd-ceph\x2d3.mount                                                               loaded active mounted   /var/lib/ceph/osd/ceph-3                                                      
  var-lib-ceph-osd-ceph\x2d6.mount                                                               loaded active mounted   /var/lib/ceph/osd/ceph-6                                                      
● ceph-osd@0.service                                                                             loaded failed failed    Ceph object storage daemon osd.0                                              
  ceph-osd@3.service                                                                             loaded active running   Ceph object storage daemon osd.3                                              
  ceph-osd@6.service                                                                             loaded active running   Ceph object storage daemon osd.6                                              
  ceph-osd.target                                                                                loaded active active    ceph target allowing to start/stop all ceph-osd@.service instances 

删除故障硬盘

//在k8sceph70201上关停服务
sudo systemctl stop ceph-osd@0.service

//在ceph客户端操作如下
 ceph osd down osd.0
//回显内容 osd.0 is already down. 
ceph osd out osd.0
//回显内容 osd.0 is already out. 
ceph osd crush remove osd.0
//回显内容 removed item id 0 name 'osd.0' from crush map
ceph osd rm 0
//回显内容 removed osd.0
ceph auth del osd.0
//回显内容 updated
ceph osd tree
//回显内容 
ID   CLASS  WEIGHT    TYPE NAME              STATUS  REWEIGHT  PRI-AFF
 -1         38.38762  root default                                    
 -3          7.27737      host k8sceph70201                           
  3    hdd   3.63869          osd.3              up   1.00000  1.00000
  6    hdd   3.63869          osd.6              up   1.00000  1.00000
 -5          2.72910      host k8sceph70202                           
  1    hdd   0.90970          osd.1              up   1.00000  1.00000
  4    hdd   0.90970          osd.4              up   1.00000  1.00000
  7    hdd   0.90970          osd.7              up   1.00000  1.00000
 -7         10.91606      host k8sceph70203                           
  2    hdd   3.63869          osd.2              up   1.00000  1.00000
  5    hdd   3.63869          osd.5              up   1.00000  1.00000
  8    hdd   3.63869          osd.8            down   1.00000  1.00000

添加硬盘

ceph-deploy disk zap  /dev/sdb 
ceph-deploy --overwrite-conf  osd create  /dev/sdb
相关推荐
petaexpress13 小时前
5种常见的k8s云原生数据管理方案详解
云原生·kubernetes·k8s云原生
颜淡慕潇15 小时前
【K8S系列】深入解析 Kubernetes 中的 Deployment
后端·云原生·容器·kubernetes
petaexpress1 天前
k8s微服务架构就是云原生吗?两者是什么关系
微服务·云原生·架构·kubernetes·k8s
wenyue11211 天前
Enhancing K8s Gateway API with Easegress Without Changing a Single Line of Code
容器·kubernetes·gateway
sxy1993sxy20182 天前
k8s rainbond centos7/win10 -20241124
云原生·容器·kubernetes
Python私教2 天前
Python 使用 Token 认证方案连接 Kubernetes (k8s) 的详细过程
开发语言·python·kubernetes
风霜不见闲沉月2 天前
k8s网络服务
linux·容器·kubernetes
夏沫的梦3 天前
kubernetes起源与介绍
kubernetes
胡八一3 天前
解决k8s拉取私有镜像401 Unauthorized 问题
云原生·容器·kubernetes
筑梦之路3 天前
银河麒麟v10 x86架构二进制方式kubeadm+docker+cri-docker搭建k8s集群(证书有效期100年) —— 筑梦之路
docker·容器·kubernetes