ceph数据迁移数据迁移与bacula配置调整优化实战

斯普信专业组2025-04-07 8:12

#作者：猎人

文章目录

1.1ceph数据迁移&&bacula配置调整

为了规避因单个OSD利用率达90%而导致整个集群错误无法访问，ceph集群现已在原ceph块设备的基础上建立了cephfs文件系统用于存储数据，需要把原备份到块设备上的设备迁移到新建的cephfs上，迁移完毕后把ceph rbd回收，后续所有的备份数据直接写到cephfs。

  • rbd分配的指定空间满了之后需要新创建，cephpfs的空间可以横向扩容，也可以避免单个OSD引起集群故障

1.2在备份服务器的ceph-client上mount cephfs文件系统

  • T机房（1xx.xxx.xxx.x3）：ceph-fuse -m 1x.xxx.xx.x9:6789 -r /tt_bacula /cephfs/
  • Y机房（1x.xxx.xx.x1）：ceph-fuse -m 1x.xxx.xx.51:6789 /cephfs/

1.2.1迁移数据

  • T：/data1/d0*/下的数全部数据迁移到/cephfs/tt/backup_d0*/
  • Y：/data1/d0*/下的数全部数据迁移到/cephfs/yz/backup_d0*/
    特别注意：在做数据恢复的时候需要把数据还原到原/data1/backup_disk0*/下才能恢复数据

1.2.2 调整bacula-sd配置

1.2.2.1T机房

bacula-sd.conf中添加7个Device F8/9/10/11/12/13/14分别指向/cephfs/tt/下的7个目录

bacula-sd.conf新加配置如下：

复制代码 
Autochanger {
  Name = F8
  Device = F8-Dev1, F8-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F-Dev1
  Media Type = File8
  Archive Device = /cephfs/tt/backup_d3/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F8-Dev2
  Media Type = File8
  Archive Device = /cephfs/tt/backup_d3/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F9
  Device = F9-Dev1, F9-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F9-Dev1
  Media Type = File9
  Archive Device = /cephfs/tt/backup_d1/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F9-Dev2
  Media Type = File9
  Archive Device = /cephfs/tt/backup_d1/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F10
  Device = F10-Dev1, F10-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F10-Dev1
  Media Type = File10
  Archive Device = /cephfs/tt/backup_d2/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F10-Dev2
  Media Type = File10
  Archive Device = /cephfs/tt/backup_d2/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F11
  Device = F11-Dev1, F11-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F11-Dev1
  Media Type = File11
  Archive Device = /cephfs/tt/backup_d4/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F11-Dev2
  Media Type = File11
  Archive Device = /cephfs/tt/backup_d4/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F12
  Device = F12-Dev1, F12-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F12-Dev1
  Media Type = File12
  Archive Device = /cephfs/tt/backup_d5/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F12-Dev2
  Media Type = File12
  Archive Device = /cephfs/tt/backup_d5/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F13
  Device = F13-Dev1, F13-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F13-Dev1
  Media Type = File13
  Archive Device = /cephfs/tt/backup_d6/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F13-Dev2
  Media Type = File13
  Archive Device = /cephfs/tt/backup_d6/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F14
  Device = F14-Dev1, F14-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F14-Dev1
  Media Type = File14
  Archive Device = /cephfs/tt/backup_d7/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F14-Dev2
  Media Type = File14
  Archive Device = /cephfs/tt/backup_d7/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
1.2.2.2Y机房

bacula-sd.conf中添加1个Device F3分别指向/cephfs/yz/目录

bacula-sd.conf新加配置如下：

复制代码 
Autochanger {
  Name = F3
  Device = F3-Dev1, F3-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}

Device {
  Name = F3-Dev1
  Media Type = File3
  Archive Device = /cephfs/yz
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 5
}

Device {
  Name = F3-Dev2
  Media Type = File3
  Archive Device = /cephfs/yz
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 5
}
1.2.2.3调整bacula-dir配置

  1. 把前面新添加的7个SD加到bacula-dir.conf中

  2. T机房：把原来Job中的 Storage 都换成对应新建的Storage，例如原Storage = tt-7（/data1/backup_d7/）替换成现在Storage = tt-14（/cephfs/tt/backup_d7/）

  3. Y机房：把原来Job中的 Storage 都换成对应新建的Storage = yz-3（亦庄把yz-1和yz-2在此做一个合并，合并到yz-3）
    bacula-dir.conf新加配置如下：

    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-8
    SDPort = 9103
    Address = xxxxxxxx
    Device = F8
    Password = "TFDk"
    Media Type = File8
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-9
    SDPort = 9103
    Address = xxxxxxxx
    Device = F9
    Password = "TFDk"
    Media Type = File9
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-10
    SDPort = 9103
    Address = xxxxxxxx
    Device = F10
    Password = "TFDk"
    Media Type = File10
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-11
    SDPort = 9103
    Address = xxxxxxxx
    Device = F11
    Password = "TFDk"
    Media Type = File11
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-12
    SDPort = 9103
    Address = xxxxxxxx
    Device = F12
    Password = "TFDk"
    Media Type = File12
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-13
    SDPort = 9103
    Address = xxxxxxxx
    Device = F13
    Password = "TFDk"
    Media Type = File13
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-14
    SDPort = 9103
    Address = xxxxxxxx
    Device = F14
    Password = "TFDk"
    Media Type = File14
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = yz-3
    SDPort = 9103
    Address = xxx
    Device = F3
    Password = "TFDk"
    Media Type = File3
    }

相关推荐
胡斌附体14 分钟前
vue添加loading后修复页面渲染问题
前端·javascript·vue.js·渲染·v-if·异步加载
酷爱码1 小时前
css中的 vertical-align与line-height作用详解
前端·css
沐土Arvin1 小时前
深入理解 requestIdleCallback：浏览器空闲时段的性能优化利器
开发语言·前端·javascript·设计模式·html
专注VB编程开发20年1 小时前
VB.NET关于接口实现与简化设计的分析，封装其他类
java·前端·数据库
小妖6662 小时前
css 中 content: “\e6d0“ 怎么变成图标的？
前端·css
L耀早睡2 小时前
mapreduce打包运行
大数据·前端·spark·mapreduce
HouGISer2 小时前
副业小程序YUERGS，从开发到变现
前端·小程序
outstanding木槿3 小时前
react中安装依赖时的问题 【集合】
前端·javascript·react.js·node.js
霸王蟹3 小时前
React中useState中更新是同步的还是异步的?
前端·javascript·笔记·学习·react.js·前端框架
霸王蟹3 小时前
React Hooks 必须在组件最顶层调用的原因解析
前端·javascript·笔记·学习·react.js
热门推荐
01KGG转MP3工具|非KGM文件|解密音频02YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】03从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑04【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！05DeepSeek各版本说明与优缺点分析06Coze扣子平台完整体验和实践（附国内和国际版对比）07Ubuntu24.04安装中文输入法08YOLOv5改进 | 添加CA注意力机制 + 增加预测层 + 更换损失函数之GIoU09苍穹外卖面试总结10第一届 “帕鲁杯“ writeup