ceph数据迁移数据迁移与bacula配置调整优化实战

#作者:猎人

文章目录

1.1ceph数据迁移&&bacula配置调整

为了规避因单个OSD利用率达90%而导致整个集群错误无法访问,ceph集群现已在原ceph块设备的基础上建立了cephfs文件系统用于存储数据,需要把原备份到块设备上的设备迁移到新建的cephfs上,迁移完毕后把ceph rbd回收,后续所有的备份数据直接写到cephfs。

  • rbd分配的指定空间满了之后需要新创建,cephpfs的空间可以横向扩容,也可以避免单个OSD引起集群故障

1.2在备份服务器的ceph-client上mount cephfs文件系统

  • T机房(1xx.xxx.xxx.x3):ceph-fuse -m 1x.xxx.xx.x9:6789 -r /tt_bacula /cephfs/
  • Y机房(1x.xxx.xx.x1):ceph-fuse -m 1x.xxx.xx.51:6789 /cephfs/

1.2.1迁移数据

  • T:/data1/d0*/下的数全部数据迁移到/cephfs/tt/backup_d0*/
  • Y:/data1/d0*/下的数全部数据迁移到/cephfs/yz/backup_d0*/
    特别注意:在做数据恢复的时候需要把数据还原到原/data1/backup_disk0*/下才能恢复数据

1.2.2 调整bacula-sd配置

1.2.2.1T机房

bacula-sd.conf中添加7个Device F8/9/10/11/12/13/14分别指向/cephfs/tt/下的7个目录

bacula-sd.conf新加配置如下:

复制代码
Autochanger {
  Name = F8
  Device = F8-Dev1, F8-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F-Dev1
  Media Type = File8
  Archive Device = /cephfs/tt/backup_d3/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F8-Dev2
  Media Type = File8
  Archive Device = /cephfs/tt/backup_d3/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F9
  Device = F9-Dev1, F9-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F9-Dev1
  Media Type = File9
  Archive Device = /cephfs/tt/backup_d1/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F9-Dev2
  Media Type = File9
  Archive Device = /cephfs/tt/backup_d1/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F10
  Device = F10-Dev1, F10-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F10-Dev1
  Media Type = File10
  Archive Device = /cephfs/tt/backup_d2/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F10-Dev2
  Media Type = File10
  Archive Device = /cephfs/tt/backup_d2/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F11
  Device = F11-Dev1, F11-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F11-Dev1
  Media Type = File11
  Archive Device = /cephfs/tt/backup_d4/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F11-Dev2
  Media Type = File11
  Archive Device = /cephfs/tt/backup_d4/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F12
  Device = F12-Dev1, F12-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F12-Dev1
  Media Type = File12
  Archive Device = /cephfs/tt/backup_d5/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F12-Dev2
  Media Type = File12
  Archive Device = /cephfs/tt/backup_d5/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F13
  Device = F13-Dev1, F13-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F13-Dev1
  Media Type = File13
  Archive Device = /cephfs/tt/backup_d6/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F13-Dev2
  Media Type = File13
  Archive Device = /cephfs/tt/backup_d6/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}

Autochanger {
  Name = F14
  Device = F14-Dev1, F14-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}
Device {
  Name = F14-Dev1
  Media Type = File14
  Archive Device = /cephfs/tt/backup_d7/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
Device {
  Name = F14-Dev2
  Media Type = File14
  Archive Device = /cephfs/tt/backup_d7/
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
1.2.2.2Y机房

bacula-sd.conf中添加1个Device F3分别指向/cephfs/yz/目录

bacula-sd.conf新加配置如下:

复制代码
Autochanger {
  Name = F3
  Device = F3-Dev1, F3-Dev2
  Changer Command = ""
  Changer Device = /dev/null
}

Device {
  Name = F3-Dev1
  Media Type = File3
  Archive Device = /cephfs/yz
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 5
}

Device {
  Name = F3-Dev2
  Media Type = File3
  Archive Device = /cephfs/yz
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 5
}
1.2.2.3调整bacula-dir配置
  1. 把前面新添加的7个SD加到bacula-dir.conf中

  2. T机房:把原来Job中的 Storage 都换成对应新建的Storage,例如原Storage = tt-7(/data1/backup_d7/)替换成现在Storage = tt-14(/cephfs/tt/backup_d7/)

  3. Y机房:把原来Job中的 Storage 都换成对应新建的Storage = yz-3(亦庄把yz-1和yz-2在此做一个合并,合并到yz-3)
    bacula-dir.conf新加配置如下:

    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-8
    SDPort = 9103
    Address = xxxxxxxx
    Device = F8
    Password = "TFDk"
    Media Type = File8
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-9
    SDPort = 9103
    Address = xxxxxxxx
    Device = F9
    Password = "TFDk"
    Media Type = File9
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-10
    SDPort = 9103
    Address = xxxxxxxx
    Device = F10
    Password = "TFDk"
    Media Type = File10
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-11
    SDPort = 9103
    Address = xxxxxxxx
    Device = F11
    Password = "TFDk"
    Media Type = File11
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-12
    SDPort = 9103
    Address = xxxxxxxx
    Device = F12
    Password = "TFDk"
    Media Type = File12
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-13
    SDPort = 9103
    Address = xxxxxxxx
    Device = F13
    Password = "TFDk"
    Media Type = File13
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = tt-14
    SDPort = 9103
    Address = xxxxxxxx
    Device = F14
    Password = "TFDk"
    Media Type = File14
    }
    Storage {
    Maximum Concurrent Jobs = 10
    Name = yz-3
    SDPort = 9103
    Address = xxx
    Device = F3
    Password = "TFDk"
    Media Type = File3
    }

相关推荐
10年前端老司机3 小时前
React无限级菜单:一个项目带你突破技术瓶颈
前端·javascript·react.js
阿芯爱编程7 小时前
2025前端面试题
前端·面试
前端小趴菜058 小时前
React - createPortal
前端·vue.js·react.js
晓13138 小时前
JavaScript加强篇——第四章 日期对象与DOM节点(基础)
开发语言·前端·javascript
菜包eo9 小时前
如何设置直播间的观看门槛,让直播间安全有效地运行?
前端·安全·音视频
烛阴9 小时前
JavaScript函数参数完全指南:从基础到高级技巧,一网打尽!
前端·javascript
chao_78910 小时前
frame 与新窗口切换操作【selenium 】
前端·javascript·css·selenium·测试工具·自动化·html
天蓝色的鱼鱼10 小时前
从零实现浏览器摄像头控制与视频录制:基于原生 JavaScript 的完整指南
前端·javascript
三原11 小时前
7000块帮朋友做了2个小程序加一个后台管理系统,值不值?
前端·vue.js·微信小程序
popoxf11 小时前
在新版本的微信开发者工具中使用npm包
前端·npm·node.js