docker环境安装mongoDB实现平滑迁移实战

docker环境安装mongoDB实现平滑迁移实战

• 一、备份原始数据（从别的服务器备份到当前服务器）
• 二、数据迁移
• 三、迁移过程日志打印
• 四、验证迁移数据准确性

一、备份原始数据（从别的服务器备份到当前服务器）

使用mongodump工具对原始mongoDB进行全量备份，在备份期间，确保没有新的数据写入，否则这部分数据迁移不到。

docker exec -it 12b9d0ccf6c5 mongodump -h xxx.xxx.xx.xx -u admin -p 123456 -o /data

二、数据迁移

使用mongorestore 工具 将备份的数据文件下载入新的mongDB容器即可：

docker exec -it 502a471acc14 mongorestore -h localhost -u madmin -p 123456 /data

三、迁移过程日志打印

2024-10-16T09:36:03.599+0000    WARNING: On some systems, a password provided directly using --password may be visible to system status programs such as `ps` that may be invoked by other users. Consider omitting the password to provide it via stdin, or using the --config option to specify a configuration file with the password.
2024-10-16T09:36:03.610+0000    preparing collections to restore from
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/.bash_history", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with subdirectory "db/.mongodb", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/WiredTiger", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/WiredTiger.lock", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/WiredTiger.turtle", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/WiredTiger.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/WiredTigerHS.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/_mdb_catalog.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/collection-0--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/collection-2--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/collection-4--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/collection-7--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with subdirectory "db/diagnostic.data", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-1--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-3--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-5--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-6--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-8--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/index-9--833647413404653651.wt", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with subdirectory "db/journal", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/mongod.lock", skipping...
2024-10-16T09:36:03.610+0000    don't know what to do with file "/data/db/sizeStorer.wt", skipping...
2024-10-16T09:36:03.612+0000    reading metadata for admin.crawler_data from /data/admin/crawler_data.metadata.json
2024-10-16T09:36:03.612+0000    reading metadata for admin.crawler_data_title from /data/admin/crawler_data_title.metadata.json
2024-10-16T09:36:03.612+0000    reading metadata for admin.sequence from /data/admin/sequence.metadata.json
2024-10-16T09:36:03.643+0000    restoring admin.crawler_data from /data/admin/crawler_data.bson
2024-10-16T09:36:03.649+0000    restoring admin.crawler_data_title from /data/admin/crawler_data_title.bson
2024-10-16T09:36:03.659+0000    restoring db.storage from /data/db/storage.bson
2024-10-16T09:36:03.660+0000    finished restoring admin.crawler_data_title (76 documents, 0 failures)
2024-10-16T09:36:03.669+0000    finished restoring db.storage (1 document, 0 failures)
2024-10-16T09:36:03.669+0000    restoring admin.sequence from /data/admin/sequence.bson
2024-10-16T09:36:03.680+0000    finished restoring admin.sequence (1 document, 0 failures)
2024-10-16T09:36:06.610+0000    [........................]  admin.crawler_data  176MB/12.1GB  (1.4%)
2024-10-16T09:36:09.611+0000    [........................]  admin.crawler_data  432MB/12.1GB  (3.5%)
2024-10-16T09:36:12.610+0000    [#.......................]  admin.crawler_data  690MB/12.1GB  (5.6%)
2024-10-16T09:36:15.610+0000    [#.......................]  admin.crawler_data  983MB/12.1GB  (7.9%)
2024-10-16T09:36:18.610+0000    [##......................]  admin.crawler_data  1.25GB/12.1GB  (10.3%)
2024-10-16T09:36:21.610+0000    [###.....................]  admin.crawler_data  1.53GB/12.1GB  (12.6%)
2024-10-16T09:36:24.610+0000    [###.....................]  admin.crawler_data  1.70GB/12.1GB  (14.0%)
2024-10-16T09:36:27.611+0000    [###.....................]  admin.crawler_data  1.89GB/12.1GB  (15.6%)
2024-10-16T09:36:30.610+0000    [####....................]  admin.crawler_data  2.13GB/12.1GB  (17.6%)
2024-10-16T09:36:33.610+0000    [####....................]  admin.crawler_data  2.44GB/12.1GB  (20.1%)
2024-10-16T09:36:36.610+0000    [#####...................]  admin.crawler_data  2.65GB/12.1GB  (21.9%)
2024-10-16T09:36:39.610+0000    [#####...................]  admin.crawler_data  2.92GB/12.1GB  (24.1%)
2024-10-16T09:36:42.610+0000    [######..................]  admin.crawler_data  3.17GB/12.1GB  (26.1%)
2024-10-16T09:36:45.610+0000    [######..................]  admin.crawler_data  3.42GB/12.1GB  (28.2%)
2024-10-16T09:36:48.610+0000    [#######.................]  admin.crawler_data  3.70GB/12.1GB  (30.5%)
2024-10-16T09:36:51.610+0000    [#######.................]  admin.crawler_data  3.96GB/12.1GB  (32.6%)
2024-10-16T09:36:54.610+0000    [########................]  admin.crawler_data  4.20GB/12.1GB  (34.6%)
2024-10-16T09:36:57.610+0000    [########................]  admin.crawler_data  4.46GB/12.1GB  (36.8%)
2024-10-16T09:37:00.610+0000    [#########...............]  admin.crawler_data  4.68GB/12.1GB  (38.6%)
2024-10-16T09:37:03.610+0000    [#########...............]  admin.crawler_data  4.92GB/12.1GB  (40.5%)
2024-10-16T09:37:06.610+0000    [##########..............]  admin.crawler_data  5.17GB/12.1GB  (42.7%)
2024-10-16T09:37:09.610+0000    [##########..............]  admin.crawler_data  5.38GB/12.1GB  (44.3%)
2024-10-16T09:37:12.610+0000    [###########.............]  admin.crawler_data  5.62GB/12.1GB  (46.4%)
2024-10-16T09:37:15.610+0000    [###########.............]  admin.crawler_data  5.86GB/12.1GB  (48.3%)
2024-10-16T09:37:18.611+0000    [############............]  admin.crawler_data  6.07GB/12.1GB  (50.1%)
2024-10-16T09:37:21.610+0000    [############............]  admin.crawler_data  6.32GB/12.1GB  (52.1%)
2024-10-16T09:37:24.610+0000    [############............]  admin.crawler_data  6.49GB/12.1GB  (53.5%)
2024-10-16T09:37:27.610+0000    [#############...........]  admin.crawler_data  6.66GB/12.1GB  (54.9%)
2024-10-16T09:37:30.610+0000    [#############...........]  admin.crawler_data  6.82GB/12.1GB  (56.2%)
2024-10-16T09:37:33.610+0000    [#############...........]  admin.crawler_data  6.95GB/12.1GB  (57.3%)
2024-10-16T09:37:36.611+0000    [##############..........]  admin.crawler_data  7.10GB/12.1GB  (58.5%)
2024-10-16T09:37:39.610+0000    [##############..........]  admin.crawler_data  7.25GB/12.1GB  (59.8%)
2024-10-16T09:37:42.610+0000    [##############..........]  admin.crawler_data  7.37GB/12.1GB  (60.7%)
2024-10-16T09:37:45.610+0000    [##############..........]  admin.crawler_data  7.55GB/12.1GB  (62.2%)
2024-10-16T09:37:48.610+0000    [###############.........]  admin.crawler_data  7.82GB/12.1GB  (64.5%)
2024-10-16T09:37:51.610+0000    [###############.........]  admin.crawler_data  7.95GB/12.1GB  (65.5%)
2024-10-16T09:37:54.610+0000    [################........]  admin.crawler_data  8.16GB/12.1GB  (67.3%)
2024-10-16T09:37:57.610+0000    [################........]  admin.crawler_data  8.44GB/12.1GB  (69.6%)
2024-10-16T09:38:00.610+0000    [#################.......]  admin.crawler_data  8.60GB/12.1GB  (70.9%)
2024-10-16T09:38:03.610+0000    [#################.......]  admin.crawler_data  8.81GB/12.1GB  (72.7%)
2024-10-16T09:38:06.610+0000    [#################.......]  admin.crawler_data  9.01GB/12.1GB  (74.3%)
2024-10-16T09:38:09.610+0000    [##################......]  admin.crawler_data  9.23GB/12.1GB  (76.1%)
2024-10-16T09:38:12.610+0000    [##################......]  admin.crawler_data  9.44GB/12.1GB  (77.8%)
2024-10-16T09:38:15.610+0000    [###################.....]  admin.crawler_data  9.65GB/12.1GB  (79.6%)
2024-10-16T09:38:18.610+0000    [###################.....]  admin.crawler_data  9.79GB/12.1GB  (80.7%)
2024-10-16T09:38:21.610+0000    [###################.....]  admin.crawler_data  9.91GB/12.1GB  (81.7%)
2024-10-16T09:38:24.610+0000    [###################.....]  admin.crawler_data  10.0GB/12.1GB  (82.8%)
2024-10-16T09:38:27.610+0000    [####################....]  admin.crawler_data  10.2GB/12.1GB  (84.4%)
2024-10-16T09:38:30.610+0000    [####################....]  admin.crawler_data  10.4GB/12.1GB  (85.7%)
2024-10-16T09:38:33.610+0000    [####################....]  admin.crawler_data  10.5GB/12.1GB  (86.7%)
2024-10-16T09:38:36.610+0000    [#####################...]  admin.crawler_data  10.7GB/12.1GB  (88.2%)
2024-10-16T09:38:39.610+0000    [#####################...]  admin.crawler_data  10.8GB/12.1GB  (89.5%)
2024-10-16T09:38:42.610+0000    [#####################...]  admin.crawler_data  11.0GB/12.1GB  (90.5%)
2024-10-16T09:38:45.610+0000    [######################..]  admin.crawler_data  11.1GB/12.1GB  (91.7%)
2024-10-16T09:38:48.610+0000    [######################..]  admin.crawler_data  11.3GB/12.1GB  (93.0%)
2024-10-16T09:38:51.610+0000    [######################..]  admin.crawler_data  11.4GB/12.1GB  (94.0%)
2024-10-16T09:38:54.610+0000    [######################..]  admin.crawler_data  11.5GB/12.1GB  (94.9%)
2024-10-16T09:38:57.610+0000    [#######################.]  admin.crawler_data  11.6GB/12.1GB  (95.9%)
2024-10-16T09:39:00.610+0000    [#######################.]  admin.crawler_data  11.8GB/12.1GB  (96.9%)
2024-10-16T09:39:03.611+0000    [#######################.]  admin.crawler_data  11.9GB/12.1GB  (97.8%)
2024-10-16T09:39:06.610+0000    [#######################.]  admin.crawler_data  12.1GB/12.1GB  (99.4%)
2024-10-16T09:39:08.107+0000    [########################]  admin.crawler_data  12.1GB/12.1GB  (100.0%)
2024-10-16T09:39:08.107+0000    finished restoring admin.crawler_data (12791925 documents, 0 failures)
2024-10-16T09:39:08.107+0000    restoring users from /data/admin/system.users.bson
2024-10-16T09:39:08.143+0000    no indexes to restore for collection admin.crawler_data
2024-10-16T09:39:08.143+0000    no indexes to restore for collection admin.crawler_data_title
2024-10-16T09:39:08.143+0000    no indexes to restore for collection admin.sequence
2024-10-16T09:39:08.144+0000    12792003 document(s) restored successfully. 0 document(s) failed to restore.

到这里数据就迁移到新的容器成功了。

四、验证迁移数据准确性

迁移后验证数据一致性，在两个新旧mongDB环境分别查询集合下的数据量统计，看是否一致，如果一致，则迁移成功。

db.crawler_data.countDocuments();
db.crawler_data_title.countDocuments();