备份恢复
备份恢复的总体流程
-
elasticsearch.yaml中配置的备份路径,拷贝到新的机器上。
bash# 备份路径 path.repo: ["/data/elasticsearch/snapshots/snapshot01","/data/elasticsearch/snapshots/snapshot02"]
-
将备份路径通过nfs挂载到新的集群实例的每个节点上
-
创建snapshot repo
-
创建snapshot repo
bashesname=elastic espassword=xxxxxx curl -X PUT -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @snapshot-repo.json "http://localhost:9200/_snapshot/recover_repo"
-
snapshot-repo.json
如下所示,
location
为备份路径bash{ "type": "fs", "settings": { "compress": true, "location": "/data/els_backup_new" } }
-
-
从snapshot repo恢复数据
bashesname=elastic espassword=xxxxxx # close duplicated index # curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword} "http://localhost:9200/.security-7/_close" # snapshot的名字 snapshot=snapshot_2024-11-05_01-00 curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @recover-settings.json "http://localhost:9200/_snapshot/recover_repo/$snapshot/_restore?wait_for_completion=true" | python -m json.tool
其中
recover-settings.json
:bash{ "indices": "*,-.*" }
snapshot的名称可以通过以下接口获取:
bashesname=elastic espassword=xxxxx curl -X GET -H 'Content-Type: application/json' --user ${esname}:${espassword} "http://localhost:9200/_snapshot/recover_repo"
-
查看迁移进度
bashesname=elastic espassword=xxxx curl -X GET --user ${esname}:${espassword} "http://localhost:9200/_recovery/"
响应输出格式为json,可以通过
grep -v '100.0%'
来查看未恢复完成的索引。
备份恢复成功输出
bash
[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh
{"acknowledged":true,"shards_acknowledged":false,"indices":{}} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27914 100 27914 0 0 61 0 0:07:37 0:07:30 0:00:07 7721
{
"snapshot": {
"indices": [
// 恢复的索引
],
"shards": {
"failed": 0,
"successful": 1714,
"total": 1714
},
"snapshot": "snapshot_2024-11-05_01-00"
}
}
备份恢复的常见问题:
索引冲突
bash
[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 743 100 743 0 0 354 0 0:00:02 0:00:02 --:--:-- 354
{
"error": {
"reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
"root_cause": [
{
"reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
"type": "snapshot_restore_exception"
}
],
"type": "snapshot_restore_exception"
},
"status": 500
}
出现这种情况,需要把索引关闭:
bash
esname=elastic
espassword=xxx
# 关闭索引
curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword} "http://localhost:9200/.security-7/_close"
其中.security-7
是索引的名称
bash
4.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-.logs-deprecation.elasticsearch-default-2024.11.06-000001] is empty or not defined
at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
[2024-11-06T17:00:44,102][ERROR][o.e.x.i.IndexLifecycleRunner] [node-1] policy [ilm-history-ilm-policy] for index [.ds-ilm-history-5-2024.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-ilm-history-5-2024.11.06-000001] is empty or not defined
at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]