elasticsearch备份恢复

备份恢复

备份恢复的总体流程

  1. elasticsearch.yaml中配置的备份路径,拷贝到新的机器上。

    bash 复制代码
    # 备份路径
    path.repo: ["/data/elasticsearch/snapshots/snapshot01","/data/elasticsearch/snapshots/snapshot02"]
  2. 将备份路径通过nfs挂载到新的集群实例的每个节点上

  3. 创建snapshot repo

    • 创建snapshot repo

      bash 复制代码
      esname=elastic
      espassword=xxxxxx
      curl -X PUT -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @snapshot-repo.json  "http://localhost:9200/_snapshot/recover_repo" 
    • snapshot-repo.json

      如下所示, location为备份路径

      bash 复制代码
      {
          "type": "fs",
          "settings": {
              "compress": true,
              "location": "/data/els_backup_new"
          }
      }
  4. 从snapshot repo恢复数据

    bash 复制代码
    esname=elastic
    espassword=xxxxxx
    
    # close duplicated index
    # curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/.security-7/_close"
    
    # snapshot的名字
    snapshot=snapshot_2024-11-05_01-00
    
    curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword} -d @recover-settings.json  "http://localhost:9200/_snapshot/recover_repo/$snapshot/_restore?wait_for_completion=true" | python -m json.tool

    其中recover-settings.json:

    bash 复制代码
    {
    "indices": "*,-.*"
    }

    snapshot的名称可以通过以下接口获取:

    bash 复制代码
    esname=elastic
    espassword=xxxxx
    curl -X GET -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/_snapshot/recover_repo" 
  5. 查看迁移进度

    bash 复制代码
    esname=elastic
    espassword=xxxx
    
    curl -X GET --user ${esname}:${espassword}   "http://localhost:9200/_recovery/" 

    响应输出格式为json,可以通过grep -v '100.0%'来查看未恢复完成的索引。

备份恢复成功输出

bash 复制代码
[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh 
{"acknowledged":true,"shards_acknowledged":false,"indices":{}}  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27914  100 27914    0     0     61      0  0:07:37  0:07:30  0:00:07  7721
{
    "snapshot": {
        "indices": [
            // 恢复的索引
        ],
        "shards": {
            "failed": 0,
            "successful": 1714,
            "total": 1714
        },
        "snapshot": "snapshot_2024-11-05_01-00"
    }
}

备份恢复的常见问题:

索引冲突

bash 复制代码
[portal@es-node1 tmp]$ bash recover-es-from-snapshot.sh 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   743  100   743    0     0    354      0  0:00:02  0:00:02 --:--:--   354
{
    "error": {
        "reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
        "root_cause": [
            {
                "reason": "[recover_repo:snapshot_2024-11-05_01-00/l_5Y6dxoSvGAhAHH5zjwGQ] cannot restore index [.security-7] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name",
                "type": "snapshot_restore_exception"
            }
        ],
        "type": "snapshot_restore_exception"
    },
    "status": 500
}

出现这种情况,需要把索引关闭:

bash 复制代码
esname=elastic
espassword=xxx
# 关闭索引
curl -X POST -H 'Content-Type: application/json' --user ${esname}:${espassword}  "http://localhost:9200/.security-7/_close"

其中.security-7是索引的名称

bash 复制代码
4.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-.logs-deprecation.elasticsearch-default-2024.11.06-000001] is empty or not defined
	at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
[2024-11-06T17:00:44,102][ERROR][o.e.x.i.IndexLifecycleRunner] [node-1] policy [ilm-history-ilm-policy] for index [.ds-ilm-history-5-2024.11.06-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [.ds-ilm-history-5-2024.11.06-000001] is empty or not defined
	at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:89) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:226) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:418) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:349) [x-pack-ilm-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:186) [x-pack-core-7.17.6.jar:7.17.6]
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:220) [x-pack-core-7.17.6.jar:7.17.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
相关推荐
1.01^10001 小时前
[2-02-02].第03节:环境搭建 - Win10搭建ES集群环境
elasticsearch
无级程序员1 小时前
大数据平台之ranger与ldap集成,同步用户和组
大数据·hadoop
lifallen2 小时前
Paimon 原子提交实现
java·大数据·数据结构·数据库·后端·算法
TDengine (老段)3 小时前
TDengine 数据库建模最佳实践
大数据·数据库·物联网·时序数据库·tdengine·涛思数据
张先shen3 小时前
Elasticsearch RESTful API入门:全文搜索实战(Java版)
java·大数据·elasticsearch·搜索引擎·全文检索·restful
Elastic 中国社区官方博客3 小时前
Elasticsearch 字符串包含子字符串:高级查询技巧
大数据·数据库·elasticsearch·搜索引擎·全文检索·lucene
张先shen4 小时前
Elasticsearch RESTful API入门:全文搜索实战
java·大数据·elasticsearch·搜索引擎·全文检索·restful
天翼云开发者社区4 小时前
Doris-HDFS LOAD常见问题汇总(二)
大数据·doris
简婷187019987754 小时前
源网荷储 + 零碳园区:一场关于能源与未来的双向奔赴
大数据·人工智能·能源
2201_756776775 小时前
网络安全初级
大数据·elasticsearch·搜索引擎