python elasticsearch_dsl PIT Point in time API 查询

小兜全糖(xdqt)2024-12-21 16:10

默认情况下，搜索请求针对目标索引的最新可见数据（称为时间点）执行。elasticsearchpit（时间点）是一种轻量级视图，可以查看数据在启动时的状态。在某些情况下，最好使用同一时间点执行多个搜索请求。例如，如果在search_after请求之间发生刷新，那么这些请求的结果可能不一致，因为在搜索之间发生的更改仅在最近的时间点可见。

python 复制代码

from elasticsearch_dsl import Search
from elasticsearch_dsl import connections
from elasticsearch_dsl import Q as esQ
conn = connections.create_connection(hosts=['192.168.214.133'],port=31920,http_auth="elastic:test")

response = conn.open_point_in_time(index="test",keep_alive="1m")
# 获取到id
pit_id = response['id']
search = Search(using=conn)
search.query=esQ("match_all")
search = search.extra(pit={"id": pit_id, "keep_alive": "1m"})
search = search.sort('-_shard_doc')
search = search.extra(size=1)

def loop_value(search):
    try:
        print(search.to_dict())
        response = search.execute()
        value = response['hits']['hits'][-1]['sort']
        print(response['hits']['hits'])
        if value[0]>0:
            search = search.extra(search_after=value)
            loop_value(search)
    except Exception as e:
        print(str(e))
loop_value(search)
conn.close_point_in_time(body={"id": pit_id})

对应的dsl

bash 复制代码

POST /test/_pit?keep_alive=10m

DELETE /_pit
{
  "id":"gLuKBAEFZWxsaXMWVUpRaEYxY2VTZVd2eFlYZWZLbEU3QQAWOGxDOW1rQzZTMTJJVVJubDJxM196dwAAAAAAAAAN8RZpUTdEYlBaclJ0ZThIb0YxTWQ3d0tRAAEWVUpRaEYxY2VTZVd2eFlYZWZLbEU3QQAA"
}


GET _search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "_shard_doc": {
        "order": "desc"
      }
    }
  ],"pit": {
    "id": "gLuKBAEFZWxsaXMWVUpRaEYxY2VTZVd2eFlYZWZLbEU3QQAWOGxDOW1rQzZTMTJJVVJubDJxM196dwAAAAAAAAAZxBZpUTdEYlBaclJ0ZThIb0YxTWQ3d0tRAAEWVUpRaEYxY2VTZVd2eFlYZWZLbEU3QQAA",
    "keep_alive": "1m"
  }
}

https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html