Elasticsearch Python DSL 客户端开发

Elastic 中国社区官方博客2026-09-10 19:50

在今天的练习中，我们来展示如何使用 Python 客户端来开发 DSL 请求。

安装

我们可以参考之前的文章 "如何在 Linux，MacOS 及 Windows 上进行安装 Elasticsearch" 来安装自己的 Elasticsearch 及 Kibana。

写入数据

我们在 Kibana 中写入如下的数据：

复制代码 
PUT my-index
{
  "mappings": {
    "properties": {
      "title": {
        "type": "text"
      },
      "description": {
        "type": "text"
      },
      "category": {
        "type": "keyword"
      },
      "tags": {
        "type": "keyword"
      },
      "lines": {
        "type": "integer"
      }
    }
  }
}

POST my-index/_bulk
{"index":{"_id":"1"}}
{"title":"Python Elasticsearch Client","description":"Learn how to search Elasticsearch using Python.","category":"search","tags":["python","elasticsearch"],"lines":120}
{"index":{"_id":"2"}}
{"title":"Python Search Tutorial","description":"A beginner tutorial for Python search applications.","category":"search","tags":["python","search"],"lines":250}
{"index":{"_id":"3"}}
{"title":"Python Query Examples","description":"Examples of Python queries for Elasticsearch.","category":"search","tags":["python","query"],"lines":180}
{"index":{"_id":"4"}}
{"title":"Advanced Python Search","description":"Advanced techniques for building search applications with Python.","category":"search","tags":["python","search"],"lines":420}
{"index":{"_id":"5"}}
{"title":"Python Beta Features","description":"This document describes beta features in Python search.","category":"search","tags":["python","beta"],"lines":500}
{"index":{"_id":"6"}}
{"title":"Elasticsearch Search Guide","description":"A guide to Elasticsearch search and aggregations.","category":"search","tags":["elasticsearch","search"],"lines":350}
{"index":{"_id":"7"}}
{"title":"Python Machine Learning","description":"Using Python for machine learning and data analysis.","category":"analytics","tags":["python","machine-learning"],"lines":600}
{"index":{"_id":"8"}}
{"title":"Python Logging","description":"How to implement logging in Python applications.","category":"development","tags":["python","logging"],"lines":150}
{"index":{"_id":"9"}}
{"title":"Python Search Beta","description":"Experimental beta implementation of Python search.","category":"search","tags":["python","experimental"],"lines":700}
{"index":{"_id":"10"}}
{"title":"Python Search Performance","description":"Benchmarking Python search performance with Elasticsearch.","category":"search","tags":["python","performance"],"lines":550}

搜索

标准 Python

复制代码 
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from elasticsearch import Elasticsearch

client = Elasticsearch(
    "https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

response = client.search(
    index="my-index",
    body={
      "query": {
        "bool": {
          "must": [{"match": {"title": "python"}}],
          "must_not": [{"match": {"description": "beta"}}],
          "filter": [{"term": {"category": "search"}}]
        }
      },
      "aggs" : {
        "per_tag": {
          "terms": {"field": "tags"},
          "aggs": {
            "max_lines": {"max": {"field": "lines"}}
          }
        }
      }
    }
)

for hit in response['hits']['hits']:
    print(hit['_score'], hit['_source']['title'])

for tag in response['aggregations']['per_tag']['buckets']:
    print(tag['key'], tag['max_lines']['value'])

Async Python

复制代码 
import asyncio
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from elasticsearch import AsyncElasticsearch
from elasticsearch_dsl import AsyncSearch, Q

client = AsyncElasticsearch(
    "https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

async def example():
    response = await client.search(
        index="my-index",
        body={
          "query": {
            "bool": {
              "must": [{"match": {"title": "python"}}],
              "must_not": [{"match": {"description": "beta"}}],
              "filter": [{"term": {"category": "search"}}]
            }
          },
          "aggs" : {
            "per_tag": {
              "terms": {"field": "tags"},
              "aggs": {
                "max_lines": {"max": {"field": "lines"}}
              }
            }
          }
        }
    )

    for hit in response['hits']['hits']:
        print(hit['_score'], hit['_source']['title'])

    for tag in response['aggregations']['per_tag']['buckets']:
        print(tag['key'], tag['max_lines']['value'])

asyncio.run(example())

这种方法的问题在于，它冗长、容易出现语法错误，例如嵌套不正确，而且难以修改（例如添加另一个过滤条件），写起来也绝对谈不上有趣。

让我们使用 DSL 模块重写这个示例：

复制代码 
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from elasticsearch import Elasticsearch
from elasticsearch.dsl import Search, query, aggs

client = Elasticsearch(
    "https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

s = Search(using=client, index="my-index") \
    .query(query.Match("title", "python"))   \
    .filter(query.Term("category", "search")) \
    .exclude(query.Match("description", "beta"))

s.aggs.bucket('per_tag', aggs.Terms(field="tags")) \
    .metric('max_lines', aggs.Max(field='lines'))

response = s.execute()

for hit in response:
    print(hit.meta.score, hit.title)

for tag in response.aggregations.per_tag.buckets:
    print(tag.key, tag.max_lines.value)

及

复制代码 
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from elasticsearch import Elasticsearch
from elasticsearch.dsl import Search, query, aggs

client = Elasticsearch(
    "https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

s = Search(using=client, index="my-index") \
    .query(query.Match("title", "python"))   \
    .filter(query.Term("category", "search")) \
    .exclude(query.Match("description", "beta"))

s.aggs.bucket('per_tag', aggs.Terms(field="tags")) \
    .metric('max_lines', aggs.Max(field='lines'))

response = s.execute()

for hit in response:
    print(hit.meta.score, hit.title)

for tag in response.aggregations.per_tag.buckets:
    print(tag.key, tag.max_lines.value)

正如你所看到的，DSL 模块处理了以下事项：

  • 从类创建适当的 Query 对象

  • 将多个查询组合成复合 bool 查询

  • 将 term 查询放入 bool 查询的过滤上下文中

  • 提供便捷的响应数据访问方式

  • 不再到处出现花括号或方括号

持久化

让我们创建一个简单的 Python 类，用于表示博客系统中的一篇文章：

标准 Python

复制代码 
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from datetime import datetime
from elasticsearch.dsl import Document, Date, Integer, Keyword, Text, connections, mapped_field

# Define a default Elasticsearch client
connections.create_connection(
    hosts="https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

class Article(Document):
    title: str = mapped_field(Text(analyzer='snowball', fields={'raw': Keyword()}))
    body: str = mapped_field(Text(analyzer='snowball'))
    tags: list[str] = mapped_field(Keyword())
    published_from: datetime
    lines: int

    class Index:
        name = 'blog'
        settings = {
          "number_of_shards": 2,
        }

    def save(self, **kwargs):
        self.lines = len(self.body.split())
        return super(Article, self).save(** kwargs)

    def is_published(self):
        return datetime.now() > self.published_from

# create the mappings in elasticsearch
Article.init()

# create and save and article
article = Article(meta={'id': 42}, title='Hello world!', tags=['test'])
article.body = ''' looong text '''
article.published_from = datetime.now()
article.save()

article = Article.get(id=42)
print(article.is_published())

# Display cluster health
print(connections.get_connection().cluster.health())

Async Python

复制代码 
import asyncio
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from datetime import datetime
from elasticsearch.dsl import AsyncDocument, Date, Integer, Keyword, Text, async_connections, mapped_field

# Define a default Elasticsearch client
async_connections.create_connection(
    hosts="https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

class Article(AsyncDocument):
    title: str = mapped_field(Text(analyzer='snowball', fields={'raw': Keyword()}))
    body: str = mapped_field(Text(analyzer='snowball'))
    tags: list[str] = mapped_field(Keyword())
    published_from: datetime
    lines: int

    class Index:
        name = 'blog'
        settings = {
          "number_of_shards": 2,
        }

    async def save(self, **kwargs):
        self.lines = len(self.body.split())
        return await super(Article, self).save(** kwargs)

    def is_published(self):
        return datetime.now() > self.published_from

async def example():
    # create the mappings in elasticsearch
    await Article.init()

    # create and save and article
    article = Article(meta={'id': 42}, title='Hello world!', tags=['test'])
    article.body = ''' looong text '''
    article.published_from = datetime.now()
    await article.save()

    article = await Article.get(id=42)
    print(article.is_published())

    # Display cluster health
    print(await async_connections.get_connection().cluster.health())

asyncio.run(example())

在这个示例中，你可以看到：

  • 提供一个默认连接

  • 使用 Python 类型提示定义字段，并在必要时提供额外的 mapping 配置

  • 设置索引名称

  • 定义自定义方法

  • 重写内置的 .save() 方法，以便接入持久化生命周期

  • 从 Elasticsearch 中检索和保存对象

  • 访问底层客户端，以使用其他 API

你可以在持久化章节中了解更多信息。

预构建的分面搜索

如果你已经定义了 Document，就可以创建一个分面搜索类，以简化搜索和过滤。

标准 Python

复制代码 
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)
warnings.filterwarnings("ignore", message=".*verify_certs=False.*")

from elasticsearch.dsl import FacetedSearch, TermsFacet, DateHistogramFacet, connections
from example6 import Article

connections.create_connection(
    hosts="https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

class BlogSearch(FacetedSearch):
    doc_types = [Article, ]
    # fields that should be searched
    fields = ['tags', 'title', 'body']

    facets = {
        # use bucket aggregations to define facets
        'tags': TermsFacet(field='tags'),
        'publishing_frequency': DateHistogramFacet(field='published_from', calendar_interval='month')
    }

# empty search
bs = BlogSearch()
response = bs.execute()

for hit in response:
    print(hit.meta.score, getattr(hit, 'title', 'N/A'))

for (tag, count, selected) in response.facets.tags:
    print(tag, ' (SELECTED):' if selected else ':', count)

for (month, count, selected) in response.facets.publishing_frequency:
    print(month.strftime('%B %Y'), ' (SELECTED):' if selected else ':', count)

Async Python

复制代码 
import asyncio
import warnings
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)

from elasticsearch.dsl import AsyncFacetedSearch, TermsFacet, DateHistogramFacet, async_connections
from example5 import Article

async_connections.create_connection(
    hosts="https://localhost:9200",
    api_key="d25PamhLQUJHblNmOTlpVmpydWc6NkJLNXA2TDFQUzFiRzRKZ2wxYy1ZZw==",
    verify_certs=False,
)

class BlogSearch(AsyncFacetedSearch):
    doc_types = [Article, ]
    # fields that should be searched
    fields = ['tags', 'title', 'body']

    facets = {
        # use bucket aggregations to define facets
        'tags': TermsFacet(field='tags'),
        'publishing_frequency': DateHistogramFacet(field='published_from', interval='month')
    }

async def example():
    # empty search
    bs = BlogSearch()
    response = await bs.execute()

    for hit in response:
        print(hit.meta.score, hit.title)

    for (tag, count, selected) in response.facets.tags:
        print(tag, ' (SELECTED):' if selected else ':', count)

    for (month, count, selected) in response.facets.publishing_frequency:
        print(month.strftime('%B %Y'), ' (SELECTED):' if selected else ':', count)

asyncio.run(example())

你可以在 faceted_search 章节中找到更多详细信息。

按查询更新

让我们继续使用博客文章这个简单示例，并假设每篇文章都有一定数量的点赞。在这个示例中，假设我们希望将所有匹配某个标签且不匹配某个描述的文章的点赞数增加 1。如果将其写成 dict，代码如下：

标准 Python

复制代码 
from elasticsearch import Elasticsearch

client = Elasticsearch()

response = client.update_by_query(
    index="my-index",
    body={
      "query": {
        "bool": {
          "must": [{"match": {"tag": "python"}}],
          "must_not": [{"match": {"description": "beta"}}]
        }
      },
      "script"={
        "source": "ctx._source.likes++",
        "lang": "painless"
      }
    },
  )

Async Python

复制代码 
from elasticsearch import AsyncElasticsearch

client = AsyncElasticsearch()

async def example():
    response = await client.update_by_query(
        index="my-index",
        body={
          "query": {
            "bool": {
              "must": [{"match": {"tag": "python"}}],
              "must_not": [{"match": {"description": "beta"}}]
            }
          },
          "script"={
            "source": "ctx._source.likes++",
            "lang": "painless"
          }
        },
      )

现在，使用 DSL，我们可以这样表达这个查询：

标准 Python

复制代码 
from elasticsearch import AsyncElasticsearch
from elasticsearch.dsl import AsyncSearch, AsyncUpdateByQuery
from elasticsearch.dsl.query import Match

client = AsyncElasticsearch()

async def example():
    ubq = UpdateByQuery(using=client, index="my-index") \
          .query(Match("title", "python"))   \
          .exclude(Match("description", "beta")) \
          .script(source="ctx._source.likes++", lang="painless")

    response = await ubq.execute()

Async Python

复制代码 
from elasticsearch import Elasticsearch
from elasticsearch.dsl import Search, UpdateByQuery
from elasticsearch.dsl.query import Match

client = Elasticsearch()
ubq = UpdateByQuery(using=client, index="my-index") \
      .query(Match("title", "python"))   \
      .exclude(Match("description", "beta")) \
      .script(source="ctx._source.likes++", lang="painless")

response = ubq.execute()

正如你所看到的，Update By Query 对象提供了 Search 对象所带来的许多便利，同时还允许你以相同的方式，通过指定的脚本更新搜索结果。

ES|QL 查询

DSL 模块提供了与 ES|QL 查询构建器的集成，其中包含两个所有 Document 子类都可使用的方法：esql_from() 和 esql_execute()。使用上面的 Article 文档，我们可以通过以下 ES|QL 查询，搜索最多 10 篇标题中包含 "world" 的文章：

标准 Python

复制代码 
from elasticsearch.esql import functions

query = Article.esql_from().where(functions.match(Article.title, 'world')).limit(10)
for a in Article.esql_execute(query):
    print(a.title)

Aysnc Python

复制代码 
from elasticsearch.esql import functions

async def example():
    query = Article.esql_from().where(functions.match(Article.title, 'world')).limit(10)
    async for a in Article.esql_execute(query):
        print(a.title)

查看 ES|QL 查询构建器章节，了解更多关于如何在 Python 中构建 ES|QL 查询的信息。

从标准客户端迁移

你不必为了获得 DSL 模块带来的好处而迁移整个应用程序。你可以逐步开始：从现有的 dict 创建一个 Search 对象，使用 API 对其进行修改，然后再将其序列化回 dict：

标准 Python

复制代码 
body = {...}

# Convert to Search object
s = Search.from_dict(body)

# Add some filters, aggregations, queries, ...
s.filter(query.Term("tags", "python"))

# Convert back to dict to plug back into existing code
body = s.to_dict()

Async Python

复制代码 
body = {...}

# Convert to Search object
s = Search.from_dict(body)

# Add some filters, aggregations, queries, ...
s.filter(query.Term("tags", "python"))

# Convert back to dict to plug back into existing code
body = s.to_dict()
相关推荐
风哥2号1 小时前
数据库教程FGMT03‑生产环境Linux+Oracle19c+ASM安装配置与项目实战
linux·数据库
hughnz2 小时前
石油工程的端到端数字化转型：演化还是革命
大数据·人工智能·科技
龙亘川2 小时前
旅游强国建设|一网统管智慧旅游服务模块，赋能节假日文旅数字化治理
大数据·数据库·人工智能·科技·智慧城市·旅游
计算机编程-吉哥2 小时前
脑肿瘤MRI智能识别系统：基于深度学习的像素级脑肿瘤语义分割平台【计算机毕业设计选题推荐】
人工智能·python·深度学习·算法·毕业设计·课程设计·大数据毕业设计选题推荐
星火跨境XINGHUOS2 小时前
技术向辨伪：从商标查证到源码资产，如何核验网络上的z真假“星火跨境“
数据库
小木_.2 小时前
Python 离线识别滑块缺口距离，项目推荐
开发语言·python·滑块识别·人机验证·滑块缺口·缺口识别
阿里云大数据AI技术2 小时前
Al Search x ES Agent Builder：让数据活起来，从搜索走向行动
人工智能·elasticsearch·agent
Cenxi2 小时前
Python字符串方法练习手册
人工智能·python
liliangcsdn2 小时前
因子权重矩阵处理-因子权重收缩Shrinkage算法的探索
开发语言·python·机器学习
热门推荐
01如何新建文件夹？ 电脑新建文件夹的4种方法02国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）03GitHub 镜像站点04历年考研数学一、数学二、数学三真题试卷及答案PDF052026 国产 AI 大模型横评：DeepSeek、通义千问、Kimi、文心一言、星火、豆包谁更能打？06微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新07蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接08ZCode 周末送额度活动开启：3 亿 Token 免费领取09Agnes AI 免费 API 接入指南：文本、生图、生视频，一套接口全免费10AI 编程 IDE 全景解析 2026：Agent 全面接管开发链路