Elasticsearch8.x版本Java客户端Elasticsearch Java API Client中常用API练习

Es的java API客户端

在Es7.15版本之后，es官方将它的高级客户端RestHighLevelClient标记为弃用状态。同时推出了全新的java API客户端Elasticsearch Java API Client，该客户端也将在Elasticsearch8.0及以后版本中成为官方推荐使用的客户端。

Elasticsearch Java API Client支持除Vector title search API和Find structure API之外的所有Elasticsearch API。且支持所有API数据类型，并且不再有原始JSON Value属性。它是针对Elasticsearch8.0及之后版本的客户端。

maven中引入依赖坐标

  <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>8.12.2</version>
        </dependency>
    <dependency>
            <groupId>co.elastic.clients</groupId>
            <artifactId>elasticsearch-java</artifactId>
            <version>8.12.2</version>
            <exclusions>
                <exclusion>
                    <groupId>org.elasticsearch.client</groupId>
                    <artifactId>elasticsearch-rest-client</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

• 创建连接

@Bean
    public ElasticsearchClient elasticsearchClient(@Value("${elasticsearch.xxxx}") String serverUrl,
                                                   @Value("${elasticsearch.xxxxx}") String apiKey) {

        RestClient restClient = RestClient.builder(
                        HttpHost.create(serverUrl))
                .setDefaultHeaders(new Header[]{new BasicHeader("Authorization", "ApiKey " + apiKey),})
                .build();

        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());

        return new ElasticsearchClient(transport);

    }

• 索引index

  @Test
    public void create() throws IOException {
        // 创建低级客户端
        RestClient restClient = RestClient.builder(
                new HttpHost("localhost", 9200)
        ).build();
        // 使用Jackson映射器创建传输层
        ElasticsearchTransport transport = new RestClientTransport(
                restClient, new JacksonJsonpMapper()
        );
        // 创建API客户端
        ElasticsearchClient client = new ElasticsearchClient(transport);
        // 创建索引
        CreateIndexResponse createIndexResponse = client.indices().create(c -> c.index("user_test"));
        // 响应状态
        Boolean acknowledged = createIndexResponse.acknowledged();
        System.out.println("索引操作 = " + acknowledged);

        // 关闭ES客户端
        transport.close();
        restClient.close();
    }

• 查询索引

@Test
    public void query() throws IOException {
        RestClient restClient = RestClient.builder(
                new HttpHost("localhost",9200)
        ).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 查询索引
        GetIndexResponse getIndexResponse = client.indices().get(e -> e.index("user_test"));
        System.out.println("getIndexResponse.result() = " + getIndexResponse.result());
        System.out.println("getIndexResponse.result().keySet() = " + getIndexResponse.result().keySet());

        transport.close();
        restClient.close();
    } 

如果查询的index不存在会在控制台抛出index_not_found_exception

• 删除索引

 @Test
    public void delete() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 删除索引
        DeleteIndexResponse deleteIndexResponse = client.indices().delete(e -> e.index("user_test"));
        System.out.println("删除操作 = " + deleteIndexResponse.acknowledged());

        transport.close();
        restClient.close();
    }

文档document的操作

• 添加document

@Test
    public void addDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 向user对象中添加数据
        User user = new User("java客户端", "男", 18);
        // 向索引中添加数据
        CreateResponse createResponse = client.create(e -> e.index("user_test").id("1001").document(user));
        System.out.println("createResponse.result() = " + createResponse.result());

        transport.close();
        restClient.close();
    }

注：index中参数为文档所属的索引名，id中参数为当文档的id，document为文档数据，新版本支持直接传入java对象。

• 查询document

 @Test
    public void queryDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 构建请求
        GetResponse<User> getResponse = client.get(e -> e.index("user_test").id("1001"), User.class);
        System.out.println("getResponse.source().toString() = " + getResponse.source().toString());

        transport.close();
        restClient.close();
    }

注：如果查不到控制台抛出NullPointerException

• 修改document

@Test
    public void modifyDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 使用map集合封装需要修改的内容
        Map<String, Object> map = new HashMap<>();
        map.put("name", "java客户端aaa");
        // 构建请求
        UpdateResponse<User> updateResponse = client.update(e -> e.index("user_test").id("1001").doc(map), User.class);
        System.out.println("updateResponse.result() = " + updateResponse.result());

        transport.close();
        restClient.close();
    }

• 删除document

    @Test
    public void removeDocument() throws  IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 构建请求
        DeleteResponse deleteResponse = client.delete(e -> e.index("user_test").id("1001"));
        System.out.println("deleteResponse.result() = " + deleteResponse.result());

        transport.close();
        restClient.close();
    }

• 批量添加document

   @Test
    public void batchAddDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 构建一个批量数据集合
        List<BulkOperation> list = new ArrayList<>();
        list.add(new BulkOperation.Builder().create(
                d -> d.document(new User("test2", "男", 19)).id("1002").index("user_test")).build());
        list.add(new BulkOperation.Builder().create(
                d -> d.document(new User("test3", "男", 20)).id("1003").index("user_test")).build());
        list.add(new BulkOperation.Builder().create(
                d -> d.document(new User("test4", "女", 21)).id("1004").index("user_test")).build());
        // 调用bulk方法执行批量插入操作
        BulkResponse bulkResponse = client.bulk(e -> e.index("user_test").operations(list));
        System.out.println("bulkResponse.items() = " + bulkResponse.items());

        transport.close();
        restClient.close();
    }

批量添加的核心是需要构建一个泛型为BulkOperation的ArrayList集合，实质上是将多个请求包装到一个集合中，进行统一请求，进行构建请求时调用bulk方法，实现批量添加效果。

• 批量删除

    @Test
    public void batchDeleteDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 构建一个批量数据集合
        List<BulkOperation> list = new ArrayList<>();
        list.add(new BulkOperation.Builder().delete(
                d -> d.id("1002").index("user_test")).build());
        list.add(new BulkOperation.Builder().delete(
                d -> d.id("1003").index("user_test")).build());
        // 调用bulk方法执行批量插入操作
        BulkResponse bulkResponse = client.bulk(e -> e.index("user_test").operations(list));
        System.out.println("bulkResponse.items() = " + bulkResponse.items());

        transport.close();
        restClient.close();
    }

• 全量查询

    @Test
    public void queryAllDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 全量查询
        SearchResponse<User> searchResponse = client.search(e -> e.index("user_test").query(q -> q.matchAll(m -> m)), User.class);
        HitsMetadata<User> hits = searchResponse.hits();
        for (Hit<User> hit : hits.hits()) {
            System.out.println("user = " + hit.source().toString());
        }
        System.out.println("searchResponse.hits().total().value() = " + searchResponse.hits().total().value());

        transport.close();
        restClient.close();
    }

• 分页查询

@Test
    public void pagingQueryDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 分页查询
        SearchResponse<User> searchResponse = client.search(
                s -> s.index("user_test")
                        .query(q -> q.matchAll(m -> m))
                        .from(2)
                        .size(2)
                , User.class);
        searchResponse.hits().hits().forEach(h -> System.out.println(h.source().toString()));

        transport.close();
        restClient.close();
    }

分页查询就是在全量查询的基础上增加了从第几条开始，每页显示几条

• 排序查询

    @Test
    public void sortQueryDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 排序查询
        SearchResponse<User> searchResponse = client.search(
                s -> s.index("user_test")
                        .query(q -> q.matchAll(m -> m))
                        .sort(o -> o.field(f -> f.field("age").order(SortOrder.Asc)))
                , User.class);
        searchResponse.hits().hits().forEach(h -> System.out.println(h.source().toString()));

        transport.close();
        restClient.close();
    }

• 条件查询

    @Test
    public void conditionQueryDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 条件查询
        SearchResponse<User> searchResponse = client.search(
                s -> s.index("user_test").query(q -> q.matchAll(m -> m))
                        .sort(o -> o.field(f -> f.field("age").order(SortOrder.Asc)))
                        .source(r -> r.filter(f -> f.includes("name", "age").excludes("")))
                , User.class);
        searchResponse.hits().hits().forEach(h -> System.out.println(h.source().toString()));

        transport.close();
        restClient.close();
    }

includes是显示的字段，excludes是排除的字段

• 组合查询

  @Test
    public void combinationQueryDocument() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 组合查询
        SearchResponse<User> searchResponse = client.search(
                s -> s.index("user_test").query(q -> q.bool(b -> b
                        .must(m -> m.match(u -> u.field("age").query(21)))
                        .must(m -> m.match(u -> u.field("sex").query("男")))
                        .mustNot(m -> m.match(u -> u.field("sex").query("女")))
                ))
                , User.class);
        searchResponse.hits().hits().forEach(h -> System.out.println(h.source().toString()));

        transport.close();
        restClient.close();
    }

    @Test
    public void combinationQueryDocument2() throws IOException {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        ElasticsearchClient client = new ElasticsearchClient(transport);

        // 组合查询
        SearchResponse<User> searchResponse = client.search(
                s -> s.index("user_test").query(q -> q.bool(b -> b
                        .should(h -> h.match(u -> u.field("age").query(19)))
                        .should(h -> h.match(u -> u.field("sex").query("男")))
                ))
                , User.class);
        searchResponse.hits().hits().forEach(h -> System.out.println(h.source().toString()));

        transport.close();
        restClient.close();
    }

接入向量查询

将数据可视化为向量空间 在我们实施 KNN 搜索之前，我们需要了解构建它的一个基本概念，即 "向量（vector）"。 向量只是数组。 在编程中我们可以有字符串、对象等数组，而向量只能包含数值。 因此向量是数值数组。 向量最酷的地方在于，我们可以将它们绘制在图表上，找出它们之间的距离，看看它们之间的距离有多近。

Elasticsearch 支持两种 kNN 搜索方法：

• 使用带有向量函数的 script_score 查询的 Exact，brute-force kNN
• 使用 knn 搜索选项近似 kNN（Approximate kNN）
• 在大多数情况下，你会希望使用近似 kNN。 近似 kNN 以较慢的索引和不完美的准确性为代价提供较低的延迟。

参考文档：blog.csdn.net/UbuntuTouch...

向量转换,将String->List double

public static List<Float> getDirectionValues(ElasticsearchClient client,String str) {
    ArrayList<Map<String, JsonData>> docs = new ArrayList<>();
    Map<String, JsonData> data = new HashMap<>();
    data.put("text_field", JsonData.of(str));
    docs.add(data);
    InferTrainedModelResponse modelResponse = null;
    try {
        modelResponse = esClient.ml()
                .inferTrainedModel(new InferTrainedModelRequest.Builder()
                        .modelId("model")
                        .docs(docs).build());
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    List<FieldValue> fieldValues = modelResponse.inferenceResults().get(0).predictedValue();

    return fieldValues.stream().map(e -> ((float)(e.doubleValue()))).collect(Collectors.toList());
}

向量配置：k=5范围为5，

KnnQuery knnQuery = KnnQuery.of(m -> m.field("xxxx")
        .queryVector(asrVector)
        .k(5)
        .numCandidates(100));

查询

SearchResponse<Class> response = elasticsearchClient.search(s -> s.index("xxxx")
        .knn(knnQuery), Class.class);