ES深入浅出系列（五）基于ES实践用户中台

DannyIdea2023-11-22 10:33

本文我们主要是搭建一个基于ES存储的用户中台工程，用于熟悉对于ES的知识点。

项目结构：

项目的maven依赖内容：

父模块依赖：

xml 复制代码 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>qiyu-live-user-center</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <description>用户中台</description>
    <modules>
        <module>qiyu-live-user-center-provider</module>
    </modules>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.0.4</version>
        <relativePath/>
    </parent>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <springboot.version>3.0.4</springboot.version>
        <qiyu-mysql.version>8.0.28</qiyu-mysql.version>
        <lombok.version>1.18.20</lombok.version>
    </properties>

    <dependencies>
        <!-- junit start -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
        </dependency>
    </dependencies>


    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.alibaba.cloud</groupId>
                <artifactId>spring-cloud-alibaba-dependencies</artifactId>
                <version>2022.0.0.0-RC1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
</project>

接着是子模块依赖：

xml 复制代码 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.example</groupId>
        <artifactId>qiyu-live-user-center</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <artifactId>qiyu-live-user-center-provider</artifactId>

    <properties>
        <elasticsearch.version>6.5.4</elasticsearch.version>
        <elasticsearch.rest.high.level.version>6.5.4</elasticsearch.rest.high.level.version>
        <fastjson.version>1.2.47</fastjson.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elasticsearch.rest.high.level.version}</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>${fastjson.version}</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>${lombok.version}</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

</project>

接着是ES的配置类：

java 复制代码 
package org.qiyu.live.user.center.provider.config;

import jakarta.annotation.Resource;
import lombok.extern.slf4j.Slf4j;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @Author idea
 * @Date: Created in 10:29 2023/11/5
 * @Description es配置类
 */
@Configuration
@EnableConfigurationProperties(EsProperties.class)
@Slf4j
public class EsConfiguration {

    @Resource
    private EsProperties esProperties;

    @Bean
    public RestHighLevelClient restHighLevelClient() {
        HttpHost httpHost = new HttpHost(esProperties.getEsAddress(), esProperties.getPort());
        RestClientBuilder restClientBuilder = RestClient.builder(httpHost);
        log.info("初始化es客户端工具");
        return new RestHighLevelClient(restClientBuilder);
    }

}

该配置类需要读取一个自定义的配置Properties对象：

java 复制代码 
package org.qiyu.live.user.center.provider.config;

import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;

/**
 * @Author idea
 * @Date: Created in 10:30 2023/11/5
 * @Description
 */
@Data
@ConfigurationProperties(prefix = "qiyu.live.es")
public class EsProperties {

    private String esAddress;
    private Integer port;
}

这个配置对象用于映射我们application.properties配置文件中的es地址配置信息，如下所示：

ini 复制代码 
qiyu.live.es.esAddress=127.0.0.1
qiyu.live.es.port=9200

这里你可能会疑惑，es的index和type配置在哪里了呢？这里我用了一个枚举对象去管理这部分的内容了：

java 复制代码 
package org.qiyu.live.user.center.provider.config;

import lombok.AllArgsConstructor;
import lombok.Getter;

/**
 * @Author idea
 * @Date: Created in 10:36 2023/11/5
 * @Description
 */
@AllArgsConstructor
@Getter
public enum EsTypeEnum {

    USER("user","qiyu_live_user");
    private String index;
    private String type;
}

接下来便是我们的dao层的代码内容，首先是dao层的接口：

java 复制代码 
package org.qiyu.live.user.center.provider.dao;

import org.qiyu.live.user.center.provider.dao.po.UserPO;

import java.util.List;

/**
 * @Author idea
 * @Date: Created in 10:26 2023/11/5
 * @Description
 */
public interface IUserDao {

    UserPO queryByUserId(Long userId);

    List<UserPO> queryByUsername(String username);

    UserPO insertOne(UserPO userPO);

    UserPO updateOne(UserPO userPO);

    void deleteOne(Long userId);

    List<UserPO> batchQuery(List<Long> userIds);
}

接着是dao层的接口实现：

java 复制代码 
package org.qiyu.live.user.center.provider.dao.impl;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.serializer.SimplePropertyPreFilter;
import jakarta.annotation.Resource;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.qiyu.live.user.center.provider.config.EsProperties;
import org.qiyu.live.user.center.provider.config.EsTypeEnum;
import org.qiyu.live.user.center.provider.dao.IUserDao;
import org.qiyu.live.user.center.provider.dao.po.UserPO;
import org.qiyu.live.user.center.provider.utils.EsResponseUtils;
import org.springframework.stereotype.Repository;

import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

/**
 * @Author idea
 * @Date: Created in 10:28 2023/11/5
 * @Description
 */
@Repository
public class UserDaoImpl implements IUserDao {

    @Resource
    private RestHighLevelClient restHighLevelClient;
    @Resource
    private EsProperties esProperties;

    @Override
    public UserPO queryByUserId(Long userId) {
        SearchRequest searchRequest = new SearchRequest(EsTypeEnum.USER.getIndex());
        searchRequest.types(EsTypeEnum.USER.getType());
        SearchSourceBuilder builder = new SearchSourceBuilder();
        builder.query(QueryBuilders.idsQuery().addIds(String.valueOf(userId)));
        searchRequest.source(builder);
        try {
            SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            return EsResponseUtils.resultOne(response, UserPO.class);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public List<UserPO> queryByUsername(String username) {
        SearchRequest searchRequest = new SearchRequest(EsTypeEnum.USER.getIndex());
        searchRequest.types(EsTypeEnum.USER.getType());
        SearchSourceBuilder builder = new SearchSourceBuilder();
        builder.query(QueryBuilders.wildcardQuery("username", "*" + username + "*"));
        searchRequest.source(builder);
        try {
            SearchResponse response = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
            return EsResponseUtils.resultList(response,UserPO.class);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public UserPO insertOne(UserPO userPO) {
        IndexRequest indexRequest = new IndexRequest(EsTypeEnum.USER.getIndex(), EsTypeEnum.USER.getType(), userPO.getUserId());
        indexRequest.source(JSON.toJSONString(userPO), XContentType.JSON);
        try {
            restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
            return userPO;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public UserPO updateOne(UserPO userPO) {
        UpdateRequest updateRequest = new UpdateRequest(EsTypeEnum.USER.getIndex(), EsTypeEnum.USER.getType(), userPO.getUserId());
        SimplePropertyPreFilter filter = new SimplePropertyPreFilter();
        filter.getExcludes().add("registryTime");
        filter.getExcludes().add("updateTime");
        Map<String, Object> updateMap = JSON.parseObject(JSON.toJSONString(userPO), Map.class);
        updateRequest.doc(updateMap);
        try {
            restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
            return userPO;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public void deleteOne(Long userId) {
        DeleteRequest deleteRequest = new DeleteRequest(EsTypeEnum.USER.getIndex(), EsTypeEnum.USER.getType(), String.valueOf(userId));
        try {
            restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public List<UserPO> batchQuery(List<Long> userIds) {
        SearchRequest searchRequest = new SearchRequest(EsTypeEnum.USER.getIndex());
        searchRequest.types(EsTypeEnum.USER.getType());
        SearchSourceBuilder builder = new SearchSourceBuilder();
        List<String> idList = userIds.stream().map(id -> String.valueOf(id)).collect(Collectors.toList());
        String[] idArr = new String[idList.size()];
        int size = idList.size();
        for (int i = 0; i < size; i++) {
            idArr[i] = idList.get(i);
        }
        builder.query(QueryBuilders.idsQuery().addIds(idArr));
        searchRequest.source(builder);
        try {
            SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            return EsResponseUtils.resultList(response, UserPO.class);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

可以看到，dao层内部的es实现，是依赖于我们先前在配置类里面往spring容器中注入的 restHighLevelClient 对象的bean。

至于ES的PO对象映射，则是如下所示：

java 复制代码 
package org.qiyu.live.user.center.provider.dao.po;

import lombok.Data;
import org.qiyu.live.user.center.provider.config.EsId;

import java.util.Date;

/**
 * @Author idea
 * @Date: Created in 10:25 2023/11/5
 * @Description
 */
@Data
public class UserPO {

    @EsId
    private String userId;
    private String username;
    private String tel;
    private Integer sex;
    private Date registryTime;
    private Date updateTime;
}

这里映射的时候，用到了一个自定义注解：

java 复制代码 
package org.qiyu.live.user.center.provider.config;


import java.lang.annotation.*;

/**
 * @Author idea
 * @Date: Created in 11:17 2023/11/5
 * @Description
 */
@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface EsId {

}

该注解主要是用于对象从es中解析的时候，将es中的 _id 字段值赋予到userId字段上。

关于es查询结果转换部分，我封装了一个自定义的Utils工具去完成这部分的效果：

java 复制代码 
package org.qiyu.live.user.center.provider.utils;

import com.alibaba.fastjson.JSON;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.search.SearchHit;
import org.qiyu.live.user.center.provider.config.EsId;

import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.List;

/**
 * @Author idea
 * @Date: Created in 11:09 2023/11/5
 * @Description
 */
public class EsResponseUtils {

    public static <T> T resultOne(SearchResponse response, Class<T> clazz) {
        SearchHit[] result = response.getHits().getHits();
        T value = null;
        if (result.length > 0) {
            SearchHit documentFields = result[0];
            String itemJson = documentFields.getSourceAsString();
            value = JSON.parseObject(itemJson, clazz);
            setId(clazz, value, documentFields);
        }
        return value;
    }

    public static <T> List<T> resultList(SearchResponse response, Class<T> clazz) {
        SearchHit[] result = response.getHits().getHits();
        List<T> resultList = new ArrayList<>(result.length);
        if (result.length > 0) {
            for (SearchHit documentFields : result) {
                String itemJson = documentFields.getSourceAsString();
                T value = JSON.parseObject(itemJson, clazz);
                setId(clazz, value, documentFields);
                resultList.add(value);
            }
        }
        return resultList;
    }

    private static void setId(Class clazz, Object value, SearchHit documentFields) {
        Field[] fields = clazz.getDeclaredFields();
        for (Field field : fields) {
            field.setAccessible(true);
            if (field.isAnnotationPresent(EsId.class)) {
                try {
                    field.set(value, documentFields.getId());
                } catch (IllegalAccessException e) {
                    throw new RuntimeException(e);
                }
            }
        }
    }
}

好了，到这里，dao层部分的内容基本就完善了，关于service层的部分，其实主要是对dao层的封装，比较简单，这里就不花过多篇幅去介绍了。

最后是我们的springboot启动类代码：

java 复制代码 
package org.qiyu.live.user.center.provider;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
 * @Author idea
 * @Date: Created in 10:09 2023/11/5
 * @Description
 */
@SpringBootApplication
public class UserCenterProviderApplication {
    
    public static void main(String[] args) {
        SpringApplication.run(UserCenterProviderApplication.class, args);
    }

}

非常简单基础的一个SpringBoot+Es的练手代码案例，希望对大家有所帮助。

相关推荐
wn5312 分钟前
【Go - 类型断言】
服务器·开发语言·后端·golang
希冀12326 分钟前
【操作系统】1.2操作系统的发展与分类
后端
GoppViper1 小时前
golang学习笔记29——golang 中如何将 GitHub 最新提交的版本设置为 v1.0.0
笔记·git·后端·学习·golang·github·源代码管理
爱上语文2 小时前
Springboot的三层架构
java·开发语言·spring boot·后端·spring
serve the people2 小时前
springboot 单独新建一个文件实时写数据，当文件大于100M时按照日期时间做文件名进行归档
java·spring boot·后端
罗政7 小时前
[附源码]超简洁个人博客网站搭建+SpringBoot+Vue前后端分离
vue.js·spring boot·后端
拾光师9 小时前
spring获取当前request
java·后端·spring
Lill_bin10 小时前
深入理解ElasticSearch集群：架构、高可用性与数据一致性
大数据·分布式·elasticsearch·搜索引擎·zookeeper·架构·全文检索
Java小白笔记10 小时前
关于使用Mybatis-Plus 自动填充功能失效问题
spring boot·后端·mybatis
JOJO___12 小时前
Spring IoC 配置类 总结
java·后端·spring·java-ee
热门推荐
01RAG 实践- Ollama+RagFlow 部署本地知识库02组基轨迹建模 GBTM的介绍与实现（Stata 或 R）032024年高教社杯数学建模国赛C题超详细解题思路分析04Coze扣子平台完整体验和实践（附国内和国际版对比）05苍穹外卖面试总结06【2024数模国赛赛题思路公开】国赛B题思路丨附可运行代码丨无偿自提07yolov8实战第五天——yolov8+ffmpg实时视频流检测并进行实时推流——（推流，保姆教学）08CCF-CSP认证考试 202406-3 文本分词 100分题解09【2024高教社杯全国大学生数学建模竞赛】B题 生产过程中的决策问题——解题思路 代码 论文10浏览器插件——ModHeader 的分享和学习