JavaWeb_LeadNews_Day7-ElasticSearch, Mongodb

Y_cen2023-08-28 8:47

JavaWeb_LeadNews_Day7-ElasticSearch, Mongodb

elasticsearch

安装配置

https://blog.csdn.net/Y_cen/article/details/131856995

app文章搜索

创建索引库

  • 使用postman添加映射
    Put请求, Json格式 ： http://192.168.174.133:9200/app_info_article

    json 复制代码 
    {
    "mappings":{
        "properties":{
            "id":{
                "type":"long"
            },
            "publishTime":{
                "type":"date"
            },
            "layout":{
                "type":"integer"
            },
            "images":{
                "type":"keyword",
                "index": false
            },
            "staticUrl":{
                "type":"keyword",
                "index": false
            },
            "authorId": {
                "type": "long"
            },
            "authorName": {
                "type": "text"
            },
            "title":{
                "type":"text",
                "analyzer":"ik_smart"
            },
            "content":{
                "type":"text",
                "analyzer":"ik_smart"
            }
        }
    }
}

  • 依赖

    xml 复制代码 
    <!--elasticsearch-->
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.12.1</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client</artifactId>
    <version>7.12.1</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.12.1</version>
</dependency>

  • 导入数据

    java 复制代码 
    @Autowired
private ApArticleMapper apArticleMapper;

@Autowired
private RestHighLevelClient restHighLevelClient;

/**
 * 注意：数据量的导入，如果数据量过大，需要分页导入
 * @throws Exception
 */
@Test
public void init() throws Exception {
    // 1. 查询所有符合条件的文章数据
    List<SearchArticleVo> searchArticleVos = apArticleMapper.loadArticleList();
    // 2. 批量导入到es索引库
    BulkRequest bulkRequest = new BulkRequest("app_info_article");
    for (SearchArticleVo searchArticleVo : searchArticleVos) {
        IndexRequest indexRequest = new IndexRequest().id(searchArticleVo.getId().toString()).source(JSON.toJSONString(searchArticleVo), XContentType.JSON);
        // 批量添加数据
        bulkRequest.add(indexRequest);
    }
    restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
}

app文章搜索

思路分析

具体实现

  • 配置
    没有使用数据库, 所以无需数据源的自动配置

    yml 复制代码 
    spring:
    autoconfigure:
        exclude: org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
elasticsearch:
    host: 192.168.174.133
    port: 9200

  • 业务代码

    java 复制代码 
    @Service
public class ArticleSearchServiceImpl implements ArticleSearchService {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    /**
     * es文章分页检索
     * @param dto
     * @return
     */
    @Override
    public ResponseResult search(UserSearchDto dto) throws IOException {
        // 1. 检查参数
        if(dto == null || StringUtils.isBlank(dto.getSearchWords())){
            return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);
        }
        // 2. 设置查询条件
        SearchRequest searchRequest = new SearchRequest("app_info_article");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        // 布尔查询
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();

        // 2.1 关键字的分词之后查询
        QueryStringQueryBuilder queryStringQueryBuilder = QueryBuilders.queryStringQuery(dto.getSearchWords()).field("title").field("content").defaultOperator(Operator.OR);
        boolQueryBuilder.must(queryStringQueryBuilder);

        // 2.2 查询小于mindate的数据
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("publishTime").lt(dto.getMinBehotTime().getTime());
        boolQueryBuilder.filter(rangeQueryBuilder);

        // 2.3 分页查询
        searchSourceBuilder.from(0);
        searchSourceBuilder.size(dto.getPageSize());

        // 2.4 按照发布时间倒序查询
        searchSourceBuilder.sort("publishTime", SortOrder.DESC);

        // 2.5 设置高亮 title
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("title");
        highlightBuilder.preTags("<font style='color: red; font-size: inherit;'>");
        highlightBuilder.postTags("</font>");
        searchSourceBuilder.highlighter(highlightBuilder);

        searchSourceBuilder.query(boolQueryBuilder);
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        // 3. 结果封装返回
        List<Map> list = new ArrayList();
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            Map map = JSON.parseObject(json, Map.class);
            // 处理高亮
            if(hit.getHighlightFields() != null && hit.getHighlightFields().size() > 0){
                Text[] titles = hit.getHighlightFields().get("title").getFragments();
                String title = StringUtils.join(titles);
                // 高亮标题
                map.put("h_title", title);
            }else{
                // 原始标题
                map.put("h_title", map.get("title"));
            }
            list.add(map);
        }

        return ResponseResult.okResult(list);
    }
}

  • 网关配置
    app端网关

    yml 复制代码 
    #搜索微服务
- id: leadnews-search
    uri: lb://leadnews-search
    predicates:
        - Path=/search/**
    filters:
        - StripPrefix= 1

新增文章创建索引

思路分析

具体实现

  • Producer
    SearchArticleVo需要静态地址, 所以需要在FreeMarker生成静态文章, 上传到MinIo之后再发送创建索引的消息

    java 复制代码 
    @Service
@Slf4j
@Transactional
public class ArticleFreemarkerServiceImpl implements ArticleFreemarkerService {

    ...
    @Override
    @Async
    public void buildArticleToMinIO(ApArticle article, String content) {
        if(StringUtils.isNotBlank(content)) {
            ...

            // kafka: 新增文章创建索引, 发送消息
            createArticleESIndex(article, content, path);
        }
    }

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    private void createArticleESIndex(ApArticle article, String content, String path) {

        SearchArticleVo searchArticleVo = new SearchArticleVo();
        BeanUtils.copyProperties(article, searchArticleVo);
        searchArticleVo.setContent(content);
        searchArticleVo.setStaticUrl(path);

        kafkaTemplate.send(ArticleConstants.ARTICLE_ES_SYNC_TOPIC, JSON.toJSONString(searchArticleVo));
    }
}

  • Listener

    java 复制代码 
    @Component
@Slf4j
public class SyncArticleListener {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @KafkaListener(topics = ArticleConstants.ARTICLE_ES_SYNC_TOPIC)
    public void onMessage(String message)
    {
        if(StringUtils.isNotBlank(message)){

            log.info("SyncArticleListener, message={}", message);

            SearchArticleVo vo = JSON.parseObject(message, SearchArticleVo.class);
            IndexRequest indexRequest = new IndexRequest("app_info_article");
            indexRequest.id(vo.getId().toString());
            indexRequest.source(message, XContentType.JSON);
            try {
                restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
            } catch (IOException e) {
                log.error("sync es error={}", e);
                throw new RuntimeException(e);
            }

        }
    }

}

MongoDB

安装配置

shell 复制代码 
# 拉取镜像
docker pull mongo
# 创建容器
docker run -di --name mongo -p 27017:27017 -v ~/data/mongodata:/data mongo

SpringBoot集成MongoDB

  • 依赖

    xml 复制代码 
    <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>

  • 配置

    yml 复制代码 
    spring:
    data:
        mongodb:
            host: 192.168.174.133
            port: 27017
            database: leadnews-history

  • 使用

    java 复制代码 
    public class MongoTest {


    @Autowired
    private MongoTemplate mongoTemplate;

    //保存
    @Test
    public void saveTest(){
        ApAssociateWords apAssociateWords = new ApAssociateWords();
        apAssociateWords.setAssociateWords("黑马头条");
        apAssociateWords.setCreatedTime(new Date());
        mongoTemplate.save(apAssociateWords);
    }

    //查询一个
    @Test
    public void saveFindOne(){
        ApAssociateWords apAssociateWords = mongoTemplate.findById("64e46fd4f3a760442bf50527", ApAssociateWords.class);
        System.out.println(apAssociateWords);
    }

    //条件查询
    @Test
    public void testQuery(){
        Query query = Query.query(Criteria.where("associateWords").is("黑马头条"))
                .with(Sort.by(Sort.Direction.DESC,"createdTime"));
        List<ApAssociateWords> apAssociateWordsList = mongoTemplate.find(query, ApAssociateWords.class);
        System.out.println(apAssociateWordsList);
    }

    @Test
    public void testDel(){
        mongoTemplate.remove(Query.query(Criteria.where("associateWords").is("黑马头条")),ApAssociateWords.class);
    }
}

app文章搜索记录

保存搜索记录

思路分析

异步保存

保存数据

具体实现

java 复制代码 
public class ApUserSearchServiceImpl implements ApUserSearchService {

    @Autowired
    private MongoTemplate mongoTemplate;

    /**
     * 保存用户搜索记录
     * @param keyword
     * @param userId
     */
    @Override
    @Async
    public void insert(String keyword, Integer userId) {
        // 1. 查询当前用户的搜索关键词
        Query query = Query.query(Criteria.where("userId").is(userId).and("keyword").is(keyword));
        ApUserSearch apUserSearch = mongoTemplate.findOne(query, ApUserSearch.class);

        // 2. 存在, 更新创建时间
        if(apUserSearch != null){
            apUserSearch.setCreatedTime(new Date());
            mongoTemplate.save(apUserSearch);
            return;
        }

        // 3. 不存在, 判断当前历史记录总数量是否超过10
        apUserSearch = new ApUserSearch();
        apUserSearch.setUserId(userId);
        apUserSearch.setKeyword(keyword);
        apUserSearch.setCreatedTime(new Date());

        Query query1 = Query.query(Criteria.where("userId").is(userId));
        query1.with(Sort.by(Sort.Direction.DESC, "createTime"));
        List<ApUserSearch> apUserSearchList = mongoTemplate.find(query1, ApUserSearch.class);

        if(apUserSearchList == null || apUserSearchList.size() < 10){
            mongoTemplate.save(apUserSearch);
        }else{
            ApUserSearch lastUserSearch = apUserSearchList.get(apUserSearchList.size() - 1);
            mongoTemplate.findAndReplace(Query.query(Criteria.where("id").is(lastUserSearch.getId())), apUserSearch);
        }
    }
}

在文章搜索中调用, 因为未登录也可能有token(游客), 所以需要判断是否登录.

java 复制代码 
ApUser user = AppThreadLocalUtil.getUser();
// 异步调用, 保存搜索记录
if(user!=null && dto.getFromIndex() == 0){
    apUserSearchService.insert(dto.getSearchWords(), user.getId());
}

查询搜索历史

java 复制代码 
public ResponseResult findUserSearch() {
    // 获取当前用户
    ApUser user = AppThreadLocalUtil.getUser();
    if(user == null){
        return ResponseResult.errorResult(AppHttpCodeEnum.NEED_LOGIN);
    }
    // 根据用户查询数据, 根据时间排序
    Query query = Query.query(Criteria.where("userId").is(user.getId()));
    query.with(Sort.by(Sort.Direction.DESC, "createdTime"));
    List<ApUserSearch> apUserSearchList = mongoTemplate.find(query, ApUserSearch.class);
    return ResponseResult.okResult(apUserSearchList);
}

删除搜索历史

java 复制代码 
public ResponseResult delUserSearch(HistorySearchDto dto) {
    // 检查参数
    if(dto.getId() == null){
        return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);
    }
    // 获取当前用户
    ApUser user = AppThreadLocalUtil.getUser();
    if(user == null){
        return ResponseResult.errorResult(AppHttpCodeEnum.NEED_LOGIN);
    }
    // 删除
    Query query = Query.query(Criteria.where("id").is(dto.getId()).and("userId").is(user.getId()));
    mongoTemplate.remove(query, ApUserSearch.class);
    return ResponseResult.okResult(AppHttpCodeEnum.SUCCESS);
}

搜索联想词

java 复制代码 
public class ApAssociateWordsServiceImpl implements ApAssociateWordsService {

    @Autowired
    private MongoTemplate mongoTemplate;

    /**
     * 搜索联想词
     * @param dto
     * @return
     */
    @Override
    public ResponseResult search(UserSearchDto dto) {
        // 1. 检查参数
        if(StringUtils.isBlank(dto.getSearchWords())){
            return ResponseResult.errorResult(AppHttpCodeEnum.PARAM_INVALID);
        }
        // 2. 分页检索
        if(dto.getPageSize() > 20){
            dto.setPageSize(20);
        }
        // 3. 执行查询, 模糊查询
        Query query = Query.query(Criteria.where("associateWords").regex(".*?\\" + dto.getSearchWords() + ".*"));
        query.limit(dto.getPageSize());
        List<ApAssociateWords> apAssociateWordsList = mongoTemplate.find(query, ApAssociateWords.class);
        return ResponseResult.okResult(apAssociateWordsList);
    }
}

来源

黑马程序员. 黑马头条

Gitee

https://gitee.com/yu-ba-ba-ba/leadnews

相关推荐
Sam-August27 分钟前
【分布式架构实战】Spring Cloud 与 Dubbo 深度对比：从架构到实战，谁才是微服务的王者？
java·spring cloud·dubbo
麦兜*36 分钟前
MongoDB 常见错误解决方案：从连接失败到主从同步问题
java·数据库·spring boot·redis·mongodb·容器
ytadpole1 小时前
揭秘设计模式：命令模式-告别混乱，打造优雅可扩展的代码
java·设计模式
计算机学姐1 小时前
基于Python的旅游数据分析可视化系统【2026最新】
vue.js·后端·python·数据分析·django·flask·旅游
用户3721574261351 小时前
Java 教程：轻松实现 Excel 与 CSV 互转 （含批量转换）
java
清风6666661 小时前
基于51单片机手机无线蓝牙APP控制风扇调速设计
单片机·mongodb·智能手机·毕业设计·51单片机·课程设计
Hello.Reader1 小时前
Kafka 设计与实现动机、持久化、效率、生产者/消费者、事务、复制、日志压缩与配额
分布式·kafka
叫我阿柒啊2 小时前
Java全栈开发实战：从基础到微服务的深度解析
java·微服务·kafka·vue3·springboot·jwt·前端开发
该用户已不存在2 小时前
你没有听说过的7个Windows开发必备工具
前端·windows·后端
凯尔萨厮2 小时前
Java学习笔记三（封装）
java·笔记·学习
热门推荐
01UV安装并设置国内源02A股预测还能更准？开源大模型Kronos带你跑通预测+回测全流程03解决 WSL Ubuntu 中 /etc/resolv.conf 自动重置问题04UV 工具安装与国内镜像源配置指南052025 年高教社杯全国大学生数学建模竞赛C 题 NIPT 的时点选择与胎儿的异常判定 完整成品思路模型代码分享，全网首发高质量！！！06教你如何认证 Gemini 教育优惠的二次验证，薅个 1年的 Gemini Pro 会员07KGG转MP3工具|非KGM文件|解密音频082025年数学建模国赛C题超详细解题思路09突破百度网盘的下载限速，两种方法教会你【超详细】10Linux下V2Ray安装配置指南