CoAlbum 引入ES

1.部署

[ES 部署](#ES 部署)

[Kibana 部署](#Kibana 部署)

2.Mapping

[3.集成 ES](#3.集成 ES)

[冷启动图片库](#冷启动图片库)

1.部署

ES 部署

jdk 11 选择 Elasticsearch 7.17.x（LTS）

镜像

bash 复制代码

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.15

容器

bash 复制代码

mkdir -p /mydata/elasticsearch-7.17.15/{config,data,plugins}

echo "http.host: 0.0.0.0" > /mydata/elasticsearch-7.17.15/config/elasticsearch.yml
chmod -R 777 /mydata/elasticsearch-7.17.15

docker run --name elasticsearch-7.17.15 -d \
  -p 9200:9200 -p 9300:9300 \
  -e discovery.type=single-node \
  -e ES_JAVA_OPTS="-Xms256m -Xmx512m" \
  -v /mydata/elasticsearch-7.17.15/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
  -v /mydata/elasticsearch-7.17.15/data:/usr/share/elasticsearch/data \
  -v /mydata/elasticsearch-7.17.15/plugins:/usr/share/elasticsearch/plugins \
  --restart=unless-stopped \
  docker.elastic.co/elasticsearch/elasticsearch:7.17.15

http.host: 0.0.0.0（ Elasticsearch 的配置项，表示允许通过任意 IP 访问其 HTTP API，默认只监听 127.0.0.1）

对挂载根目录赋权（开发机可用：chmod -R 777）。不赋权，常见报错为 AccessDeniedException（例如无法在 /usr/share/elasticsearch/data/nodes 下创建目录）

ES部署完成，访问端口

Kibana 部署

bash 复制代码

docker pull kibana:7.17.15

docker run --name kibana-7.17.15 \
-e ELASTICSEARCH_HOSTS=http://192.168.40.128:9200 \
-p 5601:5601 \
-d kibana:7.17.15

2.Mapping

javascript 复制代码

{
  "mappings": {
    "properties": {
      "id": {
        "type": "long"
      },
      "url": {
        "type": "keyword"
      },
      "name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "introduction": {
        "type": "text"
      },
      "tags": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "category": {
        "type": "keyword"
      },
      "picSize": {
        "type": "long"
      },
      "picWidth": {
        "type": "integer"
      },
      "picHeight": {
        "type": "integer"
      },
      "picScale": {
        "type": "double"
      },
      "picFormat": {
        "type": "keyword"
      },
      "userId": {
        "type": "long"
      },
      "editTime": {
        "type": "date",
        "format": "strict_date_optional_time||epoch_millis"
      },
      "spaceId": {
        "type": "long"
      }
    }
  }
}

3.集成 ES

引入依赖

bash 复制代码

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.17.15</version>
</dependency>

配置客户端

java 复制代码

@Configuration
public class ElasticSearchConfig {
    @Value("${search.host.address}")
    private String hostName;
    public static final RequestOptions COMMON_OPTIONS;
    static {
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
//        builder.addHeader("Authorization", "Bearer " + TOKEN);
//        builder.setHttpAsyncResponseConsumerFactory(
//                new HttpAsyncResponseConsumerFactory
//                        .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    }
    @Bean
    public RestHighLevelClient esRestClient() {
        System.out.println(hostName);
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost(hostName, 9200, "http")
                )
        );
        return client;
    }
}

4.API

分析标签

DSL语句

javascript 复制代码

{
  "size": 0,
  "query": {
    "bool": {
      "must_not": [
        {
          "exists": {
            "field": "spaceId",
            "boost": 1.0
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1.0
    }
  },
  "aggregations": {
    "all_tags": {
      "terms": {
        "field": "tags.keyword",
        "size": 10,
        "min_doc_count": 1,
        "shard_min_doc_count": 0,
        "show_term_doc_count_error": false,
        "order": [
          {
            "_count": "desc"
          },
          {
            "_key": "asc"
          }
        ]
      }
    }
  }
}

代码实现

java 复制代码

    public List<SpaceTagAnalyzeResponse> getTagList(int size,Long spaceId) throws IOException {
        SearchRequest searchRequest = new SearchRequest(EsConstant.PICTURE_INDEX);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        if(spaceId!=null){
            sourceBuilder.query(QueryBuilders.matchQuery("spaceId",spaceId));
        }else{
            //公共空间，查询不存在spaceId的doc
            sourceBuilder.query(QueryBuilders.boolQuery().mustNot(QueryBuilders.existsQuery("spaceId")));
        }
        TermsAggregationBuilder allTags = AggregationBuilders.terms("all_tags").field("tags.keyword").size(size);
        sourceBuilder.aggregation(allTags);
        // 不返回 hits，只返回聚合
        sourceBuilder.size(0);
        searchRequest.source(sourceBuilder);
        System.out.println("DSL:");
        System.out.println(sourceBuilder.toString());
        // 执行查询
        SearchResponse response = esRestClient.search(searchRequest, ElasticSearchConfig.COMMON_OPTIONS);
        // 解析聚合结果
        Terms terms = response.getAggregations().get("all_tags");
        List<SpaceTagAnalyzeResponse> collect = terms.getBuckets().stream().map(bucket -> {
            SpaceTagAnalyzeResponse tag = new SpaceTagAnalyzeResponse();
            tag.setTag(bucket.getKeyAsString());
            tag.setCount(bucket.getDocCount());
            return tag;
        }).collect(Collectors.toList());
        return collect;
    }

批量上传图片

java 复制代码

    @Override
    public boolean bulkSavePicturesToEs(List<PictureVO> pictures) throws IOException {
        //bulk 批量操作
        BulkRequest bulkRequest=new BulkRequest();
        for (PictureVO pictureVO : pictures) {
            //保存在哪个index下
            IndexRequest indexRequest = new IndexRequest(EsConstant.PICTURE_INDEX);
            //设置id
            indexRequest.id(pictureVO.getId().toString());
            //将sku的检索信息 转为 json
            String picJson= JSONUtil.toJsonStr(pictureVO);
            //设置数据元 并指定 数据源格式
            indexRequest.source(picJson, XContentType.JSON);
            //将 索引请求 加入到 bulk中
            bulkRequest.add(indexRequest);
        }
        //执行 bulk
        BulkResponse bulkResponse = esRestClient.bulk(bulkRequest, ElasticSearchConfig.COMMON_OPTIONS);

        boolean hasFailure = bulkResponse.hasFailures();
        List<String> collect = Arrays.stream(bulkResponse.getItems()).map(item -> {
            return item.getId();
        }).collect(Collectors.toList());
        if(hasFailure){
            log.error("部分图片保存至es失败，失败Id:{}",collect);
        }
        return ! hasFailure;
    }

冷启动图片库

将数据库中审核通过的所有图片索引到 ES 中

java 复制代码

    @Override
    public boolean saveAllPassedPic() throws IOException {
        List<Picture> pictures = pictureService.list();
        //审核通过
        List<PictureVO> collect = pictures.stream()
                .filter(picture -> {return picture.getReviewStatus()==1;})
                .map(PictureVO::objToVo).collect(Collectors.toList());
        return this.bulkSavePicturesToEs(collect);
    }

图片搜索

java 复制代码

    /**
     * ES作为数据源
     */
    @PostMapping("/list/page/vo/search")
    public BaseResponse<Page<PictureVO>> listPictureVOByPageFromES(@RequestBody PictureQueryRequest pictureQueryRequest,
                                                                      HttpServletRequest request) {

        try {
            return ResultUtils.success(pictureSearchService.searchPicture(pictureQueryRequest, request));
        } catch (IOException e) {
            throw new BusinessException(ErrorCode.SYSTEM_ERROR);
        }
    }

关键词：同时搜索名称和简介
标签
分类
编辑时间（开始时间与结束时间）
图片名称
图片宽度
图片高度
图片格式
分页

java 复制代码

    @Override
    public Page<PictureVO> searchPicture(PictureQueryRequest pictureQueryRequest, HttpServletRequest request) throws IOException {
        // 限制爬虫
        ThrowUtils.throwIf(pictureQueryRequest.getPageSize() > 20, ErrorCode.PARAMS_ERROR);
        // === 构建查询 ===
        SearchRequest searchRequest = new SearchRequest(EsConstant.PICTURE_INDEX);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        // 1. 关键词搜索：name 和 introduction
        if(StrUtil.isNotBlank(pictureQueryRequest.getSearchText())){
            MultiMatchQueryBuilder multiMatchQuery = QueryBuilders.multiMatchQuery(pictureQueryRequest.getSearchText(), "name", "introduction")
                    .type(MultiMatchQueryBuilder.Type.BEST_FIELDS);
            boolQuery.must(multiMatchQuery);
        }
        // 2. 分类精确匹配
        if(StrUtil.isNotBlank(pictureQueryRequest.getCategory())){
            boolQuery.must(QueryBuilders.termQuery("category", pictureQueryRequest.getCategory()));
        }
        // 3. 编辑时间范围
        boolQuery.must(QueryBuilders.rangeQuery("editTime")
                .gte(pictureQueryRequest.getStartEditTime())
                .lte(pictureQueryRequest.getEndEditTime()));

        // 4. 标签 AND 逻辑：必须包含所有 mustTags 中的标签
        if(pictureQueryRequest.getTags()!=null){
            for (String tag : pictureQueryRequest.getTags()) {
                boolQuery.must(QueryBuilders.termQuery("tags.keyword", tag));
            }
        }
        //TODO 图片宽高条件筛选
        // 格式筛选
        
        // 空间权限校验
        Long spaceId = pictureQueryRequest.getSpaceId();
        if (spaceId == null) {
            // 公开图库
            // 普通用户默认只能看到审核通过的数据
            boolQuery.mustNot(QueryBuilders.existsQuery("spaceId"));
        } else {
            // 私有空间
            User loginUser = userService.getLoginUser(request);
            Space space = spaceService.getById(spaceId);
            ThrowUtils.throwIf(space == null, ErrorCode.NOT_FOUND_ERROR, "空间不存在");
            if (!loginUser.getId().equals(space.getUserId())) {
                throw new BusinessException(ErrorCode.NO_AUTH_ERROR, "没有空间权限");
            }
            boolQuery.must(QueryBuilders.termQuery("spaceId",spaceId));
        }

        sourceBuilder.query(boolQuery);

        // 5. 分页
        int from = (pictureQueryRequest.getCurrent() - 1) * pictureQueryRequest.getPageSize();
        sourceBuilder.from(from);
        sourceBuilder.size(pictureQueryRequest.getPageSize());

        // 6. 排序（可选）
        if (StrUtil.isNotBlank(pictureQueryRequest.getSortField())){
            sourceBuilder.sort(pictureQueryRequest.getSortField(),
                    "descend".equals(pictureQueryRequest.getSortOrder())? SortOrder.DESC:SortOrder.ASC);
        }
        searchRequest.source(sourceBuilder);

        // 7. 执行查询
        SearchResponse response = esRestClient.search(searchRequest,ElasticSearchConfig.COMMON_OPTIONS);
        System.out.println(sourceBuilder.toString());

        // 8. 处理结果
        return buildSearchResult(response,pictureQueryRequest.getCurrent(),pictureQueryRequest.getPageSize());
    }

    private Page<PictureVO> buildSearchResult(SearchResponse searchResponse,int pageNum,int pageSize){
        // 1. 获取总记录数
        long totalHits = searchResponse.getHits().getTotalHits().value;

        // 2. 获取当前页的搜索结果（SearchHit 数组）
        SearchHits hits = searchResponse.getHits();
        SearchHit[] searchHits = hits.getHits();

        // 3. 将 SearchHit 转为 PictureVO 列表
        List<PictureVO> pictureVOList = new ArrayList<>();
        for (SearchHit hit : searchHits) {
            PictureVO pictureVO = JSON.parseObject(hit.getSourceAsString(), PictureVO.class);
            pictureVOList.add(pictureVO);
        }

        // 4. 构造 MyBatis-Plus 的 Page 对象
        Page<PictureVO> page = new Page<>(pageNum, pageSize, totalHits);
        page.setRecords(pictureVOList);

        return page;
    }

1.部署

ES 部署

Kibana 部署

2.Mapping

3.集成 ES

4.API

分析标签

批量上传图片

冷启动 图片库

图片搜索

冷启动图片库