
👋 大家好,欢迎来到我的技术博客!
💻 作为一名热爱 Java 与软件开发的程序员,我始终相信:清晰的逻辑 + 持续的积累 = 稳健的成长 。
📚 在这里,我会分享学习笔记、实战经验与技术思考,力求用简单的方式讲清楚复杂的问题。
🎯 本文将围绕Gradle 这个话题展开,希望能为你带来一些启发或实用的参考。
🌱 无论你是刚入门的新手,还是正在进阶的开发者,希望你都能有所收获!
文章目录
- [Gradle - 与 Elasticsearch 集成:构建搜索服务项目 🚀](#Gradle - 与 Elasticsearch 集成:构建搜索服务项目 🚀)
-
- [为什么需要 Elasticsearch? 🤔](#为什么需要 Elasticsearch? 🤔)
- [Elasticsearch 在 Gradle 项目中的角色 🧠](#Elasticsearch 在 Gradle 项目中的角色 🧠)
-
- [如何在 Gradle 中集成 Elasticsearch?](#如何在 Gradle 中集成 Elasticsearch?)
- [实现方案概览 🛠️](#实现方案概览 🛠️)
- [环境准备 🧰](#环境准备 🧰)
- [创建基础 Gradle 项目 🧱](#创建基础 Gradle 项目 🧱)
- [配置 Docker Compose 🐳](#配置 Docker Compose 🐳)
-
- `docker-compose.yml`
- [Elasticsearch 配置文件 (可选)](#Elasticsearch 配置文件 (可选))
- [编写应用程序代码 🧾](#编写应用程序代码 🧾)
-
- 主应用入口
- 数据模型 (Model)
- 评论模型 (Review)
- [Elasticsearch Repository (使用 Spring Data Elasticsearch)](#Elasticsearch Repository (使用 Spring Data Elasticsearch))
- [Service 层](#Service 层)
- [Controller 层](#Controller 层)
- 配置文件
- [Elasticsearch Settings 文件 (可选)](#Elasticsearch Settings 文件 (可选))
- [运行和测试 🧪](#运行和测试 🧪)
- [编写集成测试 🧪](#编写集成测试 🧪)
- [高级优化与最佳实践 🌟](#高级优化与最佳实践 🌟)
-
- [1. 配置优化](#1. 配置优化)
- [2. 性能调优](#2. 性能调优)
- [3. 安全性](#3. 安全性)
- [4. 监控与维护](#4. 监控与维护)
- [5. 集成与部署](#5. 集成与部署)
- [与其他技术栈的集成 🔗](#与其他技术栈的集成 🔗)
-
- [与 Spring Boot 集成](#与 Spring Boot 集成)
- [与 React/Vue/Angular 前端集成](#与 React/Vue/Angular 前端集成)
- [与 Kafka 集成](#与 Kafka 集成)
- [结论 🏁](#结论 🏁)
Gradle - 与 Elasticsearch 集成:构建搜索服务项目 🚀
在当今数据驱动的世界里,强大的搜索能力是许多应用程序的核心需求。无论是电商平台的商品搜索、内容管理系统的信息检索,还是数据分析平台的实时查询,高效的搜索引擎都能极大地提升用户体验和业务价值。Elasticsearch 作为一款基于 Apache Lucene 构建的开源、分布式、RESTful 搜索和分析引擎,凭借其高可扩展性、实时性以及丰富的功能,成为了构建搜索服务的首选技术栈之一。
本文将深入探讨如何将 Gradle 构建工具与 Elasticsearch 集成,以支持和加速搜索服务项目的开发、构建和部署。我们将从基础概念出发,逐步介绍如何在 Gradle 项目中配置 Elasticsearch 依赖、构建和运行 Elasticsearch 实例、执行集成测试以及自动化部署流程。通过详细的代码示例和实际应用场景,帮助你掌握这一强大的组合。
为什么需要 Elasticsearch? 🤔
在传统的数据库系统中,虽然可以通过 SQL 查询进行数据检索,但面对复杂的全文搜索、模糊匹配、相关性排序、聚合分析等需求时,其性能和灵活性往往显得不足。Elasticsearch 专门针对这些问题进行了优化:
- 全文搜索:支持复杂的文本分析和分词,提供强大的全文检索能力。
- 实时性:数据一旦索引,即可被快速检索,几乎实现实时搜索。
- 分布式架构:天然支持水平扩展,能够处理 PB 级别的数据。
- 丰富的 API:提供 RESTful API,易于集成到各种应用中。
- 强大的聚合分析:支持多种聚合操作,用于数据分析和报表生成。
对于需要构建搜索服务的应用程序来说,Elasticsearch 提供了一个成熟且高效的解决方案。
Elasticsearch 在 Gradle 项目中的角色 🧠
在 Gradle 项目中,Elasticsearch 主要扮演以下几个角色:
- 开发环境支持:提供本地运行的 Elasticsearch 实例,方便开发者在本地调试和测试。
- 测试环境:在单元测试或集成测试中,提供一个干净、隔离的 Elasticsearch 环境。
- 构建过程:在构建过程中,可能需要将数据导入到 Elasticsearch,或者执行特定的索引管理任务。
- 部署支持:在 CI/CD 流程中,协助配置和部署 Elasticsearch 集群。
如何在 Gradle 中集成 Elasticsearch?
虽然 Gradle 本身并不直接提供与 Elasticsearch 的集成,但我们可以借助 Gradle 的强大功能和社区提供的插件来实现。常见的做法包括:
- 使用插件 :利用如
com.github.davidmc24.gradle.plugin.avaje或org.elasticsearch.gradle等第三方 Gradle 插件来简化 Elasticsearch 的配置和管理。 - 自定义任务:编写自定义的 Gradle 任务来启动、停止、管理 Elasticsearch 实例或执行数据导入。
- 使用 Docker:通过 Gradle 任务启动 Docker 容器中的 Elasticsearch 实例,这是一种非常流行且灵活的方法。
实现方案概览 🛠️
本文将重点介绍 使用 Docker 容器化方式集成 Elasticsearch 的方法,因为它具有以下优势:
- 环境一致性:确保开发、测试和生产环境使用相同版本的 Elasticsearch。
- 易于部署:Docker 镜像可以轻松地打包和部署到任何支持 Docker 的环境中。
- 资源隔离:容器化确保了 Elasticsearch 实例与主机系统的隔离。
- 快速启动:Docker 能够快速拉取和启动 Elasticsearch 实例。
我们将通过以下步骤实现集成:
- 准备 Gradle 项目结构。
- 配置 Docker Compose:定义 Elasticsearch 服务及其配置。
- 创建 Gradle 任务:用于启动、停止和管理 Elasticsearch 容器。
- 编写集成测试:验证与 Elasticsearch 的交互。
- 构建和部署:将项目打包并部署到目标环境。
环境准备 🧰
在开始之前,请确保你已经安装了以下工具:
- Java JDK:Gradle 项目通常需要 Java。确保已安装 JDK 11 或更高版本。
- Gradle:安装 Gradle 构建工具。
- Docker :安装 Docker 引擎,用于运行 Elasticsearch 容器。Docker 官方网站
- Docker Compose :用于定义和运行多容器 Docker 应用。通常随 Docker Desktop 一起安装。Docker Compose 文档
创建基础 Gradle 项目 🧱
首先,创建一个新的 Gradle 项目:
bash
mkdir elasticsearch-search-service
cd elasticsearch-search-service
gradle init --type java-application
这将创建一个基本的 Java 应用程序结构。修改 build.gradle 文件,添加必要的依赖和插件:
build.gradle
groovy
plugins {
id 'java'
id 'application'
}
group = 'com.example'
version = '0.0.1-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
// Core application dependencies
implementation 'org.springframework.boot:spring-boot-starter-web:3.2.0' // Using Spring Boot for simplicity
implementation 'org.springframework.boot:spring-boot-starter-data-elasticsearch:3.2.0'
implementation 'org.elasticsearch.client:elasticsearch-java:8.11.3' // Elasticsearch Java Client
implementation 'co.elastic.clients:elasticsearch-java:8.11.3' // Alternative Java Client (from Elastic)
implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
// Testing dependencies
testImplementation 'org.springframework.boot:spring-boot-starter-test:3.2.0'
testImplementation 'org.testcontainers:testcontainers:1.19.3' // For testing with containers
testImplementation 'org.testcontainers:elasticsearch:1.19.3' // Testcontainers for Elasticsearch
testImplementation 'org.junit.jupiter:junit-jupiter:5.10.0'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
// Configure the main class for the application
application {
mainClass = 'com.example.search.SearchServiceApplication'
}
// Define a custom task to start Elasticsearch via Docker Compose
task startElasticsearch(type: Exec) {
commandLine 'docker-compose', 'up', '-d', 'elasticsearch'
workingDir '.'
dependsOn 'build'
onlyIf { !isElasticsearchRunning() }
}
// Define a custom task to stop Elasticsearch via Docker Compose
task stopElasticsearch(type: Exec) {
commandLine 'docker-compose', 'down', '-v'
workingDir '.'
onlyIf { isElasticsearchRunning() }
}
// Helper function to check if Elasticsearch container is running
def isElasticsearchRunning() {
try {
def result = exec {
commandLine 'docker', 'ps', '--format', '{{.Names}}'
ignoreExitCode = true
standardOutput = new ByteArrayOutputStream()
}
def output = result.standardOutput.toString()
return output.contains('elasticsearch-search-service_elasticsearch_1') // Adjust name based on your compose service name
} catch (Exception e) {
return false
}
}
// Ensure that tests run after Elasticsearch is started
test {
dependsOn startElasticsearch
finalizedBy stopElasticsearch // Stop ES after tests
// Configure test environment variables if needed
environment 'ELASTICSEARCH_HOST', 'localhost'
environment 'ELASTICSEARCH_PORT', '9200'
}
// Add a task to wait for Elasticsearch to be ready
task waitForElasticsearch {
doLast {
def maxRetries = 30
def retryCount = 0
def ready = false
while (!ready && retryCount < maxRetries) {
try {
def url = new URL("http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=1s")
def connection = url.openConnection()
connection.connectTimeout = 1000
connection.readTimeout = 1000
def responseCode = connection.responseCode
if (responseCode == 200) {
ready = true
println "Elasticsearch is ready!"
} else {
println "Elasticsearch not ready yet. Response Code: $responseCode"
}
} catch (Exception e) {
println "Elasticsearch not ready yet. Error: ${e.message}"
}
if (!ready) {
Thread.sleep(1000)
retryCount++
}
}
if (!ready) {
throw new RuntimeException("Elasticsearch did not become ready within $maxRetries seconds")
}
}
}
// Make waitForElasticsearch a dependency of startElasticsearch
startElasticsearch.dependsOn waitForElasticsearch
配置 Docker Compose 🐳
接下来,创建 docker-compose.yml 文件,定义 Elasticsearch 服务及其配置:
docker-compose.yml
yaml
version: '3.8'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.3 # Use the appropriate version
container_name: elasticsearch-search-service_elasticsearch_1 # Ensure consistent naming
ports:
- "9200:9200" # HTTP port
- "9300:9300" # Transport port (internal communication)
environment:
- discovery.type=single-node # Single node cluster for development
- xpack.security.enabled=false # Disable security for simplicity (NOT recommended for production)
- ELASTIC_USERNAME=elastic # Set username (required if security enabled)
- ELASTIC_PASSWORD=changeme # Set password (required if security enabled)
- ES_JAVA_OPTS=-Xms1g -Xmx1g # JVM heap size
volumes:
- esdata:/usr/share/elasticsearch/data # Persistent volume for data
- ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml # Custom config (optional)
networks:
- elastic-network
volumes:
esdata: # Named volume for Elasticsearch data
networks:
elastic-network:
driver: bridge # Bridge network for container communication
Elasticsearch 配置文件 (可选)
创建 elasticsearch/config/elasticsearch.yml 文件,用于自定义 Elasticsearch 配置(例如,调整日志级别、设置网络绑定地址等):
yaml
# elasticsearch.yml
# Cluster name
cluster.name: search-service-cluster
# Node name (automatically assigned if not specified)
node.name: elasticsearch-node-1
# Network settings
network.host: 0.0.0.0
http.port: 9200
# Discovery settings (for single node)
discovery.type: single-node
# Security settings (disable for dev, enable for prod)
xpack.security.enabled: false
# Heap size (adjust according to your system)
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
# Path for data storage (already configured via docker-compose)
# path.data: /usr/share/elasticsearch/data
# Path for logs (already configured via docker-compose)
# path.logs: /usr/share/elasticsearch/logs
编写应用程序代码 🧾
主应用入口
创建 src/main/java/com/example/search/SearchServiceApplication.java:
java
package com.example.search;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class SearchServiceApplication {
public static void main(String[] args) {
SpringApplication.run(SearchServiceApplication.class, args);
}
}
数据模型 (Model)
创建 src/main/java/com/example/search/model/Product.java:
java
package com.example.search.model;
import co.elastic.clients.elasticsearch.core.search.Hit;
import co.elastic.clients.json.JsonData;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import org.springframework.data.elasticsearch.annotations.GeoPointField;
import org.springframework.data.elasticsearch.annotations.Setting;
import java.util.List;
@Document(indexName = "products")
@Setting(settingPath = "/elasticsearch/settings/product-settings.json") // Optional: Custom index settings
public class Product {
@Id
private String id;
@Field(type = FieldType.Text, analyzer = "standard", searchAnalyzer = "standard")
private String name;
@Field(type = FieldType.Text, analyzer = "standard", searchAnalyzer = "standard")
private String description;
@Field(type = FieldType.Double)
private Double price;
@Field(type = FieldType.Keyword)
private String category;
@Field(type = FieldType.Date)
private String releaseDate;
@Field(type = FieldType.Nested)
private List<Review> reviews;
@GeoPointField
private String location; // Geo-point field for geospatial queries
// Constructors
public Product() {}
public Product(String id, String name, String description, Double price, String category, String releaseDate, List<Review> reviews, String location) {
this.id = id;
this.name = name;
this.description = description;
this.price = price;
this.category = category;
this.releaseDate = releaseDate;
this.reviews = reviews;
this.location = location;
}
// Getters and Setters
public String getId() { return id; }
public void setId(String id) { this.id = id; }
public String getName() { return name; }
public void setName(String name) { this.name = name; }
public String getDescription() { return description; }
public void setDescription(String description) { this.description = description; }
public Double getPrice() { return price; }
public void setPrice(Double price) { this.price = price; }
public String getCategory() { return category; }
public void setCategory(String category) { this.category = category; }
public String getReleaseDate() { return releaseDate; }
public void setReleaseDate(String releaseDate) { this.releaseDate = releaseDate; }
public List<Review> getReviews() { return reviews; }
public void setReviews(List<Review> reviews) { this.reviews = reviews; }
public String getLocation() { return location; }
public void setLocation(String location) { this.location = location; }
@Override
public String toString() {
return "Product{" +
"id='" + id + '\'' +
", name='" + name + '\'' +
", description='" + description + '\'' +
", price=" + price +
", category='" + category + '\'' +
", releaseDate='" + releaseDate + '\'' +
", reviews=" + reviews +
", location='" + location + '\'' +
'}';
}
}
评论模型 (Review)
创建 src/main/java/com/example/search/model/Review.java:
java
package com.example.search.model;
import co.elastic.clients.elasticsearch.core.search.Hit;
import co.elastic.clients.json.JsonData;
public class Review {
private String reviewerName;
private Integer rating; // 1-5 scale
private String comment;
// Constructors
public Review() {}
public Review(String reviewerName, Integer rating, String comment) {
this.reviewerName = reviewerName;
this.rating = rating;
this.comment = comment;
}
// Getters and Setters
public String getReviewerName() { return reviewerName; }
public void setReviewerName(String reviewerName) { this.reviewerName = reviewerName; }
public Integer getRating() { return rating; }
public void setRating(Integer rating) { this.rating = rating; }
public String getComment() { return comment; }
public void setComment(String comment) { this.comment = comment; }
@Override
public String toString() {
return "Review{" +
"reviewerName='" + reviewerName + '\'' +
", rating=" + rating +
", comment='" + comment + '\'' +
'}';
}
}
Elasticsearch Repository (使用 Spring Data Elasticsearch)
创建 src/main/java/com/example/search/repository/ProductRepository.java:
java
package com.example.search.repository;
import com.example.search.model.Product;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.annotations.Query;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;
import java.util.List;
import java.util.Optional;
@Repository
public interface ProductRepository extends ElasticsearchRepository<Product, String> {
// Custom query methods can be added here
// For example, find products by category
List<Product> findByCategory(String category);
// Find products with price range
List<Product> findByPriceBetween(Double minPrice, Double maxPrice);
// Find products by name (using match query)
@Query("{\"match\": {\"name\": \"?0\"}}")
Page<Product> findByName(String name, Pageable pageable);
// Find products by name with fuzzy search
@Query("{\"fuzzy\": {\"name\": {\"value\": \"?0\", \"fuzziness\": \"AUTO\"}}}")
Page<Product> findByNameFuzzy(String name, Pageable pageable);
// Find products by description (using multi_match query)
@Query("{\"multi_match\": {\"query\": \"?0\", \"fields\": [\"description\"]}}")
Page<Product> findByDescription(String description, Pageable pageable);
// Find products by geo-distance (requires location field)
// @Query("{\"geo_distance\": {\"distance\": \"10km\", \"location\": {\"lat\": ?0, \"lon\": ?1}}}")
// Page<Product> findByLocationNear(Double lat, Double lon, Pageable pageable);
// Find products with nested review fields
@Query("{\"nested\": {\"path\": \"reviews\", \"query\": {\"range\": {\"reviews.rating\": {\"gte\": ?0}}}}}")
Page<Product> findByReviewsRatingGreaterThanEqual(Integer minRating, Pageable pageable);
// Find products sorted by price
Page<Product> findAllByOrderByPriceAsc(Pageable pageable);
// Find products sorted by price descending
Page<Product> findAllByOrderByPriceDesc(Pageable pageable);
// Find products by category and price range
List<Product> findByCategoryAndPriceBetween(String category, Double minPrice, Double maxPrice);
// Find product by id (inherited from ElasticsearchRepository)
Optional<Product> findById(String id);
// Save a single product
Product save(Product product);
// Save multiple products
Iterable<Product> saveAll(Iterable<Product> products);
// Delete a product by id
void deleteById(String id);
// Delete a product
void delete(Product product);
// Check if a product exists by id
boolean existsById(String id);
// Count total products
long count();
// Find all products (with pagination)
Page<Product> findAll(Pageable pageable);
}
Service 层
创建 src/main/java/com/example/search/service/ElasticsearchService.java:
java
package com.example.search.service;
import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch.core.*;
import co.elastic.clients.elasticsearch.core.bulk.BulkResponse;
import co.elastic.clients.elasticsearch.core.search.Hit;
import co.elastic.clients.elasticsearch.indices.CreateIndexRequest;
import co.elastic.clients.elasticsearch.indices.CreateIndexResponse;
import co.elastic.clients.elasticsearch.indices.DeleteIndexRequest;
import co.elastic.clients.elasticsearch.indices.GetIndexRequest;
import co.elastic.clients.elasticsearch.indices.GetIndexResponse;
import co.elastic.clients.elasticsearch.indices.PutMappingRequest;
import co.elastic.clients.elasticsearch.indices.PutMappingResponse;
import co.elastic.clients.json.JsonData;
import com.example.search.model.Product;
import com.example.search.model.Review;
import com.example.search.repository.ProductRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@Service
public class ElasticsearchService {
private final ElasticsearchClient elasticsearchClient;
private final ProductRepository productRepository;
@Value("${elasticsearch.index.name:products}") // Default index name
private String INDEX_NAME;
@Autowired
public ElasticsearchService(ElasticsearchClient elasticsearchClient, ProductRepository productRepository) {
this.elasticsearchClient = elasticsearchClient;
this.productRepository = productRepository;
}
/**
* Checks if the index exists
*/
public boolean indexExists() throws IOException {
GetIndexRequest request = new GetIndexRequest.Builder()
.index(INDEX_NAME)
.build();
GetIndexResponse response = elasticsearchClient.indices().get(request);
return response.result().containsKey(INDEX_NAME);
}
/**
* Creates the index with mapping if it doesn't exist
*/
public void createIndexIfNotExists() throws IOException {
if (!indexExists()) {
CreateIndexRequest request = new CreateIndexRequest.Builder()
.index(INDEX_NAME)
.build();
CreateIndexResponse response = elasticsearchClient.indices().create(request);
System.out.println("Created index: " + response.index());
}
}
/**
* Deletes the index
*/
public void deleteIndex() throws IOException {
DeleteIndexRequest request = new DeleteIndexRequest.Builder()
.index(INDEX_NAME)
.build();
elasticsearchClient.indices().delete(request);
System.out.println("Deleted index: " + INDEX_NAME);
}
/**
* Adds a single product to Elasticsearch
*/
public void addProduct(Product product) throws IOException {
IndexRequest<Product> request = new IndexRequest.Builder<Product>()
.index(INDEX_NAME)
.id(product.getId())
.document(product)
.build();
IndexResponse response = elasticsearchClient.index(request);
System.out.println("Indexed product with ID: " + response.id());
}
/**
* Adds multiple products to Elasticsearch
*/
public void addProducts(List<Product> products) throws IOException {
BulkRequest.Builder bulkBuilder = new BulkRequest.Builder();
for (Product product : products) {
bulkBuilder.operations(op -> op
.index(idx -> idx
.index(INDEX_NAME)
.id(product.getId())
.document(product)
)
);
}
BulkResponse response = elasticsearchClient.bulk(bulkBuilder.build());
if (response.errors()) {
System.err.println("Bulk indexing had errors:");
for (BulkResponseItem item : response.items()) {
if (item.error() != null) {
System.err.println("Error for document " + item.id() + ": " + item.error().reason());
}
}
} else {
System.out.println("Successfully indexed " + products.size() + " products.");
}
}
/**
* Updates an existing product
*/
public void updateProduct(String productId, Product updatedProduct) throws IOException {
UpdateRequest<Product, Product> request = new UpdateRequest.Builder<Product, Product>()
.index(INDEX_NAME)
.id(productId)
.doc(updatedProduct)
.build();
UpdateResponse<Product> response = elasticsearchClient.update(request, Product.class);
System.out.println("Updated product with ID: " + response.id());
}
/**
* Deletes a product by ID
*/
public void deleteProduct(String productId) throws IOException {
DeleteRequest request = new DeleteRequest.Builder()
.index(INDEX_NAME)
.id(productId)
.build();
DeleteResponse response = elasticsearchClient.delete(request);
System.out.println("Deleted product with ID: " + response.id());
}
/**
* Searches for products by name (exact match)
*/
public List<Product> searchProductsByName(String name) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.query(q -> q.match(m -> m.field("name").query(name)))
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Searches for products by name with fuzzy matching
*/
public List<Product> searchProductsByNameFuzzy(String name) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.query(q -> q.fuzzy(f -> f.field("name").value(name)))
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Searches for products by category
*/
public List<Product> searchProductsByCategory(String category) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.query(q -> q.term(t -> t.field("category").value(category)))
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Searches for products by price range
*/
public List<Product> searchProductsByPriceRange(Double minPrice, Double maxPrice) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.query(q -> q.range(r -> r.field("price").gte(minPrice).lte(maxPrice)))
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Performs a general search using a query string
*/
public List<Product> searchProducts(String queryString) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.query(q -> q.multiMatch(m -> m.query(queryString).fields("name", "description")))
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Gets a product by its ID
*/
public Product getProductById(String id) throws IOException {
GetRequest request = new GetRequest.Builder()
.index(INDEX_NAME)
.id(id)
.build();
GetResponse<Product> response = elasticsearchClient.get(request, Product.class);
return response.found() ? response.source() : null;
}
/**
* Gets all products (with pagination)
*/
public List<Product> getAllProducts(int from, int size) throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.from(from)
.size(size)
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
return response.hits().hits().stream()
.map(Hit::source)
.collect(Collectors.toList());
}
/**
* Counts the number of products
*/
public long countProducts() throws IOException {
CountRequest request = new CountRequest.Builder()
.index(INDEX_NAME)
.build();
CountResponse response = elasticsearchClient.count(request);
return response.count();
}
/**
* Performs a complex search with aggregations (e.g., average price by category)
*/
public Map<String, Double> getAveragePriceByCategory() throws IOException {
SearchRequest request = new SearchRequest.Builder()
.index(INDEX_NAME)
.aggregations("avg_price_by_category", agg -> agg
.terms(t -> t.field("category"))
.aggs("avg_price", a -> a.avg(v -> v.field("price")))
)
.build();
SearchResponse<Product> response = elasticsearchClient.search(request, Product.class);
// Extract aggregation results (simplified)
// In a real app, you'd parse the response more carefully
// For now, returning empty map as a placeholder
return Collections.emptyMap(); // Placeholder
}
/**
* Indexes sample products (for demo purposes)
*/
public void indexSampleProducts() throws IOException {
// Sample data
List<Product> sampleProducts = new ArrayList<>();
sampleProducts.add(new Product(
"1",
"Laptop XYZ",
"High-performance laptop with SSD storage.",
1200.0,
"Electronics",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.singletonList(new Review("Alice", 5, "Great laptop!")),
"40.7128,-74.0060" // New York City coordinates
));
sampleProducts.add(new Product(
"2",
"Smartphone ABC",
"Latest smartphone with advanced camera features.",
800.0,
"Electronics",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.singletonList(new Review("Bob", 4, "Good phone, battery life could be better.")),
"34.0522,-118.2437" // Los Angeles coordinates
));
sampleProducts.add(new Product(
"3",
"Coffee Mug",
"Ceramic coffee mug, perfect for morning brew.",
15.0,
"Home & Kitchen",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.singletonList(new Review("Charlie", 5, "Beautiful mug!")),
"51.5074,-0.1278" // London coordinates
));
addProducts(sampleProducts);
System.out.println("Indexed sample products.");
}
}
Controller 层
创建 src/main/java/com/example/search/controller/ProductController.java:
java
package com.example.search.controller;
import com.example.search.model.Product;
import com.example.search.service.ElasticsearchService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import java.io.IOException;
import java.util.List;
@RestController
@RequestMapping("/api/products")
public class ProductController {
private final ElasticsearchService elasticsearchService;
@Autowired
public ProductController(ElasticsearchService elasticsearchService) {
this.elasticsearchService = elasticsearchService;
}
/**
* Index a new product
*/
@PostMapping
public ResponseEntity<String> indexProduct(@RequestBody Product product) {
try {
elasticsearchService.addProduct(product);
return ResponseEntity.ok("Product indexed successfully.");
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Failed to index product: " + e.getMessage());
}
}
/**
* Search products by name
*/
@GetMapping("/search/name/{name}")
public ResponseEntity<List<Product>> searchByName(@PathVariable String name) {
try {
List<Product> products = elasticsearchService.searchProductsByName(name);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Search products by name with fuzzy matching
*/
@GetMapping("/search/name/fuzzy/{name}")
public ResponseEntity<List<Product>> searchByNameFuzzy(@PathVariable String name) {
try {
List<Product> products = elasticsearchService.searchProductsByNameFuzzy(name);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Search products by category
*/
@GetMapping("/search/category/{category}")
public ResponseEntity<List<Product>> searchByCategory(@PathVariable String category) {
try {
List<Product> products = elasticsearchService.searchProductsByCategory(category);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Search products by price range
*/
@GetMapping("/search/price")
public ResponseEntity<List<Product>> searchByPriceRange(
@RequestParam Double minPrice,
@RequestParam Double maxPrice) {
try {
List<Product> products = elasticsearchService.searchProductsByPriceRange(minPrice, maxPrice);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* General search
*/
@GetMapping("/search")
public ResponseEntity<List<Product>> search(@RequestParam String q) {
try {
List<Product> products = elasticsearchService.searchProducts(q);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Get a product by ID
*/
@GetMapping("/{id}")
public ResponseEntity<Product> getProductById(@PathVariable String id) {
try {
Product product = elasticsearchService.getProductById(id);
if (product != null) {
return ResponseEntity.ok(product);
} else {
return ResponseEntity.notFound().build();
}
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Get all products (paginated)
*/
@GetMapping
public ResponseEntity<List<Product>> getAllProducts(
@RequestParam(defaultValue = "0") int from,
@RequestParam(defaultValue = "10") int size) {
try {
List<Product> products = elasticsearchService.getAllProducts(from, size);
return ResponseEntity.ok(products);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Get product count
*/
@GetMapping("/count")
public ResponseEntity<Long> getProductCount() {
try {
Long count = elasticsearchService.countProducts();
return ResponseEntity.ok(count);
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(null);
}
}
/**
* Index sample products (for demo purposes)
*/
@PostMapping("/sample")
public ResponseEntity<String> indexSampleProducts() {
try {
elasticsearchService.indexSampleProducts();
return ResponseEntity.ok("Sample products indexed successfully.");
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Failed to index sample products: " + e.getMessage());
}
}
/**
* Delete a product by ID
*/
@DeleteMapping("/{id}")
public ResponseEntity<String> deleteProduct(@PathVariable String id) {
try {
elasticsearchService.deleteProduct(id);
return ResponseEntity.ok("Product deleted successfully.");
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Failed to delete product: " + e.getMessage());
}
}
}
配置文件
创建 src/main/resources/application.properties:
properties
# Application properties
# Server configuration
server.port=8080
# Elasticsearch configuration (Spring Boot auto-configures this if present)
# spring.elasticsearch.uris=http://localhost:9200
# spring.elasticsearch.username=elastic
# spring.elasticsearch.password=changeme
# Custom properties
elasticsearch.index.name=products
# Logging
logging.level.com.example.search=DEBUG
Elasticsearch Settings 文件 (可选)
创建 src/main/resources/elasticsearch/settings/product-settings.json:
json
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"analysis": {
"analyzer": {
"custom_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": ["lowercase", "stop"]
}
}
}
},
"mappings": {
"properties": {
"name": {
"type": "text",
"analyzer": "custom_analyzer"
},
"description": {
"type": "text",
"analyzer": "custom_analyzer"
},
"price": {
"type": "double"
},
"category": {
"type": "keyword"
},
"releaseDate": {
"type": "date"
},
"reviews": {
"type": "nested"
},
"location": {
"type": "geo_point"
}
}
}
}
运行和测试 🧪
1. 启动 Elasticsearch 容器
使用 Gradle 任务启动 Elasticsearch:
bash
./gradlew startElasticsearch
或者手动运行:
bash
docker-compose up -d elasticsearch
2. 确认 Elasticsearch 启动
检查容器是否正在运行:
bash
docker ps
应该能看到 elasticsearch-search-service_elasticsearch_1 容器。
3. 启动应用
在另一个终端窗口中,运行 Spring Boot 应用:
bash
./gradlew bootRun
或者使用 Maven (如果转换为 Maven):
bash
mvn spring-boot:run
4. 测试 API
应用启动后,可以使用 curl 或 Postman 等工具测试 API:
索引样本产品
bash
curl -X POST http://localhost:8080/api/products/sample
搜索产品
bash
# 搜索名称包含 "Laptop"
curl -X GET http://localhost:8080/api/products/search/name/Laptop
# 搜索类别为 "Electronics"
curl -X GET http://localhost:8080/api/products/search/category/Electronics
# 搜索价格在 100 到 1000 之间
curl -X GET "http://localhost:8080/api/products/search/price?minPrice=100&maxPrice=1000"
# 通用搜索
curl -X GET "http://localhost:8080/api/products/search?q=laptop"
# 获取所有产品
curl -X GET http://localhost:8080/api/products
# 获取产品总数
curl -X GET http://localhost:8080/api/products/count
# 获取特定 ID 的产品
curl -X GET http://localhost:8080/api/products/1
5. 停止服务
停止 Elasticsearch 容器:
bash
./gradlew stopElasticsearch
或者手动运行:
bash
docker-compose down -v
编写集成测试 🧪
为了确保与 Elasticsearch 的集成正确无误,编写集成测试是必不可少的。我们将使用 Testcontainers 来在测试环境中启动一个 Elasticsearch 容器。
添加测试依赖
确保 build.gradle 中包含了以下测试依赖:
groovy
// ... existing dependencies ...
testImplementation 'org.testcontainers:testcontainers:1.19.3'
testImplementation 'org.testcontainers:elasticsearch:1.19.3'
testImplementation 'org.junit.jupiter:junit-jupiter:5.10.0'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
编写集成测试
创建 src/test/java/com/example/search/IntegrationTest.java:
java
package com.example.search;
import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch.core.IndexResponse;
import co.elastic.clients.elasticsearch.core.SearchResponse;
import co.elastic.clients.elasticsearch.core.search.Hit;
import com.example.search.model.Product;
import com.example.search.model.Review;
import com.example.search.repository.ProductRepository;
import com.example.search.service.ElasticsearchService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.testcontainers.elasticsearch.ElasticsearchContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import java.io.IOException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Collections;
import java.util.List;
import java.util.UUID;
import static org.assertj.core.api.Assertions.assertThat;
@SpringBootTest
@Testcontainers
class IntegrationTest {
// Start Elasticsearch container using Testcontainers
@Container
static final ElasticsearchContainer elasticsearch = new ElasticsearchContainer("docker.elastic.co/elasticsearch/elasticsearch:8.11.3")
.withEnv("discovery.type", "single-node")
.withEnv("xpack.security.enabled", "false");
@Autowired
private ElasticsearchService elasticsearchService;
@Autowired
private ProductRepository productRepository;
@BeforeEach
void setUp() throws IOException {
// Ensure the index exists before each test
elasticsearchService.createIndexIfNotExists();
}
@Test
void testAddProduct() throws IOException {
String productId = UUID.randomUUID().toString();
Product product = new Product(
productId,
"Test Product",
"This is a test product description.",
99.99,
"Test Category",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.singletonList(new Review("Test User", 5, "Great product!")),
"37.7749,-122.4194" // San Francisco coordinates
);
elasticsearchService.addProduct(product);
// Verify the product was indexed
Product retrievedProduct = elasticsearchService.getProductById(productId);
assertThat(retrievedProduct).isNotNull();
assertThat(retrievedProduct.getName()).isEqualTo("Test Product");
assertThat(retrievedProduct.getPrice()).isEqualTo(99.99);
}
@Test
void testSearchProductsByName() throws IOException {
String productId = UUID.randomUUID().toString();
Product product = new Product(
productId,
"Laptop XYZ",
"High-performance laptop.",
1200.0,
"Electronics",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.emptyList(),
null
);
elasticsearchService.addProduct(product);
List<Product> results = elasticsearchService.searchProductsByName("Laptop XYZ");
assertThat(results).hasSize(1);
assertThat(results.get(0).getName()).isEqualTo("Laptop XYZ");
}
@Test
void testSearchProductsByCategory() throws IOException {
String productId = UUID.randomUUID().toString();
Product product = new Product(
productId,
"Smartphone ABC",
"Latest smartphone.",
800.0,
"Electronics",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.emptyList(),
null
);
elasticsearchService.addProduct(product);
List<Product> results = elasticsearchService.searchProductsByCategory("Electronics");
assertThat(results).hasSize(1);
assertThat(results.get(0).getCategory()).isEqualTo("Electronics");
}
@Test
void testSearchProductsByPriceRange() throws IOException {
String productId = UUID.randomUUID().toString();
Product product = new Product(
productId,
"Budget Item",
"Cheap item.",
20.0,
"Other",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.emptyList(),
null
);
elasticsearchService.addProduct(product);
List<Product> results = elasticsearchService.searchProductsByPriceRange(10.0, 30.0);
assertThat(results).hasSize(1);
assertThat(results.get(0).getPrice()).isEqualTo(20.0);
}
@Test
void testGetAllProducts() throws IOException {
// Add multiple products
List<Product> products = List.of(
new Product(UUID.randomUUID().toString(), "Product A", "Desc A", 100.0, "Cat1", LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE), Collections.emptyList(), null),
new Product(UUID.randomUUID().toString(), "Product B", "Desc B", 200.0, "Cat2", LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE), Collections.emptyList(), null)
);
elasticsearchService.addProducts(products);
List<Product> allProducts = elasticsearchService.getAllProducts(0, 10);
assertThat(allProducts).hasSize(2);
}
@Test
void testCountProducts() throws IOException {
long initialCount = elasticsearchService.countProducts();
assertThat(initialCount).isEqualTo(0);
// Add one product
Product product = new Product(
UUID.randomUUID().toString(),
"Single Product",
"For counting.",
50.0,
"Test",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.emptyList(),
null
);
elasticsearchService.addProduct(product);
long newCount = elasticsearchService.countProducts();
assertThat(newCount).isEqualTo(1);
}
@Test
void testDeleteProduct() throws IOException {
String productId = UUID.randomUUID().toString();
Product product = new Product(
productId,
"To Be Deleted",
"This will be deleted.",
1000.0,
"Delete Test",
LocalDateTime.now().format(DateTimeFormatter.ISO_LOCAL_DATE),
Collections.emptyList(),
null
);
elasticsearchService.addProduct(product);
// Verify it exists
Product retrieved = elasticsearchService.getProductById(productId);
assertThat(retrieved).isNotNull();
// Delete it
elasticsearchService.deleteProduct(productId);
// Verify it's gone
Product deleted = elasticsearchService.getProductById(productId);
assertThat(deleted).isNull();
}
}
运行集成测试
运行测试:
bash
./gradlew test
Testcontainers 会自动拉取 Elasticsearch 镜像并在测试运行期间启动一个临时容器。测试结束后,容器会被自动清理。
高级优化与最佳实践 🌟
1. 配置优化
- JVM Heap Size :在
docker-compose.yml中调整ES_JAVA_OPTS,确保 Elasticsearch 有足够的内存。 - 索引设置:根据数据特点和查询模式,调整索引的分片数、副本数等设置。
- 分析器:根据语言和业务需求配置自定义分析器。
- 安全配置:在生产环境中,务必启用安全功能(用户名/密码、SSL/TLS)。
2. 性能调优
- 批量操作:在索引大量数据时,使用批量操作(Bulk API)而非单个索引请求。
- 缓存策略:合理利用 Elasticsearch 的查询缓存和字段数据缓存。
- 查询优化:编写高效的查询语句,避免全表扫描。
- 硬件资源:确保足够的 CPU、内存和 SSD 存储空间。
3. 安全性
- 访问控制:限制对 Elasticsearch 的访问,仅允许授权的应用和服务。
- 数据加密:在传输层(TLS)和存储层(加密磁盘)实施数据保护。
- 审计日志:启用审计日志以跟踪敏感操作。
4. 监控与维护
- 指标监控:使用 Prometheus、Grafana 等工具监控 Elasticsearch 集群健康状况。
- 日志管理:集中收集和分析 Elasticsearch 日志。
- 备份策略:定期备份重要数据和索引。
- 版本升级:关注 Elasticsearch 版本更新,及时进行兼容性测试和升级。
5. 集成与部署
- CI/CD 流水线:将 Elasticsearch 的启动、测试和部署集成到 CI/CD 流程中。
- Kubernetes 部署:如果使用 Kubernetes,可以利用 StatefulSet 和 Operator 来管理 Elasticsearch 集群。
- 环境变量:通过环境变量配置 Elasticsearch 的连接信息和参数,提高灵活性。
与其他技术栈的集成 🔗
与 Spring Boot 集成
如上所述,Spring Boot 通过 spring-boot-starter-data-elasticsearch 和 elasticsearch-java 客户端提供了对 Elasticsearch 的无缝集成。Spring Data Elasticsearch 提供了 Repository 模式,简化了数据访问。
与 React/Vue/Angular 前端集成
前端应用可以通过 RESTful API 与后端 Spring Boot 应用通信,进而与 Elasticsearch 交互。例如,前端发送搜索请求到 /api/products/search?q=query,后端处理请求并返回结果。
与 Kafka 集成
可以将 Kafka 作为数据流管道,实时消费数据并同步到 Elasticsearch 中,实现近实时的搜索能力。
结论 🏁
通过将 Gradle 与 Elasticsearch 集成,我们可以构建出功能强大、性能优越的搜索服务。本文从基础概念出发,详细介绍了如何在 Gradle 项目中配置和使用 Elasticsearch,包括 Docker 容器化部署、Spring Boot 集成、数据模型设计、API 开发、集成测试以及最佳实践。
关键要点总结如下:
- 容器化部署:使用 Docker Compose 简化了 Elasticsearch 的本地和开发环境部署。
- Gradle 任务:通过自定义 Gradle 任务,实现了 Elasticsearch 的启动、停止和等待就绪。
- Spring Boot 集成:利用 Spring Boot 的自动配置和 Spring Data Elasticsearch,简化了数据访问层的开发。
- 测试保障:通过 Testcontainers 进行集成测试,确保了组件间的协同工作。
- 最佳实践:涵盖了性能优化、安全性、监控和部署等方面的建议。
Elasticsearch 与 Gradle 的结合,不仅提升了开发效率,也为构建现代化、可扩展的搜索服务奠定了坚实的基础。随着技术的发展,这种集成方式将继续演进,为开发者提供更多便利和可能性。
参考文献与链接:
- Elasticsearch 官方文档 📘
- Spring Data Elasticsearch 官方文档 🔄
- Testcontainers 官方网站 🧪
- Docker 官方文档 🐳
- Gradle 官方文档 📚
Mermaid 图表:Gradle 与 Elasticsearch 集成架构图
Development Workflow
Production Deployment
CI/CD Pipeline
Kubernetes/Orchestration
Deployed Elasticsearch Cluster
Search Service
Testing
Test Containers
Elasticsearch Test Instance
JUnit Tests
Spring Data Elasticsearch
API Layer
Elasticsearch
Gradle Build
Custom Tasks
Start Elasticsearch Container
Stop Elasticsearch Container
Wait for Elasticsearch Ready
Docker Compose
Elasticsearch Server
Search Service Application
Spring Boot App
Elasticsearch Java Client
Elasticsearch API
这个图表展示了从 Gradle 构建、容器化部署、开发测试到生产部署的完整流程,以及各个组件之间的关系。
希望这篇博客能帮助你更好地理解和实践 Gradle 与 Elasticsearch 的集成!🚀
🙌 感谢你读到这里!
🔍 技术之路没有捷径,但每一次阅读、思考和实践,都在悄悄拉近你与目标的距离。
💡 如果本文对你有帮助,不妨 👍 点赞 、📌 收藏 、📤 分享 给更多需要的朋友!
💬 欢迎在评论区留下你的想法、疑问或建议,我会一一回复,我们一起交流、共同成长 🌿
🔔 关注我,不错过下一篇干货!我们下期再见!✨