es的优势

破局缘2023-11-22 8:49

系列文章目录

提示：这里可以添加系列文章的所有文章的目录，目录需要自己手动添加

例如：第一章 Python 机器学习入门之pandas的使用

提示：写完文章后，目录可以自动生成，如何生成可参考右边的帮助文档

文章目录

前言

总结es优势

一、es为什么比mysql快

  1. es是一个基于Lucene引擎库，基于内存，查询速度比mysql，这个是在存储方式的比较
  2. 第二是数据存储方式，倒排索引，存储方式，可以快速找到数据的大概位置，文档列表，利用二叉查询方法进行寻找方式
  3. es支持复杂的语法格式，寻找附近酒店，进行分页
  4. 缺点
  5. 过于占内存

二、使用步骤

1.引入库

代码如下（示例）：

java 复制代码 
package com.cn;

import com.alibaba.fastjson.JSON;
import com.cn.mapper.ESMapper;
import com.cn.pojo.Hotel;
import com.cn.pojo.TbHotel;
import com.cn.pojo.vo.TbHotelVo;
import org.apache.http.HttpHost;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.junit.Test;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class EsTest {
    @Autowired
    private RestHighLevelClient client;

    @Autowired
    private ESMapper esMapper;


    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.36.128:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }

    /**
     * 判断索引是否存在
     * @throws IOException
     */
    @Test
    public void  getIndexRequest() throws IOException {
        GetIndexRequest request = new GetIndexRequest("tiantian");
        boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);
        System.out.println(exists?"不能再":"我问");
    }

    /**
     * 批量导入文档
     * @throws IOException
     */
    @Test
    public void c() throws IOException {
        List<Hotel> hotels = new ArrayList<>();
        for (int i = 0; i < 50; i++) {
            Hotel hotel = new Hotel();
            hotel.setId(Long.valueOf(i));
            hotel.setAge(i);
            hotel.setAddress("我的地址方式"+i);
            hotel.setTime(new Date());
            hotel.setName("将来"+i);
            hotel.setLatLon("23.5","3"+i);
            hotel.setLocation("23.5","3"+i);
            hotels.add(hotel);
        }

        //批量导入
        BulkRequest bulkRequest = new BulkRequest();
        //转化文档
        for (Hotel hotel : hotels) {
            bulkRequest.add(new IndexRequest("tiantian")
                    .id(hotel.getId().toString())
                    .source(JSON.toJSONString(hotel), XContentType.JSON));
        }
        //发送请求
        BulkResponse bulk = client.bulk(bulkRequest, RequestOptions.DEFAULT);
        System.out.println(bulk);
    }

    /**
     * 导入数据信息
     * @throws IOException
     */
    @Test
    public void bulkRequest() throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
        List<TbHotel> tbHotels = esMapper.selectList(null);
        for (TbHotel tbHotel : tbHotels) {
            TbHotelVo tbHotelVo = new TbHotelVo(tbHotel);
            bulkRequest.add(new IndexRequest("hotel")
                    .id(tbHotelVo.getId().toString())
                    .source(JSON.toJSONString(tbHotelVo),XContentType.JSON)
            );
        }
        client.bulk(bulkRequest,RequestOptions.DEFAULT);
    }

    /**
     * 查询所有文档信息
     * @throws IOException
     */
    @Test
    public void testMatchAll() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        request.source().query(QueryBuilders.matchAllQuery());
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        for (SearchHit hit : response.getHits().getHits()) {
            String source = hit.getSourceAsString();
            System.out.println(source);
        }
    }

    /**
     * 准确查询
     * @throws IOException
     */
    @Test
    public void matchQuery() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        request.source().query(QueryBuilders.termQuery("price","189"));
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        for (SearchHit hit : response.getHits().getHits()) {
            String sourceAsString = hit.getSourceAsString();
            System.out.println(sourceAsString);
        }
    }

    /**
     * 布尔查询方式
     * 查询名字叫做如家
     * 价格200
     * 地址在上海
     */
    @Test
    public void boolQuery() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        boolQuery.mustNot(QueryBuilders.rangeQuery("price").gte("400"));
        boolQuery.must(QueryBuilders.matchQuery("city","上海"));
        boolQuery.must(QueryBuilders.matchQuery("name","如家"));
        request.source().query(boolQuery);
        for (SearchHit hit : client.search(request, RequestOptions.DEFAULT).getHits().getHits()) {
            String sourceAsString = hit.getSourceAsString();
            System.out.println(sourceAsString);
        }
    }

    /**
     * bool查询方式
     * @throws IOException
     */
    @Test
    public void testBoole() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        boolQuery.must(QueryBuilders.matchQuery("name","如家"));
        request.source().query(boolQuery);
        for (SearchHit hit : client.search(request, RequestOptions.DEFAULT).getHits()) {
            System.out.println(hit.getSourceAsString());
        }
    }

    /**
     * 根据经纬度查询方式
     * @throws IOException
     */
    @Test
    public void geoPoint() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        request.source().sort(SortBuilders.geoDistanceSort("lonAndLat",new GeoPoint("31.2,121.5"))
                .order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS));
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        for (SearchHit hit : response.getHits().getHits()) {
            TbHotel tbHotel = JSON.parseObject(hit.getSourceAsString(), TbHotel.class);
            System.out.println(tbHotel.getLonAndLat());
        }

    }

    
    @Test
    public void testMat() throws IOException {
        SearchRequest request = new SearchRequest("hotel");
        request.source().query(QueryBuilders.matchQuery("name","如家"));
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        for (SearchHit hit : response.getHits().getHits()) {
            System.out.println(hit.getSourceAsString());
        }
    }
}

2. es查询语法

代码如下（示例）：

java 复制代码 
//添加索引
PUT /tiantian
{
  "mappings": {
    "properties": {
      "name":{
        "type": "text"
        , "analyzer": "ik_smart"
      },
      "age":{
        "type": "integer"
        , "index": false
      }
    }
  }
}

//查询索引
GET /tiantian

//查询文档信息
GET /tiantian/_doc/1

//添加字段信息
PUT /tiantian/_mapping
{
  "properties":{
    "address":{
      "type":"text",
      "index":false
    }
  }
}

//添加数据文档数据信息
POST /tiantian/_doc/1
{
  "address":"巴黎",
  "name":"太牛龙",
  "age":123
}

//查询经纬度
GET /hotel/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "price": {
        "order": "desc"
      }
    },
    {
      "_geo_distance": {
        "lonAndLat": {
          "lat": 31.2,
          "lon": 121.5
        },
        "order": "asc"
      }
    }
  ]
}

POST /hotel/_update/1902197537
{
    "doc": {
        "isAD": true
    }
}
POST /hotel/_update/2056126831
{
    "doc": {
        "isAD": true
    }
}
POST /hotel/_update/1989806195
{
    "doc": {
        "isAD": true
    }
}
POST /hotel/_update/2056105938
{
    "doc": {
        "isAD": true
    }
}


GET /hotel/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "name": "外滩"
        }
      },
      "functions": [
        {
          "filter": { 
            "term": {
              "id": "1"
            }
          },
          "weight": 10
        }
      ],
      "boost_mode": "multiply"
    }
  },
  "from": 1,
  "size": 10
}



GET /hotel/_search
{
  "query": {
    "function_score": {
      "query": {
       "match": {
         "name": "如家"
       }
      },
      "functions": [
        {
          "filter": {
            "term": {
              "name": "339952837"
            }
          }
          , "weight": 10
        }
      ],
      "boost_mode": "sum"
    }
  }
}

GET /hotel/_search
{
  "query": {
    "match_all": {}
  }
}

GET /hotel

GET /hotel/_search
{
  "size": 0, 
  "aggs": {
    "brandAgg": {
      "terms": {
        "field": "brand.keyword"
      }
    }
  }
  
}

GET /hotel/_search
{
  "query": {
    "match": {
      "name": "东方明珠"
    }
  }
}

三，api功能

java 复制代码 
@Override
    public PageResult search(RequestParams params) throws IOException {
        //1.准备发送
        SearchRequest request = new SearchRequest("hotel");
        //2.准备布尔条件
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        //3.判断是否存在
        String key = params.getKey();
        if (key==null || "".equals(key)){
            boolQuery.must(QueryBuilders.matchAllQuery());
        }else {
            boolQuery.must(QueryBuilders.matchQuery("name",key));
        }
        Integer size = params.getSize();
        Integer page = params.getPage();
        request.source().from((page - 1) * size).size(size);
        request.source().query(boolQuery);

        String location = params.getLocation();

        //得出具体路径
        if (location!=null && !location.equals("")){
            request.source().sort(SortBuilders
                    .geoDistanceSort("lonAndLat",new GeoPoint(location))
                    .order(SortOrder.ASC)
                    .unit(DistanceUnit.KILOMETERS));
        }

        //相关性算法
        FunctionScoreQueryBuilder scoreQuery = QueryBuilders.functionScoreQuery(
                boolQuery,
                new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{
                        new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                                QueryBuilders.termQuery("isAD", true),
                                ScoreFunctionBuilders.weightFactorFunction(10)
                        )
                }
        );
        request.source().query(scoreQuery);
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        return  handleResponse(response);
    }

根据经纬度查询方式

java 复制代码 
//得出具体路径
        if (location!=null && !location.equals("")){
            request.source().sort(SortBuilders
                    .geoDistanceSort("lonAndLat",new GeoPoint(location))
                    .order(SortOrder.ASC)
                    .unit(DistanceUnit.KILOMETERS));
        }

相关性算法

java 复制代码 
       //相关性算法
        FunctionScoreQueryBuilder scoreQuery = QueryBuilders.functionScoreQuery(
                boolQuery,
                new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{
                        new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                                QueryBuilders.termQuery("isAD", true),
                                ScoreFunctionBuilders.weightFactorFunction(10)
                        )
                }
        );

总结

  1. es高效，速度快，基于lu的内存数据库，采用倒序索引的方式，倒叙索引好处，我们之前数据库根据id找到值，倒排索引的方式，es默认创建索引，根据值找到，对应的文档列表，文档列表根据二分查找方式，找到对应的值
  2. es强大的api功能普通基于内存的数据库的方式，比如redis功能，适合做缓存，没有强大的api，不能做复杂的功能，es有强大api，分页，查询，联合查询，经纬度查询，相关性质查询方式
相关推荐
花菜回锅肉43 分钟前
hadoop分布式文件系统常用命令
大数据·hadoop·分布式
Yz98762 小时前
Hive基础
大数据·linux·数据仓库·hive·hadoop·bigdata
AORO_BEIDOU3 小时前
抢抓5G机遇，AORO A23防爆手机如何直击园区巡检挑战？
大数据·5g·智能手机·信息与通信
Shaidou_Data3 小时前
信息技术引领未来：大数据治理的实践与挑战
大数据·人工智能·数据清洗·信息技术·数据治理技术
Elastic 中国社区官方博客3 小时前
开始使用 Elastic AI Assistant 进行可观察性和 Microsoft Azure OpenAI
大数据·人工智能·elasticsearch·microsoft·搜索引擎·全文检索·azure
青云交4 小时前
大数据新视界 -- 大数据大厂之 Impala 性能优化：新技术融合的无限可能（下）（12/30）
大数据·性能优化·impala·技术创新·新技术融合·电商案例·跨行业应用
weixin_442643424 小时前
FileLink跨网文件安全摆渡系统——企业数据流转的安全桥梁
大数据·网络·安全·filelink文件摆渡系统
OBOO鸥柏5 小时前
OBOO鸥柏“触摸屏广告一体机交互”亮相2024中国珠海航展
大数据·人工智能·科技·交互
我是琦琦琦琦6 小时前
flink 同步oracle11g数据表到pg库
大数据·postgresql·oracle·flink
myheartgo-on6 小时前
PySpark——Python与大数据
大数据·python·信息可视化
热门推荐
01智能车联网安全发展形势、挑战02Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04玄机平台应急响应—webshell查杀05软件工程从理论到实践客观题汇总(头歌第一章至第八章)06组基轨迹建模 GBTM的介绍与实现（Stata 或 R）07HP惠普星15青春版/惠普小欧笔记本电脑15s-du1008tx原装出厂Win11系统08全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）09基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测10Python数据分析案例31——中国A股的月份效应研究(方差分析，虚拟变量回归)