es自定义分词器支持数字字母分词，中文分词器jieba支持添加禁用词和扩展词典

专职2023-08-20 12:51

自定义分析器，分词器

json 复制代码

PUT http://xxx.xxx.xxx.xxx:9200/test_index/
{
  "settings": {
    "analysis": {
      "analyzer": {
        "char_test_analyzer": {
          "tokenizer": "char_test_tokenizer",
          "filter": [
            "lowercase"
          ]
        }
      },
      "tokenizer": {
        "char_test_tokenizer": {
          "type": "ngram",
          "min_gram": 1,
          "max_gram": 2
        }
      }
    }
  },
  "mappings": {
    "test_zysf_index": {
      "properties": {
        "text": {
          "type": "text",
          "analyzer": "char_test_analyzer"
        }
      }
    }
  }
}

所有字段检索

json 复制代码

{
	"query": {
		"multi_match": {
			"query": "河北保定",
			"type": "cross_fields",
			"fields": ["*"],
			"operator": "AND"
    }
	}
}

高亮搜索

json 复制代码

{
  "highlight": {
    "fields": {
        "name": {},
        "content": {},
    },
    "pre_tags": ["<em>"],
    "post_tags": ["</em>"]
  }
}

分词测试

GET /test_index/_analyze

json 复制代码

{
  "analyzer": "char_test_analyzer",
  "text": "adfsjdsa12345646abADS"
}

jieba中文分词支持添加禁用词和扩展词库功能

创建索引：PUT http://xxxx:9200/test_index

json 复制代码

{
	"settings": {
		"analysis": {
			"filter": {
				"jieba_stopword": {
					"type": "stop",
					"stopwords_path": "/home/stopwords.txt"
				}
			},
			"tokenizer": {
	        	"jieba_tokenizer": {
		          	"type": "jieba_index",
		          	"user_dict": "/home/user.dict"
        		}
      		},
			"analyzer": {
				"my_jieba": {
					"filter": [
						"lowercase",
						"jieba_stopword"
					],
					"tokenizer": "jieba_tokenizer"
				}
			}
		}
	},
	"mappings": {
    	"test_index": {
      		"properties": {
        		"name": {
          			"type": "text",
					"index": true,
					"analyzer": "my_jieba",
					"similarity": "BM25"
        		}
      		}
    	}
  	}
}

分词测试：

GET http://xxxxxx:9200/test_index/_analyze

json 复制代码

{
  "analyzer": "my_jieba",
  "text": "中国你好，我爱你中国"
}