基于脚手架微服务的视频点播系统-脚手架开发部分elasticsearch与libcurl的简单使用与二次封装及bug修复-完结
- 1.ElasticClient的使用
-
- 1.1ES检索原理
- 1.2ES核心概念
- [1.3 Kibana访问es进行测试](#1.3 Kibana访问es进行测试)
- 1.4典型操作
- 1.5ES客户端SDK
- 1.6二次封装
- 2.libcurl的使用与封装
- 3.bug修复
- 4.打包脚手架项目
1.ElasticClient的使用
Elasticsearch， 简称ES，它是个开源分布式搜索引擎，它的特点有：分布式，零配置，⾃动发现，索引⾃动分⽚，索引副本机制，restful⻛格接⼝，多数据源，⾃动搜索负载等。它可以近乎实时的存储、检索数据；本⾝扩展性很好，可以扩展到上百台服务器，处理PB级别的数据。es也使⽤Java开发并使⽤Lucene作为其核⼼来实现所有索引和搜索的功能，但是它的⽬的是通过简单的RESTful API来隐藏Lucene的复杂性，从⽽让全⽂搜索变得简单。
Elasticsearch是⾯向⽂档(document oriented)的，这意味着它可以存储整个对象或⽂档(document)。然⽽它不仅仅是存储，还会索引(index)每个⽂档的内容使之可以被搜索。在Elasticsearch中，你可以对⽂档（⽽⾮成⾏成列的数据）进⾏索引、搜索、排序、过滤。
1.1ES检索原理
正排索引
正排索引，也称为前向索引，是⼀种将⽂档或数据记录按照某种特定顺序组织的索引机制。在正排索引中，索引的键通常是⽂档的标识符，如⽂档ID，⽽索引的值则包含⽂档的详细信息，例如标题、内容摘要、发布⽇期等。这种结构使得正排索引⾮常适合执⾏基于特定标识符的查找操作。正排索引的优点在于能够直接根据⽂档ID快速访问⽂档，适合于需要按照⽂档顺序进⾏操作的场景。
倒排索引
倒排索引，⼜称反向索引或逆向索引，是按照⽂档中的词汇来组织数据的索引⽅法。在倒排索引中，每个独特的词汇都会有⼀个索引条⽬，该条⽬包含指向包含该词 的所有⽂档的指针或引⽤。这使得倒排索引⾮常适合全⽂搜索，能够快速找到包含特定关键词的⽂档。倒排索引则适合于全⽂搜索，可以快速找到包含特定关键词的所有⽂档，索引的⼤⼩相对较⼩，因为它只记录关键词和⽂档的映射关系。但是，倒排索引不能直接通过索引访问⽂档，需要结合正排索引来获取⽂档的详细信息。
1.2ES核心概念
ES中的一些概念与传统数据库可以做如下类比
这里需要注意的一点是，在es7.x版本之后，类型这个概念已经被废弃，也就是每个索引下只有一个类型就是_doc。我们访问一个索引中的文档时格式如下 :/index/_doc/文档id。
1.2.1索引(index)
⼀个索引就是⼀个拥有⼏分相似特征的⽂档的集合。⽐如说，你可以有⼀个客⼾数据的索引，⼀个产品⽬录的索引，还有⼀个订单数据的索引。⼀个索引由⼀个名字来标识（必须全部是⼩写字⺟的），并且当我们要对应于这个索引中的⽂档进⾏索引、搜索、更新和删除的时候，都要使⽤到这个名字。在⼀个集群中，可以定义任意多的索引。
1.2.2类型(Field)
在⼀个索引中，你可以定义⼀种或多种类型。⼀个类型是你的索引的⼀个逻辑上的分类/分区，其语义完全由你来定。通常，会为具有⼀组共同字段的⽂档定义⼀个类型。⽐如说，我们假设你运营⼀个博客平台并且将你所有的数据存储到⼀个索引中。在这个索引中，你可以为⽤⼾数据定义⼀个类型，为博客数据定义另⼀个类型，为评论数据定义另⼀个类型...
1.2.3字段(Field)
字段相当于是数据表的字段，对⽂档数据根据不同属性进⾏的分类标识。
| 分类 | 类型 | 备注 |
|---|---|---|
| 字符串 | text, keyword | text会被分词生成索引；keyword不会被分词生成索引，只能精确值搜索 |
| 整形 | integer, long, short, byte | |
| 浮点 | double, float | |
| 逻辑 | boolean | true或false |
| 日期 | date, date_nanos | "2018-01-13" 或 "2018-01-13 12:10:30" 或者时间戳，即1970到现在的秒数/毫秒数 |
| 二进制 | binary | 二进制通常只存储，不索引 |
| 范围 | range |
1.2.4映射(mapping)
映射是在处理数据的⽅式和规则⽅⾯做⼀些限制，如某个字段的数据类型、默认值、分析器、是否被索引等等，这些都是映射⾥⾯可以设置的，其它就是处理es⾥⾯数据的⼀些使⽤规则设置也叫做映射，按着最优规则处理数据对性能提⾼很⼤，因此才需要建⽴映射，并且需要思考如何建⽴映射才能对性能更好。
| 名称 | 数值 | 备注 |
|---|---|---|
| enabled | true(默认) / false | 是否仅作存储，不做搜索和分析 |
| index | true(默认) / false | 是否构建倒排索引（决定了是否分词，是否被索引） |
| index_option | ||
| dynamic | true（缺省） / false | 控制mapping的自动更新 |
| doc_value | true(默认) / false | 是否开启doc_value，用于聚合和排序分析，分词字段不能使用 |
| fielddata | "fielddata": {"format": "disabled"} | 是否为text类型启动fielddata，实现排序和聚合分析 |
| store | true / false(默认) | 针对分词字段，参与排序或聚合时能提高性能，不分词字段统一建议使用doc_value |
| coerce | true(默认) / false | 是否开启自动数据类型转换功能，比如：字符串转数字，浮点转整型 |
| analyzer | "analyzer": "ik" | 指定分词器，默认分词器为standard analyzer |
| boost | "boost": 1.23 | 字段级别的分数加权，默认值是1.0 |
| fields | "fields": { "name": { "type": "text", "index": true }, ... } | 对一个字段提供多种索引模式，同一个字段的值，一个分词，一个不分词 |
| data_detection | true(默认) / false | 是否自动识别日期类型 |
1.2.5文档(document)
⼀个⽂档是⼀个可被索引的基础信息单元。⽐如，你可以拥有某⼀个客⼾的⽂档，某⼀个产品的⼀个⽂档或者某个订单的⼀个⽂档。⽂档以JSON（Javascript Object Notation）格式来表⽰，⽽JSON是⼀个到处存在的互联⽹数据交互格式。在⼀个index/type⾥⾯，你可以存储任意多的⽂档。⼀个⽂档必须被索引或者赋予⼀个索引的type。
1.3 Kibana访问es进行测试
通过⽹⻚访问kibana： http://192.168.65.128:5601/ ，注意：将链接中的IP地址改换成为你的主机IP地址。
默认账户密码为:elastic:123456
在开发工具中进行以下实验即可:
1.3.1创建索引
css
PUT /user
{
"settings": {
"analysis": {
"analyzer": {
"ikmax": {
"type": "custom",
"tokenizer": "ik_max_word"
}
}
}
},
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "text",
"boost": 3,
"analyzer": "ikmax"
},
"age": {
"type": "integer"
},
"phone": {
"type": "keyword",
"boost": 1
},
"skills": {
"type": "text"
},
"birth": {
"type": "date",
"index": false,
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
}1.3.2新增数据
css
POST /user/_doc/_bulk
{"index":{"_id":"1"}}
{"user_id" : "USER4b862aaa-2df8654a-7eb4bb65-e3507f66","nickname" : "昵称
1","phone" : "⼿机号1","description" : "签名1","avatar_id" : "头像1"}
{"index":{"_id":"2"}}
{"user_id" : "USER14eeeaa5-442771b9-0262e455-e4663d1d","nickname" : "昵称
2","phone" : "⼿机号2","description" : "签名2","avatar_id" : "头像2"}
{"index":{"_id":"3"}}
{"user_id" : "USER484a6734-03a124f0-996c169d-d05c1869","nickname" : "昵称
3","phone" : "⼿机号3","description" : "签名3","avatar_id" : "头像3"}
{"index":{"_id":"4"}}
{"user_id" : "USER186ade83-4460d4a6-8c08068f-83127b5d","nickname" : "昵称
4","phone" : "⼿机号4","description" : "签名4","avatar_id" : "头像4"}
{"index":{"_id":"5"}}
{"user_id" : "USER6f19d074-c33891cf-23bf5a83-57189a19","nickname" : "昵称
5","phone" : "⼿机号5","description" : "签名5","avatar_id" : "头像5"}
{"index":{"_id":"6"}}
{"user_id" : "USER97605c64-9833ebb7-d0455353-35a59195","nickname" : "昵称
6","phone" : "⼿机号6","description" : "签名6","avatar_id" : "头像6"}1.3.3查看并搜索数据
css
GET /user/_doc/_search?pretty
{
"query" : {
"bool" : {
"must_not" : [
{
"terms" : {
"user_id.keyword" : [
"USER4b862aaa-2df8654a-7eb4bb65-e3507f66",
"USER14eeeaa5-442771b9-0262e455-e4663d1d",
"USER484a6734-03a124f0-996c169d-d05c1869"
]
}
}
],
"should" : [
{
"match" : {
"user_id" : "昵称"
}
},
{
"match" : {
"nickname" : "昵称"
}
},
{
"match" : {
"phone" : "昵称"
}
}
]
}
}
}1.3.4删除索引
css
DELETE /user1.4典型操作
ES中的操作，是基于Restful⻛格的接⼝，使⽤HTTP协议进⾏通信，通信的时候正⽂采⽤JSON格式进⾏序列化。
因此，在组织请求的过程中，更多关注的⽅⾯：
- 请求⽅法
- 请求的URI路径
- 请求正⽂中的各个关键字段
1.4.1索引
创建索引
css
PUT /student
{
"settings": {
"analysis": {
"analyzer": {
"ikmax": {
"type": "custom",
"tokenizer": "ik_max_word"
}
}
}
},
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "text",
"boost": 3,
"analyzer": "ikmax"
},
"age": {
"type": "integer"
},
"phone": {
"type": "keyword",
"boost": 1
},
"skills": {
"type": "text"
},
"birth": {
"type": "date",
"index": false,
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
}删除索引
css
DELETE /student1.4.2新增数据
1.4.2.1单数据新增
URI组成： /索引名称/⽂档类型名称[/当前⽂档ID]
css
POST /student/_doc/2
{
"name":"lisi",
"age": 22,
"phone":"15522222222",
"skills":[
"C++",
"Python"
],
"birth":"2018-08-19 22:22:33"
}1.4.2.2批量数据新增
正⽂格式要求： 每个⽂档包含两⾏内容：1. 索引信息； 2. ⽂档信息
css
POST /student/_doc/_bulk
{"index":{"_id":"1"}}
{"name":"zhangsan","age":18,"phone":"15511111111","skills":["C++",
"Java"],"birth":"2017-08-19 22:22:22"}
{"index":{"_id":"6"}}
{"name":"lisi","age":19,"phone":"15522222222","skills":["C++",
"Python"],"birth":"2018-08-19 22:22:33"}
{"index":{"_id":"3"}}
{"name":"wangwu","age":30,"phone":"15533333333","skills":["Php",
"Python"],"birth":"2028-08-19 22:22:44"}
{"index":{"_id":"4"}}
{"name":"lisi w","age":20,"phone":"15544444444","skills":["Php",
"Java"],"birth":"2038-08-19 22:22:55"}
{"index":{"_id":"5"}}
{"name":"lisi z","age":10,"phone":"15555555555","skills":["YYY",
"BBBB"],"birth":"2048-08-19 22:22:11"}1.4.2.3查询所有数据
css
POST /student/_doc/_search
{
"query": {
"match_all": {}
}
}1.4.3删除
1.4.3.1删除指定id的数据
css
DELETE /student/_doc/11.4.3.2批量删除指定id的数据
css
POST /student/_doc/_bulk
{ "delete": { "_id": "2" } }
{ "delete": { "_id": "3" } }1.4.3.3根据条件进行数据的删除
css
POST /student/_doc/_delete_by_query
{
"query": {
"match": {
"name": "lisi"
}
}
}1.4.4更新
1.4.4.1单数据更新
css
POST /student/_update/1
{
"doc": {
"phone": "18888888888",
"age": 19
}
}1.4.4.2批量数据更新
css
POST /student/_doc/_bulk
{"update":{"_id":"1"}}
{ "doc": { "phone": "11111111111" } }
{"update":{"_id":"2"}}
{ "doc": { "phone": "22222222222" } }
{"update":{"_id":"3"}}
{ "doc": { "phone": "33333333333" } }除了以上两种⽅式进⾏更新，也可以以新增单个数据的⽅式进⾏全字段更新。
1.4.5查询
单条件查询
1.term查询,单字段精确匹配(如果字段没有keyword属性时比如下方例子去掉.keyword，便是一种模糊匹配，有则都是精确匹配)
css
POST /student/_doc/_search
{
"query" : {
"term" : {
"name.keyword" : "lisi"
}
}
}2.terms单字段多值精确匹配
css
{
"query" : {
"terms" : {
"phone" : [
"15511111111",
"15522222222"
]
}
}
}3.match单字段模糊匹配
css
{
"query" : {
"match" : {
"name" : "z"
}
}
}4.数字类型区间匹配
css
{
"query" : {
"range" : {
"age" : {
"gte" : 10,
"lte" : 20
}
}
}5.多字段模糊匹配
css
{
"query" : {
"multi_match" : {
"fields" : [
"name",
"phone"
],
"query" : "zhangsan"
}
}
}多条件查询
bool检索
• must：必须匹配的条件
• must_not ： 必须过滤掉的条件
• should： 匹配⼀个或多个都可以
minimum_should_match：should中⾄少应该匹配n个条件
css
{
"query" : {
"bool" : {
"must" : [
{
"match" :
{
"skills" : "C++"
}
},
],
"must_not" : [
{
"terms" : {
"phone" : [
"15522222222",
"15555555555"
]
}
}
],
"should" : [
{
"match" : {
"name" : "lisi"
}
},
{
"terms" : {
"phone" : [
"15511111111",
"15555555555"
]
}
}
]
"minimum_should_match" : 1,
}
}
}sort检索
css
{
"query" : {
"bool" : {
"minimum_should_match" : 1,
"should" : [
{
"match" : {
"name" : "lisi"
}
},
{
"terms" : {
"phone" :
[
"15511111111",
"15533333333"
]
}
}
]
}
},
"sort" : [
{
"name.keyword" : "desc"
},
{
"age" : "asc"
}
]
}
#结果⽰例
[
{
"age" : 18,
"birth" : "2017-08-19 22:22:22",
"name" : "zhangsan",
"phone" : "15511111111",
"skills" : [
"C++",
"Java"
]
},
{
"age" : 10,
"birth" : "2048-08-19 22:22:11",
"name" : "lisi z",
"phone" : "15555555555",
"skills" : [
"YYY",
"BBBB"
]
},
{
"age" : 20,
"birth" : "2038-08-19 22:22:55",
"name" : "lisi w",
"phone" : "15544444444",
"skills" : [
"Php",
"Java"
]
},
{
"age" : 19,
"birth" : "2018-08-19 22:22:33",
"name" : "lisi",
"phone" : "15522222222",
"skills" : [
"C++",
"Python"
]
}
]分⻚查找
对检索按指定⽅式进⾏排序，并限制获取结果数量，以及偏移量
css
{
"query" :
{
"bool" :
{
"minimum_should_match" : 1,
"should" :
[
{
"match" :
{
"name" : "lisi"
}
},
{
"terms" :
{
"phone" :
[
"15511111111",
"15533333333"
]
}
}
]
}
},
"size" : 10,
"from" : 2,
"sort" :
[
{
"name.keyword" : "desc"
},
{
"age" : "asc"
}
]
} [
{
"age" : 18,
"birth" : "2017-08-19 22:22:22",
"name" : "zhangsan",
"phone" : "15511111111",
"skills" :
[
"C++",
"Java"
]
},
{
"age" : 10,
"birth" : "2048-08-19 22:22:11",
"name" : "lisi z",
"phone" : "15555555555",
"skills" :
[
"YYY",
"BBBB"
]
}1.5ES客户端SDK
ES C++的客⼾端选择并不多， 我们这⾥使⽤elasticlient库, 下⾯进⾏安装。
1.5.1接口介绍
1.响应结构
cpp
namespace cpr {
class Response {
public:
long status_code{};
std::string text{};
Header header{};
Url url{};
double elapsed{};
Cookies cookies{};
Error error{};
std::string raw_header{};
std::string status_line{};
std::string reason{};
};
}2.典型接口
cpp
namespace elasticlient {
class Client {
// http://user:password@localhost:9200/
Client(const std::vector<std::string> &hostUrlList,
std::int32_t timeout = 6000);
/**
* Perform search on nodes until it is successful. Throws exception if all
nodes
* has failed to respond.
* \param indexName specification of an Elasticsearch index.
* \param docType specification of an Elasticsearch document type.
* \param body Elasticsearch request body.
* \param routing Elasticsearch routing. If empty, no routing has been used.
* \return cpr::Response if any of node responds to request.
* \throws ConnectionException if all hosts in cluster failed to respond.
*/
cpr::Response search(const std::string &indexName,
const std::string &docType,
const std::string &body,
const std::string &routing = std::string());
/**
* Get document with specified id from cluster. Throws exception if all nodes
* has failed to respond.
* \param indexName specification of an Elasticsearch index.
* \param docType specification of an Elasticsearch document type.
* \param id Id of document which should be retrieved.
* \param routing Elasticsearch routing. If empty, no routing has been used.
* *
\return cpr::Response if any of node responds to request.
* \throws ConnectionException if all hosts in cluster failed to respond.
*/
cpr::Response get(const std::string &indexName,
const std::string &docType,
const std::string &id = std::string(),
const std::string &routing = std::string());
/**
* Index new document to cluster. Throws exception if all nodes has failed to
respond.
* \param indexName specification of an Elasticsearch index.
* \param docType specification of an Elasticsearch document type.
* \param body Elasticsearch request body.
* \param id Id of document which should be indexed. If empty, id will be
generated
* automatically by Elasticsearch cluster.
* \param routing Elasticsearch routing. If empty, no routing has been used.
* *
\return cpr::Response if any of node responds to request.
* \throws ConnectionException if all hosts in cluster failed to respond.
*/
cpr::Response index(const std::string &indexName,
const std::string &docType,
const std::string &id,
const std::string &body,
const std::string &routing = std::string());
/**
* Delete document with specified id from cluster. Throws exception if all
nodes
* has failed to respond.
* \param indexName specification of an Elasticsearch index.
* \param docType specification of an Elasticsearch document type.
* \param id Id of document which should be deleted.
* \param routing Elasticsearch routing. If empty, no routing has been used.
* *
\return cpr::Response if any of node responds to request.
* \throws ConnectionException if all hosts in cluster failed to respond.
*/
cpr::Response remove(const std::string &indexName,
const std::string &docType,
const std::string &id,
const std::string &routing = std::string());
} }1.5.2使用样例
cpp
#include <cpr/response.h>
#include <elasticlient/client.h>
#include <iostream>
const std::string index_name = "student";
const std::string doc_type = "_doc";
const std::string doc_id = "default_id";
void add_index(elasticlient::Client& client){
std::string body = R"(
{
"settings": {
"analysis": {
"analyzer": {
"ikmax": {
"type": "custom",
"tokenizer": "ik_max_word"
}
}
}
},
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "text",
"boost": 3,
"analyzer": "ikmax"
},
"age": {
"type": "integer"
},
"phone": {
"type": "keyword",
"boost": 1
},
"skills": {
"type": "text"
},
"birth": {
"type": "date",
"index": false,
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
})";
auto resp = client.performRequest(elasticlient::Client::HTTPMethod::PUT,index_name,body);
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "创建索引失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}
}
void add_data(elasticlient::Client& client){
std::string body = R"({
"name": "张三",
"age": 19,
"phone": "13888888888",
"skills": ["java","php","go"],
"birth": "2007-05-21 12:35:32"})";
auto resp = client.index(index_name,doc_type,"1",body);
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "添加数据失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}
}
void mod_data(elasticlient::Client& client){
//注意必须是全量插入，否则会导致除了更新字段其他字段全部被删除
std::string body = R"({
"name": "张三",
"age": 19,
"phone": "13333333333",
"skills": ["java","php","go"],
"birth": "2007-05-21 12:35:32"})";
//插入进行数据修改，也就是先删除原有数据再进行新增，一般不使用update方法对数据进行更新，此三方库中也并不提供update方法
auto resp = client.index(index_name,doc_type,"1",body);
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "修改数据失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}
}
void search_data(elasticlient::Client& client){
//按照测试函数的顺序执行这几个测试函数，会因为请求过快而查不到对应数据
std::string query = R"({"query": {"match_all": {}}})";
auto resp = client.search(index_name,doc_type,query);
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "搜索数据失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}else{
std::cout << "搜索结果:" << resp.text << std::endl;
}
}
void remove_data(elasticlient::Client& client){
auto resp = client.remove(index_name,doc_type,"1");
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "删除数据失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}
}
void remove_index(elasticlient::Client& client){
auto resp = client.performRequest(elasticlient::Client::HTTPMethod::DELETE,index_name,"");
if(resp.status_code < 200 || resp.status_code >= 300){
std::cerr << "删除索引失败,响应码:" << resp.status_code << ",响应信息:" << resp.text << std::endl;
}
}
int main(){
//设置服务端访问url
const std::string url = "http://elastic:123456@192.168.30.128:9200/";//记得一定要加上最后的/不然会报错
//创建elasticlient客户端
elasticlient::Client client({url});
//测试
add_index(client);//如果存在会创建失败
add_data(client);
mod_data(client);
search_data(client);
remove_data(client);
remove_index(client);
return 0;
}编译运行:
bash
source:source.cc
g++ -std=c++17 $^ -o $@ -lcpr -ljsoncpp -lelasticlient
clean:
rm -f source1.6二次封装
对于ES的操作其实都是通过resful⻛格API实现的，⽽在代码中我们更多是组织出合适的json格式的正⽂，⽽如果使⽤json来进⾏组织，则要求使⽤⼈员对ES各项请求的格式与字段都有深⼊了解才⾏，为了简化使⽤要求，我们的⽬标就是封装出⼀套类能够直接序列化出指定功能的json正⽂。
ES中的操作⽆⾮也就是数据的增删改查操作，我们要做的就是将这些操作中所⽤到的json结构定义出来。
具体要搞出来的结构如下:
其实就是按照我们上面的使用样例的格式走的，因为我们index的分词器设置也就只有设置一个中文分词器而已，所以我们就不搞那么复杂了，具体封装如下:
cpp
#pragma once
#include "limeutil.h"
#include "limelog.h"
#include <unordered_map>
#include <cpr/response.h>
#include <elasticlient/client.h>
#include <iostream>
#include <optional>
namespace limeelastic {
class Base
{
public:
using Ptr = std::shared_ptr<Base>;
Base(const std::string& key);
template<typename T>
void add(const std::string& key, const T& value)//键值对类型数据
{
_value[key] = value;
}
template<typename T>
void append(const std::string& key,const T& value)//数组类型数据
{
_value[key].append(value);
}
virtual std::string to_string();//将对象转化为json字符串
const std::string& get_key() const;//获取当前对象存储值的名称
virtual const Json::Value get_value() const;//获取当前对象存储的值
protected:
std::string _key;//当前对象存储值的名称
Json::Value _value;//当前对象存储的值
};
class Array;
class Object : public Base
{
public:
using Ptr = std::shared_ptr<Object>;
Object(const std::string& key);
void addElement(const Base::Ptr& element);
std::shared_ptr<Object> newObject(const std::string& key);
std::shared_ptr<Array> newArray(const std::string& key);
template<typename T>
std::shared_ptr<T> getElement(const std::string& key)
{
auto it = _elements.find(key);
if (it!= _elements.end()) {
//使用dynamic_pointer_cast向下安全转换类型,仅适用于shared_ptr与weak_ptr
return std::dynamic_pointer_cast<T>(it->second);
}
return nullptr;
}
const Json::Value get_value() const;
std::string to_string() const;
private:
std::unordered_map<std::string, Base::Ptr> _elements;
};
class Array : public Base
{
public:
using Ptr = std::shared_ptr<Array>;
Array(const std::string& key);
void addElement(const Base::Ptr& element);
std::shared_ptr<Object> newObject(const std::string& key);
std::shared_ptr<Array> newArray(const std::string& key);
const Json::Value get_value() const;
std::string to_string() const;
private:
std::vector<Base::Ptr> _elements;
};
class Tokenizer : public Object{
public:
using Ptr = std::shared_ptr<Tokenizer>;
Tokenizer(const std::string& key);
void setTokenizer(const std::string& tokenizer);//设定分词器
void setType(const std::string& type);//设定分词类型
};
class Analyzer : public Object{
public:
using Ptr = std::shared_ptr<Analyzer>;
Analyzer();
Tokenizer::Ptr setTokenizer(const std::string& tokenizer);
};
class Analysis : public Object{
public:
using Ptr = std::shared_ptr<Analysis>;
Analysis();
Analyzer::Ptr setAnalyzer();
};
class Settings : public Object{
public:
using Ptr = std::shared_ptr<Settings>;
Settings();
Analysis::Ptr setAnalysis();
};
class Field : public Object{
public:
using Ptr = std::shared_ptr<Field>;
Field(const std::string& key);
void setType(const std::string& type);//设置字段类型
void setIndex(bool index);//设置是否索引
void setBoost(double boost);//设置权重
void setAnalyzer(const std::string& analyzer);//设置分词器
void setFormat(const std::string& format);//设置格式化器-对于日期这类的特殊字段
};
class Properties : public Object{
public:
using Ptr = std::shared_ptr<Properties>;
Properties();
Field::Ptr addField(const std::string& key);
};
class Mappings : public Object{
public:
using Ptr = std::shared_ptr<Mappings>;
Mappings();
void setDynamic(bool dynamic);//设置是否动态映射
Properties::Ptr setProperties();
};
class Term : public Object{
public:
using Ptr = std::shared_ptr<Term>;
Term(const std::string& field_name);
template<typename T>
void setValue(const T& value)
{
this->add(_field_name, value);
}//设置查询值
private:
std::string _field_name;
};
class Terms : public Object{
public:
using Ptr = std::shared_ptr<Terms>;
Terms(const std::string& field_name);
void addValue(const std::string& value)
{
this->append(_field_name, value);
}//添加查询值
private:
std::string _field_name;
};
class Match : public Object{
public:
using Ptr = std::shared_ptr<Match>;
Match(const std::string& field_name);
template<typename T>
void setValue(const T& value)
{
this->add(_field_name, value);
}//设置查询值
private:
std::string _field_name;
};
class MultiMatch : public Object{
public:
using Ptr = std::shared_ptr<MultiMatch>;
MultiMatch();
void addField(const std::string& field_name);//追加查询字段
template<typename T>
void setQuery(const T& query)
{
this->add("query", query);
}//设置查询语句
};
class Range : public Object{
public:
using Ptr = std::shared_ptr<Range>;
Range(const std::string& field_name);
template<typename T>
void setRange(const T& gte, const T& lte)
{
_sub->add("gte", gte);
_sub->add("lte", lte);
}//设置区间
private:
Object::Ptr _sub;
};
//查询操作的基础类
class QObject : public Object{
public:
using Ptr = std::shared_ptr<QObject>;
QObject(const std::string& key);
Term::Ptr term(const std::string& field_name);//单字段精确匹配
Terms::Ptr terms(const std::string& field_name);//多字段精确匹配
Match::Ptr match(const std::string& field_name);//单字段模糊匹配
MultiMatch::Ptr multi_match();//多字段模糊匹配
Range::Ptr range(const std::string& field_name);//数字字段区间匹配
};
class QArray : public Array{
public:
using Ptr = std::shared_ptr<QArray>;
QArray(const std::string& key);
Term::Ptr term(const std::string& field_name);//单字段精确匹配
Terms::Ptr terms(const std::string& field_name);//多字段精确匹配
Match::Ptr match(const std::string& field_name);//单字段模糊匹配
MultiMatch::Ptr multi_match();//多字段模糊匹配
Range::Ptr range(const std::string& field_name);//数字字段区间匹配
private:
std::string _key;
};
class Must : public QArray{
public:
using Ptr = std::shared_ptr<Must>;
Must();
};
class Should : public QArray{
public:
using Ptr = std::shared_ptr<Should>;
Should();
};
class MustNot : public QArray{
public:
using Ptr = std::shared_ptr<MustNot>;
MustNot();
};
class BoolQuery : public QObject{
public:
using Ptr = std::shared_ptr<BoolQuery>;
BoolQuery();
void minimum_should_match(int num);
Must::Ptr setMust();
Should::Ptr setShould();
MustNot::Ptr setMustNot();
};
class Query : public QObject{
public:
using Ptr = std::shared_ptr<Query>;
Query();
void match_all();
BoolQuery::Ptr setBoolQuery();
Must::Ptr setMust();
Should::Ptr setShould();
MustNot::Ptr setMustNot();
};
class Request {
public:
Request(const std::string &index, const std::string &type, const std::string &op, const std::string &id);
void set_index(const std::string &index);
void set_type(const std::string &type);
void set_op(const std::string &op);
void set_id(const std::string &id);
const std::string& get_index() const;
const std::string& get_type() const;
const std::string& get_op() const;
const std::string& get_id() const;
protected:
std::string _index;
std::string _type;
std::string _op;
std::string _id;
};
class Indexer : public Object,public Request{
public:
using Ptr = std::shared_ptr<Indexer>;
Indexer(const std::string &index);
Settings::Ptr setSettings();
Tokenizer::Ptr setTokenizer(const std::string& tokenizer);//设定分词器
Mappings::Ptr setMappings();
Field::Ptr addField(const std::string& key);//添加字段
};
class Inserter : public Object, public Request {
public:
using ptr = std::shared_ptr<Inserter>;
Inserter(const std::string &index, const std::string &id);
};
class Updater : public Object, public Request {
public:
using ptr = std::shared_ptr<Updater>;
Updater(const std::string &index, const std::string &id);
Object::Ptr doc();
};
class Deleter : public Object, public Request {
public:
using ptr = std::shared_ptr<Deleter>;
Deleter(const std::string &index, const std::string &id);
};
class Searcher: public QObject, public Request{
public:
using Ptr = std::shared_ptr<Searcher>;
Searcher(const std::string &index);
Query::Ptr setQuery();
void size(size_t count);
void from(size_t offest);
void addSort(const std::string& field_name, bool asc = true);
};
class BaseClinet{
public:
using Ptr = std::shared_ptr<BaseClinet>;
BaseClinet() = default;
virtual ~BaseClinet() = default;
virtual bool addIndex(const Indexer& indexer) = 0;
virtual bool addData(const Inserter& inserter) = 0;
virtual bool updateData(const Updater& updater) = 0;
virtual bool deleteData(const Deleter& deleter) = 0;
virtual bool deleteIndex(const std::string& index) = 0;
virtual std::optional<Json::Value> searchData(const Searcher& searcher) = 0;
};
class EscClient : public BaseClinet{
public:
using Ptr = std::shared_ptr<EscClient>;
EscClient(const std::vector<std::string>& hosts);
bool addIndex(const Indexer& indexer) override;
bool addData(const Inserter& inserter) override;
bool updateData(const Updater& updater) override;
bool deleteData(const Deleter& deleter) override;
bool deleteIndex(const std::string& index) override;
std::optional<Json::Value> searchData(const Searcher& searcher) override;
private:
std::shared_ptr<elasticlient::Client> _client;
};
}// namespace limeelastic
cpp
#include "limeelastic.h"
namespace limeelastic {
Base::Base(const std::string& key) :_key(key) {}
std::string Base::to_string()
{
Json::Value root;
root[_key] = _value;
return *limeutil::LimeJson::serialize(root,true);
}
const std::string& Base::get_key() const
{
return _key;
}
const Json::Value Base::get_value() const
{
return _value;
}
Object::Object(const std::string& key) : Base(key) {}
void Object::addElement(const Base::Ptr& element)
{
_elements[element->get_key()] = element;
}
std::shared_ptr<Object> Object::newObject(const std::string& key)
{
auto is_exist = this->getElement<Object>(key);
if (is_exist) {
return is_exist;
}
auto obj = std::make_shared<Object>(key);
this->addElement(obj);
return obj;
}
std::shared_ptr<Array> Object::newArray(const std::string& key)
{
auto is_exist = this->getElement<Array>(key);
if (is_exist) {
return is_exist;
}
auto arr = std::make_shared<Array>(key);
this->addElement(arr);
return arr;
}
const Json::Value Object::get_value() const
{
Json::Value root = _value;
for (auto& element : _elements) {
root[element.first] = element.second->get_value();
}
return root;
}
std::string Object::to_string() const
{
Json::Value root = _value;//当前对象存储的值
for (auto& element : _elements) {//子对象存储的值
root[element.first] = element.second->get_value();
}
return *limeutil::LimeJson::serialize(root,true);
}
Array::Array(const std::string& key) : Base(key) {}
void Array::addElement(const Base::Ptr& element)
{
_elements.push_back(element);
}
std::shared_ptr<Object> Array::newObject(const std::string& key)
{
//这里不管原本是否存在目标元素，添加重复了是使用者的问题
auto obj = std::make_shared<Object>(key);
this->addElement(obj);
return obj;
}
std::shared_ptr<Array> Array::newArray(const std::string& key)
{
//这里不管原本是否存在目标元素，添加重复了是使用者的问题
auto arr = std::make_shared<Array>(key);
this->addElement(arr);
return arr;
}
const Json::Value Array::get_value() const
{
Json::Value root = _value;
for (auto& element : _elements) {
root.append(element->get_value());
}
return root;
}
std::string Array::to_string() const
{
Json::Value root = _value;
for (auto& element : _elements) {
root.append(element->get_value());
}
return *limeutil::LimeJson::serialize(root,true);
}
Tokenizer::Tokenizer(const std::string& key) : Object(key) {
this->setTokenizer("ik_max_word");//默认设置
this->setType("custom");
}
void Tokenizer::setTokenizer(const std::string& tokenizer)
{
this->add("tokenizer", tokenizer);
}
void Tokenizer::setType(const std::string& type)
{
this->add("type", type);
}
Analyzer::Analyzer() : Object("analyzer") {}
Tokenizer::Ptr Analyzer::setTokenizer(const std::string& tokenizer)
{
auto is_exist = this->getElement<Tokenizer>("tokenizer");
if (is_exist) {
return is_exist;
}
auto tokenizer_obj = std::make_shared<Tokenizer>(tokenizer);
this->addElement(tokenizer_obj);
return tokenizer_obj;
}
Analysis::Analysis() : Object("analysis") {}
Analyzer::Ptr Analysis::setAnalyzer()
{
auto is_exist = this->getElement<Analyzer>("analyzer");
if (is_exist) {
return is_exist;
}
auto analyzer_obj = std::make_shared<Analyzer>();
this->addElement(analyzer_obj);
return analyzer_obj;
}
Settings::Settings() : Object("settings") {}
Analysis::Ptr Settings::setAnalysis()
{
auto is_exist = this->getElement<Analysis>("analysis");
if (is_exist) {
return is_exist;
}
auto analysis_obj = std::make_shared<Analysis>();
this->addElement(analysis_obj);
return analysis_obj;
}
Field::Field(const std::string& key) : Object(key) {
this->setType("text");//默认设置字段类型为text
}
void Field::setType(const std::string& type)
{
this->add("type", type);
}
void Field::setIndex(bool index)
{
this->add("index", index);
}
void Field::setBoost(double boost)
{
this->add("boost", boost);
}
void Field::setAnalyzer(const std::string& analyzer)
{
this->add("analyzer", analyzer);
}
void Field::setFormat(const std::string& format)
{
this->add("format", format);
}
Properties::Properties() : Object("properties") {}
Field::Ptr Properties::addField(const std::string& key)
{
//对字段进行查找，如果存在则不添加
auto is_exist = this->getElement<Field>(key);
if (is_exist) {
return is_exist;
}
auto field_obj = std::make_shared<Field>(key);
this->addElement(field_obj);
return field_obj;
}
Mappings::Mappings() : Object("mappings") {}
void Mappings::setDynamic(bool dynamic)
{
this->add("dynamic", dynamic);
}
Properties::Ptr Mappings::setProperties()
{
auto is_exist = this->getElement<Properties>("properties");
if (is_exist) {
return is_exist;
}
auto properties_obj = std::make_shared<Properties>();
this->addElement(properties_obj);
return properties_obj;
}
Term::Term(const std::string& field_name)
: Object("term")
, _field_name(field_name)
{}
Terms::Terms(const std::string& field_name)
: Object("terms")
, _field_name(field_name)
{}
Match::Match(const std::string& field_name)
: Object("match")
, _field_name(field_name)
{}
MultiMatch::MultiMatch()
: Object("multi_match")
{}
void MultiMatch::addField(const std::string& field_name)
{
this->append("fields", field_name);
}
Range::Range(const std::string& field_name)
: Object("range")
{
_sub = std::make_shared<Object>(field_name);
this->addElement(_sub);
}
QObject::QObject(const std::string& key) : Object(key) {}
Term::Ptr QObject::term(const std::string& field_name)
{
auto is_exist = this->getElement<Term>("term");
if (is_exist) {
return is_exist;
}
auto term_obj = std::make_shared<Term>(field_name);
this->addElement(term_obj);
return term_obj;
}
Terms::Ptr QObject::terms(const std::string& field_name)
{
auto is_exist = this->getElement<Terms>("terms");
if (is_exist) {
return is_exist;
}
auto terms_obj = std::make_shared<Terms>(field_name);
this->addElement(terms_obj);
return terms_obj;
}
Match::Ptr QObject::match(const std::string& field_name)
{
auto is_exist = this->getElement<Match>("match");
if (is_exist) {
return is_exist;
}
auto match_obj = std::make_shared<Match>(field_name);
this->addElement(match_obj);
return match_obj;
}
MultiMatch::Ptr QObject::multi_match()
{
auto is_exist = this->getElement<MultiMatch>("multi_match");
if (is_exist) {
return is_exist;
}
auto multi_match_obj = std::make_shared<MultiMatch>();
this->addElement(multi_match_obj);
return multi_match_obj;
}
Range::Ptr QObject::range(const std::string& field_name)
{
auto is_exist = this->getElement<Range>("range");
if (is_exist) {
return is_exist;
}
auto range_obj = std::make_shared<Range>(field_name);
this->addElement(range_obj);
return range_obj;
}
QArray::QArray(const std::string& key)
: Array(key)
, _key(key)
{}
Term::Ptr QArray::term(const std::string& field_name)
{
//不考虑重复添加
auto term_obj = std::make_shared<Term>(field_name);
auto obj = this->newObject("");
obj->addElement(term_obj);
return term_obj;
}
Terms::Ptr QArray::terms(const std::string& field_name)
{
//不考虑重复添加
auto terms_obj = std::make_shared<Terms>(field_name);
auto obj = this->newObject("");
obj->addElement(terms_obj);
return terms_obj;
}
Match::Ptr QArray::match(const std::string& field_name)
{
//不考虑重复添加
auto match_obj = std::make_shared<Match>(field_name);
auto obj = this->newObject("");
obj->addElement(match_obj);
return match_obj;
}
MultiMatch::Ptr QArray::multi_match()
{
//不考虑重复添加
auto multi_match_obj = std::make_shared<MultiMatch>();
auto obj = this->newObject("");
obj->addElement(multi_match_obj);
return multi_match_obj;
}
Range::Ptr QArray::range(const std::string& field_name)
{
//不考虑重复添加
auto range_obj = std::make_shared<Range>(field_name);
auto obj = this->newObject("");
obj->addElement(range_obj);
return range_obj;
}
Must::Must() : QArray("must") {}
Should::Should() : QArray("should") {}
MustNot::MustNot() : QArray("must_not") {}
BoolQuery::BoolQuery() : QObject("bool") {}
void BoolQuery::minimum_should_match(int num)
{
this->add("minimum_should_match", num);
}
Must::Ptr BoolQuery::setMust()
{
auto is_exist = this->getElement<Must>("must");
if (is_exist) {
return is_exist;
}
auto must_obj = std::make_shared<Must>();
this->addElement(must_obj);
return must_obj;
}
Should::Ptr BoolQuery::setShould()
{
auto is_exist = this->getElement<Should>("should");
if (is_exist) {
return is_exist;
}
auto should_obj = std::make_shared<Should>();
this->addElement(should_obj);
return should_obj;
}
MustNot::Ptr BoolQuery::setMustNot()
{
auto is_exist = this->getElement<MustNot>("must_not");
if (is_exist) {
return is_exist;
}
auto must_not_obj = std::make_shared<MustNot>();
this->addElement(must_not_obj);
return must_not_obj;
}
Query::Query() : QObject("query") {}
void Query::match_all()
{
this->add("match_all",Json::Value(Json::ValueType::objectValue));
}
BoolQuery::Ptr Query::setBoolQuery()
{
auto is_exist = this->getElement<BoolQuery>("bool");
if (is_exist) {
return is_exist;
}
auto bool_obj = std::make_shared<BoolQuery>();
this->addElement(bool_obj);
return bool_obj;
}
Must::Ptr Query::setMust()
{
auto is_exist = this->getElement<Must>("must");
if (is_exist) {
return is_exist;
}
auto must_obj = std::make_shared<Must>();
this->addElement(must_obj);
return must_obj;
}
Should::Ptr Query::setShould()
{
auto is_exist = this->getElement<Should>("should");
if (is_exist) {
return is_exist;
}
auto should_obj = std::make_shared<Should>();
this->addElement(should_obj);
return should_obj;
}
MustNot::Ptr Query::setMustNot()
{
auto is_exist = this->getElement<MustNot>("must_not");
if (is_exist) {
return is_exist;
}
auto must_not_obj = std::make_shared<MustNot>();
this->addElement(must_not_obj);
return must_not_obj;
}
Request::Request(const std::string &index, const std::string &type, const std::string &op, const std::string &id)
: _index(index)
, _type(type)
, _op(op)
, _id(id)
{}
void Request::set_index(const std::string &index) { _index = index; };
void Request::set_type(const std::string &type) { _type = type; };
void Request::set_op(const std::string &op) { _op = op; };
void Request::set_id(const std::string &id) { _id = id; };
const std::string& Request::get_index() const { return _index; };
const std::string& Request::get_type() const { return _type; };
const std::string& Request::get_op() const { return _op; };
const std::string& Request::get_id() const { return _id; };
Indexer::Indexer(const std::string &index)
: Object("")
, Request(index, "_doc", "_index", index)
{}
Settings::Ptr Indexer::setSettings()
{
auto is_exist = this->getElement<Settings>("settings");
if (is_exist) {
return is_exist;
}
auto settings_obj = std::make_shared<Settings>();
this->addElement(settings_obj);
return settings_obj;
}
Tokenizer::Ptr Indexer::setTokenizer(const std::string& tokenizer)
{
return this->setSettings()->setAnalysis()->setAnalyzer()->setTokenizer(tokenizer);
}
Mappings::Ptr Indexer::setMappings()
{
auto is_exist = this->getElement<Mappings>("mappings");
if (is_exist) {
return is_exist;
}
auto mappings_obj = std::make_shared<Mappings>();
this->addElement(mappings_obj);
return mappings_obj;
}
Field::Ptr Indexer::addField(const std::string& key)
{
return this->setMappings()->setProperties()->addField(key);
}
Inserter::Inserter(const std::string &index, const std::string &id)
: Object("")
, Request(index, "_doc", "_insert", id)
{}
Updater::Updater(const std::string &index, const std::string &id)
: Object("")
, Request(index, "_doc", "_update", id)
{}
Object::Ptr Updater::doc()
{
auto is_exist = this->getElement<Object>("doc");
if (is_exist) {
return is_exist;
}
auto doc_obj = std::make_shared<Object>("doc");
this->addElement(doc_obj);
return doc_obj;
}
Deleter::Deleter(const std::string &index, const std::string &id)
: Object("")
, Request(index, "_doc", "_delete", id)
{}
Searcher::Searcher(const std::string &index)
: QObject("")
, Request(index, "_doc", "_search", "")
{}
Query::Ptr Searcher::setQuery()
{
auto is_exist = this->getElement<Query>("query");
if (is_exist) {
return is_exist;
}
auto query_obj = std::make_shared<Query>();
this->addElement(query_obj);
return query_obj;
}
void Searcher::size(size_t count) {
this->add("size", count);
}
void Searcher::from(size_t offset) {
this->add("from", offset);
}
void Searcher::addSort(const std::string& field_name, bool asc) {
Json::Value sort_obj;
sort_obj[field_name] = asc ? "asc" : "desc";
this->append("sort", sort_obj);//重复添加排序规则是使用者的问题
}
EscClient::EscClient(const std::vector<std::string>& hosts)
: _client(std::make_shared<elasticlient::Client>(hosts))
{}
bool EscClient::addIndex(const Indexer& indexer)
{
std::string index_name = indexer.get_index();
std::string body = indexer.to_string();
auto resp = _client->performRequest(elasticlient::Client::HTTPMethod::PUT,index_name,body);
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("创建索引失败,响应码:{},响应信息:{}", resp.status_code, resp.text);
return false;
}
return true;
}
bool EscClient::addData(const Inserter& inserter)
{
std::string index_name = inserter.get_index();
std::string doc_type = inserter.get_type();
std::string id = inserter.get_id();
std::string body = inserter.to_string();
auto resp = _client->index(index_name,doc_type,id,body);
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("添加数据失败,响应码:{},响应信息:{}", resp.status_code, resp.text);
return false;
}
return true;
}
bool EscClient::updateData(const Updater& updater)
{
std::string index_name = updater.get_index();
std::string op = updater.get_op();
std::string id = updater.get_id();
std::string body = updater.to_string();
std::string url = index_name + "/" + op + "/" + id;
auto resp = _client->performRequest(elasticlient::Client::HTTPMethod::POST,url,body);
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("更新数据失败,响应码:{},响应信息:{}", resp.status_code, resp.text);
return false;
}
return true;
}
bool EscClient::deleteData(const Deleter& deleter)
{
std::string index_name = deleter.get_index();
std::string doc_type = deleter.get_type();
std::string id = deleter.get_id();
auto resp = _client->remove(index_name,doc_type,id);
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("删除数据失败,响应码:{},响应信息:{}", resp.status_code, resp.text);
return false;
}
return true;
}
bool EscClient::deleteIndex(const std::string& index)
{
std::string index_name = index;
auto resp = _client->performRequest(elasticlient::Client::HTTPMethod::DELETE,index_name,"");
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("删除索引失败,响应码:{},响应信息:{}", resp.status_code, resp.text);
return false;
}
return true;
}
std::optional<Json::Value> EscClient::searchData(const Searcher& searcher)
{
std::string index_name = searcher.get_index();
std::string doc_type = searcher.get_type();
std::string body = searcher.to_string();
auto resp = _client->search(index_name,doc_type,body);
if(resp.status_code < 200 || resp.status_code >= 300){
ERR("搜索数据失败,响应码:{},响应信息:{},请求正文:{}", resp.status_code, resp.text,body);
return std::nullopt;
}
//解析响应数据
auto json_resp = limeutil::LimeJson::deserialize(resp.text);
if(!json_resp)
{
ERR("解析响应数据失败:{}", resp.text);
return std::nullopt;
}
//获取查询结果
if(json_resp->isNull() || (*json_resp)["hits"].isNull() || (*json_resp)["hits"]["hits"].isNull())
{
ERR("无法正确解析结果:{}", resp.text);
return std::nullopt;
}
Json::Value result;
int sz = (*json_resp)["hits"]["hits"].size();
for(int i=0;i<sz;i++)
{
result.append((*json_resp)["hits"]["hits"][i]["_source"]);
}
return result;
}
} // namespace limeelastic使用样例
cpp
#include "../../source/limeelastic.h"
limeelastic::Indexer test_indexer(){
limeelastic::Indexer indexer("student");
//设置字段是否动态映射
indexer.setMappings()->setDynamic(false);
//添加字段
auto field = indexer.addField("name");
field->setType("text");
field->setBoost(3.0);
field->setAnalyzer("ikmax");
auto field2 = indexer.addField("age");
field2->setType("integer");
auto field3 = indexer.addField("phone");
field3->setType("keyword");
field3->setBoost(1.0);
auto field4 = indexer.addField("birthday");
field4->setType("date");
field4->setIndex(false);
field4->setFormat("yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis");
auto filed5 = indexer.addField("skills");
filed5->setType("text");
//设置分词器
auto tokenizer = indexer.setTokenizer("ikmax");
//打印索引配置
//std::cout << indexer.to_string() << std::endl;
return indexer;
}
void test_query()
{
auto searchQuery = std::make_shared<limeelastic::Searcher>("student");
auto query = searchQuery->setQuery();
//设置match_all查询
query->match_all();
//设置term查询
auto term = query->term("name.keyword");
term->setValue("lisi");
//设置terms查询
auto terms = query->terms("phone");
terms->addValue("15511111111");
terms->addValue("15522222222");
//设置match查询
auto match = query->match("name");
match->setValue("z");
//设置multi_match查询
auto multi_match = query->multi_match();
multi_match->setQuery("zhangsan");
multi_match->addField("name");
multi_match->addField("phone");
//设置range查询
auto range = query->range("age");
range->setRange(10,20);
//打印查询配置
std::cout << searchQuery->to_string() << std::endl;
}
void test_bool()
{
auto searchQuery = std::make_shared<limeelastic::Searcher>("student");
auto boolQuery = searchQuery->setQuery()->setBoolQuery();
boolQuery->minimum_should_match(1);
//设置bool查询
auto must = boolQuery->setMust();
auto match = must->match("skills");
match->setValue("c++");
auto should = boolQuery->setShould();
auto match2 = should->match("name");
match2->setValue("lisi");
auto terms1 = should->terms("phone");
terms1->addValue("15511111111");
terms1->addValue("15555555555");
auto must_not = boolQuery->setMustNot();
auto terms = must_not->terms("phone");
terms->addValue("15522222222");
terms->addValue("15555555555");
//设置sort
searchQuery->addSort("age", true);
searchQuery->addSort("birthday", false);
//打印查询配置
std::cout << searchQuery->to_string() << std::endl;
}
int main(){
limelog::limelog_init();
limeelastic::EscClient esc_client({"http://elastic:123456@192.168.30.128:9200/"});
// esc_client.addIndex(test_indexer());
// limeelastic::Inserter inserter("student","1");
// inserter.add("name","张三");
// inserter.add("age",20);
// inserter.add("phone","13888888888");
// inserter.append("skills","c++");
// inserter.append("skills","php");
// inserter.append("skills","go");
// inserter.add("birthday","2007-05-21 12:35:32");
// esc_client.addData(inserter);
// limeelastic::Updater updater("student","1");
// updater.doc()->add("phone","13333333333");
// esc_client.updateData(updater);
limeelastic::Searcher searcher("student");
searcher.setQuery()->match("name")->setValue("lisi");
searcher.size(2);
searcher.from(2);
auto result = esc_client.searchData(searcher);
if(result)
{
INF("查询到的目标数据为:{}",*limeutil::LimeJson::serialize(*result,true));
// //进行数据删除
// limeelastic::Deleter deleter("student","1");
// esc_client.deleteData(deleter);
// //进行索引删除
// esc_client.deleteIndex("student");
}
return 0;
}编译构建:
cpp
test:test.cc ../../source/limeelastic.cc ../../source/limeutil.cc ../../source/limelog.cc
g++ $^ -o $@ -std=c++17 -lcpr -ljsoncpp -lelasticlient -lspdlog -lpthread -lfmt
clean:
rm test这样一来就不需要我们自己再去一个一个的组织格式然后发给elastic服务端了，方便很多。
2.libcurl的使用与封装
libcurl 是⼀个跨平台、开源的客⼾端⽹络传输库，⽀持多种协议（如 HTTP、HTTPS、FTP、SMTP 等），⼴泛应⽤于⽹络通信、数据抓取、API 交互等场景。
安装命令如下:
bash
sudo apt install libcurl4-openssl-dev2.1启用邮箱授权码(以163邮箱为例)
因为我们是通过libcurl实现邮件推送客⼾端，因此先在这⾥启⽤邮箱客⼾端授权码，这样才能便于使⽤。
163邮箱官网地址:https://mail.163.com/
2.2使用
2.2.1头文件与链接库
cpp
#include <curl/curl.h>
cpp
-lcurl -lssl -lcrypto2.2.2核心接口
cpp
#define CURL_GLOBAL_SSL (1<<0) /* no purpose since since 7.57.0 */
#define CURL_GLOBAL_WIN32 (1<<1)
#define CURL_GLOBAL_ALL (CURL_GLOBAL_SSL|CURL_GLOBAL_WIN32)
#define CURL_GLOBAL_NOTHING 0
#define CURL_GLOBAL_DEFAULT CURL_GLOBAL_ALL
#define CURL_GLOBAL_ACK_EINTR (1<<2)
// 初始化全局配置
CURLcode curl_global_init(long flags);
// 销毁全局配置
void curl_global_cleanup(void);
// 创建并初始化curl操作句柄
CURL *curl_easy_init(void);
// 设置curl操作选项
CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
// 同步阻塞函数，会⼀次性执⾏所有通过curl_easy_setopt 设置的选项
// 如： URL,请求⽅法,回调函数等
CURLcode curl_easy_perform(CURL *curl);
// 仅清除通过 curl_easy_setopt 设置的选项，恢复为初始状态
void curl_easy_reset(CURL *curl);
// 关闭所有与该句柄相关的连接，释放所有资源
void curl_easy_cleanup(CURL *curl);
// 获取上次函数执⾏错误信息
const char *curl_easy_strerror(CURLcode);2.2.3链表操作
cpp
struct curl_slist {
char *data;
struct curl_slist *next;
};
struct curl_slist *curl_slist_append(struct curl_slist *, const char *);
void curl_slist_free_all(struct curl_slist *);2.2.4http选项
| 选项 | 功能 | 备注 |
|---|---|---|
| CURLOPT_URL | 设置请求 URL | 所有请求的基础配置 |
| CURLOPT_READFUNCTION | 处理请求数据的回调函数 | 添加请求正文 |
| CURLOPT_WRITEDATA | 设置给请求回调函数的用户数据 | 文件句柄或数据对象 |
| CURLOPT_WRITEFUNCTION | 处理响应数据的回调函数 | 下载文件或保存 API 响应 |
| CURLOPT_POSTFIELDS | 指定 POST 请求正文 | 提交表单或 JSON 数据 |
| CURLOPT_HTTPHEADER | 自定义 HTTP 头部 | 设置认证头或内容类型 |
| CURLOPT_FOLLOWLOCATION | 自动跟随重定向 | 处理短链接或跳转页面 |
| CURLOPT_VERBOSE | 输出调试信息 | 开发阶段问题排查 |
| CURLOPT_SSL_VERIFYPEER | 控制是否验证对端证书 | 生产环境不推荐使用 |
| CURLOPT_TIMEOUT | 设置传输超时时间 | 请求到完成 |
| CURLOPT_CONNECTTIMEOUT | 设置连接超时时间 | 握手超时时间 |
2.2.5SMTP选项
| 选项 | 功能 | 典型值示例 |
|---|---|---|
| CURLOPT_URL | SMTP服务器地址 | "smtps://smtp.example.com:465" |
| CURLOPT_MAIL_FROM | 发件人地址 | "user@example.com" |
| CURLOPT_MAIL_RCPT | 收件人列表（链表） | curl_slist.append(recipients, "to@example.com") |
| CURLOPT_READFUNCTION | 处理请求数据的回调函数 | 添加请求正文 |
| CURLOPT_READDATA | 通信正文内容 | 代码块 1 From: from@example.com\r\n 2 To: to@example.com\r\n 3 Subject: title\r\n 4 Content-Type: text/html\r\n 5 \r\n 6 Body\r\n |
| CURLOPT_SSL_VERIFYPEER | 启用服务器证书验证 | 1L（启用）或 0L（禁用） |
| CURLOPT_USERNAME | SMTP登录用户名 | "user@example.com" |
| CURLOPT_PASSWORD | SMTP登录密码 | |
| CURLOPT_USE_SSL | 控制SSL/TLS加密行为 | |
| CURLOPT_UPLOAD | 启用上传模式 | 1L（启用）或 0L（禁用），与CURLOPT_READDATA & CURLOPT_READFUNCTION搭配使用 |
2.3二次封装
因为这里的使用基本是按照流程走的，所以使用样例我们也放到二次封装处即可。我们的二次封装主要也就以下几个内容:
验证码发送客户端类：
- 发送邮件函数sendCode
- 设置邮件正文函数
具体实现如下:
cpp
#pragma once
#include <curl/curl.h>
#include <iostream>
#include <sstream>
#include <memory>
namespace limemail {
struct EmailCodeSettings{
std::string username;
std::string password;
std::string url;
std::string from;
};
class CodeClient{
public:
CodeClient() = default;
virtual ~CodeClient() = default;
virtual bool sendCode(const std::string& to, const std::string& code) = 0;
};
class EmailClient : public CodeClient{
public:
using Ptr = std::shared_ptr<EmailClient>;
EmailClient(const EmailCodeSettings& settings);
bool sendCode(const std::string& to, const std::string& code) override;
~EmailClient();
private:
std::stringstream requestBody(const std::string& to, const std::string& code);
static size_t callback(char *buffer, size_t size, size_t nitems, void *userdata);
private:
EmailCodeSettings _settings;//不要保存curl句柄，因为他是线程不安全的，最好是每次使用时直接获取与释放
std::string _subject = "limeplayer登录/注册验证码通知";
};
} // namespace limemail
cpp
#include "limemail.h"
#include "limelog.h"
namespace limemail {
const std::string htmlTemplate = R"(<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Limeplayer登录/注册验证码通知</title>
<style>
body {
font-family: 'Arial', 'Helvetica Neue', sans-serif;
margin: 0;
padding: 20px;
background: linear-gradient(135deg, #fffdbf 0%, #c9f7c9 100%);
min-height: 100vh;
}
.container {
max-width: 600px;
margin: 0 auto;
background: white;
padding: 40px;
border-radius: 16px;
box-shadow: 0 10px 30px rgba(0,0,0,0.1);
position: relative;
overflow: hidden;
}
/* 夏日装饰元素 */
.container::before {
content: '';
position: absolute;
top: 0;
right: 0;
width: 120px;
height: 120px;
background: linear-gradient(45deg, #ffeb3b 0%, #ff9800 100%);
border-radius: 0 0 0 100px;
z-index: 0;
}
.lemon {
position: absolute;
width: 40px;
height: 40px;
background: #fff176;
border-radius: 50%;
top: 25px;
right: 25px;
z-index: 1;
box-shadow: 0 0 10px rgba(255, 193, 7, 0.5);
}
.lemon::before {
content: '';
position: absolute;
width: 5px;
height: 15px;
background: #388e3c;
left: 18px;
top: -10px;
border-radius: 3px;
}
.lemon::after {
content: '';
position: absolute;
width: 30px;
height: 30px;
border-radius: 50%;
border: 2px dashed rgba(255, 255, 255, 0.7);
top: 5px;
left: 5px;
}
.content {
position: relative;
z-index: 2;
}
h2 {
color: #388e3c;
margin-top: 0;
font-size: 24px;
border-left: 4px solid #ff9800;
padding-left: 15px;
}
p {
line-height: 1.6;
color: #424242;
}
.code-container {
text-align: center;
margin: 30px 0;
padding: 20px;
background: #f9fbe7;
border-radius: 10px;
border: 2px dashed #ff9800;
}
.code {
font-size: 40px;
color: #ff6f00;
font-weight: bold;
letter-spacing: 8px;
margin: 10px 0;
text-shadow: 1px 1px 3px rgba(0,0,0,0.1);
}
.expiry {
color: #f44336;
font-size: 14px;
font-weight: bold;
display: flex;
align-items: center;
justify-content: center;
}
.expiry::before {
content: '⏰';
margin-right: 5px;
font-size: 16px;
}
.footer {
margin-top: 40px;
padding-top: 20px;
border-top: 1px solid #e0e0e0;
color: #757575;
font-size: 12px;
text-align: center;
}
.lemon-slice {
display: inline-block;
font-size: 18px;
margin: 0 3px;
}
</style>
</head>
<body>
<div class="container">
<div class="lemon"></div>
<div class="content">
<h2>尊敬的用户{{USER_NAME}}，您好！</h2>
<p>感谢您使用Limeplayer！您正在进行的操作需要验证身份，请输入以下验证码：</p>
<div class="code-container">
<div class="code">{{VERIFICATION_CODE}}</div>
</div>
<p class="expiry">此验证码将在 5 分钟内失效，请尽快使用</p>
<p>如果这不是您本人的操作，请立即忽略此邮件。</p>
<div class="footer">
<p>
<span class="lemon-slice">🍋</span>
系统自动发送，请勿回复
<span class="lemon-slice">🍋</span>
</p>
</div>
</div>
</div>
</body>
</html>)";
EmailClient::EmailClient(const EmailCodeSettings& settings)
:_settings(settings)
{
//初始化全局配置
auto ret = curl_global_init(CURL_GLOBAL_DEFAULT);
if (ret != CURLE_OK) {
ERR("curl全局初始化失败, 错误信息: {}", curl_easy_strerror(ret));
exit(-1);//直接退出程序
}
}
bool EmailClient::sendCode(const std::string& to, const std::string& code)
{
//构造操作句柄
auto curl = curl_easy_init();
if (curl == nullptr) {
ERR("获取curl操作句柄失败");
return false;
}
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 15L); // 连接超时15秒
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 30L); // 传输超时30秒
//设置请求参数：url，username，password，body, from, to
auto ret = curl_easy_setopt(curl, CURLOPT_URL, _settings.url.c_str());
if(ret != CURLE_OK)
{
ERR("设置请求url失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
ret = curl_easy_setopt(curl, CURLOPT_USERNAME, _settings.username.c_str());
if(ret != CURLE_OK)
{
ERR("设置用户名失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
ret = curl_easy_setopt(curl, CURLOPT_PASSWORD, _settings.password.c_str());
if(ret != CURLE_OK)
{
ERR("设置密码失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
ret = curl_easy_setopt(curl, CURLOPT_MAIL_FROM, _settings.from.c_str());
if(ret != CURLE_OK)
{
ERR("设置发件人失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
struct curl_slist *cs = nullptr;
cs = curl_slist_append(cs, to.c_str());
ret = curl_easy_setopt(curl, CURLOPT_MAIL_RCPT, cs);
if(ret != CURLE_OK)
{
ERR("设置收件人失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
std::stringstream ss = requestBody(to, code);
ret = curl_easy_setopt(curl, CURLOPT_READDATA, (void*)&ss);
if(ret != CURLE_OK)
{
ERR("设置请求体失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
//设置回调函数
ret = curl_easy_setopt(curl, CURLOPT_READFUNCTION, EmailClient::callback);
if(ret != CURLE_OK)
{
ERR("设置回调函数失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
ret = curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
if(ret != CURLE_OK)
{
ERR("设置上传模式失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
//执行请求
ret = curl_easy_perform(curl);
if(ret != CURLE_OK)
{
ERR("执行请求失败, 错误信息: {}", curl_easy_strerror(ret));
return false;
}
//释放资源
curl_slist_free_all(cs);
curl_easy_cleanup(curl);
DBG("发送验证码成功:{}-{}", to, code);
return true;
}
EmailClient::~EmailClient()
{
//释放全局资源
curl_global_cleanup();
}
std::stringstream EmailClient::requestBody(const std::string& to, const std::string& code)
{
std::stringstream ss;
ss << "From:"<< "LimePlayer" << "\r\n"
<< "To:" << to << "\r\n"
<< "Subject:"<< _subject <<"\r\n"
<< "Content-Type: text/html; charset=utf-8\r\n"
<< "\r\n";
std::string html = htmlTemplate;
size_t pos = html.find("{{VERIFICATION_CODE}}");
if (pos != std::string::npos) {
html.replace(pos, std::string("{{VERIFICATION_CODE}}").length(), code);
}
// 用户名替换
pos = html.find("{{USER_NAME}}");
if (pos != std::string::npos) {
html.replace(pos, std::string("{{USER_NAME}}").length(), to);
}
ss << html << "\r\n";
return ss;
}
size_t EmailClient::callback(char *buffer, size_t size, size_t nitems, void *userdata)
{
std::stringstream *ss = (std::stringstream*)userdata;
ss->read(buffer, size * nitems);
return ss->gcount();
}
}// namespace limemail使用样例
cpp
#include "../../source/limemail.h"
#include "../../source/limelog.h"
int main()
{
limelog::limelog_init();
limemail::EmailCodeSettings settings{
.username = "xiu7987knd@163.com",
.password = "111",//输入你开启smtp的授权码
.url = "smtps://smtp.163.com:465",
.from = "xiu7987knd@163.com"
};
auto client = std::make_unique<limemail::EmailClient>(settings);
client->sendCode("ib30583239@126.com","114514");
return 0;
}编译运行:
cpp
mail_test:mail_test.cc ../../source/limelog.cc ../../source/limemail.cc
g++ $^ -o $@ -std=c++17 -lpthread -lcurl -lssl -lcrypto -lgflags -lfmt -lpthread -lspdlog
clean:
rm -f mail_test运行成功之后你应该会在你的目标邮箱中收到如下一封邮件:
3.bug修复
3.1util⼯具类功能补充
描述：在后面设置缓存过期时间的时候需要⽣成⼀个指定范围的随机数，但是当前没有接⼝提供
解决：在Random类中新增⼀个⽣成随机数的接⼝
cpp
size_t LimeRandom::number(size_t min, size_t max) {
std::random_device rd;
std::mt19937 generator(rd());
std::uniform_int_distribution<int> distribution(min, max);
return distribution(generator);
}3.2RPC封装Bug与补充
功能补充
描述：⽬前封装信道管理，是提前创建好信道使⽤时直接获取，信道的使⽤协议是固定的，但是在有
些情况下固定协议的信道⽆法⽀持更加灵活的操作。
解决：在信道管理的同时也增加节点地址管理，并且提供⼀个获取节点地址的接口
bug修复
描述：当前代码中rpc信道的默认超时时间有些短，稍有处理时间有些⻓的接⼝就会导致请求超时失败。
解决：创建信道时将信道的请求超时时间设置的⻓⼀些
cpp
#pragma once
#include <brpc/channel.h>
#include <brpc/server.h>
#include <mutex>
#include <unordered_map>
#include <optional>
#include "limelog.h"
namespace limerpc
{
using ChannelPtr = std::shared_ptr<brpc::Channel>;
// 服务信道管理类
class RpcChannels
{
public:
using ptr = std::shared_ptr<RpcChannels>;
RpcChannels();
// 获取服务信道
ChannelPtr get_channel();
// 增加服务信道
void add_channel(const std::string &addr);
// 删除服务信道
void remove_channel(const std::string &addr);
std::optional<std::string> selectAddr();
private:
std::mutex _mtx; // 互斥锁
uint32_t _idx; // 服务信道索引-轮询下标
std::vector<std::pair<std::string, ChannelPtr>> _channels; // 服务信道列表
};
// 服务管理类
class SvcRpcChannels
{
public:
using ptr = std::shared_ptr<SvcRpcChannels>;
SvcRpcChannels() = default;
// 设置服务关心
void set_match(const std::string &service_name);
// 新增结点
void add_node(const std::string &service_name, const std::string &node_addr);
// 删除结点
void remove_node(const std::string &service_name, const std::string &node_addr);
// 获取服务信道
ChannelPtr get_channel(const std::string &service_name);
private:
std::mutex _mtx;
std::unordered_map<std::string, RpcChannels::ptr> _svc_channels_map; // 服务名称-服务信道管理映射表
};
// 异步回调⼯⼚类
class ClosureFactory
{
public:
using callback_t = std::function<void()>;
static google::protobuf::Closure *create(callback_t &&cb);
private:
struct Object
{
using ptr = std::shared_ptr<Object>;
callback_t callback;
};
static void asyncCallback(const Object::ptr obj);
};
// 服务器⼯⼚类
class RpcServer{
public:
// 默认svc是堆上new出来的对象，将管理权移交给rpc服务器进⾏管理
static std::shared_ptr<brpc::Server> create(int port,google::protobuf::Service *svc);
};
} // namespace limerpc
cpp
#include "limerpc.h"
namespace limerpc{
RpcChannels::RpcChannels():_idx(0){
}
// 轮询获取服务信道
ChannelPtr RpcChannels::get_channel(){
std::unique_lock<std::mutex> lock(_mtx);
if(_channels.size() == 0){
return ChannelPtr();
}
size_t index = _idx % _channels.size();
_idx++;
return _channels[index].second;
}
// 增加服务信道
void RpcChannels::add_channel(const std::string &addr){
std::unique_lock<std::mutex> lock(_mtx);
for(const auto &item : _channels){
if(item.first == addr){
//信道已存在，直接返回
return;
}
}
//1.定义并设置channel的配置
brpc::ChannelOptions options;
options.protocol = "baidu_std"; //使用baidu_std协议
options.timeout_ms = 30000; //设置超时时间为30秒
//2.创建并初始化channel-channel可以理解为客⼾端到服务器的⼀条通信线路
ChannelPtr channel = std::make_shared<brpc::Channel>();
channel->Init(addr.c_str(), &options);
//3.将channel添加到_channels中
_channels.push_back(std::make_pair(addr, channel));
}
// 删除服务信道
void RpcChannels::remove_channel(const std::string &addr){
std::unique_lock<std::mutex> lock(_mtx);
//找到后从_channels中删除
for(auto it = _channels.begin(); it != _channels.end(); it++){
if(it->first == addr){
_channels.erase(it);
return;
}
}
}
std::optional<std::string> RpcChannels::selectAddr(){
std::unique_lock<std::mutex> lock(_mtx);
if(_channels.size() == 0){
return std::nullopt;
}
size_t index = _idx % _channels.size();
_idx++;
return _channels[index].first;
}
//设置服务关心
void SvcRpcChannels::set_match(const std::string &service_name){
std::unique_lock<std::mutex> lock(_mtx);
_svc_channels_map[service_name] = std::make_shared<RpcChannels>();
}
//新增结点
void SvcRpcChannels::add_node(const std::string &service_name, const std::string &node_addr){
//判断是否为关心的服务
auto it = _svc_channels_map.find(service_name);
if(it == _svc_channels_map.end()){
DBG("服务:{} 未设置关注，忽略添加", service_name);
return;
}
//增加服务信道
it->second->add_channel(node_addr);
}
//删除结点
void SvcRpcChannels::remove_node(const std::string &service_name, const std::string &node_addr){
//判断是否为关心的服务
auto it = _svc_channels_map.find(service_name);
if(it == _svc_channels_map.end()){
DBG("服务:{} 未设置关注，忽略删除", service_name);
return;
}
//删除服务信道
it->second->remove_channel(node_addr);
}
//获取服务信道
ChannelPtr SvcRpcChannels::get_channel(const std::string &service_name){
//判断是否为关心的服务
auto it = _svc_channels_map.find(service_name);
if(it == _svc_channels_map.end()){
DBG("服务:{} 未设置关注，无法获取信道", service_name);
return ChannelPtr();
}
//获取服务信道
return it->second->get_channel();
}
//因为google::protobuf::Closure的回调函数不允许传入多个参数，所以需要定义一个Object类来包装回调函数和参数
google::protobuf::Closure* ClosureFactory::create(callback_t &&cb){
ClosureFactory::Object::ptr obj = std::make_shared<ClosureFactory::Object>();
obj->callback = std::move(cb);
return google::protobuf::NewCallback(ClosureFactory::asyncCallback, obj);
}
//异步回调函数执行
void ClosureFactory::asyncCallback(const ClosureFactory::Object::ptr obj){
obj->callback();
}
//服务端创建
std::shared_ptr<brpc::Server> RpcServer::create(int port,google::protobuf::Service *svc){
//1.定义服务器配置对象ServerOptions
brpc::ServerOptions options;
options.idle_timeout_sec = -1;//设置超时时间为-1，表示不超时
//2.创建服务器对象
std::shared_ptr<brpc::Server> server = std::make_shared<brpc::Server>();
//3.注册服务
if (server->AddService(svc, brpc::SERVER_OWNS_SERVICE) != 0) {
ERR("服务注册失败");
exit(-1);//直接退出
}
//4.启动服务器
if (server->Start(port, &options) != 0) {
ERR("服务启动失败");
exit(-1);//直接退出
}
return server;
}
}3.3MQ封装Bug
描述：条件控制问题，⽐如在交换机队列还没有创建绑定完毕，就开始订阅队列消息
解决后修正的代码如下:
cpp
#pragma once
#include <amqpcpp.h>
#include <ev.h>
#include <amqpcpp/libev.h>
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <functional>
namespace limemq {
const std::string DIRECT = "direct";
const std::string FANOUT = "fanout";
const std::string TOPIC = "topic";
const std::string HEADERS = "headers";
const std::string DELAYED = "delayed";
const std::string DLX_PREFIX = "dlx-";
const std::string DEAD_LETTER_EXCHANGE = "x-dead-letter-exchange";
const std::string DEAD_LETTER_BINDING_KEY = "x-dead-letter-routing-key";
const std::string MESSAGE_TTL = "x-message-ttl";
struct declare_settings {
std::string exchange;
std::string exchange_type;// 交换机类型： direct、fanout、topic、headers、delayed
std::string queue;
std::string binding_key;
size_t delayed_ttl = 0;
std::string dlx_exchange() const;
std::string dlx_queue() const;
std::string dlx_binding_key() const;
};
extern AMQP::ExchangeType exchange_type(const std::string &type);
using MessageCallback = std::function<void(const char*, size_t)>;
class MQClient {
public:
using ptr = std::shared_ptr<MQClient>;
MQClient(const std::string &url); // 构造成员，启动事件循环
~MQClient(); // 发送异步请求，结束事件循环，等待异步线程结束
//声明交换机，声明队列，并绑定交换机和队列，如果是延时队列，则还要创建配套的死信交换机和队列
//声明交换机和队列以及绑定成功后，需要等待，等待实际交换机和队列声明成功，再返回
void declare(const declare_settings &settings);
bool publish(const std::string &exchange, const std::string &routing_key, const std::string &body);
void consume(const std::string &queue, const MessageCallback &callback);
void wait();
private:
static void callback(struct ev_loop *loop, ev_async *watcher, int32_t revents);
// 声明常规交换机和队列，并进行绑定
void _declared(const declare_settings &settings, AMQP::Table &args, bool is_dlx = false);
private:
std::mutex _declared_mtx;
std::mutex _mtx;
std::condition_variable _cv;
struct ev_loop *_ev_loop;
struct ev_async _ev_async;
AMQP::LibEvHandler _handler;
AMQP::TcpConnection _connection;
AMQP::TcpChannel _channel;
std::thread _async_thread;
};
class Publisher {
public:
using ptr = std::shared_ptr<Publisher>;
//对成员进行初始化，并声明套件内的交换机和队列
Publisher(const MQClient::ptr &mq_client, const declare_settings &settings);
bool publish(const std::string &body);
private:
MQClient::ptr _mq_client;
declare_settings _settings;
};
class Subscriber {
public:
using ptr = std::shared_ptr<Subscriber>;
//对成员进行初始化，并声明套件内的交换机和队列
Subscriber(const MQClient::ptr &mq_client, const declare_settings &settings);
//如果是延时队列，则实际订阅的是配套的死信队列
void consume(MessageCallback &&callback);
private:
MQClient::ptr _mq_client;
declare_settings _settings;
MessageCallback _callback;
};
}
cpp
#include "limemq.h"
#include "limelog.h"
namespace limemq {
std::string declare_settings::dlx_exchange() const {
return DLX_PREFIX + exchange;
}
std::string declare_settings::dlx_queue() const {
return DLX_PREFIX + queue;
}
std::string declare_settings::dlx_binding_key() const {
return DLX_PREFIX + binding_key;
}
AMQP::ExchangeType exchange_type(const std::string &type) {
if(type == DIRECT)
{
//直接匹配
return AMQP::direct;
}
else if(type == FANOUT)
{
//广播类型
return AMQP::fanout;
}
else if(type == TOPIC)
{
//主题类型
return AMQP::topic;
}
else if(type == HEADERS)
{
//头部匹配
return AMQP::headers;
}
else if(type == DELAYED)
{
return AMQP::direct;
}
else
{
WRN("未知交换机类型:{}, 使用默认类型direct", type);
return AMQP::direct;
}
}
MQClient::MQClient(const std::string &url)
:_ev_loop(EV_DEFAULT),
_handler(_ev_loop),
_connection(&_handler, AMQP::Address(url)),
_channel(&_connection),
_async_thread(std::thread([this](){ ev_run(_ev_loop); }))
{}
void MQClient::declare(const declare_settings &settings)
{
AMQP::Table args;
if(settings.exchange_type == DELAYED)
{
//声明死信交换机和死信队列
_declared(settings, args, true);
args["x-dead-letter-exchange"] = settings.dlx_exchange();
args["x-dead-letter-routing-key"] = settings.dlx_binding_key();
args["x-message-ttl"] = settings.delayed_ttl;
}
//声明延时队列与交换机
_declared(settings, args, false);
}
void MQClient::_declared(const declare_settings &settings, AMQP::Table &args, bool is_dlx)
{
//用于互斥
std::unique_lock<std::mutex> declared_lock(_declared_mtx);
//用于条件控制
std::unique_lock<std::mutex> lock(_mtx);
//定义交换机名，队列名，绑定键
std::string exchange = is_dlx ? settings.dlx_exchange() : settings.exchange;
std::string queue = is_dlx ? settings.dlx_queue() : settings.queue;
std::string binding_key = is_dlx ? settings.dlx_binding_key() : settings.binding_key;
//5.声明exchange(直接交换)和queue
_channel.declareExchange(exchange,exchange_type(settings.exchange_type))
.onSuccess([=](){
//交换机声明成功，声明队列
_channel.declareQueue(queue,args)
.onSuccess([=](const std::string &name,uint32_t messagecount,uint32_t consumercount){
std::cout<<"队列声明成功: "<< name<<", 消息数: "<< messagecount<<", 消费者数: "<< consumercount<<std::endl;
//队列声明成功，绑定交换机和队列
_channel.bindQueue(exchange,queue,binding_key)
.onSuccess([=](){
//成功绑定后通知等待线程
_cv.notify_all();
})
.onError([=](const char* message){
std::cerr<<"队列绑定失败: "<< message <<std::endl;
exit(-1);//绑定失败直接退出
});
})
.onError([=](const char* message){
std::cerr<<"队列声明失败: "<< message <<std::endl;
exit(-1);//声明失败直接退出
});
})
.onError([=](const char* message){
std::cerr<<"交换机声明失败: "<< message <<std::endl;
exit(-1);//声明失败直接退出
});
_cv.wait(lock);
}
bool MQClient::publish(const std::string &exchange, const std::string &routing_key, const std::string &body)
{
return _channel.publish(exchange, routing_key, body);
}
void MQClient::consume(const std::string &queue, const MessageCallback &callback)
{
std::unique_lock<std::mutex> declared_lock(_declared_mtx);
std::unique_lock<std::mutex> lock(_mtx);
_channel.consume(queue)
.onReceived([=](const AMQP::Message &message,uint64_t deliveryTag,bool redelivered){
callback(message.body(), message.bodySize());
//确认消息
_channel.ack(deliveryTag);
})
.onError([=](const char* message){
std::cerr<<"订阅消息失败: "<< message <<std::endl;
exit(-1);//订阅失败直接退出
})
.onSuccess([=](){
std::cout<<"订阅成功"<<std::endl;
_cv.notify_all();
});
_cv.wait(lock);
}
void MQClient::callback(struct ev_loop *loop, ev_async *watcher, int32_t revents)
{
//此接口官方规定不能跨线程调用
ev_break(loop, EVBREAK_ALL);
}
void MQClient::wait()
{
//等待异步线程结束，主线程无事可干时等待异步线程结束
_async_thread.join();
}
MQClient::~MQClient(){
//当mqclient对象被销毁时，关闭异步线程
ev_async_init(&_ev_async, MQClient::callback);
ev_async_start(_ev_loop, &_ev_async);
ev_async_send(_ev_loop, &_ev_async);
_async_thread.join();// 等待异步线程结束
}
Publisher::Publisher(const MQClient::ptr &mq_client, const declare_settings &settings)
:_mq_client(mq_client),
_settings(settings)
{
_mq_client->declare(_settings);
}
bool Publisher::publish(const std::string &body)
{
return _mq_client->publish(_settings.exchange, _settings.binding_key, body);
}
Subscriber::Subscriber(const MQClient::ptr &mq_client, const declare_settings &settings)
:_mq_client(mq_client),
_settings(settings)
{
_mq_client->declare(_settings);
}
void Subscriber::consume(MessageCallback &&callback)
{
_callback = callback;
//如果是延时队列，顶订阅的是死信队列消息，否则订阅的是常规队列消息
if (_settings.exchange_type == DELAYED) {
_mq_client->consume(_settings.dlx_queue(), _callback);
}else {
_mq_client->consume(_settings.queue, _callback);
}
}
} // namespace limemq4.打包脚手架项目
因为我们并没有深入了解学习过cmake，所以我们这里直接按照下面的步骤对我们的项目仅打包:
4.1在项目根目录创建如下CMakeLists.txt文件
bash
# 1. 设置cmake所需版本
cmake_minimum_required(VERSION 3.1.3)
# 2. 设置工程项目名称（内部会生成一系列的内置变量）
project(lime_scaffold VERSION 1.0)
# 3. 添加生成库目标（说明通过哪些文件生成哪个库）
file(GLOB_RECURSE BASE_FILES ${CMAKE_CURRENT_SOURCE_DIR}/source/*.cc) # CMAKE_CURRENT_SOURCE_DIR cmakelists.txt文件在哪里，就指向哪个路径
add_library(${PROJECT_NAME} STATIC ${BASE_FILES})
# 4. 设置编译特性
target_compile_features(${PROJECT_NAME} PUBLIC cxx_std_17)
# 5. 设置生成库属性：默认生成位置，版本信息
set_target_properties(${PROJECT_NAME} PROPERTIES
ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib) # CMAKE_CURRENT_BINARY_DIR 在哪里执行cmake，就指向哪个路径
set_target_properties(${PROJECT_NAME} PROPERTIES
VERSION ${PROJECT_VERSION} # 完整版本号（1.0）
SOVERSION ${PROJECT_VERSION_MAJOR} # 主版本号（1）
)
# 6. 依赖检查
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake) # 声明依赖检测子模块的所在路径
include(Finddepends)
# 1. 编写依赖检查子模块（查找系统中的指定库，并将库添加到链接选项中）
find_my_depends()
# 7. 设置安装头文件到指定位置
install(
DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/source/
DESTINATION include/${PROJECT_NAME}
FILES_MATCHING PATTERN "*.h"
)
# 8. 设置安装库文件到指定位置
install(
TARGETS ${PROJECT_NAME}
EXPORT ${PROJECT_NAME}Targets # 声明目标名称
ARCHIVE DESTINATION lib #静态库安装路径
INCLUDES DESTINATION include/${PROJECT_NAME} #头文件安装路径
)
# 9. 设置安装子模块到指定位置
install(
DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/cmake/
DESTINATION lib/cmake/${PROJECT_NAME}/modules
FILES_MATCHING PATTERN "Find*.cmake"
)
# 10. 设置导出目标信息
install(
EXPORT ${PROJECT_NAME}Targets
FILE ${PROJECT_NAME}Targets.cmake
DESTINATION lib/cmake/${PROJECT_NAME}
NAMESPACE ${PROJECT_NAME}::
)
# 11. 处理 Config.cmake.in 模板
# 1. 需要提前编写好模板内容
include(CMakePackageConfigHelpers)
# 处理 Config.cmake.in 模板，生成最终的配置文件，解析 @PACKAGE_INIT@ 变量
configure_package_config_file(
${CMAKE_CURRENT_SOURCE_DIR}/cmake/${PROJECT_NAME}Config.cmake.in
${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
INSTALL_DESTINATION lib/cmake/${PROJECT_NAME}
)
# 12. 设置版本配置文件
write_basic_package_version_file(
${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake
VERSION ${PROJECT_VERSION}
COMPATIBILITY SameMajorVersion
)
# 13. 安装配置文件
install(
FILES
${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake
DESTINATION lib/cmake/${PROJECT_NAME}
)4.2在根目录下创建名为cmake的文件夹，并添加两个新文件
cpp
///home/dev/workspace/cpp-microservice-scaffold/cmake/Finddepends.cmake
# 定义变量，保存所有需要查找的库的名称
set(libs
amqpcpp
ev
brpc
leveldb
protobuf
dl
ssl
crypto
gflags
pthread
cpr
jsoncpp
elasticlient
cpprest
etcd-cpp-api
fdfsclient
fastcommon
avcodec
avformat
avutil
gtest
curl
odb-mysql
odb
odb-boost
hiredis
redis++
spdlog
fmt
boost_system
)
# 编写辅助宏：查找库
macro(find_libraries lib)
find_library(${lib}_LIBRARY ${lib} PATHS /usr/lib /usr/lib/x86_64-linux-gnu /usr/local/lib)
if(${lib}_LIBRARY)
set(${lib}_FOUND TRUE)
if(NOT TARGET lime_scaffold::${lib})
add_library(lime_scaffold::${lib} INTERFACE IMPORTED)
set_target_properties(lime_scaffold::${lib} PROPERTIES
INTERFACE_LINK_LIBRARIES "${${lib}_LIBRARY}"
)
message(STATUS "找到依赖库: " ${lib} " - " ${${lib}_LIBRARY})
endif()
endif()
endmacro()
function(find_my_depends)
foreach(lib ${libs})
find_libraries(${lib})
if(NOT ${lib}_FOUND)
message(FATAL_ERROR "丢失关键依赖库: ${lib}")
endif()
endforeach()
target_link_libraries(${PROJECT_NAME} PUBLIC ${libs})
endfunction()
cpp
///home/dev/workspace/cpp-microservice-scaffold/cmake/lime_scaffoldConfig.cmake.in
@PACKAGE_INIT@ # CMake 提供的初始化宏
# 添加自定义Find模块路径
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_LIST_DIR}/modules)
# 声明所有传递依赖（自动触发find_package）
include("${CMAKE_CURRENT_LIST_DIR}/modules/Finddepends.cmake")
find_my_depends()
# 包含目标导出文件
include(${CMAKE_CURRENT_LIST_DIR}/lime_scaffoldTargets.cmake)
# 可选：验证组件或其他配置
check_required_components(lime_scaffold)4.3在根目下创建一个名为build的文件夹，进入并依次执行如下bash命令
bash
cmake .. -DCMAKE_INSTALL_PREFIX=/usr
make && sudo make install执行完毕后，查看对应文件与库是否安装到对应位置:
bash
dev@ca01086a6c5d:~/workspace/cpp-videoplayer-server$ ls /usr/include/lime_scaffold/
limeelastic.h limeetcd.h limefds.h limeffmpeg.h limelog.h limemail.h limemq.h limeodb.h limeredis.h limerpc.h limeutil.h
dev@ca01086a6c5d:~/workspace/cpp-videoplayer-server$ ls /usr/lib/liblime_scaffold.a
/usr/lib/liblime_scaffold.a出现如上结果则说明我们项目已经打包完毕。至此我们脚手架的编写到此结束。开始最后一部分服务端的编写。