18.基于Consul的服务发现和ConsulManager管理

192.168.10.14 prometheus、consul

192.168.10.100 各类服务

一、基于Consul的服务发现

Consul 是由 HashiCorp 开发的一个支持多数据中心的分布式服务发现和键值对存储服务的开源软件，是一个通用的服务发现和注册中心工具，被大量应用于基于微服务的软件架构当中。
过api将exporter服务注册到 Consul，然后配置 Prometheus 从 Consul 中发现实例。关于 Consul 本身的使用可以查看官方文档 https://learn.hashicorp.com/consul 了解更多。

1.1 二进制安装consul（二选一）

在页面 https://www.consul.io/downloads 下载符合自己系统的安装文件，比如我们这里是 Linux 系统，使用下面命令下载安装即可：

wget https://releases.hashicorp.com/consul/1.18.1/consul_1.18.1_linux_amd64.zip

apt install unzip -y
unzip consul_1.18.1_linux_amd64.zip
mv consul /usr/local/bin
consul version

# 启动consul
为了查看更多的日志信息，我们可以在 dev 模式下运行 Consul，如下所示：
consul agent -dev -client 0.0.0.0

启动命令后面使用 -client 参数指定了客户端绑定的 IP 地址，默认为 127.0.0.1

1.2 docker安装Consul(二选一)

# docker 运行
docker run -d --name consul -p 8500:8500 consul:1.14.5

# consul http访问地址
http://192.168.10.100:8500/ui/dc1/services

1.3 通过api注册到Consul

# 第一种方式：使用命令注册
curl -X PUT -d '{"id": "node1","name": "node_exporter","address": "node_exporter","port": 9100,"tags": ["exporter"],"meta": {"job": "node_exporter","instance": "Prometheus服务器"},"checks": [{"http": "http://192.168.10.14:9100/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

# 第二种方式：把json数据放在文件中，使用这个json文件注册
mkdir /data/consul
cd /data/consul

cat > node_exporter.json<<"EOF"
  {
    "id": "node2",
    "name": "node_exporter",
    "address": "192.168.10.100",
    "port": 9100,
    "tags": ["exporter"],
    "meta": {
      "job": "node_exporter",
      "instance": "test服务器"
    },
    "checks": [{
      "http": "http://192.168.10.14:9100/metrics",
      "interval": "10s"
    }]
  } 
EOF

curl --request PUT --data @node_exporter.json http://localhost:8500/v1/agent/service/register

注册的 2 个 demo 服务之外，Consul agent 还会将自己注册为一个名为 consul 的服务，我们可以在浏览器中访问 http://192.168.11.61:8500 查看注册的服务。

在 Consul UI 页面中可以看到有 consul 和 node_exporter 两个 Service 服务。

1.4 配置Prometheus

配置 Prometheus 通过 Consul 来自动发现 node_porter服务。
在 Prometheus 的配置文件 prometheus.yml 文件中的 scrape_configs 部分添加如下所示的抓取配置：

备份源文件:

cd /data/docker-prometheus
cp -a prometheus/prometheus.yml prometheus/prometheus.yml.bak
ls -l prometheus/prometheus.yml.bak

使用以下配置

cat > prometheus/prometheus.yml<<"EOF"     
# 全局配置
global:
  scrape_interval:     15s # 将搜刮间隔设置为每15秒一次。默认是每1分钟一次。
  evaluation_interval: 15s # 每15秒评估一次规则。默认是每1分钟一次。

# Alertmanager 配置
alerting:
  alertmanagers:
  - static_configs:
    - targets: ['alertmanager:9093']

# 报警(触发器)配置
rule_files:
  - "alert.yml"
  - "rules/*.yml"

# 搜刮配置
scrape_configs:
  - job_name: 'prometheus'
    # 覆盖全局默认值，每15秒从该作业中刮取一次目标
    scrape_interval: 15s
    static_configs:
    - targets: ['localhost:9090']
  - job_name: 'alertmanager'
    # 覆盖全局默认值，每15秒从该作业中刮取一次目标
    scrape_interval: 15s
    static_configs:
    - targets: ['alertmanager:9093']

  - job_name: 'consul_exporter'
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*exporter.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
#  Spring Boot 2.x 应用数据采集配置
  - job_name: 'consul_springboot_demo'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 5s
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*springboot.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
#http配置
  - job_name: "consul-blackbox_http"
    metrics_path: /probe
    params:
      module: [http_2xx]
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*blackbox_http.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
      - source_labels: [__meta_consul_service_address]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 192.168.10.100:9115
#tcp检查配置
  - job_name: "consul_blackbox_tcp"
    metrics_path: /probe
    params:
      module: [tcp_connect]
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*blackbox_tcp.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
      - source_labels: [__meta_consul_service_address]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 192.168.10.100:9115

#icmp检查配置
  - job_name: "consul_blackbox_icmp"
    metrics_path: /probe
    params:
      module: [icmp]
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*blackbox_icmp.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
      - source_labels: [__meta_consul_service_address]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 192.168.10.100:9115

#域名检测
  - job_name: consul_domain_exporter
    scrape_interval: 10s
    metrics_path: /probe
    consul_sd_configs:
      - server: '192.168.10.14:8500'
        services: []
    relabel_configs:
      - source_labels: [__meta_consul_tags]
        regex: .*domain.*
        action: keep
      - regex: __meta_consul_service_metadata_(.+)
        action: labelmap
      - source_labels: [__meta_consul_service_address]
        target_label: __param_target
      - target_label: __address__
        replacement: 192.168.10.100:9222
EOF

通过 consul_sd_configs 配置用于自动发现的 Consul 服务地址，服务名为[]，我们通过relabel_configs的过滤规则只接收指定的exporter

curl -X POST http://localhost:9090/-/reload
配置完成后重新启动 Prometheus，然后重新查看 Prometheus 页面上的 targets 页面，验证上面的配置是否存在：

http://192.168.10.14:9090/targets?search=

正常情况下是可以看到会有一个 exporter 的任务，下面有 2 个自动发现的抓取目标。

1.5 创建添加脚本

prometheus上使用预先准备好的脚本，一次添加多个targets:

cat >/data/consul/api.sh <<"EOF"
#nginx
curl -X PUT -d '{"id": "nginx1","name": "nginx_exporter","address": "192.168.10.100","port": 9113,"tags": ["exporter"],"meta": {"job": "nginx_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9113/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#rabbitmq
curl -X PUT -d '{"id": "rabbitmq1","name": "rabbitmq_exporter","address": "192.168.10.100","port": 9419,"tags": ["exporter"],"meta": {"job": "rabbitmq_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9419/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#redis
curl -X PUT -d '{"id": "redis1","name": "redis_exporter","address": "192.168.10.100","port": 9121,"tags": ["exporter"],"meta": {"job": "redis_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9121/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#mongodb
curl -X PUT -d '{"id": "mongodb1","name": "mongodb_exporter","address": "192.168.10.100","port": 9216,"tags": ["exporter"],"meta": {"job": "mongodb_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9216/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#mysql
curl -X PUT -d '{"id": "mysql1","name": "mysqld_exporter","address": "192.168.10.100","port": 9104,"tags": ["exporter"],"meta": {"job": "mysqld_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9104/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#cadvisor
curl -X PUT -d '{"id": "cadvisor1","name": "cadvisor","address": "cadvisor","port": 8080,"tags": ["exporter"],"meta": {"job": "cadvisor","instance": "Prometheus服务器","env":"test"},"checks": [{"http": "http://192.168.11.61:8080/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register
curl -X PUT -d '{"id": "cadvisor2","name": "cadvisor","address": "192.168.10.100","port": 8080,"tags": ["exporter"],"meta": {"job": "cadvisor","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:8080/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#springboot
curl -X PUT -d '{"id": "springboot1","name": "springboot","address": "192.168.10.100","port": 8081,"tags": ["springboot"],"meta": {"job": "springboot","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:8081/actuator/prometheus", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register


#process_exporter
curl -X PUT -d '{"id": "process1","name": "process_exporter","address": "192.168.10.100","port": 9256,"tags": ["exporter"],"meta": {"job": "process_exporter","instance": "test服务器","env":"test"},"checks": [{"http": "http://192.168.10.100:9256/metrics", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#http
curl -X PUT -d '{"id": "http1","name": "blackbox_http","address": "https://www.jd.com","tags": ["blackbox_http"],"checks": [{"http": "http://192.168.10.100:9115", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#tcp
curl -X PUT -d '{"id": "tcp1","name": "blackbox_tcp","address": "192.168.11.61:9090","tags": ["blackbox_tcp"],"checks": [{"http": "http://192.168.10.100:9115", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

#icmp
curl -X PUT -d '{"id": "icmp1","name": "blackbox_icmp","address": "192.168.10.100","tags": ["blackbox_icmp"],"checks": [{"http": "http://192.168.10.100:9115", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register


#domin
curl -X PUT -d '{"id": "domain1","name": "domain_exporter","address": "baidu.com","tags": ["domain"],"checks": [{"http": "http://192.168.10.100:9222", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register
EOF

# 新增domin:id和adress需要修改

curl -X PUT -d '{"id": "domain2","name": "domain_exporter","address": "9you.com","tags": ["domain"],"checks": [{"http": "http://192.168.10.100:9222", "interval": "5s"}]}'  http://localhost:8500/v1/agent/service/register

执行脚本

sh /data/consul/api.sh

# 检查
http://192.168.10.14:9090/targets?search=

# 服务注册
http://192.168.10.14:8500/ui/dc1/services

1.6 consul删除服务

curl --request PUT http://127.0.0.1:8500/v1/agent/service/deregister/ID

# 服务ID就是例如："id": "domain2"，id不能重复；微服务ID

二、ConsulManager

官方gitee地址
1、ConsulManager需要依赖Consul，请先完成Consul的部署。（暂时最高支持Consul v1.14.5）（docs/Consul部署说明.md）

2、使用docker-compose来部署ConsulManager

下载：wget https://starsl.cn/static/img/docker-compose.yml（仓库根目录下docker-compose.yml）

vim docker-compose.yml

version: "3.2"
services:
  flask-consul:
    image: swr.cn-south-1.myhuaweicloud.com/starsl.cn/flask-consul:latest
    container_name: flask-consul
    hostname: flask-consul
    restart: always
    volumes:
      - /usr/share/zoneinfo/PRC:/etc/localtime
    environment:
      consul_token: xxxxx-xxxxx-xxxxx
      consul_url: http://192.168.10.14:8500/v1
      admin_passwd: password
      log_level: INFO
  nginx-consul:
    image: swr.cn-south-1.myhuaweicloud.com/starsl.cn/nginx-consul:latest
    container_name: nginx-consul
    hostname: nginx-consul
    restart: always
    ports:
      - "1026:1026"
    volumes:
      - /usr/share/zoneinfo/PRC:/etc/localtime

修改3个环境变量：
consul_token：consul的登录token（https://github.com/starsliao/ConsulManager/blob/main/docs/Consul部署说明.md？）,当然也可以不获取token，这样consul使用无密码登录（不安全）。
consul_url：consul的URL(http开头，/v1要保留)
admin_passwd：登录ConsulManager Web的admin密码 
●  启动：docker-compose pull && docker-compose up -d 
●  访问：http://{IP}:1026，使用配置的变量 admin_passwd 登录 

参考：https://github.com/starsliao/ConsulManager/blob/main/docs/FAQ.md

http://192.168.10.14:1026,账号密码：admin/password

可以通过管理服务，新增和删除都行

新增