安全通信设置：使用 OpenSSL 为 Logstash 和 Filebeat 提供 SSL 证书

Elastic 中国社区官方博客2024-02-03 12:05

在为 Elasticsearch 采集数据时，我们经常使用到 Filebeat 及 Logstash。在我们之前的很多教程中，我们通常不为 Filebeat 和 Logstash 之前的通信做安全配置。

如何为 Filebeat 及 Logstash 直接建立安全的链接？这个在很多的情况下是非常有用的。在我之前的文章 "Elasticsearch：为日志分析设置安全的 Elasticsearch 管道"，我有涉及到这个问题。最主要的是，它是针对 Elastic Stack 8.x 而设计的。在今天的文章中，我将针对最普通的情况来进行展示。

本指南提供了有关使用 OpenSSL 生成和配置 SSL 证书的详细说明，以增强 Logstash 和 Filebeat 之间通信的安全性。 概述的步骤包括为证书颁发机构 (CA) 生成私钥、为 CA 创建自签名 X509 证书以及为 Logstash 和 Filebeat 生成密钥和证书。 还包括使用这些证书的 Filebeat 和 Logstash 的配置步骤。

为 CA 生成私钥

复制代码 
# Generate a private key for the Certificate Authority (CA)
openssl genrsa 2048 > /path/to/your/ca-key.pem

$ pwd
/Users/liuxg/demos/filebeat_logstash
$ openssl genrsa 2048 > ca-key.pem
$ ls
ca-key.pem

为 CA 生成 X509 证书

复制代码 
# Generate a self-signed X509 certificate for the CA
openssl req -new -x509 -nodes -days 365000 \
    -key /path/to/your/ca-key.pem \
    -out /path/to/your/ca-cert.pem

openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

$ pwd
/Users/liuxg/demos/filebeat_logstash
$ ls
ca-key.pem
$ openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Elastic
Organizational Unit Name (eg, section) []:Community
Common Name (e.g. server FQDN or YOUR name) []:Xiaoguo
Email Address []:x@y.com
$ ls
ca-cert.pem ca-key.pem

为 Logstash 和 Filebeat 创建证书和密钥

复制代码 
# Generate a private key and certificate for Logstash and Filebeat using the CA
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout /path/to/your/server.key \
    -out /path/to/your/server.crt \
    -CA /path/to/your/ca-cert.pem \
    -CAkey /path/to/your/ca-key.pem

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout ./server.key \
    -out ./server.crt \
    -CA ./ca-cert.pem \
    -CAkey ./ca-key.pem

$ pwd
/Users/liuxg/demos/filebeat_logstash
$ ls
ca-cert.pem ca-key.pem
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
>     -keyout ./server.key \
>     -out ./server.crt \
>     -CA ./ca-cert.pem \
>     -CAkey ./ca-key.pem
........+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...........+..+.+...+.........+..+.............+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........+..........+.....+...+.........+.+..+......+.......+...............+..+...+.......+...+...+.....+......+......+.......+..+....+..+..........+..+......+......+.+..+.+..............+...+.......+..+...+.......+...+........+...+......+....+..+.......+.....+...............+......+................+......+...+......+.....+.+..+...+..........+........+................+...............+......+..+.+..+.........+...+.......+......+...+.....+.+...............+...+........+...+.........+...+.........+....+.....+.+........+...+.........+..........+......+..+.............+.....+....+...+..+.............+......+......+.........+..+...+....+...+............+...+..+.+......+.....+.+..+...+..................+.....................+....+..............+.+..+..........+..+...+....+...+......+.................+...+....+........+...+.+...........+..........+.....+...............+....+......+..............+...+......+......+...+.......+...+.....+...+.............+..+.......+.....+.+..+....+...+......+.....+..........+..+.........+.........+.+........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+.........+.........+.....+.+.....+.+.........+..+....+........+...+.......+........+...+......+...+...+............+..........+...+..+..........+.....+.............+..+...+...+....+...+..............+....+..+......................+..+...+...+...+.+.........+......+...+..+.+...+.....+.........+...+...+.+...+..+.+........+............+..........+..+.+.....................+......+........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Elastic
Organizational Unit Name (eg, section) []:Community 
Common Name (e.g. server FQDN or YOUR name) []:localhost
Email Address []:x@y.com

按照同样的方法，我们生成相应的 client 证书：

复制代码 
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout client.key \
    -out client.crt \
    -CA ./ca-cert.pem \
    -CAkey ./ca-key.pem

$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
>     -keyout ./server.key \
>     -out ./server.crt \
>     -CA ./ca-cert.pem \
>     -CAkey ./ca-key.pem
........+...+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...........+..+.+...+.........+..+.............+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........+..........+.....+...+.........+.+..+......+.......+...............+..+...+.......+...+...+.....+......+......+.......+..+....+..+..........+..+......+......+.+..+.+..............+...+.......+..+...+.......+...+........+...+......+....+..+.......+.....+...............+......+................+......+...+......+.....+.+..+...+..........+........+................+...............+......+..+.+..+.........+...+.......+......+...+.....+.+...............+...+........+...+.........+...+.........+....+.....+.+........+...+.........+..........+......+..+.............+.....+....+...+..+.............+......+......+.........+..+...+....+...+............+...+..+.+......+.....+.+..+...+..................+.....................+....+..............+.+..+..........+..+...+....+...+......+.................+...+....+........+...+.+...........+..........+.....+...............+....+......+..............+...+......+......+...+.......+...+.....+...+.............+..+.......+.....+.+..+....+...+......+.....+..........+..+.........+.........+.+........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+.........+.........+.....+.+.....+.+.........+..+....+........+...+.......+........+...+......+...+...+............+..........+...+..+..........+.....+.............+..+...+...+....+...+..............+....+..+......................+..+...+...+...+.+.........+......+...+..+.+...+.....+.........+...+...+.+...+..+.+........+............+..........+..+.+.....................+......+........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Elastic
Organizational Unit Name (eg, section) []:Community 
Common Name (e.g. server FQDN or YOUR name) []:localhost
Email Address []:x@y.com
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
>     -keyout client.key \
>     -out client.crt \
>     -CA ./ca-cert.pem \
>     -CAkey ./ca-key.pem
....+...+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.........+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+....+.....+....+...+...+..+..................+.+..............+.+.....................+...+..+.............+..+.......+..+......+............+......+.+......+.........+.........+......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
..+.....+..................+.+..+............+.+..+....+......+...+.....+.......+...+.....+......+...+.............+........+.+.....+....+..+.......+......+..+....+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.....+......+.+...+..+....+.....+.............+........+.......+...+............+.....+..................+.......+..+.+.........+...............+.....+.+..+...+...+.+......+.....+.......+.....+...+.........+......+....+...+...........+....+...........+......+.........+....+.....+......+......+.+.....+..........+.....+............+.+......+...+...+...+..+.......+..+...+......+.+......+.....+....+...+..+...............+....+...............+..+.........+...+.+.....+................+.....+.+...+......+.....+......+.......+..+...+......+.+......+...+.....+.........+......................+...+.....+...+...+......+.........+.+.....+....+.....+...............+....+...+.....+............+.+..+.......+..+...+...+............+..........+..+................+..+...+...+.+...+........+...+.+...........+.........+......+..................+....+.....+............+....+.....+......+....+..+....+.....+.+.....+...+...............+....+.....+..........+..+................+..+............+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Elastic
Organizational Unit Name (eg, section) []:Community
Common Name (e.g. server FQDN or YOUR name) []:localhost
Email Address []:x@y.com
$ ls
ca-cert.pem                           filebeat-8.12.0-darwin-aarch64.tar.gz
ca-key.pem                            logstash-8.12.0
client.crt                            logstash-8.12.0-darwin-aarch64.tar.gz
client.key                            server.crt
filebeat-8.12.0-darwin-aarch64        server.key

配置 Filebeat（或任何 Beat）

复制代码 
output.logstash:
  hosts: ["localhost:5044"]
  
  # Specify the path to the CA certificate for verification
  ssl.certificate_authorities: ["/path/to/your/ca-cert.pem"]
  
  # Specify the path to the client certificate
  ssl.certificate: "/path/to/your/server.crt"
  
  # Specify the path to the client private key
  ssl.key: "/path/to/your/server.key"

配置 Logstash

复制代码 
input {
  beats {
    port => 5044
    ssl => true
    
    # Specify the path to the CA certificate for verification
    ssl_certificate_authorities => ["/path/to/your/ca-cert.pem"]
    
    # Specify the path to the server certificate
    ssl_certificate => "/path/to/your/server.crt"
    
    # Specify the path to the server private key
    ssl_key => "/path/to/your/server.key"
    
    # Set peer verification mode to force_peer
    ssl_verify_mode => "force_peer"
  }
}

确保将 /path/to/your/ 替换为密钥和证书的实际所需路径。

测试

我们在当前的目录下安装 Filebeat 及 Logstash：

filebeat_ssl.yml

复制代码 
$ pwd
/Users/liuxg/demos/filebeat_logstash
$ ls 
ca-cert.pem                           logstash-8.12.0
ca-key.pem                            logstash-8.12.0-darwin-aarch64.tar.gz
filebeat-8.12.0-darwin-aarch64        server.crt
filebeat-8.12.0-darwin-aarch64.tar.gz server.key
$ cd filebeat-8.12.0-darwin-aarch64
$ ls filebeat_ssl.yml 
filebeat_ssl.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - ./logfile.log
output.logstash:
  hosts: ["localhost:5044"]
  ssl.certificate_authorities: ["../ca-cert.pem"]
  ssl.certificate: "../server.crt"
  ssl.key: "../server.key"
logging:
  level: info
  to_files: true
  files:
  path: .
  name: filebeat.log
  keepfiles: 7
  permissions: 0644

我们在 filebeat 安装目录下创建一个叫做 logfile.log 的文件。它的内容如下：

logfile.log

复制代码 
This is a test log

logstash.conf

复制代码 
input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => "../ca-cert.pem"
    ssl_certificate => "../server.crt"
    ssl_key => "../server.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  stdout { codec => rubydebug }
  file {
    path => "../outputfile.log"
  }
}

我们首先运行 logstash：

复制代码 
./bin/logstash -f logstash.conf

我们再接下来运行 Filebeat：

我们回到 Logstash 运行的 terminal 中进行查看：

从上面的输出中，我们可以看到日志被正确地采集到 Logstash。我们可以更进一步输出到 Elasticsearch。在这里，我们就不再累述了。

相关推荐
pipip.44 分钟前
UDP————套接字socket
linux·网络·c++·网络协议·udp
阳洞洞2 小时前
https和http有什么区别
网络协议·http·https
安全系统学习5 小时前
网络安全之SQL RCE漏洞
安全·web安全·网络安全·渗透测试
孞㐑￥5 小时前
Linux之Socket 编程 UDP
linux·服务器·c++·经验分享·笔记·网络协议·udp
you秀6 小时前
HTTPS通信流程：SSL/TLS握手全解析
网络协议·https·ssl
GISer_Jing6 小时前
Git协作开发：feature分支、拉取最新并合并
大数据·git·elasticsearch
2501_915921438 小时前
iOS IPA 混淆实测分析：从逆向视角验证加固效果与防护流程
websocket·网络协议·tcp/ip·http·网络安全·https·udp
2501_915918418 小时前
打造可观测的 iOS CICD 流程：调试、追踪与质量保障全记录
websocket·网络协议·tcp/ip·http·网络安全·https·udp
hanniuniu1311 小时前
AI时代API挑战加剧，API安全厂商F5护航企业数字未来
人工智能·安全
zhulangfly11 小时前
API接口安全-1：身份认证之传统Token VS JWT
安全
热门推荐
01手机电脑之间快速传输图片视频文件，不压缩画质、不限制大小的方法！02Coze实战第13讲：飞书多维表格读取+豆包生图模型，轻松批量生成短剧封面03Coze扣子平台完整体验和实践（附国内和国际版对比）04华为昇腾 910B 部署 DeepSeek-R1 蒸馏系列模型详细指南05免费可用！最强AI数字人对口型神器：让照片开口说话唱歌，支持多人对口型+全身动作，1分钟学会！（附保姆级教程）06扣子（coze）实战|我用扣子搭建了一个自动分析小红薯笔记内容的AI应用|详细步骤拆解07Coze（扣子）智能体工作流：自动批量生成书单号视频，1分钟100个，书单号博主都在用！08C#调用WechatOCR.exe实现本地OCR文字识别09DeepSeek各版本说明与优缺点分析10Coze 全方位入门剖析 - 免费打造自己的 AI 超级个体