【EFK】Linux集群部署Elasticsearch最新版本8.x

【EFK】Linux集群部署Elasticsearch最新版本8.x

摘要

The Elastic Stack,包括ElasticsearchKibanaBeatsLogstash(也成为ELK Stack

Elasticsearch:简称ES,是一个开源的高扩展的分布式全文搜索引擎,是整个Elastic Stack技术栈的核心。它可以近乎实时地存储、检索数据;本身扩展性很好,可以扩展到上百台服务器,处理PB级的数据。

本文主要讲解如何部署Elasticsearch,使用最新版8.15.3

环境准备

环境信息

主机名 操作系统版本 IP地址
elk1 Centos7 192.168.30.133
elk2 Centos7 192.168.30.134
elk3 Centos7 192.168.30.135

系统初始化

shell 复制代码
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时
#查看selinux状态
getenforce
#根据规划设置主机名
hostnamectl set-hostname
#添加hosts
cat >> /etc/hosts << EOF
192.168.30.133 elk1
192.168.30.134 elk2
192.168.30.135 elk3
EOF
# 创建用户,因为elastic相关服务不允许root启动
groupadd elk
useradd elk -g elk
# 创建数据及⽇志⽂件并授权
mkdir -pv /opt/elk/
chown -R elk:elk /opt/elk/

启动先决条件

  • 调整进程最大打开文件数数量
shell 复制代码
#临时设置
ulimit -n 65535
#永久设置,退出重连生效
vi /etc/security/limits.conf
elk - memlock unlimited
elk - nproc 4096    ##noproc 是代表最大进程数
elk - nofile 65535  ##nofile 是代表最大文件打开数
#验证
ulimit -n
  • 调整进程最大虚拟内存区域数量
shell 复制代码
#临时设置
sysctl -w vm.max_map_count=262144
#永久设置
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p

下载&安装

shell 复制代码
mkdir -p /opt/elk/ && cd /opt/elk/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.15.3-linux-x86_64.tar.gz
tar -xzf elasticsearch-8.15.3-linux-x86_64.tar.gz
cd elasticsearch-8.15.3/

修改elasticsearch.yml

vi config/elasticsearch.yml

yaml 复制代码
#必改配置
cluster.name: elk-cluster # 集群名称
node.name: elk1 # 集群节点名称
network.host: 0.0.0.0 # 监听地址
discovery.seed_hosts: ["192.168.30.133", "192.168.30.134","192.168.30.135"] # 集群节点列表
cluster.initial_master_nodes: ["elk1"] # 首次启动指定的Master节点
#可选配置
path.data: data # 数据目录
path.logs: logs # 日志目录
bootstrap.memory_lock: false # 锁内存,尽量不使⽤交换内存
http.port: 9200 # 监听端口
http.cors.allow-origin: "*" # Only use unrestricted value for local development
# Use a specific origin value in production, like `http.cors.allow-origin: "https://<my-website-domain.example>"`
http.cors.enabled: true
http.cors.allow-credentials: true
http.cors.allow-methods: OPTIONS, POST
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept
action.auto_create_index: '*'

控制台启动

shell 复制代码
# 不允许root用户启动
chown -R elk:elk /opt/elk/
su elk
#控制台启动
./bin/elasticsearch
#后台启动
./bin/elasticsearch -d
  • 启动成功截图
  • elastic密码: Ne5=s3QNimmzOey0D=kF
  • CA证书指纹: c493885f81be0090bc625aba200706439038ab34b1f26b183565676b681c9dfd

Linux服务启动

修改/etc/systemd/system/elasticsearch.service

shell 复制代码
[Unit]
Description=Elasticsearch
After=network.target

[Service]
User=elk
Group=elk
LimitNOFILE=65535
LimitNPROC=4096
LimitMEMLOCK=infinity
ExecStart=/opt/elk/elasticsearch-8.15.3/bin/elasticsearch

[Install]
WantedBy=multi-user.target
  • 注册服务开机启动: systemctl enable elasticsearch.service
  • 手动启动服务: systemctl start elasticsearch.service
  • 查看ES服务状态: systemctl status elasticsearch.service

访问验证

查看集群信息

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 529

{
  "name" : "node-1",
  "cluster_name" : "elk-cluster",
  "cluster_uuid" : "fPVHn9FzSz6FzaVznIqj-Q",
  "version" : {
    "number" : "8.15.3",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "f97532e680b555c3a05e73a74c28afb666923018",
    "build_date" : "2024-10-09T22:08:00.328917561Z",
    "build_snapshot" : false,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

查看es健康状态

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/_cat/health

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked

1731134586 06:43:06 elk-cluster green 1 1 45 45 0 0 0 0 - 100.0%
  • green: 集群所有数据都处于正常状态
  • yellow: 集群所有数据都可以访问,但一些数据的副本还没有分配
  • red: 集群部分数据不可访问

查看集群节点

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cat/nodes?pretty"

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked

192.168.4.103 10 97 1    cdfhilmrstw * node-1

查询集群状态

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cluster/health?pretty"

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 466

{
  "cluster_name" : "elk-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 45,
  "active_shards" : 45,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

生成service token

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XPOST https://localhost:9200/_security/service/elastic/kibana/credential/token/mytoken

返回

shell 复制代码
{"created":true,"token":{"name":"mytoken","value":"AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46cXo1bUxFZ0pSMldGWm9nTUlfeTA2UQ"}}

验证service token

运行命令

shell 复制代码
curl -ik -H "Authorization: Bearer AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46REtzdWE5cVBSTFdHbW1OS1hrNzA5QQ" https://localhost:9200/_security/_authenticate

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 395

{"username":"elastic/kibana","roles":[],"full_name":"Service account - elastic/kibana","email":null,"token":{"name":"mytoken","type":"_service_account_index"},"metadata":{"_elastic_service_account":true},"enabled":true,"authentication_realm":{"name":"_service_account","type":"_service_account"},"lookup_realm":{"name":"_service_account","type":"_service_account"},"authentication_type":"token"}

IK分词器下载

shell 复制代码
wget https://release.infinilabs.com/analysis-ik/stable/elasticsearch-analysis-ik-8.15.3.zip
# 把zip包的内容解压到elasticsearch-analysis-ik-8.15.3目录
unzip elasticsearch-analysis-ik-8.15.3.zip -d elasticsearch-analysis-ik-8.15.3
# 将ik文件夹移动到ES安装目录下的plugins文件夹下
mv elasticsearch-analysis-ik-8.15.3 elasticsearch-8.15.3/plugins/
  • 重启Elasticsearch
相关推荐
意疏1 小时前
【Linux 篇】Docker 的容器之海与镜像之岛:于 Linux 系统内探索容器化的奇妙航行
linux·docker
BLEACH-heiqiyihu1 小时前
RedHat7—Linux中kickstart自动安装脚本制作
linux·运维·服务器
我的K84092 小时前
Flink整合Hudi及使用
linux·服务器·flink
1900432 小时前
linux6:常见命令介绍
linux·运维·服务器
Camellia-Echo3 小时前
【Linux从青铜到王者】Linux进程间通信(一)——待完善
linux·运维·服务器
Linux运维日记3 小时前
k8s1.31版本最新版本集群使用容器镜像仓库Harbor
linux·docker·云原生·容器·kubernetes
我是唐青枫3 小时前
Linux dnf 包管理工具使用教程
linux·运维·服务器
编程修仙4 小时前
Collections工具类
linux·windows·python
芝麻团坚果4 小时前
对subprocess启动的子进程使用VSCode python debugger
linux·ide·python·subprocess·vscode debugger
梦幻通灵4 小时前
ES分词环境实战
大数据·elasticsearch·搜索引擎