【EFK】Linux集群部署Elasticsearch最新版本8.x

【EFK】Linux集群部署Elasticsearch最新版本8.x

摘要

The Elastic Stack,包括ElasticsearchKibanaBeatsLogstash(也成为ELK Stack

Elasticsearch:简称ES,是一个开源的高扩展的分布式全文搜索引擎,是整个Elastic Stack技术栈的核心。它可以近乎实时地存储、检索数据;本身扩展性很好,可以扩展到上百台服务器,处理PB级的数据。

本文主要讲解如何部署Elasticsearch,使用最新版8.15.3

环境准备

环境信息

主机名 操作系统版本 IP地址
elk1 Centos7 192.168.30.133
elk2 Centos7 192.168.30.134
elk3 Centos7 192.168.30.135

系统初始化

shell 复制代码
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时
#查看selinux状态
getenforce
#根据规划设置主机名
hostnamectl set-hostname
#添加hosts
cat >> /etc/hosts << EOF
192.168.30.133 elk1
192.168.30.134 elk2
192.168.30.135 elk3
EOF
# 创建用户,因为elastic相关服务不允许root启动
groupadd elk
useradd elk -g elk
# 创建数据及⽇志⽂件并授权
mkdir -pv /opt/elk/
chown -R elk:elk /opt/elk/

启动先决条件

  • 调整进程最大打开文件数数量
shell 复制代码
#临时设置
ulimit -n 65535
#永久设置,退出重连生效
vi /etc/security/limits.conf
elk - memlock unlimited
elk - nproc 4096    ##noproc 是代表最大进程数
elk - nofile 65535  ##nofile 是代表最大文件打开数
#验证
ulimit -n
  • 调整进程最大虚拟内存区域数量
shell 复制代码
#临时设置
sysctl -w vm.max_map_count=262144
#永久设置
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p

下载&安装

shell 复制代码
mkdir -p /opt/elk/ && cd /opt/elk/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.15.3-linux-x86_64.tar.gz
tar -xzf elasticsearch-8.15.3-linux-x86_64.tar.gz
cd elasticsearch-8.15.3/

修改elasticsearch.yml

vi config/elasticsearch.yml

yaml 复制代码
#必改配置
cluster.name: elk-cluster # 集群名称
node.name: elk1 # 集群节点名称
network.host: 0.0.0.0 # 监听地址
discovery.seed_hosts: ["192.168.30.133", "192.168.30.134","192.168.30.135"] # 集群节点列表
cluster.initial_master_nodes: ["elk1"] # 首次启动指定的Master节点
#可选配置
path.data: data # 数据目录
path.logs: logs # 日志目录
bootstrap.memory_lock: false # 锁内存,尽量不使⽤交换内存
http.port: 9200 # 监听端口
http.cors.allow-origin: "*" # Only use unrestricted value for local development
# Use a specific origin value in production, like `http.cors.allow-origin: "https://<my-website-domain.example>"`
http.cors.enabled: true
http.cors.allow-credentials: true
http.cors.allow-methods: OPTIONS, POST
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept
action.auto_create_index: '*'

控制台启动

shell 复制代码
# 不允许root用户启动
chown -R elk:elk /opt/elk/
su elk
#控制台启动
./bin/elasticsearch
#后台启动
./bin/elasticsearch -d
  • 启动成功截图
  • elastic密码: Ne5=s3QNimmzOey0D=kF
  • CA证书指纹: c493885f81be0090bc625aba200706439038ab34b1f26b183565676b681c9dfd

Linux服务启动

修改/etc/systemd/system/elasticsearch.service

shell 复制代码
[Unit]
Description=Elasticsearch
After=network.target

[Service]
User=elk
Group=elk
LimitNOFILE=65535
LimitNPROC=4096
LimitMEMLOCK=infinity
ExecStart=/opt/elk/elasticsearch-8.15.3/bin/elasticsearch

[Install]
WantedBy=multi-user.target
  • 注册服务开机启动: systemctl enable elasticsearch.service
  • 手动启动服务: systemctl start elasticsearch.service
  • 查看ES服务状态: systemctl status elasticsearch.service

访问验证

查看集群信息

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 529

{
  "name" : "node-1",
  "cluster_name" : "elk-cluster",
  "cluster_uuid" : "fPVHn9FzSz6FzaVznIqj-Q",
  "version" : {
    "number" : "8.15.3",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "f97532e680b555c3a05e73a74c28afb666923018",
    "build_date" : "2024-10-09T22:08:00.328917561Z",
    "build_snapshot" : false,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

查看es健康状态

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/_cat/health

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked

1731134586 06:43:06 elk-cluster green 1 1 45 45 0 0 0 0 - 100.0%
  • green: 集群所有数据都处于正常状态
  • yellow: 集群所有数据都可以访问,但一些数据的副本还没有分配
  • red: 集群部分数据不可访问

查看集群节点

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cat/nodes?pretty"

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked

192.168.4.103 10 97 1    cdfhilmrstw * node-1

查询集群状态

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cluster/health?pretty"

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 466

{
  "cluster_name" : "elk-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 45,
  "active_shards" : 45,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

生成service token

运行命令

shell 复制代码
curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XPOST https://localhost:9200/_security/service/elastic/kibana/credential/token/mytoken

返回

shell 复制代码
{"created":true,"token":{"name":"mytoken","value":"AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46cXo1bUxFZ0pSMldGWm9nTUlfeTA2UQ"}}

验证service token

运行命令

shell 复制代码
curl -ik -H "Authorization: Bearer AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46REtzdWE5cVBSTFdHbW1OS1hrNzA5QQ" https://localhost:9200/_security/_authenticate

返回

shell 复制代码
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 395

{"username":"elastic/kibana","roles":[],"full_name":"Service account - elastic/kibana","email":null,"token":{"name":"mytoken","type":"_service_account_index"},"metadata":{"_elastic_service_account":true},"enabled":true,"authentication_realm":{"name":"_service_account","type":"_service_account"},"lookup_realm":{"name":"_service_account","type":"_service_account"},"authentication_type":"token"}

IK分词器下载

shell 复制代码
wget https://release.infinilabs.com/analysis-ik/stable/elasticsearch-analysis-ik-8.15.3.zip
# 把zip包的内容解压到elasticsearch-analysis-ik-8.15.3目录
unzip elasticsearch-analysis-ik-8.15.3.zip -d elasticsearch-analysis-ik-8.15.3
# 将ik文件夹移动到ES安装目录下的plugins文件夹下
mv elasticsearch-analysis-ik-8.15.3 elasticsearch-8.15.3/plugins/
  • 重启Elasticsearch
相关推荐
飞大圣21 分钟前
Ubuntu 18 EDK2 环境编译
linux·运维·ubuntu
冷心笑看丽美人33 分钟前
RHEL 网络配置(Linux网络服务器 09)
linux·运维·服务器·网络·bash
运维&陈同学42 分钟前
【HAProxy05】企业级反向代理HAProxy调度算法之静态算法与动态算法
linux·运维·算法·nginx·云原生·负载均衡·lvs·haproxy
Narutolxy1 小时前
深入解析 CentOS 7 上 MySQL 8.0 的最佳实践20241112
linux·mysql·centos
BillKu1 小时前
Linux(CentOS)项目总结(前后端分离)
java·linux·mysql·nginx·centos
angleboy81 小时前
【全面系统性介绍】虚拟机VM中CentOS 7 安装和网络配置指南
linux·运维·centos
孤邑1 小时前
【Linux】网络编程3
linux·服务器·网络·笔记·学习
夏天匆匆2过1 小时前
linux性能提升之sendmmsg和recvmmsg
linux·c++·单片机·网络协议·udp·tcp
Diamond技术流1 小时前
从0开始学习Linux——文件管理
linux·运维·学习·安全·文件·权限·极限编程
GDAL2 小时前
linux入门教程:perl库
linux·运维·perl