keepalived在架构中的作用
LVS和HAProxy组成负载均衡
调度器LVS(四层)+后端服务器(多)
LVS:优点是速度快,性能要求不高,但是没有对后端服务器的健康检测;
HAProxy:有后端健康检测,支持七层(支持对数据里的http的报文分析acl),稳定性不高;
高可用:A = MTBF 在线时间/ (MTBF+MTTR平均故障处理时间)A越大高可用就越强
系统故障
硬件故障:设计缺陷、wear out(损耗)、非人为不可抗拒因素
软件故障:设计缺陷 bug
实现高可用
active/passive 主/备
active/active 双主(放置不同的业务,如果有一个故障,就会把故障的业务和数据迁移到另外一个,节省了服务器的数量)
active --> HEARTBEAT --> passive (active,passive每隔一段时间都会发信息给组播地址,组播地址认为active存活,如果active没发消息启用passive)
active <--> HEARTBEAT <--> active 同上
通告:心跳 HEARTBEAT 使用VRRP协议
工作方式:抢占式,非抢占式
安全认证:
无认证
简单字符认证:预共享密钥
MD5
主/备:单虚拟路由器
主/主:主/备(虚拟路由器1),备/主(虚拟路由器2)
VRRP Virtual Router Redundancy Protocol 虚拟路由冗余协议,解决静态网关单点风险
虚拟路由器:Virtual Router
虚拟路由器标识:VRID(0-255),唯一标识虚拟路由器
VIP:Virtual IP
VMAC:Virutal MAC (00-00-5e-00-01-VRID)
物理路由器:
master:主设备
backup:备用设备
priority:优先级 VIP放置在哪取决于谁的优先级高;
keepalived 简介
vrrp 协议的软件实现,原生设计目的为了高可用 ipvs服务
基于vrrp协议完成地址流动
为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
为ipvs集群的各RS做健康状态检测
基于脚本调用接口完成脚本中定义的功能,进而影响集群事务,以此支持nginx、haproxy等服务
用户空间核心组件:
vrrp stack:VIP消息通告(hreatbeat)
checkers:监测real server
system call:实现 vrrp 协议状态转换时调用脚本的功能(有邮件服务器)
SMTP:邮件组件
IPVS wrapper:生成IPVS规则
Netlink Reflector:网络接口
WatchDog:监控进程
自动控制:
控制组件:提供keepalived.conf 的解析器,完成Keepalived配置(Control Plane)
IO复用器:针对网络目的而优化的自己的线程抽象
内存管理组件:为某些通用的内存管理功能(例如分配,重新分配,发布等)提供访问权限(Memory Mngt)
Keepalived实验环境
实验需要4个rhel7 配置环境
修改内存和内核为1;
realserver1:172.25.254.110
realserver2:172.25.254.120
KA1:172.25.254.10
KA2:172.25.254.20
VIP:172.25.254.100
打开rs1、rs2 ka1、ka2:
[root@nginx ~]# vmset.sh eth0 172.25.254.110 RS1.hui.org
[root@nginx ~]# vmset.sh eth0 172.25.254.120 RS2.hui.org
[root@nginx ~]# vmset.sh eth0 172.25.254.10 ka1.hui.org
[root@nginx ~]# vmset.sh eth0 172.25.254.20 ka2.hui.org
[root@nginx ~]# getenforce
Disabled
查看防火墙是否关闭:
[root@nginx ~]# systemctl status firewalld.service
[root@nginx ~]# systemctl stop firewalld.service [root@nginx ~]# systemctl mask firewalld.service
在rs1、rs2上:
[root@rs1 ~]# yum install httpd -y
[root@rs2 ~]# yum install httpd -y
[root@rs1 ~]# echo 172.25.254.110 > /var/www/html/index.html
[root@rs1 ~]# systemctl enable --now httpd.service
[root@rs2 ~]# echo 172.25.254.120 > /var/www/html/index.html
[root@rs2 ~]# systemctl enable --now httpd.service
Keepalived-虚拟路由的配置
KeepAlived 配置说明
配置文件组成部分
配置文件:/etc/keepalived/keepalived.conf
在ka1、ka2:
[root@ka1 ~]# yum install keepalived -y
[root@ka2 ~]# yum install keepalived.x86_64 -y
打开主配置文件
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
#全局配置
global_defs {
notification_email {
3173026775@qq.com(发生故障时发送的邮箱,可以写好几个)
}
notification_email_from keepalived@hui.org(发邮件的地址)
smtp_server 127.0.0.1 #邮件服务器地址
smtp_connect_timeout 30 #邮件服务器连接超时时间30s
router_id ka1.hui.org #机唯一标识
vrrp_skip_check_adv_addr #对所有通告报文都检查,会比较消耗性能
(用此配置后,如果收到的通告报文和上一个报文是同一 个路由器,则跳过检查,默认值为全检查)
vrrp_strict
vrrp_garp_interval 0 #报文发送延迟,0表示不延迟
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18 #指定组播IP地址范围:
}
设置虚拟路由
vrrp_instance VI_1 {
state MASTER #主
interface eth0 #网卡
virtual_router_id 100#每个虚拟路由器惟一标识,范围:0-255,每个虚拟路由器此值必须唯一
priority 100 #当前物理节点在此虚拟路由器的优先级,范围:1-254
advert_int 1
authentication {
auth_type PASS #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)
auth_pass 1111 #预共享密钥,仅前8位有效
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1(VIP)
}
}
启动服务:
[root@ka1 ~]# systemctl enable --now keepalived.service
[root@ka1 ~]# ifconfig
复制到ka2上:
[root@ka1 ~]# scp /etc/keepalived/keepalived.conf root@172.25.254.20:/etc/keepalived/keepalived.conf
可以使用抓包命令查看:
[root@ka1 ~]# tcpdump -i eth0 -nn host 224.0.0.18
ka2:
打开[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP(改)
interface eth0
virtual_router_id 100
priority 80(改)
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@ka2 ~]# systemctl enable --now keepalived.service
在rs1上登陆ka1,关掉keepalived服务ka2会开始接替ka1
[root@rs1 ~]# ssh -l root 172.25.254.10
[root@ka1 ~]# systemctl stop keepalived.service
此时ka2上就有VIP了
[root@ka2 ~]# ifconfig
eth0:1
重新启动ka1说的keepalived
[root@ka1 ~]# systemctl start keepalived.service
ka2就无IP了
启用keepalived日志功能
独立日志
打开ka1:
[root@ka1 ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
:wq
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# vim /etc/rsyslog.conf
local6.* /var/log/keepalived.log
:wq
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# systemctl restart rsyslog.service
[root@ka1 ~]# ll /var/log/keepalived.log
-rw------- 1 root root 8743 8月 14 17:23 /var/log/keepalived.log
Keepalived独立子配置文件
/etc/keepalived/keepalived.conf 文件中内容过多,不易管理
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
把vrrp_instance VI_1 {....这一段注释再写入
include "/etc/keepalived/conf.d/*.conf"
[root@ka1 ~]# mkdir -p /etc/keepalived/conf.d [root@ka1 ~]# vim /etc/keepalived/conf.d/172.25.254.100.conf
写入
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# ifconfig
Keepalived-非抢占模式和延迟抢占
即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色,这样会使vip在KA主机中来回漂移,造成网络抖动,
建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色,非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机
要关闭 VIP抢占,必须将各 keepalived 服务器state配置为BACKUP
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP 改
interface eth0
virtual_router_id 100
priority 100
advert_int 1
nopreempt !!!
authentication {
auth_type PASS
auth_pass 1111
}
先不着急启动服务,等ka2启动服务后再启动
[root@ka1 ~]# systemctl restart keepalived.service
此时VIP在ka2上;
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
nopreempt 加!
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
[root@ka2 ~]# systemctl restart keepalived.service
等ka1启动服务后:
[root@ka2 ~]# systemctl stop keepalived.service 停止服务 [root@ka2 ~]# ifconfig 无VIP
此时VIP在ka1 上;
[root@ka2 ~]# systemctl restart keepalived.service 重新启动 [root@ka2 ~]# ifconfig 还是无VIP
延迟抢占
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
preempt_delay 10s !!!!
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
默认keepalived主机之间利用多播相互通告消息,会造成网络拥塞,可以替换成单播,减少网络流量
注意:启用 vrrp_strict 时,不能启用单播
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER !
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10 !
unicast_peer { !
172.25.254.20 !
} !
}
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
[root@ka2 ~]# systemctl restart keepalived.service
[root@ka1 ~]# tcpdump -i eth0 -nn src host 172.25.254.10 and dst 172.25.254.20
Keepalived-邮件通知
[root@ka1 ~]# yum install mailx -y
[root@ka1 ~]# echo hello | mail -s test 3xxxxxxx@qq.com
[root@ka1 ~]# mail
打开浏览器QQ邮箱登陆
安全设置------>最后一个什么什么服务------>生成授权码
cpoqgqnmdprdddba
[root@ka1 ~]# vim /etc/mail.rc 在最后写上
set from=3173026775@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=3xxxxxxxx@qq.com
set smtp-auth-password=cpoqgqnmdprdddba(授权码)
set smtp-auth=login
set ssl-verify=ignore
:wq
[root@ka1 ~]# echo hello | mail -s test 3xxxxx@qq.com
ka2同理
[root@ka1 ~]# vim /etc/keepalived/mail.sh
#/bin/bash
mail_dst="3xxxxxx@qq.com"
send_message()
{
mail_sub="$HOSTNAME to be $1 vip move"
mail_msg="`date +%F\ %T`:vrrp move $HOSTNAME chage $1"
echo $mail_msg | mail -s "$mail_sub" $mail_dst
}
case $1 in
master)
send_massage master
;;
backup)
send_massage backup
;;
fault)
send_message fault
;;
*)
;;
esac
[root@ka1 ~]# chmod +x /etc/keepalived/mail.sh
配置主配置文件:
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
notify_master "/etc/keepalived/mail.sh master"
notify_backup "/etc/keepalived/mail.sh backup"
notify_fault "/etc/keepalived/mail.sh fault"
[root@ka1 ~]# /etc/keepalived/mail.sh fault
keepalived-双主模式
master/slave的单主架构,同一时间只有一个Keepalived对外提供服务,此主机繁忙,而另一台主机却很空闲,利用率低下,可以使用master/master的双主架构,解决此问题。
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf 写入
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 200
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200/24 dev eth0 label eth0:2
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
ka2同理
重启服务再看IP