pgsql集群搭建
主机ip:01--192.168.171.4 02--192.168.171.5 03--192.168.171.6
1.放行必要端口
ini
sudo firewall-cmd --permanent --zone=public \
--add-port=5432/tcp \
--add-port=6432/tcp \
--add-port=2379/tcp \
--add-port=8008/tcp \
--add-port=2380/tcp
# 重载防火墙
sudo firewall-cmd --reload
# 查看放行
sudo firewall-cmd --list-ports
2.禁用thp
ini
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo "echo never >/sys/kernel/mm/transparent_hugepage/enabled" >>/etc/rc.local
chmod +x /etc/rc.local
为什么要禁用 THP?
THP 是什么?
- Linux 内核的内存管理特性,自动将普通 4KB 内存页合并为 2MB 大页
- 目的是减少 TLB(转译后备缓冲器)缺失,提升内存访问性能
为什么数据库(尤其是 PostgreSQL)要禁用?
| 问题 | 说明 |
|---|---|
| 内存分配延迟 | THP 的内存合并/拆分是异步的,可能在运行时触发,导致不可预测的延迟 spikes |
| 写放大 | 即使只修改大页中的一小部分数据,整个 2MB 页面都会被标记为脏页,增加 I/O 负担 |
| 内存碎片 | THP 可能导致内存碎片化,反而降低性能 |
| fork 延迟 | PostgreSQL 使用多进程架构,fork 子进程时 THP 可能增加延迟 |
3.安装 etcd 与 PgBouncer
CentOS 7 的官方基础源或 EPEL 源中包含了这两个组件:
3.1安装 EPEL 源(PgBouncer 通常在此源中)
ini
yum install -y epel-release
3.2 安装 etcd 和 pgbouncer
ini
yum install -y etcd pgbouncer
4.安装 Python 3 及 Patroni
CentOS 7 自带的 python3 (3.6) 可能会在安装最新版 Patroni 时遇到依赖问题。推荐通过以下方式安装:
4.1安装 Python3 及开发依赖工具
ini
yum install -y python3 python3-devel gcc pidof
4.2 升级 pip 并通过 pip 安装 psycopg2 和 patroni
ini
pip3 install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple
pip3 install psycopg2-binary patroni[etcd3] -i https://pypi.tuna.tsinghua.edu.cn/simple
5.部署 etcd 集群
5.1修改每台机器的/etc/etcd/etcd.yml。注意将 IP 替换为当前节点的实际内网 IP:
ini
vim /etc/etcd/etcd.yml
192.168.171.4(注意修改为自己的ip)
ini
name: node1
data-dir: /var/lib/etcd/node1.etcd
listen-peer-urls: http://192.168.171.4:2380
listen-client-urls: http://192.168.171.4:2379,http://127.0.0.1:2379
initial-advertise-peer-urls: http://192.168.171.4:2380
advertise-client-urls: http://192.168.171.4:2379
initial-cluster: node1=http://192.168.171.4:2380,node2=http://192.168.171.5:2380,node3=http://192.168.171.6:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new
192.168.171.5(注意修改为自己的ip)
ini
name: node2
data-dir: /var/lib/etcd/node2.etcd
listen-peer-urls: http://192.168.171.5:2380
listen-client-urls: http://192.168.171.5:2379,http://127.0.0.1:2379
initial-advertise-peer-urls: http://192.168.171.5:2380
advertise-client-urls: http://192.168.171.5:2379
initial-cluster: node1=http://192.168.171.4:2380,node2=http://192.168.171.5:2380,node3=http://192.168.171.6:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new
192.168.171.6(注意修改为自己的ip)
ini
name: node3
data-dir: /var/lib/etcd/node3.etcd
listen-peer-urls: http://192.168.171.6:2380
listen-client-urls: http://192.168.171.6:2379,http://127.0.0.1:2379
initial-advertise-peer-urls: http://192.168.171.6:2380
advertise-client-urls: http://192.168.171.6:2379
initial-cluster: node1=http://192.168.171.4:2380,node2=http://192.168.171.5:2380,node3=http://192.168.171.6:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new
5.2修改/usr/lib/systemd/system/etcd.service
ini
vim /usr/lib/systemd/system/etcd.service
进去后按ggdG清空原有数据,粘贴下面数据
ini
[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=notify
ExecStart=/usr/bin/etcd --config-file=/etc/etcd/etcd.yml
Restart=on-failure
[Install]
WantedBy=multi-user.target
5.3找到并编辑配置文件
在 CentOS/RHEL 系统中,配置文件通常位于 /etc/etcd/etcd.conf 你需要修改以下两个参数:
ini
vim /etc/etcd/etcd.conf
| 参数名 | 当前错误配置 | 建议修改为 | 说明 |
|---|---|---|---|
| ETCD_LISTEN_CLIENT_URLS | http://localhost:2379 |
http://0.0.0.0:2379 |
0.0.0.0 表示监听所有网卡,允许外部访问 |
| ETCD_ADVERTISE_CLIENT_URLS | http://localhost:2379 |
http://<本机IP>:2379 |
告诉集群其他成员,通过这个 IP 来连接我 |
5.4 重启服务
修改完三台机器的配置后,依次执行:
ini
systemctl daemon-reload
systemctl restart etcd
启动并验证 etcd:
ini
systemctl enable etcd
systemctl start etcd
验证 etcd 健康状态 (CentOS 7 默认是 v2 API,指定 v3 执行验证)
ini
ETCDCTL_API=3 etcdctl endpoint health --endpoints=192.168.171.4:2379,192.168.171.5:2379,192.168.171.6:2379
6.配置并启动 Patroni 管理 PG14
- 在各个节点创建配置文件(例如放于/etc/patroni/patroni.yml):
ini
mkdir /etc/patroni
vim /etc/patroni/patroni.yml
192.168.171.4(修改为自己的ip)
yml
scope: pg-ha-cluster
namespace: /service
name: pg-node-1
# 修改为自己集群的ip
etcd3:
hosts:
- 192.168.171.4:2379
- 192.168.171.5:2379
- 192.168.171.6:2379
restapi:
listen: 0.0.0.0:8008
# 修改
connect_address: 192.168.171.4:8008
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
synchronous_mode: true
postgresql:
use_pg_rewind: true
use_slots: true
initdb:
- encoding: UTF8
- data-checksums
pg_hba:
- host replication replicator 0.0.0.0/0 md5
- host all all 0.0.0.0/0 md5
users:
admin:
password: 'AdminSecurePassword123'
options:
- superuser
- createdb
postgresql:
listen: 0.0.0.0:5432
connect_address: 192.168.171.4:5432
data_dir: /var/lib/pgsql/14/data
bin_dir: /usr/pgsql-14/bin
pgpass: /var/lib/pgsql/.pgpass
authentication:
replication:
username: replicator
password: 'ReplicaPassword123'
superuser:
# 你的pgsql的超级用户和密码
username: postgres
password: '123456'
parameters:
password_encryption: md5
ssl: off
shared_buffers: 128MB
effective_cache_size: 32GB
work_mem: 64MB
maintenance_work_mem: 1GB
max_wal_size: 2GB
min_wal_size: 1GB
checkpoint_completion_target: 0.9
checkpoint_timeout: 15min
max_worker_processes: 3
max_parallel_workers_per_gather: 2
max_parallel_workers: 3
yml
scope: pg-ha-cluster
namespace: /service
name: pg-node-2
etcd3:
hosts:
- 192.168.171.4:2379
- 192.168.171.5:2379
- 192.168.171.6:2379
restapi:
listen: 0.0.0.0:8008
connect_address: 192.168.171.5:8008
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
synchronous_mode: true
postgresql:
use_pg_rewind: true
use_slots: true
initdb:
- encoding: UTF8
- data-checksums
pg_hba:
- host replication replicator 0.0.0.0/0 md5
- host all all 0.0.0.0/0 md5
users:
admin:
password: 'AdminSecurePassword123'
options:
- superuser
- createdb
postgresql:
listen: 0.0.0.0:5432
connect_address: 192.168.171.5:5432
data_dir: /var/lib/pgsql/14/data
bin_dir: /usr/pgsql-14/bin
pgpass: /var/lib/pgsql/.pgpass
authentication:
replication:
username: replicator
password: 'ReplicaPassword123'
superuser:
username: postgres
password: '123456'
parameters:
password_encryption: md5
ssl: off
shared_buffers: 128MB
effective_cache_size: 32GB
work_mem: 64MB
maintenance_work_mem: 1GB
max_wal_size: 2GB
min_wal_size: 1GB
checkpoint_completion_target: 0.9
checkpoint_timeout: 15min
max_worker_processes: 3
max_parallel_workers_per_gather: 2
max_parallel_workers: 3
yml
scope: pg-ha-cluster
namespace: /service
name: pg-node-3
etcd3:
hosts:
- 192.168.171.4:2379
- 192.168.171.5:2379
- 192.168.171.6:2379
restapi:
listen: 0.0.0.0:8008
connect_address: 192.168.171.6:8008
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
synchronous_mode: true
postgresql:
use_pg_rewind: true
use_slots: true
initdb:
- encoding: UTF8
- data-checksums
pg_hba:
- host replication replicator 0.0.0.0/0 md5
- host all all 0.0.0.0/0 md5
users:
admin:
password: 'AdminSecurePassword123'
options:
- superuser
- createdb
postgresql:
listen: 0.0.0.0:5432
connect_address: 192.168.171.6:5432
data_dir: /var/lib/pgsql/14/data
bin_dir: /usr/pgsql-14/bin
pgpass: /var/lib/pgsql/.pgpass
authentication:
replication:
username: replicator
password: 'ReplicaPassword123'
superuser:
username: postgres
password: '123456'
parameters:
password_encryption: md5
ssl: off
shared_buffers: 128MB
effective_cache_size: 32GB
work_mem: 64MB
maintenance_work_mem: 1GB
max_wal_size: 2GB
min_wal_size: 1GB
checkpoint_completion_target: 0.9
checkpoint_timeout: 15min
max_worker_processes: 3
max_parallel_workers_per_gather: 2
max_parallel_workers: 3
2.确保存储目录及权限正确(不需要再生成 SSL 证书)
ini
mkdir -p /var/lib/pgsql/14/data /etc/patroni
chown -R postgres:postgres /var/lib/pgsql/
chmod 700 /var/lib/pgsql/14/data
3.配置 CentOS 7 Systemd 服务来管理 Patroni
ini
vim /etc/systemd/system/patroni.service
ini
[Unit]
Description=Patroni Orchestrator for PostgreSQL
After=network.target etcd.service
[Service]
Type=simple
User=postgres
Group=postgres
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
TimeoutSec=30
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
4.启动 Patroni:
ini
systemctl daemon-reload
systemctl enable patroni
systemctl start patroni
5.切换到 postgres 用户执行查看
ini
su - postgres -c "patronictl -c /etc/patroni/patroni.yml list"
查看日志
ini
journalctl -u patroni -n 100 --no-pager