Oracle 19c RAC 安装文档(4个节点)
📚 1.参考资料
🧭 2.环境规划
官方文档最小要求的内存是8G,但是在实验过程中发现,8G很可能导致数据库无法开启,建议至少要12GB。心跳网络不做bond,用2个网络做负载均衡。
| 服务器名 | 操作系统版本 | 物理IP | 虚拟IP | SCAN IP | 私网IP1 | 私网IP2 | 根目录空间 | 内存 | CPU核数 | 共享磁盘 |
|---|---|---|---|---|---|---|---|---|---|---|
| rac1 | CentOS 7 | 192.168.60.11 | 192.168.60.21 | 192.168.60.31~33(SCAN) | 10.0.0.11 | 10.1.0.11 | 100GB | 12GB | 4 | +OCR(10G×3) +DATA(100G) +ARCH(50G) |
| rac2 | CentOS 7 | 192.168.60.12 | 192.168.60.22 | --- | 10.0.0.12 | 10.1.0.12 | 100GB | 12GB | 4 | --- |
| rac3 | CentOS 7 | 192.168.60.13 | 192.168.60.23 | --- | 10.0.0.13 | 10.1.0.13 | 100GB | 12GB | 4 | --- |
| rac4 | CentOS 7 | 192.168.60.14 | 192.168.60.24 | --- | 10.0.0.14 | 10.1.0.14 | 100GB | 12GB | 4 | --- |
🔧3. 安装部署前准备
检查主机时区、时间、各个主机网卡名是否一致。
3.1 依赖包安装
配置本地 YUM 源
bash
mount /dev/cdrom /mnt/
cd /etc/yum.repos.d/
# 备份原有 repo 文件
mkdir bak && mv *.repo bak/
vi /etc/yum.repos.d/ol.repo
bash
[local-source]
name=ol
baseurl=file:///mnt
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
安装依赖包
bash
yum install -y bc binutils compat-libcap1 compat-libstdc++ elfutils-libelf elfutils-devel fontconfig-devel glibc glibc-devel libaio libaio-devel libX11 libXau libXi libXtst libXrender libXrender-devel libgcc xterm librdmacm-devel libstdc++ libstdc++-devel libxcb make smartmontools sysstat net-tools nfs-utils python python-configshell python-rtslib python-six targetcli libaio-devel.x86_64 compat-libstdc++* ksh* xterm tigervnc-server tigervnc-server-module device-mapper-multipath
3.2 修改 transparent_hugepage
bash
cat /etc/default/grub
# 确认 GRUB_CMDLINE_LINUX 中是否有 transparent_hugepage=madvise
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
cat /sys/kernel/mm/transparent_hugepage/enabled
# 应为:always [madvise] never
3.3 创建用户与用户组
bash
groupadd -g 1000 oinstall
groupadd -g 1001 dba
groupadd -g 1002 asmadmin
groupadd -g 1003 asmdba
groupadd -g 1004 asmoper
useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper,dba grid
useradd -u 1101 -g oinstall -G dba,asmdba,asmadmin,asmoper oracle
passwd grid
passwd oracle
mkdir -p /u01/app/19.3.0/grid
mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle/product/19.3.0/dbhome_1
mkdir -p /u01/app/oraInventory
chown -R oracle:oinstall /u01
chown -R grid:oinstall /u01/app/grid /u01/app/oraInventory /u01/app/19.3.0/grid
chmod -R 775 /u01/
| 作用 | 路径 |
|---|---|
| grid_home | /u01/app/19.3.0/grid |
| grid_base | /u01/app/grid |
| oracle_base | /u01/app/oracle |
| oracle_home | /u01/app/oracle/product/19.3.0/dbhome_1 |
3.4 配置 /etc/hosts
bash
hostnamectl set-hostname rac1 # 各节点分别设置
vi /etc/hosts
text
##ip##
192.168.60.11 rac1
192.168.60.12 rac2
192.168.60.13 rac3
192.168.60.14 rac4
##vip##
192.168.60.21 rac1vip
192.168.60.22 rac2vip
192.168.60.23 rac3vip
192.168.60.24 rac4vip
##private ip##
10.0.0.11 rac1priv1
10.0.0.12 rac2priv1
10.0.0.13 rac3priv1
10.0.0.14 rac4priv1
10.1.0.11 rac1priv2
10.1.0.12 rac2priv2
10.1.0.13 rac3priv2
10.1.0.14 rac4priv2
##scan ip##
192.168.60.31 racscan
192.168.60.32 racscan
192.168.60.33 racscan
3.5 系统文件配置
1. 修改 /etc/security/limits.conf
bash
vi /etc/security/limits.conf
添加以下内容:
text
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft nofile 65536
oracle hard nofile 65536
oracle soft memlock unlimited
oracle hard memlock unlimited
oracle soft stack 10240
oracle hard stack 32768
grid soft nproc 16384
grid hard nproc 16384
grid soft nofile 65536
grid hard nofile 65536
grid soft memlock unlimited
grid hard memlock unlimited
grid soft stack 10240
grid hard stack 32768
2. 修改 /etc/sysctl.conf
bash
vi /etc/sysctl.conf
添加以下内容:
text
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
kernel.shmall = 4194304
kernel.shmmax = 5153960755
fs.aio-max-nr = 3145728
vm.min_free_kbytes = 2115694
vm.swappiness = 10
参数说明(简要):
kernel.shmmax:单个共享内存段最大值,建议为物理内存的 60%kernel.shmall:共享内存总量限制(单位:页,通常为 4KB)vm.min_free_kbytes:系统保留最低内存(单位 KB),建议为物理内存的 0.4%vm.swappiness:控制 swap 使用倾向,值越低越倾向使用物理内存
使配置生效:
bash
sysctl -p
3. 修改 /etc/pam.d/login
bash
vi /etc/pam.d/login
添加:
text
session required pam_limits.so
4. 修改 /etc/profile
bash
vi /etc/profile
添加:
bash
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
3.6 禁用 SELinux 与防火墙
bash
setenforce 0
vi /etc/sysconfig/selinux
修改:
text
SELINUX=disabled
关闭防火墙:
bash
systemctl stop firewalld
systemctl disable firewalld
3.7 oracle、grid 用户环境变量配置
oracle 用户(所有节点)
bash
su - oracle
vi ~/.bash_profile
添加:
bash
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=orcl1 # 节点2为 orcl2,以此类推
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/19.3.0/dbhome_1
export TNS_ADMIN=$ORACLE_HOME/network/admin
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
grid 用户(所有节点)
bash
su - grid
vi ~/.bash_profile
添加:
bash
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=+ASM1 # RAC1
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/19.3.0/grid
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
3.8 修改心跳网络负载均衡
bash
vi /etc/sysctl.conf
添加:
text
net.ipv4.conf.ens224.rp_filter = 2
net.ipv4.conf.ens256.rp_filter = 2
bash
sysctl -p
3.9 关闭 RemoveIPC(Linux 7.2 以上)
bash
vi /etc/systemd/logind.conf
# 修改
RemoveIPC=no
vi /usr/lib/systemd/system/systemd-logind.service
# 修改
RemoveIPC=no
systemctl daemon-reload
systemctl restart systemd-logind
检查,没有信息则OK:
bash
loginctl show-session | grep RemoveIPC
systemctl show systemd-logind | grep RemoveIPC
3.10 MTU 值修改(官方建议)
根据官档Doc ID 2041723.1,适用于11.2.0.4及以上版本。
bash
ifconfig lo mtu 16436
vi /etc/sysconfig/network-scripts/ifcfg-lo
添加:
bash
MTU=16436
重启网络:
bash
service network restart

3.11 配置 HugePages(当物理内存 > 64G 时可配置)
参考官方文档:Doc ID 401749.1,本次测试内存仅12GB,不需要配置。
bash
vi /etc/sysctl.conf
添加:
text
vm.nr_hugepages = (所有数据库实例使用的 SGA 内存之和 / 2M) + 100
说明:
- 单位为"页",每页 2MB
- 不包括 ASM 实例内存
- 修改后需重启系统生效
查看是否生效:
bash
cat /proc/meminfo | grep Huge
示例:
text
HugePages_Total: 1544
HugePages_Free: 30
3.12 解压安装包(节点1)
bash
su - grid
cd $ORACLE_HOME
unzip /soft/grid19c.zip
3.13 配置 SSH 互信(方法1:手动)
生成密钥(oracle 和 grid 用户都要执行)
bash
ssh-keygen -t rsa -P ''
合并公钥
将各节点的 ~/.ssh/id_rsa.pub 内容复制到所有节点的 ~/.ssh/authorized_keys
测试连通性
bash
ssh rac1
ssh rac2
ssh rac3
ssh rac4
3.14 配置 SSH 互信(方法2:Oracle 脚本推荐)
bash
# root 用户在 rac1 执行
cd /u01/app/19.3.0/grid/oui/prov/resources/scripts/
./sshUserSetup.sh -user grid -hosts "rac1 rac2 rac3 rac4" -advanced -noPromptPassphrase
./sshUserSetup.sh -user oracle -hosts "rac1 rac2 rac3 rac4" -advanced -noPromptPassphrase
3.15 安装 cvuqdisk(所有节点)
bash
# 在 rac1 找到 rpm 包
cd /u01/app/19.3.0/grid/cv/rpm
rpm -ivh cvuqdisk-1.0.10-1.rpm
# 拷贝并安装到其他节点
scp cvuqdisk-1.0.10-1.rpm rac2:/tmp/
ssh rac2 "cd /tmp && rpm -ivh cvuqdisk-1.0.10-1.rpm"
3.16.Vcenter配置共享磁盘
4台虚拟机添加参数disk.EnableUUID = "TRUE"

4台虚机都新建一个SCSI


节点1配置共享硬盘,配置如下:


创建 10G3 100G 1 50G *1 ,共5个硬盘。

其他3台服务器通过现有硬盘添加

添加完要修改相应配置

3.17.使用多路径配置共享存储
1.每个节点安装multipath:
bash
yum install -y device-mapper-multipath
2.启动multipath(所有节点)
bash
modprobe dm-multipath
modprobe dm-round-robin
检查内核添加情况:
lsmod |grep multipath
将多路径软件multipath设置为开机自启动并启动
bash
systemctl enable multipathd.service
systemctl start multipathd.service
systemctl status multipathd.service
Active: active (running)
3.多路径配置
所得所有存储的lun的wwid:
bash
for i in `cat /proc/partitions |awk {'print $4'} |grep sd`; do echo "Device: $i WWID: `/usr/lib/udev/scsi_id --page=0x83 --whitelisted --device=/dev/$i` "; done |sort -k4
Device: sdb WWID: 36000c29106f5851f44c0f23109a4a4c2
Device: sdd WWID: 36000c29143f1cc65c36f2637b5507c1b
Device: sdc WWID: 36000c2929f0618f3894a77b329c24563
Device: sde WWID: 36000c29a499665b56027292e2208ddb4
Device: sdf WWID: 36000c29acedbf4f5aa8eb7336d7d3d0f
也可使用命令: /usr/lib/udev/scsi_id -g -u /dev/sdc 查看指定盘符的uuid
bash
vi /etc/multipath.conf
defaults {
user_friendly_names yes
}
devices {
device {
vendor "(COMPAQ|HP)"
product "HSV1[01]1|HSV2[01]0|HSV300|HSV4[05]0"
#getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
features "0"
hardware_handler "0"
path_selector "round-robin 0"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
no_path_retry 12
rr_min_io 1000
path_checker tur
prio alua
}
}
multipaths {
multipath {
wwid 36000c29106f5851f44c0f23109a4a4c2
alias OCR1
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 36000c29143f1cc65c36f2637b5507c1b
alias OCR2
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 36000c2929f0618f3894a77b329c24563
alias OCR3
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 36000c29a499665b56027292e2208ddb4
alias DATA
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 36000c29acedbf4f5aa8eb7336d7d3d0f
alias ARCH
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
}
blacklist {
devnode "^(sda|ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^asm/*" # Here
devnode "ofsctl" # Here
}
#启动多路径:
systemctl restart multipathd.service
启动multipath配置后,会在/dev/mapper下生成多路径逻辑盘
bash
cd /dev/mapper
ll
#查看多路径的状态:
multipath -ll
4.配置udev,更改属组
bash
vi /etc/udev/rules.d/99-oracle-asmdevices.rules
ENV{DM_NAME}=="OCR1", OWNER:="grid", GROUP:="asmadmin", MODE:="0660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"
ENV{DM_NAME}=="OCR2", OWNER:="grid", GROUP:="asmadmin", MODE:="0660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"
ENV{DM_NAME}=="OCR3", OWNER:="grid", GROUP:="asmadmin", MODE:="0660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"
ENV{DM_NAME}=="DATA", OWNER:="grid", GROUP:="asmadmin", MODE:="0660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"
ENV{DM_NAME}=="ARCH", OWNER:="grid", GROUP:="asmadmin", MODE:="0660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"
5.重启多路径和udev
bash
/usr/sbin/udevadm control --reload-rules
---验证权限
ls -l /dev/dm-*
---验证命名结果
ls -l /dev/mapper/*
有可能权限没变化,可以重启服务器,基本就可以了。
4. 安装部署
4.1 安装 Grid(图形化安装)
略,图形化这步骤比较简单,懒的贴图了。(主要是实验操作过程中忘记截图了)
4.2 安装 Oracle 数据库软件
同上。

