目录
[三、iSCSI target 配置](#三、iSCSI target 配置)
[3.3.开启iSCSI Target](#3.3.开启iSCSI Target)
[4.1.安装iSCSI Initiator客户端](#4.1.安装iSCSI Initiator客户端)
[4.2.发现Openfiler的iSCSI Target](#4.2.发现Openfiler的iSCSI Target)
[第2步:确认 iSCSI 服务已启用开机自启](#第2步:确认 iSCSI 服务已启用开机自启)
[1)创建安装用户组 dinstall](#1)创建安装用户组 dinstall)
[2)创建安装用户 dmdba](#2)创建安装用户 dmdba)
一、安装openfiler服务器
1.1.准备镜像
https://www.openfiler.com/community/download
1.2.创建虚拟机,安装Openfiler







这里默认就好,后边需要从新挂载磁盘



1.3.配置Openfiler
开机后




警告:设备sda上的分区表不可读。创建新的分区,它被初始化,造成的损失掉这个驱动器上的所有数据。此操作将重写任何以前安装的选择,带动忽视。你想要初始化这个驱动器,清除所有数据?
yes
选择自定义布局

启动分区

交换分区

根分区

默认

进行网络及主机名配置

设置时区

设置密码:测试环境,简单一点就好,123456

开始安装

等待安装过程

重启



二、基础配置
2.1.添加网卡

配置静态网卡地址:

2.2.添加硬盘


三、iSCSI target 配置
3.1.登录到openfiler的web控制端
注意,这里需要使用火狐浏览器,并且调整一下TLS协议支持版本。


将协议的最低支持版本更改为1后重试应该就能直接登录了。
账号:openfiler
密码:password


3.2.创建逻辑卷
登录后,点击Volumes标签

点击create new physical volumes后可以看到我们新加的100G盘,已经被识别为/dev/sdb,点击/dev/sdb


点击页面右下角Reset,然后点击Create。分区类型为Physical volume

点击Volume Groups

输入名称,勾选复选框,单击Add volume group




输入内容,调整磁盘大小,卷类型选择block(iSCSI,FC,etc),按照表格创建四个逻辑卷
|------------|---------------|--------------------------------|---------------------------------------------------|--------------------|
| 卷名 | 大小 | 关键设置 | Volume Description ( 卷描述) | 作用说明 |
| lun_vote | 5GB /5120MB | Filesystem:block(iSCSI,FC,etc) | DAMENG_VOTE_5G | 明确标识这是达梦集群投票盘,5G大小 |
| lun_dcr | 10GB /10240MB | Filesystem:block(iSCSI,FC,etc) | DAMENG_DCR_10G | 标识达梦集群配置注册表盘 |
| lun_data | 70GB /71680MB | Filesystem:block(iSCSI,FC,etc) | DAMENG_DATA_70G | 标识主数据文件存储 |
| lun_log | 15GB /13560MB | Filesystem:block(iSCSI,FC,etc) | DAMENG_LOG_15G | 标识Redo日志存储 |

重复执行Add Volume操作,直到使用完最后的空间



3.3.开启iSCSI Target
点击Services标签栏设置iSCSI Target 为Enable,即为开机自启,并点击Start进行开启服务

添加 iSCSI Target ,点击Add新建Target,Target IQN保持默认或自定义


3.4.映射LUN到Target
选择LUN Mapping标签,点击Map
将4个LUN(lun_vote、lun_dcr、lun_data、lun_log)全部映射


3.5.配置网络访问控制
配置本地网络
由于iSCSI是走IP网络,因此我们要允许网络中的计算机可以透过IP来访问。下面就是OpenFiler中IP网络和同一网段中其他主机的连接方法
1.进入OpenFiler中的System,并且直接拉到页面的下方

2.Network Access Configuration的地方输入这个网络访问的名称,如DAMENG_STORAGE_NET
3.输入主机的IP段。注意不可以输入单一主机的IP,这样会都无法访问。我们在这边输入192.168.75.0,表示从192.168.75.1一直到192.168.59.254都能访问,同理可以添加多张网卡。
4.在Netmask中选择255.255.255.0,并且在Type下拉列表框中选择Share,之后即可以单击Update按钮

选择完之后就更新
至此就可以在这个OpenFiler中看到被授权的网段了!!!
3.6.网络访问权限
在iSCSI Targets中,点击 Network ACL 标签

设置Access为Allow 然后点击Update
到此存储的web端的配置已经完成!!
3.7.关键:修改全局访问控制文件
通过SSH登录Openfiler服务器执行:
编辑initiators.deny文件
vi /etc/initiators.deny
将类似下面这行注释掉:
iqn.2006-01.com.openfiler:tsn.xxxxxxxx ALL

验证修改,筛选出该配置文件中没有注释的行:grep -v "^#" /etc/initiators.deny

至此,我们IP SAN存储网络的target端已经配置完全,若想完全使用IP SAN网络,还需要在对应的节点中,使用iSCSI Initiator端。
四、达梦共享集群DSC双节点配置
两个节点都要执行
4.1.安装iSCSI Initiator客户端
yum -y install iscsi-initiator-utils
这里如果安装出错,执行:wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo更改一下当前的镜像源下载地址后重新执行

4.2.发现Openfiler的iSCSI Target
iscsiadm -m discovery -t st -p 192.168.75.160
iscsiadm -m discovery -t st -p 192.168.76.160


4.3.配置双路径自动登录
正确格式
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.75.160:3260 --op update -n node.startup -v automatic
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.76.160:3260 --op update -n node.startup -v automatic


4.4.验证是否具备开机自动登陆并挂载多路径配置
第1步:确认自动登录配置是否正确写入
bash
# 查看节点的当前 startup 设置
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.75.160:3260 --op show | grep node.startup
如果输出不是 node.startup = automatic,说明配置没写入成功,重新执行:
bash
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.75.160:3260 --op update -n node.startup -v automatic
注意 :重新执行
iscsiadm -m discovery可能会把automatic重置回manual,如果后续有新的发现操作,需要重新设置。
第2步:确认 iSCSI 服务已启用开机自启
bash
systemctl is-enabled iscsid iscsi
如果显示不是 enabled,执行:
bash
systemctl enable iscsid iscsi
第3步:重启后检查是否已自动登录
重启虚拟机后执行:
bash
iscsiadm -m session
-
如果有输出(显示 Target 信息):说明自动登录 ✅ 成功了
-
如果没有输出:说明自动登录 ❌ 失败,需要查看日志:
bash
journalctl -u iscsi | tail -20
第4步:检查多路径服务状态
bash
systemctl status multipathd
-
如果状态是
active (running):多路径服务正常 -
如果状态是
inactive (dead):启动并设置开机自启:bash
systemctl start multipathd systemctl enable multipathd
第5步:确认多路径聚合是否生效
bash
multipath -ll
应该能看到类似 mpathX (WWID) dm-X NETAPP,LUN 的输出

4.5.登录target
登录
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.75.160:3260 -l
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.09faef6280d9 -p 192.168.76.160:3260 -l
4.6.验证会话
验证会话
iscsiadm -m session


4.7.验证多路径
iscsiadm -m session -P 3
dcs01
[root@dcs01 network-scripts]# iscsiadm -m session -P 3
iSCSI Transport Class version 2.0-870
version 6.2.0.874-22
Target: iqn.2006-01.com.openfiler:tsn.09faef6280d9 (non-flash)
Current Portal: 192.168.75.160:3260,1
Persistent Portal: 192.168.75.160:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:6d85e9ddacb
Iface IPaddress: 192.168.75.171
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 1
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 131072
FirstBurstLength: 262144
MaxBurstLength: 262144
ImmediateData: No
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 3 State: running
scsi3 Channel 00 Id 0 Lun: 0
Attached scsi disk sdb State: running
scsi3 Channel 00 Id 0 Lun: 1
Attached scsi disk sdc State: running
scsi3 Channel 00 Id 0 Lun: 2
Attached scsi disk sdd State: running
scsi3 Channel 00 Id 0 Lun: 3
Attached scsi disk sde State: running
Current Portal: 192.168.76.160:3260,1
Persistent Portal: 192.168.76.160:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:6d85e9ddacb
Iface IPaddress: 192.168.76.171
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 2
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 131072
FirstBurstLength: 262144
MaxBurstLength: 262144
ImmediateData: No
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 4 State: running
scsi4 Channel 00 Id 0 Lun: 0
Attached scsi disk sdf State: running
scsi4 Channel 00 Id 0 Lun: 1
Attached scsi disk sdg State: running
scsi4 Channel 00 Id 0 Lun: 2
Attached scsi disk sdh State: running
scsi4 Channel 00 Id 0 Lun: 3
Attached scsi disk sdi State: running
[root@dcs01 network-scripts]#
dcs02
[root@dcs02 network-scripts]# iscsiadm -m session -P 3
iSCSI Transport Class version 2.0-870
version 6.2.0.874-22
Target: iqn.2006-01.com.openfiler:tsn.09faef6280d9 (non-flash)
Current Portal: 192.168.75.160:3260,1
Persistent Portal: 192.168.75.160:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:ae808c7388a2
Iface IPaddress: 192.168.75.172
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 1
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 131072
FirstBurstLength: 262144
MaxBurstLength: 262144
ImmediateData: No
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 3 State: running
scsi3 Channel 00 Id 0 Lun: 0
Attached scsi disk sdb State: running
scsi3 Channel 00 Id 0 Lun: 1
Attached scsi disk sdc State: running
scsi3 Channel 00 Id 0 Lun: 2
Attached scsi disk sdd State: running
scsi3 Channel 00 Id 0 Lun: 3
Attached scsi disk sde State: running
Current Portal: 192.168.76.160:3260,1
Persistent Portal: 192.168.76.160:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:ae808c7388a2
Iface IPaddress: 192.168.76.172
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 2
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 131072
FirstBurstLength: 262144
MaxBurstLength: 262144
ImmediateData: No
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 4 State: running
scsi4 Channel 00 Id 0 Lun: 0
Attached scsi disk sdf State: running
scsi4 Channel 00 Id 0 Lun: 1
Attached scsi disk sdg State: running
scsi4 Channel 00 Id 0 Lun: 2
Attached scsi disk sdh State: running
scsi4 Channel 00 Id 0 Lun: 3
Attached scsi disk sdi State: running
[root@dcs02 network-scripts]#
|-----|----------------|----------------|--------------------|--------|
| 路径 | 源IP | 目标IP | 磁盘映射 | 说明 |
| 路径1 | 192.168.75.171 | 192.168.75.160 | sdb, sdc, sdd, sde | 第一网卡路径 |
| 路径2 | 192.168.76.172 | 192.168.76.160 | sdf, sdg, sdh, sdi | 第二网卡路径 |


4.8.配置Multipath固定别名
在两台节点上执行
安装多路径软件工具-Multipath
yum install -y device-mapper-multipath

启动服务并设置服务为开机自启
systemctl start multipathd
systemctl enable multipathd


4.8.1.获取每块磁盘的WWID
for disk in sdb sdc sdd sde; do echo "Disk /dev/{disk}:"; /lib/udev/scsi_id --whitelisted --device=/dev/{disk}; done

bash
[root@dcs01 network-scripts]#
[root@dcs01 network-scripts]# for disk in sdb sdc sdd sde; do echo "Disk /dev/${disk}:"; /lib/udev/scsi_id --whitelisted --device=/dev/${disk}; done
Disk /dev/sdb:
14f504e46494c45523233464f43512d365534522d67306a36
Disk /dev/sdc:
14f504e46494c45527570376231492d505965752d556e5056
Disk /dev/sdd:
14f504e46494c455230667251564a2d57304e632d39785031
Disk /dev/sde:
14f504e46494c4552646a315056422d5a514b6e2d43326169
[root@dcs01 network-scripts]#
bash
[root@dcs02 network-scripts]# for disk in sdb sdc sdd sde; do echo "Disk /dev/${disk}:"; /lib/udev/scsi_id --whitelisted --device=/dev/${disk}; done
Disk /dev/sdb:
14f504e46494c45523233464f43512d365534522d67306a36
Disk /dev/sdc:
14f504e46494c45527570376231492d505965752d556e5056
Disk /dev/sdd:
14f504e46494c455230667251564a2d57304e632d39785031
Disk /dev/sde:
14f504e46494c4552646a315056422d5a514b6e2d43326169
[root@dcs02 network-scripts]#
4.8.2.配置Multipath别名
编辑:vi /etc/multipath.conf


4.8.3.应用配置并验证
重启multipath服务
systemctl restart multipathd
查看多路径设备
multipath -ll
验证别名设备是否存在
ls -l /dev/mapper/rczy_*

4.8.4.验证多路径识别
检查服务状态
systemctl status multipathd
注意:如果这里执行提示sda的相关失败信息,需在vi /etc/multipath.conf中将sda块设备加入黑名单
blacklist {
devnode "^sda"
}

磁盘映射关系表
|---------------------------|----------|----------|-----|-------|
| Multipath设备 | 原始磁盘路径1 | 原始磁盘路径2 | 大小 | 达梦用途 |
| ++++rczy_vote++++ | /dev/sdd | /dev/sde | 5G | 集群投票盘 |
| ++++rczy_dcr++++ | /dev/sdb | /dev/sdc | 10G | 集群注册表 |
| ++++rczy_data++++ | /dev/sdf | /dev/sdg | 70G | 数据文件 |
| ++++rczy_log++++ | /dev/sdh | /dev/sdi | 15G | 日志文件 |
4.9.进行块设备UUID绑定
4.9.1.创建达梦数据库安装用户和组
注意两个节点的一致性
1)创建安装用户组 dinstall
groupadd -g 12349 dinstall
2)创建安装用户 dmdba
useradd -u 12345 -g dinstall -m -d /home/dmdba -s /bin/bash dmdba
3)初始化用户密码
passwd dmdba
4)之后通过系统提示进行密码设置
密码就设置为:dmdba
4.9.2.设置rczy设备权限
chown dmdba:dinstall /dev/mapper/rczy_*
chmod 660 /dev/mapper/rczy_*

4.9.3.在两节点上配置持久化规则(UDEV)
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| # 在两台节点上都创建这个文件 cat > /etc/udev/rules.d/99-dameng-perm.rules << 'EOF' # 达梦DMASM磁盘权限规则 KERNEL=="dm-*", ENV{DM_NAME}=="rczy_vote", OWNER="dmdba", GROUP="dinstall", MODE="0660" KERNEL=="dm-*", ENV{DM_NAME}=="rczy_dcr", OWNER="dmdba", GROUP="dinstall", MODE="0660" KERNEL=="dm-*", ENV{DM_NAME}=="rczy_data", OWNER="dmdba", GROUP="dinstall", MODE="0660" KERNEL=="dm-*", ENV{DM_NAME}=="rczy_log", OWNER="dmdba", GROUP="dinstall", MODE="0660" EOF |


4.9.4.应用并验证规则
1)应用规则
udevadm control --reload-rules
udevadm trigger
2)验证
ls -l /dev/mapper/rczy_*
ls -l /dev/dm-2 /dev/dm-3 /dev/dm-4 /dev/dm-5

在两台节点上执行最终验证:
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| # 在两台节点上都执行这个验证脚本 cat > /tmp/final_check.sh << 'EOF' #!/bin/bash echo "=== 节点: (hostname) 最终验证 ===" echo "1. 用户ID一致性:" echo " dmdba UID: (id -u dmdba)" echo " dmdba GID: (id -g dmdba)" echo -e "\\n2. 块设备权限:" ls -l /dev/dm-2 /dev/dm-3 /dev/dm-4 /dev/dm-5 2\>/dev/null \| awk '{print " "1" "3":"4" "9}' echo -e "\\n3. 符号链接:" ls -l /dev/mapper/rczy_\* 2\>/dev/null \| awk '{print " "9" -> "11}' echo -e "\\n4. 达梦用户访问测试:" sudo -u dmdba bash -c "ls -l /dev/mapper/rczy_\* \>/dev/null 2\>\&1 \&\& echo ' ✓ 可以访问rczy设备' \|\| echo ' ✗ 无法访问'" echo -e "\\n5. iSCSI连接:" iscsiadm -m session 2\>/dev/null \| wc -l \| awk '{print " 活跃会话: "1" 条"}' EOF chmod +x /tmp/final_check.sh /tmp/final_check.sh |

bash
[root@dcs02 mapper]# # 在两台节点上都执行这个验证脚本
[root@dcs02 mapper]#
[root@dcs02 mapper]# cat > /tmp/final_check.sh << 'EOF'
>
> #!/bin/bash
>
> echo "=== 节点: $(hostname) 最终验证 ==="
>
>
>
> echo "1. 用户ID一致性:"
>
> echo " dmdba UID: $(id -u dmdba)"
>
> echo " dmdba GID: $(id -g dmdba)"
>
>
>
> echo -e "\n2. 块设备权限:"
>
> ls -l /dev/dm-2 /dev/dm-3 /dev/dm-4 /dev/dm-5 2>/dev/null | awk '{print " "$1" "$3":"$4" "$9}'
>
>
>
> echo -e "\n3. 符号链接:"
>
> ls -l /dev/mapper/rczy_* 2>/dev/null | awk '{print " "$9" -> "$11}'
>
>
>
> echo -e "\n4. 达梦用户访问测试:"
>
> sudo -u dmdba bash -c "ls -l /dev/mapper/rczy_* >/dev/null 2>&1 && echo ' ✓ 可以访问rczy设备' || echo ' ✗ 无法访问'"
>
>
>
> echo -e "\n5. iSCSI连接:"
>
> iscsiadm -m session 2>/dev/null | wc -l | awk '{print " 活跃会话: "$1" 条"}'
>
> EOF
[root@dcs02 mapper]# chmod +x /tmp/final_check.sh
[root@dcs02 mapper]#
[root@dcs02 mapper]# /tmp/final_check.sh
=== 节点: dcs02 最终验证 ===
1. 用户ID一致性:
dmdba UID: 12345
dmdba GID: 12349
2. 块设备权限:
brw-rw----. dmdba:dinstall 17:37
brw-rw----. dmdba:dinstall 17:37
brw-rw----. dmdba:dinstall 17:37
brw-rw----. dmdba:dinstall 17:37
3. 符号链接:
/dev/mapper/rczy_data -> ../dm-3
/dev/mapper/rczy_dcr -> ../dm-4
/dev/mapper/rczy_log -> ../dm-5
/dev/mapper/rczy_vote -> ../dm-2
4. 达梦用户访问测试:
✓ 可以访问rczy设备
5. iSCSI连接:
活跃会话: 2 条
[root@dcs02 mapper]#