利用ASM功能实现零停机oracle RAC存储迁移

潇湘秦2024-01-31 20:39

在实际生产过程中，经常会遇到比如新旧存储替换等场景，如果是一般情况可能只有做数据迁移才能实现，而迁移可能就存在停机的可能

这里介绍利用ASM DG加盘踢盘功能实现零停机存储迁移；原理很简单 将新存储磁盘添加到data dg，之后将老的磁盘踢出data磁盘组，即可实现存储迁移

solaris的ZFS的pool功能可以实现类似ASM的零停机迁移本地目录存储迁移，请参考如下
利用 ZFS实现存储迁移 - 墨天轮文档

  1. 存储划盘给集群
  2. 可能需要重启机器才能识别到新加盘
    （IBM,SUN小机等也存在这种问题，需要重启才能识别新加盘，双节点轮流重启可以不影响生产）

新加四块盘磁盘的属性不同

[oracle@db4 ~]$ ll /dev/dm*

brw-rw---- 1 root disk 253, 0 Aug 31 10:04 /dev/dm-0

brw-rw---- 1 grid asmadmin 253, 1 Aug 31 10:08 /dev/dm-1

brw-rw---- 1 root disk 253, 10 Aug 31 10:04 /dev/dm-10

brw-rw---- 1 root disk 253, 11 Aug 31 10:04 /dev/dm-11

brw-rw---- 1 root disk 253, 12 Aug 31 10:04 /dev/dm-12

brw-rw---- 1 grid asmadmin 253, 2 Aug 31 10:08 /dev/dm-2

brw-rw---- 1 grid asmadmin 253, 3 Aug 31 10:08 /dev/dm-3

brw-rw---- 1 grid asmadmin 253, 4 Aug 31 10:07 /dev/dm-4

brw-rw---- 1 grid asmadmin 253, 5 Aug 31 10:08 /dev/dm-5

brw-rw---- 1 grid asmadmin 253, 6 Aug 31 10:08 /dev/dm-6

brw-rw---- 1 grid asmadmin 253, 7 Aug 31 10:07 /dev/dm-7

brw-rw---- 1 root disk 253, 8 Aug 31 10:04 /dev/dm-8

brw-rw---- 1 root disk 253, 9 Aug 31 10:04 /dev/dm-9

[oracle@db4 ~]$

[oracle@db4 ~]$ su -

查看多路径状态

[root@db4 ~]#multipath --ll

新加四块盘是没有别名的

crs3 (36006016056804c0099bbc25c79d5871a) dm-6 DGC,VRAID

size=2.0G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:6 sdal 66:80 active ready running

| `- 18:0:1:6 sdam 66:96 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 18:0:0:6 sdo 8:224 active ready running

`- 16:0:0:6 sdn 8:208 active ready running

crs2 (36006016056804c0097bbc25c191f51e4) dm-5 DGC,VRAID

size=2.0G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:0:3 sdh 8:112 active ready running

| `- 18:0:0:3 sdi 8:128 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 18:0:1:3 sdag 66:0 active ready running

`- 16:0:1:3 sdaf 65:240 active ready running

crs1 (36006016056804c0096bbc25c2e129aaf) dm-3 DGC,VRAID

size=2.0G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:2 sdad 65:208 active ready running

| `- 18:0:1:2 sdae 65:224 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:2 sdf 8:80 active ready running

`- 18:0:0:2 sdg 8:96 active ready running

dbbak (3600601605a804c00cfc1c35ca78f2198) dm-0 DGC,VRAID

size=2.0T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 18:0:0:0 sdc 8:32 active ready running

| `- 16:0:0:0 sdb 8:16 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:1:0 sdz 65:144 active ready running

`- 18:0:1:0 sdaa 65:160 active ready running

mpathm (36006016056804c000d3f0e6346a3cc75) dm-12 DGC,VRAID

size=500G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 18:0:1:11 sdaw 67:0 active ready running

| `- 16:0:1:11 sdav 66:240 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:11 sdx 65:112 active ready running

`- 18:0:0:11 sdy 65:128 active ready running

mpathl (36006016056804c000d3f0e63d1536140) dm-10 DGC,VRAID

size=500G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:0:10 sdv 65:80 active ready running

| `- 18:0:0:10 sdw 65:96 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:1:10 sdat 66:208 active ready running

`- 18:0:1:10 sdau 66:224 active ready running

mpathk (36006016056804c000c3f0e639c8cd30a) dm-9 DGC,VRAID

size=500G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:9 sdar 66:176 active ready running

| `- 18:0:1:9 sdas 66:192 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:9 sdt 65:48 active ready running

`- 18:0:0:9 sdu 65:64 active ready running

mpathj (36006016056804c000c3f0e6307e54cd5) dm-8 DGC,VRAID

size=500G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:8 sdap 66:144 active ready running

| `- 18:0:1:8 sdaq 66:160 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:8 sdr 65:16 active ready running

`- 18:0:0:8 sds 65:32 active ready running

data4 (36006016056804c00b0bbc25c0a9ff654) dm-2 DGC,VRAID

size=505G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:4 sdah 66:16 active ready running

| `- 18:0:1:4 sdai 66:32 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:4 sdj 8:144 active ready running

`- 18:0:0:4 sdk 8:160 active ready running

data3 (36006016056804c00aebbc25c43721e7f) dm-7 DGC,VRAID

size=505G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:0:7 sdp 8:240 active ready running

| `- 18:0:0:7 sdq 65:0 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 18:0:1:7 sdao 66:128 active ready running

`- 16:0:1:7 sdan 66:112 active ready running

data2 (36006016056804c00adbbc25cd11a9458) dm-4 DGC,VRAID

size=505G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 16:0:1:5 sdaj 66:48 active ready running

| `- 18:0:1:5 sdak 66:64 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:0:5 sdl 8:176 active ready running

`- 18:0:0:5 sdm 8:192 active ready running

data1 (36006016056804c00acbbc25cab85eb40) dm-1 DGC,VRAID

size=505G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw

|-+- policy='round-robin 0' prio=50 status=active

| |- 18:0:0:1 sde 8:64 active ready running

| `- 16:0:0:1 sdd 8:48 active ready running

`-+- policy='round-robin 0' prio=10 status=enabled

|- 16:0:1:1 sdab 65:176 active ready running

`- 18:0:1:1 sdac 65:192 active ready running

[root@db4 ~]#

3.---获取所有磁盘wwid

Linux 7--

for i in `cat /proc/partitions | awk {'print 4'} \| grep sd\`; do val=\`/sbin/blockdev --getsize64 /dev/i` ; val2=`expr val / 1073741824\`; echo "/dev/i:val2 \` /usr/lib/udev/scsi_id -g -u /dev/i`" ; done

Linux 6--

for i in `cat /proc/partitions | awk {'print 4'} \| grep sd\`; do val=\`/sbin/blockdev --getsize64 /dev/i` ; val2=`expr val / 1073741824\`; echo "/dev/i:val2 \` scsi_id -g -u /dev/i`" ; done

4.修改配置文件（两节点均要）

cat /etc/multipath.conf

将新加四块盘加入到配置文件

注意alisa 和wwid号

multipath {

wwid 36006016056804c000d3f0e6346a3cc75

alias data5

}

multipath {

wwid 36006016056804c000d3f0e63d1536140

alias data6

}

multipath {

wwid 36006016056804c000c3f0e639c8cd30a

alias data7

}

multipath {

wwid 36006016056804c000c3f0e6307e54cd5

alias data8

}

重启动多路径软件

#service multipathd restart

启用multipath配置后，会在/dev/mapper下生成多路径逻辑盘

#cd /dev/mapper

原来还是mpathl等 重启后应该能看的到别名

检查多路径状态

5.配置 udev(两节点)

利用udev绑定盘符(是Oracleasm绑定如果用作文件系统使用请忽略下面的步骤)

#cd /etc/udev/rules.d/

修改/etc/udev/rules.d/60-dm-permissions.rules，添加如下内容到最后面

ENV{DM_NAME}=="data5", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"

ENV{DM_NAME}=="data6", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"

ENV{DM_NAME}=="data7", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"

ENV{DM_NAME}=="data8", OWNER:="grid", GROUP:="asmadmin", MODE:="660", SYMLINK+="iscsi/oraasm-$env{DM_NAME}"

重启udev服务

Linux 6 #start_udev

Linux 7:

udevadm control --reload-rules

/sbin/udevadm trigger --type=devices --action=change

查看结果 主要看权限

#ll /dev/dm*

确认盘的权限为grid：asmadmin

6，磁盘组加盘删盘（单节点即可）

column path format a20

select name,state,type,total_mb,free_mb from v$asm_diskgroup;

检查磁盘组状态

select name,path,total_mb,free_mb,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS from v$asm_disk order by path;

select name,path,total_mb,free_mb,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS from v$asm_disk where name like 'DATA%' order by path;

检查新加盘的状态

alter diskgroup DATA add disk '/dev/mapper/data9';

alter diskgroup DATA add disk '/dev/mapper/data10';

alter diskgroup DATA add disk '/dev/mapper/data7';

alter diskgroup DATA add disk '/dev/mapper/data8';

alter diskgroup ARCH add disk '/dev/mapper/arch2';

将盘加到磁盘组

alter diskgroup DATA rebalance power 6;

调整磁盘均衡的并发(生产环境 根据实际情况调整并发度)

select * from v$asm_operation;

均衡结束后将就的磁盘剔除磁盘组 （非必须等到均衡结束）

alter diskgroup DATA drop disk 'DATA_0000','DATA_0001','DATA_0002','DATA_0003';

alter diskgroup ARCH drop disk 'ARCH_0000';

踢完结束后状态

如果旧盘需要重新回收继续使用，记得修改**/etc/multipath.conf 和/etc/udev/rules.d/60-dm-permissions.rules 删掉就磁盘的信息**

墨天伦个人主页： 潇湘秦的个人主页 - 墨天轮

相关推荐
YONG823_API6 分钟前
1688商品数据采集API的测试对接步骤分享（提供免费测试key）
开发语言·数据库·爬虫·python·数据挖掘
码上一元29 分钟前
掌握 Spring 事务管理：深入理解 @Transactional 注解
数据库·spring
程序猿毕设源码分享网32 分钟前
基于springboot停车场管理系统源码和论文
数据库·spring boot·后端
YiSLWLL38 分钟前
Django+Nginx+uwsgi网站使用Channels+redis+daphne实现简单的多人在线聊天及消息存储功能
服务器·数据库·redis·python·nginx·django
.生产的驴1 小时前
Docker Seata分布式事务保护搭建 DB数据源版搭建 结合Nacos服务注册
数据库·分布式·后端·spring cloud·docker·容器·负载均衡
盖盖衍上1 小时前
4.4 MySQL 触发器（Trigger）
数据库·mysql
清心歌1 小时前
Redis入门（九）
数据库·redis
superman超哥1 小时前
Oralce数据库巡检SQL脚本
数据库·oracle·性能优化·dba·rdbms·巡检
墨城烟柳ベ旧人殇1 小时前
MySQL数据库6——SQL优化
数据库·sql·mysql
standxy1 小时前
集成金蝶云星空数据至MySQL的完整案例解析
android·数据库·mysql
热门推荐
01抖音的互联网架构02PyTorch机器学习实现液态神经网络03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04国内外大模型汇总：Open AI大模型、Google大模型、Microsoft大模型、文心一言大模型、通义千问大模型、字节豆包大模型、智普清言大模型05Coze扣子平台完整体验和实践（附国内和国际版对比）06(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明07【凡人修仙传_01】散修厉飞羽是如何出师未捷身先死的！08玄机平台应急响应—webshell查杀09RAG 实践- Ollama+RagFlow 部署本地知识库10〔AI 绘画〕Stable Diffusion 之 解决绘制多人或面部很小的人物时面部崩坏问题 篇