服务器磁盘卷组缓存cache设置介绍

工具1： storcli

a. 确认软件包是否安装

root@localhost \~\]#rpm -qa \| grep storcli storcli-1.21.06-1.noarch 备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下： #rpm -ivh storcli-xx-xx-1.noarch.rpm ### **b. 查看逻辑卷读/写缓存状态** 备注：请关注标红部分示例为：NRWTD, 参数解释如下： NR: No Read Ahead（关闭预读，即读缓存关闭） R:Read Ahead(开启预读，即开启读缓存） WB: Write Back（开启回写，即开启写缓存） AWB: Always Write Back（强制开启回写，强制开启写缓存） WT: Write Through(直写模式，关闭回写/写缓存） > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 show > > Controller = 0 > > Status = Success > > Description = None > > Virtual Drives : > > ============== > > -------------------------------------------------------------- > > DG/VD TYPE State Access Consist Cache Cac sCC Size Name > > -------------------------------------------------------------- > > 1/1 RAID10 Optl RW No NRWTD - ON 2.180 TB > > -------------------------------------------------------------- > > Cac=CacheCade\|Rec=Recovery\|OfLn=OffLine\|Pdgd=Partially Degraded\|Dgrd=Degraded > > Optl=Optimal\|RO=ReadOnly\|RW=ReadWrite\|HD=Hidden\|TRANS=TransportReady\|B=Blocked\| Consist=Consistent\|R=ReadAheadAlways\|NR=NoReadAhead\|WB=WriteBack\| AWB=AlwaysWriteBack\|WT=WriteThrough\|C=CachedIO\|D=DirectIO\|sCC=ScheduledCheck Consistency ### **c. 读缓存打开/关闭** 语法如下： storcli /cx/vx set rdcache=RA\|NoRA 参数说明： cx: x为数字，从0开始，代表控制器编号 vx: x为数字，从0开始，代表逻辑卷编号 RA: Read Ahead NoRA: No Read Ahead 示例： > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=ra > > Controller = 0 > > Status = Success > > Description = None > > Detailed Status : > > =============== > > --------------------------------------- > > VD Property Value Status ErrMsg ErrCd > > --------------------------------------- > > 1 rdCache RA Success - 0 > > --------------------------------------- > > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=nora > > Controller = 0 > > Status = Success > > Description = None > > Detailed Status : > > =============== > > --------------------------------------- > > VD Property Value Status ErrMsg ErrCd > > --------------------------------------- > > 1 rdCache NoRA Success - 0 > > --------------------------------------- ### **d. 写缓存打开/关闭** 语法如下： storcli /cx/vx set wrcache=WT\|WB\|AWB 参数说明： cx: x为数字，从0开始，代表控制器编号 vx: x为数字，从0开始，代表逻辑卷编号 WT: Write Through WB: Write Back AWB: Always Write Back 示例： > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wt > > Controller = 0 > > Status = Success > > Description = None > > Detailed Status : > > =============== > > --------------------------------------- > > VD Property Value Status ErrMsg ErrCd > > --------------------------------------- > > 1 wrCache WT Success - 0 > > --------------------------------------- > > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wb > > Controller = 0 > > Status = Success > > Description = None > > Detailed Status : > > =============== > > --------------------------------------- > > VD Property Value Status ErrMsg ErrCd > > --------------------------------------- > > 1 wrCache WB Success - 0 > > --------------------------------------- > > \[root@localhost \~\]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=awb > > Controller = 0 > > Status = Success > > Description = None > > Detailed Status : > > =============== > > --------------------------------------- > > VD Property Value Status ErrMsg ErrCd > > --------------------------------------- > > 1 wrCache AWB Success - 0 > > --------------------------------------- ## **工具2：megacli** ### **a. 确认软件包是否安装** \[root@localhost \~\]# rpm -qa \| grep -i MegaCli MegaCli-8.07.10-1.noarch 备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下： #rpm -ivh MegaCli-xx-xx-1.noarch.rpm ### **b. 查看逻辑卷读/写缓存状态** 请关注标红部分示例为： WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU, 参数解释如下： NORA: No Read Ahead（关闭预读，即读缓存关闭） RA:Read Ahead(开启预读，即开启读缓存） WB: Write Back（开启回写，即开启写缓存） ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存） WT: Write Through(直写模式，关闭回写/写缓存） 示例： > \[root@localhost \~\]# /opt/MegaRAID/MegaCli/MegaCli64 -ldinfo -l1 -a0 > > Adapter 0 -- Virtual Drive Information: > > Virtual Drive: 1 (Target Id: 1) > > Name : > > RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 > > Size : 2.180 TB > > Sector Size : 512 > > Is VD emulated : Yes > > Mirror Data : 2.180 TB > > State : Optimal > > Strip Size : 64 KB > > Number Of Drives per span:2 > > Span Depth : 5 > > Default Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU > > Current Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU > > Default Access Policy: Read/Write > > Current Access Policy: Read/Write > > Disk Cache Policy : Disk's Default > > Ongoing Progresses: > > Check Consistency : Completed 26%, Taken 493 min. > > Encryption Type : None > > Bad Blocks Exist: No > > PI type: No PI > > Is VD Cached: No > > Exit Code: 0x00 ### **c. 读/写缓存打开/关闭** 语法如下： MegaCli -LDSetProp {-Name LdNamestring} \| -RW\|RO\|Blocked\|RemoveBlocked \| WT\|WB\|ForcedWB \[-Immediate\] \|RA\|NORA\|ADRA \| DsblPI \| Cached\|Direct \| -EnDskCache\|DisDskCache \| CachedBadBBU\|NoCachedBadBBU -Lx\|-L0,1,2\|-Lall -aN\|-a0,1,2\|-aALL 参数说明： aN: N为数字，从0开始，代表控制器编号 Lx: x为数字，从0开始，代表逻辑卷编号 NORA: No Read Ahead（关闭预读，即读缓存关闭） RA:Read Ahead(开启预读，即开启读缓存） WB: Write Back（开启回写，即开启写缓存） ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存） WT: Write Through(直写模式，关闭回写/写缓存） **读缓存修改示例：** > \[root@localhost \~\]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp RA -L1 -a0 > > Set Read Policy to ReadAhead on Adapter 0, VD 1 (target id: 1) success > > Exit Code: 0x00 > > \[root@localhost \~\]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp NORA -L1 -a0 > > Set Read Policy to NoReadAhead on Adapter 0, VD 1 (target id: 1) success > > Exit Code: 0x00 **读缓存修改示例：** > \[root@localhost \~\]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WB -L1 -a0 > > Set Write Policy to WriteBack on Adapter 0, VD 1 (target id: 1) success > > Exit Code: 0x00 > > \[root@localhost \~\]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WT -L1 -a0 > > Set Write Policy to WriteThrough on Adapter 0, VD 1 (target id: 1) success > > Exit Code: 0x00 > > \[root@localhost \~\]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp ForcedWB -L1 -a0 > > Set Write Policy to Forced WriteBack on Adapter 0, VD 1 (target id: 1) success > > Exit Code: 0x00 ## **工具3：hpssacli/ssacli** 说明： 1. hpssacli工具为旧工具，目前大部分线上服务器都安装有对应的安装包，ssacli为新工具，在旧工具无法正常执行的情况下，建议使用新工具，语法完全一致。 2. ssdsmartpath（固态硬盘智能路径）和caching(写缓存）只能打开一个，在一个打开的状态下，另外一个的值必定是关闭的,例如： caching：Enable的情况下，ssdsmartpath的值必定为disabled. ### **a. 确认安装包是否安装** \[root@localhost \~\]# rpm -qa \| grep hpssacli hpssacli-2.20-11.0.x86_64 备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下： #rpm -ivh hpssacli-xx-xx.x86_64.rpm #rpm -ivh ssacli-xx-xx.x86_64.rpm ### **b. 确认硬盘控制器所在slot插槽** 说明：如下所示， 阵列卡型号为Smart Array P840ar, 插槽位置为Slot 0, 务必查询信息，插槽位置后面的命令要用。 \[root@localhost \~\]# hpssacli ctrl all show Smart Array P840ar in Slot 0 (Embedded) (sn: PVYKH0BRH710CO) ### **c. 确认逻辑卷写缓存(caching)状态** 参数说明： ctrl: Controller 简写 slot=0: 硬盘控制器所在PCIe插槽位置 ld 2: ld(logical drive 逻辑卷）,后面的数字为逻辑卷编号，顺序从1开始 show: 信息展示 说明：请关注示例中caching的值，示例中为Disabled(即关闭） > \[root@localhost \~\]# hpssacli ctrl slot=0 ld 2 show > > Smart Array P840ar in Slot 0 (Embedded) > > array B > > Logical Drive: 2 > > Size: 2.2 TB > > Fault Tolerance: 1+0 > > Heads: 255 > > Sectors Per Track: 32 > > Cylinders: 65535 > > Strip Size: 256 KB > > Full Stripe Size: 1280 KB > > Status: OK > > MultiDomain Status: OK > > Caching: Disabled > > Unique Identifier: 600508B1001C0D8EB99EB2A7EE5060E4 > > Disk Name: /dev/sdb > > Mount Points: /opt/uredis 2.2 TB Partition Number 1 > > Logical Drive Label: 06172624PVYKH0BRH710COB70A > > Mirror Group 1: > > physicaldrive 1I:1:5 (port 1I:box 1:bay 5, Solid State SATA, 480.1 GB, OK) > > physicaldrive 1I:1:6 (port 1I:box 1:bay 6, Solid State SATA, 480.1 GB, OK) > > physicaldrive 1I:1:7 (port 1I:box 1:bay 7, Solid State SATA, 480.1 GB, OK) > > physicaldrive 1I:1:8 (port 1I:box 1:bay 8, Solid State SATA, 480.1 GB, OK) > > physicaldrive 1I:1:4 (port 1I:box 1:bay 4, Solid State SATA, 480.1 GB, OK) > > Mirror Group 2: > > physicaldrive 1I:1:3 (port 1I:box 1:bay 3, Solid State SATA, 480.1 GB, OK) > > physicaldrive 2I:2:1 (port 2I:box 2:bay 1, Solid State SATA, 480.1 GB, OK) > > physicaldrive 2I:2:2 (port 2I:box 2:bay 2, Solid State SATA, 480.1 GB, OK) > > physicaldrive 2I:2:3 (port 2I:box 2:bay 3, Solid State SATA, 480.1 GB, OK) > > physicaldrive 2I:2:4 (port 2I:box 2:bay 4, Solid State SATA, 480.1 GB, OK) > > Drive Type: Data > > LD Acceleration Method: HP SSD Smart Path ### **d. 确认逻辑卷ssdsmartpath状态** 参数说明： ctrl: Controller 简写 slot=0: 硬盘控制器所在PCIe插槽位置 array b: ld(array, 磁盘阵列，也即逻辑卷）,后面的字母为逻辑卷编号，顺序从a开始 show: 信息展示 说明：请关注示例中ssdsmartpath的值，示例中为Enable(即关闭） > \[root@localhost \~\]# hpssacli ctrl slot=0 array b show > > Smart Array P840ar in Slot 0 (Embedded) > > Array: B > > Interface Type: Solid State SATA > > Unused Space: 0 MB (0.0%) > > Used Space: 4.4 TB (100.0%) > > Status: OK > > MultiDomain Status: OK > > Array Type: Data > > HP SSD Smart Path: enable ### **e. 关闭固态硬件智能路径并打开写缓存示例** \[root@localhost \~\]# hpssacli ctrl slot=0 array b modify ssdsmartpath=disable \[root@localhost \~\]# hpssacli ctrl slot=0 ld 2 modify caching=enable ### **f. 关闭写缓存并打开固态硬盘智能路径** \[root@localhost \~\]# hpssacli ctrl slot=0 ld 2 modify caching=disable \[root@localhost \~\]# hpssacli ctrl slot=0 array b modify ssdsmartpath=enable ## **工具4： arcconf** ### **a. 确认软件包是否安装** \[root@localhost \~\]# rpm -qa \| grep Arcconf Arcconf-2.04-22665.x86_64 备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下： #rpm -ivh Arcconf-xx-xx.x86_64.rpm ### **b. 确认硬盘控制器信息** 说明：控制器编号从1开始，所以示例中控制器ID为1: > \[root@localhost \~\]# /usr/Arcconf/arcconf list > > Controllers found: 1 > > ---------------------------------------------------------------------- > > Controller information > > ---------------------------------------------------------------------- > > Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN > > ---------------------------------------------------------------------- > > Controller 1: : Optimal, Slot 2147483647, RAID (Expose RAW), , FB488000, 560DA833FB488000 ### **c. 查看逻辑卷读/写缓存状态** 备注： 1. 请关注标红部分示例为：参数说明如下： Read-cache setting : Enabled(读缓存打开) Write-cache setting : Enabled(写缓存打开） 2. 示例命令行参数说明（/usr/Arcconf/arcconf getconfig 1 ld 1） getconfig: prints controller information(打印控制器相关信息） 第一个1：控制器编号 ld:logical drive(逻辑卷，即磁盘阵列） 第二个1：逻辑卷编号，从0开始，示例是第二个逻辑卷，所以为1. > \[root@localhost \~\]# /usr/Arcconf/arcconf getconfig 1 ld 1 > > Controllers found: 1 > > ---------------------------------------------------------------------- > > Logical device information > > ---------------------------------------------------------------------- > > Logical Device number 1 > > Logical Device name : data > > Block Size of member drives : 512 Bytes > > RAID level : 10 > > Unique Identifier : 6CDE0BB1 > > Status of Logical Device : Optimal > > Additional details : Quick initialized > > Size : 1014 MB > > Parity space : 1024 MB > > Stripe-unit size : 256 KB > > Interface Type : Serial ATA > > Device Type : HDD > > Read-cache setting : Enabled > > Read-cache status : On > > Write-cache setting : Enabled > > Write-cache status : On > > Partitioned : No > > Protected by Hot-Spare : No > > Bootable : No > > Failed stripes : No > > Power settings : Disabled > > -------------------------------------------------------- > > Logical Device segment information > > -------------------------------------------------------- > > Group 0, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:2) K4KZMMKL > > Group 0, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:3) K7G2L3SL > > Group 1, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:4) K4KYTSLL > > Group 1, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:5) K7G183TL ### **d. 读/写缓存设置语法** 语法如下： Usage: SETCACHE \ LOGICALDRIVE \ \ \[noprompt\] \[nologs

Usage: SETCACHE <Controller#> DEVICE <Channel# ID#> <physical mode> [nologs]

Usage: SETCACHE <Controller#> DEVICEALL <policy> [noprompt] [nologs]

Example: SETCACHE 1 LOGICALDRIVE 0 ron

Example: SETCACHE 1 DEVICE 0 0 wt

Example: SETCACHE 1 DEVICEALL drivespecific

===================================================================================

Changes a device's cache mode.

LOGICALDRIVE parameters

LogicalDrive# : Number of the logical device whose cache will be altered

Logical Modes : ron - read cache enabled

roff - read cache disabled

wt - write through

wb - write back

wbb - write back with battery/ZMM

con - cache enabled

coff - cache disabled

DEVICE parameters

Channel# ID# : Channel and ID of the device whose cache will be altered

Physical Modes : wt - write through

wb - write back

DEVICEALL parameters

Policy : disable - write back for all physical drives

enable - write through for all physical drives

drivespecific - user can set for an individual physical drive

部分参数说明：

rron: read cache enabled(打开读缓存)

roff: read cache disabled(关闭读缓存)

wt: write through(只写模式，即关闭写缓存）

wb:write back(回写模式，即打开写缓存）

con: cache enabled(打开逻辑卷缓存（打开读/写缓存前提是逻辑卷缓存已打开，在逻辑卷关闭的情况下，无法打开读/写缓存））

coff: cache disabled(关闭逻辑卷缓存）

noprompt：no prompt(没有提示，按照默认y执行，部分命令需要和用户交互，使用noprompt参数可按照默认yes答复执行）

e. 读缓存打开/关闭示例

root@localhost \~\]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 roff Controllers found: 1 Command completed successfully. \[root@localhost \~\]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 ron Controllers found: 1 Command completed successfully. ### **f. 写缓存打开/关闭示例** \[root@localhost \~\]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wt Controllers found: 1 Command completed successfully. You have new mail in /var/spool/mail/root \[root@localhost \~\]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wb Controllers found: 1 WARNING: Power failure without battery/ZMM support will lead to data loss. Do you wish to continue? Press y, then ENTER to continue or press ENTER to abort: y Command completed successfully. \[root@localhost \~\]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wbb Controllers found: 1 Command completed successfully