Add cachecade SSD to an existing virtual drive in AVAGO MegaRAID with storcli64

Increase the IO performance of big storage consisting of hard drives with the CacheCade feature with MegaRAID hardware controllers. Here are the commands to ensure adding an SSD to cache storage of hard drives and the whole output:

storcli64 /c0 add VD cachecade r0 drives=252:0 WB
storcli64 /c0/v0 set ssdcaching=on

Summary and explanation

  • The controller used is AVAGO MegaRAID SAS9361-4i
  • Three 12T hard drives and 1 SSD 1T.
  • RAID5 of 3 hard drive. The name of the virtual drive is v0.
  • The CacheCade is in RAID0. The name of the virtual drive is v1.
  • The CacheCade is in WriteBack mode. The WriteBack mode is dangerous when the CacheCade device is RAID0, because there is no redundancy, but it’s OK if the purpose of the machine is just a proxy cache. WriteBack mode leads to much more performance to gain from the cache device over slow device sotrage, but should be carefully planned when using RAID0 cache (or single) device.
  • Using storcli – the cli command line tool to manage a (LSI) MegaRAID.
  • Add CacheCade RAID0 device by the first command. It will create a virtual drive RAID0 with the name v1.
  • Ensure the caching is enabled for the virtual device it should be by explicitly setting it with the second command. Adding only a CacheCade device may not add the cache virtual device to the existing virtual devices!

Output of a real world example


The status of the virtual device and the disks before the operation:

[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None

Product Name = AVAGO MegaRAID SAS 9361-4i
Serial Number = SK11111111
SAS Address =  500605b00ec0fdb0
PCI Address = 00:01:00:00
System Time = 09/30/2020 09:13:41
Mfg. Date = 08/29/18
Controller Time = 09/30/2020 09:13:07
FW Package Build = 24.21.0-0067
BIOS Version = 6.36.00.3_4.19.08.00_0x06180203
FW Version = 4.680.00-8392
Driver Name = megaraid_sas
Driver Version = 07.710.50.00-rh1
Current Personality = RAID-Mode 
Vendor Id = 0x1000
Device Id = 0x5D
SubVendor Id = 0x1000
SubDevice Id = 0x9363
Host Interface = PCI-E
Device Interface = SAS-12G
Bus Number = 1
Device Number = 0
Function Number = 0
Drive Groups = 1

TOPOLOGY :
========

----------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace TR 
----------------------------------------------------------------------------
 0 -   -   -        -   RAID5 Optl  N  21.827 TB dflt N  N   dflt N      N  
 0 0   -   -        -   RAID5 Optl  N  21.827 TB dflt N  N   dflt N      N  
 0 0   0   252:1    5   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
 0 0   1   252:2    6   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
 0 0   2   252:3    7   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
----------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present
TR=Transport Ready

Virtual Drives = 1

VD LIST :
=======

------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC      Size Name     
------------------------------------------------------------------
0/0   RAID5 Optl  RW     Yes     RAWBD -   ON  21.827 TB storageA 
------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 4

PD LIST :
=======

---------------------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model                      Sp Type 
---------------------------------------------------------------------------------------
252:0     4 UGood -  893.75 GB SATA SSD N   N  512B SAMSUNG MZ7KM960HMJP-00005 U  -    
252:1     5 Onln  0  10.913 TB SATA HDD N   N  512B ST12000NM0007-2A1101       U  -    
252:2     6 Onln  0  10.913 TB SATA HDD N   N  512B ST12000NM0007-2A1101       U  -    
252:3     7 Onln  0  10.913 TB SATA HDD N   N  512B ST12000NM001G-2MV103       U  -    
---------------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down/PowerSave|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded


Cachevault_Info :
===============

------------------------------------
Model  State   Temp Mode MfgDate    
------------------------------------
CVPM02 Optimal 26C  -    2018/05/19 
------------------------------------



[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0/v0 show
CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC      Size Name     
------------------------------------------------------------------
0/0   RAID5 Optl  RW     Yes     RAWBD -   ON  21.827 TB storageA 
------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

The SSD is in UGood mode, which is ready to be used in a virtual drive.

Add CacheCade device and show the slow device. In the example, the slow device did not enable the SSD caching device after it was added!

[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0 add VD cachecade r0 drives=252:0 WB
CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = Add VD Succeeded


[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0/v0 show
CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC      Size Name     
------------------------------------------------------------------
0/0   RAID5 Optl  RW     Yes     RAWBD -   ON  21.827 TB storageA 
------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency



[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0/v0 set ssdcaching=on
CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None

Detailed Status :
===============

-----------------------------------------
VD Property   Value Status  ErrCd ErrMsg 
-----------------------------------------
 0 SSDCaching On    Success     0 -      
-----------------------------------------

After setting the SSDCaching property the slow virtual drive’s “Cac” column will output the mode it uses the Cache drive.

The show command for the slow virtual drive and all virtual drives. The show commands for the all virtual drives (only /c0) will show information for the CacheCade drive, too.

[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0/v0 show
CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC      Size Name     
------------------------------------------------------------------
0/0   RAID5 Optl  RW     Yes     RAWBD RW  ON  21.827 TB storageA 
------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency



[root@srv ~]# /opt/MegaRAID/storcli/storcli64 /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

CLI Version = 007.0709.0000.0000 Aug 14, 2018
Operating system = Linux 3.10.0-1127.19.1.el7.x86_64
Controller = 0
Status = Success
Description = None

Product Name = AVAGO MegaRAID SAS 9361-4i
Serial Number = SK11111111
SAS Address =  500605b00ec0fdb0
PCI Address = 00:01:00:00
System Time = 09/30/2020 09:14:29
Mfg. Date = 08/29/18
Controller Time = 09/30/2020 09:13:56
FW Package Build = 24.21.0-0067
BIOS Version = 6.36.00.3_4.19.08.00_0x06180203
FW Version = 4.680.00-8392
Driver Name = megaraid_sas
Driver Version = 07.710.50.00-rh1
Current Personality = RAID-Mode 
Vendor Id = 0x1000
Device Id = 0x5D
SubVendor Id = 0x1000
SubDevice Id = 0x9363
Host Interface = PCI-E
Device Interface = SAS-12G
Bus Number = 1
Device Number = 0
Function Number = 0
Drive Groups = 2

TOPOLOGY :
========

----------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace TR 
----------------------------------------------------------------------------
 0 -   -   -        -   RAID5 Optl  N  21.827 TB dflt N  N   dflt N      N  
 0 0   -   -        -   RAID5 Optl  N  21.827 TB dflt N  N   dflt N      N  
 0 0   0   252:1    5   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
 0 0   1   252:2    6   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
 0 0   2   252:3    7   DRIVE Onln  Y  10.913 TB dflt N  N   dflt -      N  
 1 -   -   -        -   Cac0  Optl  N  893.75 GB dflt N  N   none N      N  
 1 0   -   -        -   RAID0 Optl  N  893.75 GB dflt N  N   none N      N  
 1 0   0   252:0    4   DRIVE Onln  N  893.75 GB dflt N  N   none -      N  
----------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present
TR=Transport Ready

Virtual Drives = 2

VD LIST :
=======

------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC      Size Name     
------------------------------------------------------------------
0/0   RAID5 Optl  RW     Yes     RAWBD RW  ON  21.827 TB storageA 
1/1   Cac0  Optl  RW     Yes     RWBD  -   ON  893.75 GB          
------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|HD=Hidden|TRANS=TransportReady|B=Blocked|
Consist=Consistent|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 4

PD LIST :
=======

---------------------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model                      Sp Type 
---------------------------------------------------------------------------------------
252:0     4 Onln   1 893.75 GB SATA SSD N   N  512B SAMSUNG MZ7KM960HMJP-00005 U  -    
252:1     5 Onln   0 10.913 TB SATA HDD N   N  512B ST12000NM0007-2A1101       U  -    
252:2     6 Onln   0 10.913 TB SATA HDD N   N  512B ST12000NM0007-2A1101       U  -    
252:3     7 Onln   0 10.913 TB SATA HDD N   N  512B ST12000NM001G-2MV103       U  -    
---------------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down/PowerSave|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded


Cachevault_Info :
===============

------------------------------------
Model  State   Temp Mode MfgDate    
------------------------------------
CVPM02 Optimal 26C  -    2018/05/19 
------------------------------------

Leave a Reply

Your email address will not be published.