Linux: Megaraid SAS

From ReceptiveIT
Jump to: navigation, search

Introduction

These instructions relate to the LSI Megaraid SAS based raid controllers, and my adventures with them using Ubuntu Linux 64-bit.

The driver for these cards is quite stable, and is in the mainstream kernel (megaraid_sas). There are two management tools available for these cards, the command line MegaCli and the graphical Raid Web Console 2.

Monitoring

Monitoring your raid arrays can be done quite easily by using the command line tool, and if you combine this with a good monitoring solutions, like Zabbix, it can give you really good peace of mind.

Common Tasks

How many adapters do we have?

Before we go any further, we should make sure that we have a supported, and recognised card.

> ./MegaCli64 -adpCount

Controller Count: 1.
Exit Code: 0x01

There, we have 1 controller in our machine

Firmware

From time to time, you need to upgrade the firmware on your adapter. It is quite easy to do this with the command line tool.

Finding Current Firmware Version

> ./MegaCli64 -AdpAllInfo -aAll -NoLog
                                     
Adapter #0

==============================================================================
                   Versions
               ================
Product Name    : Intel(R) RAID Controller SRCSASPH16I
Serial No       : P268152008
FW Package Build: 7.0.1-0045
                   Mfg. Data
               ================
Mfg. Date       : 06/15/08
Rework Date     : 00/00/00
Revision No     : (
Battery FRU     : N/A
               Image Versions in Flash:
               ================
Boot Block Version : R.2.3.12
BIOS Version       : MT33
MPT Version        : MPTFW-01.18.78.00-IT
FW Version         : 1.12.120-0394
WebBIOS Version    : 1.1-33d-e_11-Rel
Ctrl-R Version     : 1.04-019A
Preboot CLI Version: 01.00-011:#%00001

Flashing Firmware

> ./MegaCli64 -adpfwflash -f 68_fw718.rom -aAll

Adapter 0: Intel(R) RAID Controller SRCSASPH16I
Vendor ID: 0x1000, Device ID: 0x0411

Package version on the controller: 7.0.1-0045 
Package version of the image file: 7.0.1-0074 
Download Completed.     
Flashing image to adapter...
Adapter 0: Flash Completed.

Exit Code: 0x00

This process puts the new flash into a queue, and it will perform an upgrade at the next boot.

Do we have any degraded drives?

> ./MegaCli64 -AdpAllInfo -aAll -NoLog

                Device Present
               ================
Virtual Drives    : 1 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 9 
  Disks           : 7 
  Critical Disks  : 0 
  Failed Disks    : 0

Command Line Help

     Intel(R) RAID Command Line Utilities Version 2  
                Ver 4.00.14 July 01, 2009

   (c)Copyright 2009, LSI Corporation, All Rights Reserved.
MegaCli -v 
MegaCli -help|-h|? 
MegaCli -adpCount 
MegaCli -AdpSetProp {CacheFlushInterval -val}|{ RebuildRate -val} 
   |{PatrolReadRate -val}|{BgiRate -val}|{CCRate -val} 
   |{ReconRate -val}|{SpinupDriveCount -val}|{SpinupDelay -val} 
   |{CoercionMode -val}|{ClusterEnable -val}|{PredFailPollInterval -val} 
   |{BatWarnDsbl -val} |{EccBucketSize -val} | {EccBucketLeakRate -val} 
   |{AbortCCOnError -val} | AlarmEnbl | AlarmDsbl | AlarmSilence 
   |{SMARTCpyBkEnbl -val} | NCQEnbl | NCQDsbl -aN|-a0,1,2|-aALL 
   |{SSDSMARTCpyBkEnbl -val} -aN|-a0,1,2|-aALL 
   |{MaintainPdFailHistoryEnbl -val} -aN|-a0,1,2|-aALL 
   |{EnblSpinDownUnConfigDrvs -val} -aN|-a0,1,2|-aALL 
   |{EnblSSDPatrolRead -val} -aN|-a0,1,2|-aALL 
MegaCli -AdpSetProp -AutoDetectBackPlaneDsbl -val -aN|-a0,1,2|-aALL 
      val - 0=Enable Auto Detect of SGPIO and i2c SEP. 
            1=Disable Auto Detect of SGPIO. 
            2=Disable Auto Detect of i2c SEP. 
            3=Disable Auto Detect of SGPIO and i2c SEP. 
MegaCli -AdpSetProp -CopyBackDsbl -val -aN|-a0,1,2|-aALL 
      val - 0=Enable Copyback. 
            1=Disable Copyback. 
MegaCli -AdpSetProp -LoadBalanceMode -val -aN|-a0,1,2|-aALL 
      val - 0=Auto Load balance mode. 
            1=Disable Load balance mode. 
MegaCli -AdpSetProp -UseFDEOnlyEncrypt -val -aN|-a0,1,2|-aALL 
      val - 0=FDE and controller encryption (if HW supports) is allowed. 
            1=Only support FDE encryption, disallow controller encryption. 
MegaCli -AdpGetProp CacheFlushInterval | RebuildRate | PatrolReadRate | BgiRate 
   | CCRate | ReconRate | SpinupDriveCount | SpinupDelay | CoercionMode
   | PredFailPollInterval | EccBucketSize | EccBucketLeakRate | EccBucketCount
   | ClusterEnable | BatWarnDsbl | AbortCCOnError | AlarmDsply 
   | AutoDetectBackPlaneDsbl| CopyBackDsbl | LoadBalanceMode | SMARTCpyBkEnbl | SSDSMARTCpyBkEnbl | MaintainPdFailHistoryEnbl | EnblSpinDownUnConfigDrvs 
   | EnblSSDPatrolRead 
   | NCQDsply -aN|-a0,1,2|-aALL 
   | UseFDEOnlyEncrypt | WBSupport  -aN|-a0,1,2|-aALL 
MegaCli -AdpAllInfo -aN|-a0,1,2|-aALL  
MegaCli -AdpGetTime -aN|-a0,1,2|-aALL  
MegaCli -AdpSetTime yyyymmdd hh:mm:ss -aN   
MegaCli -AdpSetVerify -f fileName -aN|-a0,1,2|-aALL  
MegaCli -AdpBIOS -Enbl |-Dsbl | -SOE | -BE | -Dsply -aN|-a0,1,2|-aALL 
MegaCli -AdpBootDrive {-Set -Lx}|-Get -aN|-a0,1,2|-aALL 
MegaCli -AdpAutoRbld -Enbl|-Dsbl|-Dsply -aN|-a0,1,2|-aALL
MegaCli -AdpCacheFlush -aN|-a0,1,2|-aALL
MegaCli -AdpPR -Dsbl|EnblAuto|EnblMan|Start|Stop|Info| SSDPatrolReadEnbl | SSDPatrolReadDsbl
        |{SetDelay Val}|{-SetStartTime yyyymmdd hh}|{maxConcurrentPD Val} -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -Dsbl|-Info|{-ModeConc | -ModeSeq [-ExcludeLD -LN|-L0,1,2]
  [-SetStartTime yyyymmdd hh ] [-SetDelay val ] } -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -SetStartTime yyyymmdd hh -aN|-a0,1,2|-aALL
MegaCli -AdpCcSched -SetDelay val  -aN|-a0,1,2|-aALL
MegaCli -FwTermLog -BBUoff|BBUoffTemp|BBUon|BBUGet|Dsply|Clear -aN|-a0,1,2|-aALL
MegaCli -AdpAlILog -aN|-a0,1,2|-aALL 
MegaCli -AdpShutDown -aN|-a0,1,2|-aALL
MegaCli -PDList -aN|-a0,1,2|-aALL 
MegaCli -PDGetNum -aN|-a0,1,2|-aALL 
MegaCli -pdInfo -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL  
MegaCli -PDOnline  -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PDOffline -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PDMakeGood -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PDHSP {-Set [-Dedicated [-ArrayN|-Array0,1,2...]] [-EnclAffinity] [-nonRevertible]} 
    |-Rmv -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PDRbld -Start|-Stop|-ShowProg |-ProgDsply 
       -PhysDrv [E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL  
MegaCli -PDClear -Start|-Stop|-ShowProg |-ProgDsply 
       -PhysDrv [E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL  
MegaCli -PdLocate {[-start] | -stop} -physdrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PdMarkMissing -physdrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL 
MegaCli -PdGetMissing -aN|-a0,1,2|-aALL 
MegaCli -PdReplaceMissing -physdrv[E0:S0] -arrayA, -rowB -aN 
MegaCli -PdPrpRmv [-UnDo] -physdrv[E0:S0] -aN|-a0,1,2|-aALL  
MegaCli -EncInfo -aN|-a0,1,2|-aALL 
MegaCli -PhyInfo -phyM -aN|-a0,1,2|-aALL  
MegaCli -PdFwDownload -PhysDrv[0:1,1:2,...] -f <filename> -aN|-a0,1,2|-aALL 
MegaCli -LDInfo -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL 
MegaCli -LDSetProp  {-Name LdNamestring} | -RW|RO|Blocked | WT|WB [-Immediate] |RA|NORA|ADRA 
       | Cached|Direct | -EnDskCache|DisDskCache | CachedBadBBU|NoCachedBadBBU
       -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL 
MegaCli -LDGetProp  -Cache | -Access | -Name | -DskCache -Lx|-L0,1,2|-LALL  
       -aN|-a0,1,2|-aALL 
MegaCli -LDInit {-Start [-full]}|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL 
MegaCli -LDCC {-Start [-force]}|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL 
MegaCli -LDBI -Enbl|-Dsbl|-getSetting|-Abort|-ShowProg|-ProgDsply -Lx|-L0,1,2|-LALL -aN|-a0,1,2|-aALL  
MegaCli -LDRecon {-Start -rX [{-Add | -Rmv} -Physdrv[E0:S0,...]]}|-ShowProg|-ProgDsply 
       -Lx -aN 
MegaCli -LdPdInfo -aN|-a0,1,2|-aALL 
MegaCli -LDGetNum -aN|-a0,1,2|-aALL 
MegaCli -CfgLdAdd -rX[E0:S0,E1:S1,...] [WT|WB] [NORA|RA|ADRA] [Direct|Cached]
       [CachedBadBBU|NoCachedBadBBU] [-szXXX [-szYYY ...]]
       [-strpszM] [-Hsp[E0:S0,...]] [-AfterLdX] [-Force]|[FDE|CtrlBased] -aN 
MegaCli -CfgEachDskRaid0 [WT|WB] [NORA|RA|ADRA] [Direct|Cached] 
       [CachedBadBBU|NoCachedBadBBU] [-strpszM]|[FDE|CtrlBased] -aN|-a0,1,2|-aALL
MegaCli -CfgClr -aN|-a0,1,2|-aALL 
MegaCli -CfgDsply -aN|-a0,1,2|-aALL 
MegaCli -CfgLdDel -LX|-L0,2,5...|-LALL -aN|-a0,1,2|-aALL 
MegaCli -CfgFreeSpaceinfo -aN|-a0,1,2|-aALL 
MegaCli -CfgSpanAdd -r10 -Array0[E0:S0,E1:S1] -Array1[E0:S0,E1:S1] [-ArrayX[E0:S0,E1:S1] ...] 
       [WT|WB] [NORA|RA|ADRA] [Direct|Cached] [CachedBadBBU|NoCachedBadBBU]
       [-szXXX[-szYYY ...]][-strpszM][-AfterLdX]|[FDE|CtrlBased] -aN
MegaCli -CfgSpanAdd -r50 -Array0[E0:S0,E1:S1,E2:S2,...] -Array1[E0:S0,E1:S1,E2:S2,...] 
       [-ArrayX[E0:S0,E1:S1,E2:S2,...] ...] [WT|WB] [NORA|RA|ADRA] [Direct|Cached] 
       [CachedBadBBU|NoCachedBadBBU][-szXXX[-szYYY ...]][-strpszM][-AfterLdX] | -aN
       [FDE|CtrlBased] -aN 
MegaCli -CfgAllFreeDrv -rX [-SATAOnly] [-SpanCount XXX] [WT|WB] [NORA|RA|ADRA] 
       [Direct|Cached] [CachedBadBBU|NoCachedBadBBU] [-strpszM]
       [-HspCount XX [-HspType -Dedicated|-EnclAffinity|-nonRevertible]]| -aN 
       [FDE|CtrlBased] -aN 
MegaCli -CfgSave -f filename -aN   
MegaCli -CfgRestore -f filename -aN   
MegaCli -CfgForeign -Scan | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL    
MegaCli -CfgForeign -Dsply [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL    
MegaCli -CfgForeign -Preview [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL    
MegaCli -CfgForeign -Import [x] | [-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL    
MegaCli -CfgForeign -Clear [x]|[-SecurityKey sssssssssss] -aN|-a0,1,2|-aALL    
       x - index of foreign configurations. Optional. All by default. 
MegaCli -AdpEventLog -GetEventLogInfo -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -GetEvents         -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -GetSinceShutdown  -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -GetSinceReboot    -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -IncludeDeleted    -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -GetLatest n -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -GetCCIncon -f <fileName> -LX|-L0,2,5...|-LALL -aN|-a0,1,2|-aALL 
MegaCli -AdpEventLog -Clear -aN|-a0,1,2|-aALL 
MegaCli -AdpBbuCmd -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -GetBbuStatus -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -GetBbuCapacityInfo -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -GetBbuDesignInfo -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -GetBbuProperties -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -BbuLearn -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -BbuMfgSleep -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -BbuMfgSeal -aN|-a0,1,2|-aALL  
MegaCli -AdpBbuCmd -SetBbuProperties -f <fileName> -aN|-a0,1,2|-aALL 
MegaCli -AdpFacDefSet -aN 
MegaCli -AdpFwFlash -f filename [-NoSigChk] [-NoVerChk] -aN|-a0,1,2|-aALL  
MegaCli -AdpGetConnectorMode -ConnectorN|-Connector0,1|-ConnectorAll -aN|-a0,1,2|-aALL  
MegaCli -AdpSetConnectorMode -Internal|-External|-Auto -ConnectorN|-Connector0,1|-ConnectorAll -aN|-a0,1,2|-aALL  
MegaCli -PhyErrorCounters -aN|-a0,1,2|-aALL  
MegaCli -DirectPdMapping -Enbl|-Dsbl|-Dsply -aN|-a0,1,2|-aALL  
MegaCli -PDCpyBk -Start -PhysDrv[E0:S0,E1:S1] -aN|-a0,1,2|-aALL 
MegaCli -PDCpyBk -Stop|-ShowProg|-ProgDsply -PhysDrv[E0:S0] -aN|-a0,1,2|-aALL 
MegaCli -PDReprovision -PhysDrv[E0:S0,E1:S1,...] | [-Force] -aN|-a0,1,2|-aALL 
MegaCli -DestroySecurityKey | [-Force] -aN 
MegaCli -CreateSecurityKey -SecurityKey sssssssssss | [-Passphrase sssssssssss] |[-KeyID kkkkkkkkkkk] -aN 
MegaCli -ChangeSecurityKey -OldSecurityKey sssssssssss | -SecurityKey sssssssssss| 
         [-Passphrase sssssssssss] | [-KeyID kkkkkkkkkkk] -aN
MegaCli -GetKeyID [-PhysDrv[E0:S0]] -aN 
MegaCli -SetKeyID -KeyID kkkkkkkkkkk -aN 
MegaCli -VerifySecurityKey -SecurityKey sssssssssss -aN 
MegaCli -GetPreservedCacheList -aN|-a0,1,2|-aALL 
MegaCli -DiscardPreservedCache -Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL 
      sssssssssss  - It must be between eight and thirty-two 
                     characters and contain at least one number, 
                     one lowercase letter, one uppercase 
                     letter and one non-alphanumeric character.
      kkkkkkkkkkk -  Must be less than 256 characters. 
   Note: The directly connected drives can be specified as [:S] 
   Wildcard '?' can be used to specify the enclosure ID for the drive in the 
     only enclosure without direct connected device or the direct connected 
     drives with no enclosure in the system.
   The following options may be given at the end of any command above: 
   [-Silent] [-AppLogFile filename] [-NoLog] [-page[N]] 
                 N - Number of lines per page. 

Exit Code: 0x00

Other References

http://tools.rapidsoft.de/perc/perc-cheat-sheet.html