24

When I'm inside of Linux, I can get the following information from lsblk (irrelevant drives removed from output):

NAME  MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda     8:0    0   298G  0 disk 
sdb     8:16   0   2.7T  0 disk

When I manually pull the drives out of the server, I can tell I'm physically using the following drives:

0  Seagate 320GB
1  Seagate 320GB
2  Hitachi 1TB
3  Hitachi 1TB
4  Hitachi 1TB
5  Hitachi 1TB
6  [empty]
7  [empty]

Because there is more physical storage in the server than available space in Linux, this means I'm obviously using some form of RAID system. With a bit of math, I can often figure out what type of RAID system is being used.

Is there a way for me to detect if I'm using hardware RAID from inside of Linux, and figure out all the information about it (such as type of RAID, available drives) without turning off the server, physically pulling the drives out, and reading their labels?

Can this information be gathered from inside of Linux, or is the point of hardware RAID to make the underlying system "invisible" to the operating system?

sa289
  • 1,308
  • 2
  • 17
  • 42
IQAndreas
  • 1,480
  • 2
  • 19
  • 39
  • It depends entirely on the RAID controller you are using and what support it has under linux. If you provide information about your controller you might find help here. You could also do a web search for "model# linux commands" if you don't get the information you need here. – Gene Sep 02 '15 at 20:24
  • @Gene That's unfortunate. Again, I was hoping there was a solution which does not require looking up the specific model number, but can be used regardless of what hardware you are using. – IQAndreas Sep 02 '15 at 20:29
  • Different hardware do RAID differently and have different drivers. In a lot of cases you don't have to power your system down, but at the very least you need to know what model controller or chipset you have so you can find commands to query the controller from the operating system. – Gene Sep 02 '15 at 20:42
  • 1
    You typically can find the controller or chipset information via dmesg, dmidecode, lspci, and other similar utilities. If it's built-in RAID on the motherboard knowing the make and model of the motherboard will get you 99% of the way there. – Gene Sep 02 '15 at 20:47
  • 1
    `adds an answer explaining how to figure out which RAID controller is being used from inside of Linux` - @IQAndreas but that isn't in the question. Answers on serverfault should be answer the question being asked. If you Have a different question you should edit your question. That question could almost certainly be answered yourself by a few minutes on Google. Anyway the answer to that is `lshw`. – Zoredache Sep 02 '15 at 21:34

6 Answers6

28

How to get the RAID information is going to depend entirely on the RAID controller you are using. Often, manufacturers will have tools that can be downloaded from their website which can be used to query the RAID controller and get this information.

In order to find which RAID controller you are using, try one of the following commands:

lspci

# lspci -knn | grep 'RAID bus controller'
08:00.0 RAID bus controller [0104]: 3ware Inc 9690SA SAS/SATA-II RAID PCIe [13c1:1005] (rev 01)

Here, the information we are looking for is "3ware Inc 9690SA SAS/SATA-II RAID PCIe".

lsscsi

The command is not available on Debian and Ubuntu, but a quick sudo apt-get install lsscsi will fetch it from the repos. Note, if you are not using a RAID controller, the manufacturer and model number of your harddrive will show up here instead.

# lsscsi
[2:0:0:0]    disk    AMCC     9690SA-8I  DISK  4.08  /dev/sda 
[2:0:1:0]    disk    AMCC     9690SA-8I  DISK  4.08  /dev/sdb

Here we see the manufacturer is "AMCC" and the model number of the RAID card is "9690SA-8I". A quick Google search shows that this card is also known as "AMCC 3Ware 9690SA-8I".

lshw

A third method (which gives quite a bit of output data) is to use the lshw command. Run lshw -class disk as root to only display the details about harddrives (which includes RAID information).

Finding the RAID controller tools

Now that we have the manufacturer and model number, it should be possible to find the tools on their website, or at least be able to Google details on how to find and use the tools for that specific controller.

If the manufacturer shows up in this list, see these answers for more details on how to get the RAID information for your card:

Community
  • 1
IQAndreas
  • 1,480
  • 2
  • 19
  • 39
5

Run something like lspci -knn | grep 'RAID bus controller'.

Using that output, Google (for example) for LSI Logic / Symbios Logic MegaRAID SAS 2208.

Find it uses the storcli utility to interrogate the RAID controller.

Download it and install it.

storcli64 show gives you the model of controller specifically, and the controller index, number of drive groups, and virtual drives.

storcli64 /c0/d0 show shows you first controller, first drive group. Tells you raid levels, including nesting.

storcli64 /c0/eall/sall show all shows you all the information on all the disks.

Further reference data for those commands can be found here:

http://mycusthelp.info/LSI/_cs/AnswerPreview.aspx?sSessionID=&inc=8275

Should be all you have to do. Just use the normal hardware introspection in Linux, then Google, then download and install the utilities that go with it.

@Gene's suggestions and comments are absolutely spot-on.

This answer is entirely specific to the output you get from the lspci command, and the ability for your Google search to identify the manufacturer and to grab the correct command line tool.

But it does show that you can get all of this from a server, on a command line, without halting the machine, opening the case, and pulling drives, which I hope is helpful.

Kassandry
  • 639
  • 1
  • 6
  • 15
  • Since the answer to my question is going to be hardware specific, what I did was add [a community wiki answer on how to find out what RAID controller you are using](http://serverfault.com/a/718769/97027). This answer then links to answers (including yours) that provide details for specific controllers or manufacturers. – IQAndreas Sep 03 '15 at 09:30
  • If you want to update the format of your answer to reflect this, it would be nice, but not strictly necessary. – IQAndreas Sep 03 '15 at 09:31
2

This isn't a one-size-fits-all answer and doesn't give you all the information you need, but on one Adaptec hardware RAID controller we've used it. It gave some access to the drives themselves via special devices /dev/sg1, /dev/sg2, etc.

We could run smartctl -a /dev/sg1 to get a lot of info on that physical drive including manufacturer, model number, interface, serial number, size, and other data.

As far as figuring out which controller is being used, I agree with Gene's comment about dmidecode, dmesg, and lspci - those would be my ones to try first as well.

Gryu
  • 479
  • 1
  • 6
  • 14
sa289
  • 1,308
  • 2
  • 17
  • 42
2

For creating a storage profile, I'd do the following before running lspci or downloading anything.

cat /etc/fstab         # What partitions get mounted at boot?

sudo pvs               # Are you running LVM? List the physical volumes

cat /proc/mdstat       # List software RAID devices & status

for D in /dev/sd?; do parted $D print; done
                       # If hardware RAID is being used, the "Model" field will list
                       # the Mfr. name, otherwise the hard drive Mfr.
rubo77
  • 2,282
  • 3
  • 32
  • 63
nortally
  • 381
  • 2
  • 11
1

In addition to the lspci and lshw methods described in the other answer, you can parse dmesg and use smartctl to obtain informations about the physical drives. For example, give a look at this server with a PERC H710p controller:

[root@localhost ~]# dmesg | grep -i scsi
[    0.946723] SCSI subsystem initialized
[    1.279256] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
[    2.853831] scsi host0: Avago SAS based MegaRAID driver
[    2.858058] scsi 0:2:0:0: Direct-Access     DELL     PERC H710P       3.13 PQ: 0 ANSI: 5
[    2.873982] scsi 0:2:1:0: Direct-Access     DELL     PERC H710P       3.13 PQ: 0 ANSI: 5
[    3.011376] sd 0:2:0:0: [sda] Attached SCSI disk
[    3.012031] sd 0:2:1:0: [sdb] Attached SCSI disk
[    7.915918] sd 0:2:0:0: Attached scsi generic sg0 type 0
[    7.916305] sd 0:2:1:0: Attached scsi generic sg1 type 0

You can see two arrays defined: sda and sdb. Running smartctl against such virtual devices retuns something as the following:

[root@localhost ~]# smartctl --all /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               DELL
Product:              PERC H710P
Revision:             3.13
Compliance:           SPC-3
User Capacity:        9,930,501,324,800 bytes [9.93 TB]
Logical block size:   512 bytes
Logical Unit id:      REDACTED
Serial number:        REDACTED
Device type:          disk
Local Time is:        Fri May 29 14:46:48 2020 CEST
SMART support is:     Unavailable - device lacks SMART capability.

To check the physical disk health values you can use smartctl --all -d <driver>,<disknum> <sddevice> syntax. For example, to check the first disk of the second array:

[root@localhost ~]# smartctl --all -d megaraid,0 /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     REDACTED
Serial Number:    REDACTED
LU WWN Device Id: REDACTED
...                      
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   100   253   021    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       3
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   097   097   000    Old_age   Always       -       2251
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       3
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       1
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       3
194 Temperature_Celsius     0x0022   123   117   000    Old_age   Always       -       20
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

Finally, to obtain array layout information (ie: RAID type, spare disks, etc) you had to use specific tools. megacli is a generic tool for LSI-based cards, but vendor-customized tools exist also (ie: omreport storage vdisk). For example, on a DELL server:

[root@localhost ~]# omreport storage vdisk
List of Virtual Disks in the System

Controller PERC H710P Mini (Embedded)
ID                                : 0
Status                            : Ok
Name                              : REDACTED
State                             : Ready
Hot Spare Policy violated         : Not Assigned
Encrypted                         : No
Layout                            : RAID-6
Size                              : 64.00 GB (68719411200 bytes)
T10 Protection Information Status : No
Associated Fluid Cache State      : Not Applicable
Device Name                       : /dev/sda
Bus Protocol                      : SATA
Media                             : HDD
Read Policy                       : Adaptive Read Ahead
Write Policy                      : Write Back
Cache Policy                      : Not Applicable
Stripe Element Size               : 64 KB
Disk Cache Policy                 : Enabled
shodanshok
  • 44,038
  • 6
  • 98
  • 162
0

3ware

This has been tested on the following RAID controllers: 9690SA-8I

Please let me know in the comments if it worked for your specific controller, or if the instructions need updating for your model.

Downloading

First, you need to download the tool tw_cli. This can be found on the Avago Technologies website*:

* Don't worry, it's not a third party website; 3ware was purchased by AMCC, and then later sold to the LSI Corporation, which later became part of Avago [source]. To back this up, note that 3ware.com redirects to the relevant page on avagotech.com.

Type tw_cli into the search bar, then navigate around until you find a download named CLI Linux - from the 10.2.2.1/9.5.5.1 code sets (the latest version number may be different than what is listed here)

Extract the zip file, and in either the x86 or x86_64 you will find the tw_cli binary that you need. Save this binary to your server or computer.

Running tw_cli

NOTE: (1) The command-line utility may need to run as root. (2) Run it as you would any other binary by either adding it to the path, or by running it directly from the current directory with ./tw_cli.

First, run tw_cli show to list which RAID controllers are available on your system:

# tw_cli show
Ctl   Model        (V)Ports  Drives   Units   NotOpt  RRate   VRate  BBU
------------------------------------------------------------------------
c2    9690SA-8I    6         6        2       0       1       1      Charging

Here we can see there is one controller attached, with the id of c2 (remember this id!), which has 6 physical drives attached. To get more details about which drives are attached to the unit, run tw_cli /c2 show (replacing /c2 with the ID of your controller):

# tw_cli /c2 show
Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-1    OK             -       -       -       298.013   Ri     ON     
u1    RAID-5    OK             -       -       64K     2793.94   Ri     ON     

VPort Status         Unit Size      Type  Phy Encl-Slot    Model
------------------------------------------------------------------------------
p0    OK             u0   298.09 GB SATA  0   -            ST3320613AS         
p1    OK             u0   298.09 GB SATA  1   -            ST3320613AS         
p2    OK             u1   931.51 GB SATA  2   -            Hitachi HDS721010CL 
p3    OK             u1   931.51 GB SATA  3   -            Hitachi HDS721010CL 
p4    OK             u1   931.51 GB SATA  4   -            Hitachi HDS721010CL 
p5    OK             u1   931.51 GB SATA  5   -            Hitachi HDS721010CL

The first group shows the RAID setups being used. The second group shows which drives are currently physically attached to the server.

In this particular case, I have two RAID configurations:

  • u0 -> Drives p0 and p1 are both 300 GB drives by the same manufacturer, set up to use RAID-1 (exact mirrors of each other)
  • u1 -> I have four 1TB Hitachi drives (p2 through p5) set up using RAID-5 ("one of the drives" is used to make sure data is preserved if any one of the other drives fails). This means (as we can see from the top), u1 has 3TB (or, rather, 2793.94 GB) of usable data.

Even more information about the status and health of the drives can be found by running tw_cli /c2/u0 show or tw_cli /c2/p0 show. More details and commands can be found in the man pages.

IQAndreas
  • 1,480
  • 2
  • 19
  • 39