LSI (MPT Internal) RAID Engines

Top  Previous  Next

Benefits of Directly Querying LSI RAID Controllers

Use the software to assess RAID health remotely, and not be dependent on a BIOS-based program, or a utility that only runs on the host console. Since the output can easily be parsed and scripted, the administrator can implement a phone-home system based on specific parameters.  Obviously this can't be done from a BIOS because the host isn't even running an O/S.   Limitations in LSI-supplied windows-based tools prevent you from creating customized actions based on health.
The software can tell you serial numbers of disk drives, the controller, and firmware/driver revisions.  SMARTMon-UX frees you from having to take a system down to gather patch/BIOS/driver information
If you are in a high-security area, use the -zdq command as part of a polling daemon that reports that all of the disks behind a RAID controller are online and have not been taken.  We have customers who have "national security" implications that use the software to make sure that nobody has stolen a disk drive.  Remember if you have RAID5, then somebody could take a disk drive, and the host would run normally on the degraded LUN.  Our software detects disk drive removals behind LSI-based RAID controllers.
Do you have newer 6Gbit SAS disks, and/or SATA drives?  Is everything synced up to highest supported speed?   Look at the Link Max/min rates to find out.

 

The results below show /etc/smartmon-ux -zd /dev/es/ses0

(You must give it the device name for something that is attached to a LSI internal RAID controller.  In this case, the controller is the LSISAS3800X card, which is a JBOD controller.)

 

SMARTMon-UX [Release 1.38, Build  30-OCT-2008] - Copyright 2001-2008 SANtools(R), Inc. http://www.SANtools.com

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

 

 

Port #0. /proc/mpt/ioc0    RAID SAS1068 A0     MPT 105   Firmware (1.16.00.01)   IOC 0

x86 BIOS image's version: MPTBIOS-6.12.00.00 (2006.10.31)

Bus/Dev/Fun    Board Name       Board Assembly   Board Tracer

130   3   0     SAS1068

SAS1068's phylinks are (Port 0,1,...,8): 3.0 G, 3.0 G, 3.0 G, 3.0 G, down, down, down, down

 

Firmware Settings

-----------------

SAS WWID:                       500605b0000488c0

Multi-pathing:                  Disabled

SATA Native Command Queuing:    Enabled

SATA Write Caching:             Enabled

SATA Maximum Queue Depth:       32

Device Missing Report Delay:    0 seconds

Device Missing I/O Delay:       0 seconds

Phy Parameters for Phynum:      0    1    2    3    4    5    6    7  

Link Enabled:                 Yes  Yes  Yes  Yes  Yes  Yes  Yes  Yes

Link Min Rate:                1.5  1.5  1.5  1.5  1.5  1.5  1.5  1.5

Link Max Rate:                3.0  3.0  3.0  3.0  3.0  3.0  3.0  3.0

SSP Initiator Enabled:        Yes  Yes  Yes  Yes  Yes  Yes  Yes  Yes

SSP Target Enabled:           No   No   No   No   No   No   No   No  

Port Configuration:           Auto Auto Auto Auto Auto Auto Auto Auto

Target IDs per enclosure:       1

Persistent mapping:             Enabled

Physical mapping type:          None

Target ID 0 reserved for boot:  No

Starting slot (direct attach):  0

Target IDs (physical mapping):  0

Interrupt Coalescing:           Enabled, timeout is 16 us, depth is 16

 

Persistent Mappings

-------------------

Persistent entry 0 is valid, Bus 0 Target 0 is PhysId 5000c5000040f53d

Persistent entry 1 is valid, Bus 0 Target 1 is PhysId 0523270354666c41

Persistent entry 2 is valid, Bus 0 Target 2 is PhysId 0523270354666f3e

Persistent entry 3 is valid, Bus 0 Target 3 is PhysId 0523270354666d4b

Persistent entry 4 is valid, Bus 0 Target 4 is PhysId 0523270354666c4a

Persistent entry 5 is valid, Bus 0 Target 5 is PhysId 5000c5000694c6ea

Persistent entry 6 is valid, Bus 0 Target 6 is PhysId 5000c5000694be86

Persistent entry 7 is valid, Bus 0 Target 7 is PhysId 5000c5000694bb7a

Persistent entry 8 is valid, Bus 0 Target 8 is PhysId 5000c5000694beae

Persistent entry 9 is valid, Bus 0 Target 9 is PhysId 5000c5000694c0de

Persistent entry 10 is valid, Bus 0 Target 10 is PhysId 5000c5000694bffe

Persistent entry 11 is valid, Bus 0 Target 11 is PhysId 500a0b82e0850019

Persistent entry 12 is valid, Bus 0 Target 12 is PhysId 5000c5000694c6e9

Persistent entry 13 is valid, Bus 0 Target 13 is PhysId 5000c5000694be85

Persistent entry 14 is valid, Bus 0 Target 14 is PhysId 5000c5000694bb79

Persistent entry 15 is valid, Bus 0 Target 15 is PhysId 5000c5000694bead

Persistent entry 16 is valid, Bus 0 Target 16 is PhysId 5000c5000694c0dd

Persistent entry 17 is valid, Bus 0 Target 17 is PhysId 5000c5000694bffd

Persistent entry 18 is valid, Bus 0 Target 18 is PhysId 500a0b82e0894019

SAS1068's links are 3.0 G, 3.0 G, 3.0 G, 3.0 G, down, down, down, down

 

B___T___L  Vendor   Product          Rev      SASAddress     PhyNum

0  12   0  SEAGATE  ST3146855SS      MS01  5000c5000694c6e9     0

0  13   0  SEAGATE  ST3146855SS      MS01  5000c5000694be85     1

0  14   0  SEAGATE  ST3146855SS      MS01  5000c5000694bb79     2

0  15   0  SEAGATE  ST3146855SS      MS01  5000c5000694bead     3

0  16   0  SEAGATE  ST3146855SS      MS01  5000c5000694c0dd     5

0  17   0  SEAGATE  ST3146855SS      MS01  5000c5000694bffd    11

0  18   0  LSILOGIC SYM3600-SAS      0166  500a0b82e0894019    24

RAID is not supported on this port

RAID is not supported on this port

RAID is not supported on this port

 

 

Program Ended.

 

The results below show /etc/smartmon-ux -zdL /dev/es/ses0

This particular controller doesn't support an event log, but the dump will still provide information about the firmware and chipset.

 

SMARTMon-UX [Release 1.36, Build  8-JUN-2008] - Copyright 2001-2008 SANtools(R), Inc. http://www.SANtools.com

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

 

mpt0              RAID SAS1068 A0     MPT 105   Firmware 01100001   IOC 0

This controller does not support event logging

The event log is empty for the above controller, or the feature is not supported by the firmware

 

 

Program Ended.

 

Reporting disk drives only (the -zdq command)

The results below do an efficient scan to just report physical disks seen by the operating system, as well as disk drives that are hidden behind logical disks created by RAID firmware, using the command smartmon-ux -zdq This dump was run on a LINUX host that uses a LSI controller configured in RAID-1 mode. Note that some of the disks report a physical device (/dev/hdb, /dev/sda, /dev/sdb, /dev/sdc).   Those disks are directly seen by the operating system.  The HP disk at "Bus 0 Target 5" is only seen by the RAID controller and invisible to the operating system. (Note for security reasons, the serial numbers were manually changed in this document).

[root@w13 /scratch/common]# ./smartmon-ux -zdq

SMARTMon-UX [Release 1.38, Build 30-OCT-2008] - Copyright 2001-2008 SANtools(R), Inc. http://www.SANtools.com

Discovered TSSTcorpCDW/DVD TS-L462D S/N "" on /dev/hdb (SMART unsupported)

Discovered ATA ST3500630NS S/N "9QG43RVS" on /dev/sda (Not Enabling SMART)(476940 MB)

Discovered HP DF072BAFDT S/N "BJL4P86004TB0862" at Bus 0 Target 5 (Not Enabling SMART) (70007 MB)

Discovered ATA WDC WD2500AAJS-2 S/N "WD-WMART1663509" on /dev/sdb (Not Enabling SMART)(238475 MB

Discovered LSILOGIC Logical Volume S/N "" on /dev/sdc (SMART unsupported)(69618 MB)

 

Here is the -zd dump from the same system, that reveals more about the configuration and how the disks are used.

 

[root@w13 /scratch/common]# ./smartmon-ux -zd

SMARTMon-UX [Release 1.38, Build 30-OCT-2008] - Copyright 2001-2008 SANtools(R), Inc. http://www.SANtools.com

Discovered TSSTcorpCDW/DVD TS-L462D S/N "" on /dev/hdb (SMART unsupported)

Discovered ATA ST3500630NS S/N "9QG43RVS" on /dev/sda (Not Enabling SMART)(476940 MB)

 

Discovered (1) Controllers:

 

Port #0. /proc/mpt/ioc0    RAID SAS1068 B1     MPT 105   Firmware (1.18.00)   IOC 0

x86 BIOS image's version: MPTBIOS-6.12.00.00 (2006.10.31)

Bus/Dev/Fun    Board Name       Board Assembly   Board Tracer

130   3   0     SAS1068

SAS1068's phylinks are (Port 0,1,...,8): 1.5 G, down, 3.0 G, down, down, 3.0 G, down, down

 

Firmware Settings

-----------------

SAS WWID:                       500d068000003505

Multi-pathing:                  Disabled

SATA Native Command Queuing:    Enabled

SATA Write Caching:             Enabled

SATA Maximum Queue Depth:       32

Device Missing Report Delay:    0 seconds

Device Missing I/O Delay:       0 seconds

Phy Parameters for Phynum:      0    1    2    3    4    5    6    7

Link Enabled:                 Yes  Yes  Yes  Yes  Yes  Yes  Yes  Yes

Link Min Rate:                1.5  1.5  1.5  1.5  1.5  1.5  1.5  1.5

Link Max Rate:                3.0  3.0  3.0  3.0  3.0  3.0  3.0  3.0

SSP Initiator Enabled:        Yes  Yes  Yes  Yes  Yes  Yes  Yes  Yes

SSP Target Enabled:           No   No   No   No   No   No   No   No

Port Configuration:           Auto Auto Auto Auto Auto Auto Auto Auto

Target IDs per enclosure:       1

Persistent mapping:             Disabled

Physical mapping type:          Direct Attach

Target ID 0 reserved for boot:  No

Starting slot (direct attach):  0

Target IDs (physical mapping):  0

Interrupt Coalescing:           Enabled, timeout is 16 us, depth is 4

 

Persistent Mappings

-------------------

No persistent entries found

SAS1068's phylinks are (Port 0,1,...,8): 1.5 G, down, 3.0 G, down, down, 3.0 G, down, down

 

Discovered ST3500630NS S/N "9QG43RVS" on RAID (Not Enabling SMART) (476940 MB)

Discovered WDC WD2500AAJS-22VTA0 S/N "WD-WMART1663590" on RAID (Not Enabling SMART) (238475 MB)

Discovered LSILOGIC Logical Volume S/N "" on RAID (Not Enabling SMART) (69618 MB)

1 volume is active, 2 physical disks are

 

Volume 0 is Bus 0 Target 4, Type IM (Integrated Mirroring)

Volume Name:

Volume WWID:  0a0cade5ed79d4ab

Volume State:  degraded, enabled

Volume Settings:  write caching disabled, auto configure

Volume draws from Hot Spare Pools:  0

Volume Size 69618 MB, Stripe Size 0 KB, 2 Members

Volume Device:

 Member 1 is PhysDisk 0 at (Bus 0 Target 5)

 

Discovered HP DF072BAFDT S/N "BJL4P86004TB0862" at Bus 0 Target 5 (70007 MB)  state=online PhysDisk=0

Discovered HP DF072BABUD S/N "J2YD2PCA" at Bus 0 Target 8 (70007 MB)  state=missing, out of sync PhysDisk=1

Volume 0 State:  degraded, enabled

Volume 1 State:  optimal, disabled

 

 

(Additional output follows, but was truncated as it isn't relevant to the -zd command)

There are several points of interest in this dump.

Note that the HP Disk S/N J2YD2PCA shows state=missing.  That is because this disk is no longer plugged into the system, and really is missing.  The RAID controller remembers the serial number, and smartmon-ux reports it, so you can see what was manually removed).
Some ports are running at 1.5 Gbit/sec, others are running at 3 Gbit/sec
The logical device is degraded (one disk is missing from the RAID-1 mirror)

 

Reporting statistical totals via the -zds command)

Below shows link totals on all disk and expander ports.  You would generally want to use the -zdt command to cross-reference device names and ports so you know what device is associated with each phy and expander port.

./smartmon-ux -zds

SMARTMon-UX [Release 1.52, Build 29-DEC-2010] - Copyright 2001-2010 SANtools(R), Inc. http://www.SANtools.com

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

 

Discovered (1) Controllers:

 

Port #0. mpt0              RAID SAS1068E B1    MPT 105   Firmware (1.29.00)   IOC 0

x86 BIOS image's version is MPTBIOS-6.28.00.00 (2009.02.03)

FCode image's version is MPT SAS FCode Version 1.00.41 (2006.04.13)

Seg/Bus/Dev/Fun    Board Name       Board Assembly   Board Tracer

0   1   0   0     SAS3442E         L3-00151-01B     L011901606    

Adapter Phy 0:  Link Up, No Errors

 

Adapter Phy 1:  Link Up

Invalid words:                 320

Disparity errors:              295

Loss of syncs:                 0

Phys port resets:              0

 

Adapter Phy 2:  Link Up

Invalid words:                 2468

Disparity errors:              2272

Loss of syncs:                 0

Phys port resets:              0

 

Adapter Phy 3:  Link Up

Invalid words:                 1776

Disparity errors:              1700

Loss of syncs:                 0

Phys port resets:              0

 

Adapter Phy 4:  Link Up, No Errors

 

Adapter Phy 5:  Link Up, No Errors

 

Adapter Phy 6:  Link Up, No Errors

 

Adapter Phy 7:  Link Up, No Errors

 

Expander (Handle 000d) Phy 0:  Link Up, No Errors

 

Expander (Handle 000d) Phy 1:  Link Up

Invalid words:                 820

Disparity errors:              820

Loss of syncs:                 0

Phys port resets:              0

 

Expander (Handle 000d) Phy 2:  Link Up, No Errors

 

Expander (Handle 000d) Phy 3:  Link Up, No Errors

 

Expander (Handle 000d) Phy 4:  Link Down, No Errors

 

Expander (Handle 000d) Phy 5:  Link Down, No Errors

 

Expander (Handle 000d) Phy 6:  Link Down, No Errors

 

Expander (Handle 000d) Phy 7:  Link Down, No Errors

 

Expander (Handle 000d) Phy 8:  Link Up, No Errors

 

Expander (Handle 000d) Phy 9:  Link Down, No Errors

 

Expander (Handle 000d) Phy 10:  Link Down, No Errors

 

Expander (Handle 000d) Phy 11:  Link Up, No Errors

 

Expander (Handle 000d) Phy 12:  Link Down, No Errors

 

Expander (Handle 000d) Phy 13:  Link Down, No Errors

 

Expander (Handle 000d) Phy 14:  Link Down, No Errors

 

Expander (Handle 000d) Phy 15:  Link Down, No Errors

 

Expander (Handle 000d) Phy 16:  Link Up

Invalid words:                 75008

Disparity errors:              75008

Loss of syncs:                 2

Phys port resets:              0

 

Expander (Handle 000d) Phy 17:  Link Up

Invalid words:                 75039

Disparity errors:              75039

Loss of syncs:                 2

Phys port resets:              0

 

Expander (Handle 000d) Phy 18:  Link Up, No Errors

 

Expander (Handle 000d) Phy 19:  Link Up, No Errors

 

Expander (Handle 000d) Phy 20:  Link Down, No Errors

 

Expander (Handle 000d) Phy 21:  Link Down, No Errors

 

Expander (Handle 000d) Phy 22:  Link Down, No Errors

 

Expander (Handle 000d) Phy 23:  Link Down, No Errors

 

Expander (Handle 000d) Phy 24:  Link Up, No Errors

 

Reporting topology details via the -zdt command)

This feature ties together SAS Expanders, physical ports, device names, as well as enumerate SAS topology.  The first 4 columns of errors are reported and maintained by counters inside of the LSI controller itself.   The second set of 4 columns represent counters maintained by SAS disk drives.  In the report below, the WDC disks are SATA, so they do not have internal error counters.   Only the 2 maxtor SAS drives show internal errors, as they are in a cheap 2-drive enclosure.  Note that none of the other SAS disks show any error counters.  This is because they are in a premium SAS/SATA enclosure, and the disks simply have not logged any errors yet.

 

./smartmon-ux -zdt

root@nex:/volumes/satamirror/Development/smartmon# ./smartmon-ux -zdt

SMARTMon-UX [Release 1.52, Build  4-JAN-2011] - Copyright 2001-2010 SANtools(R), Inc. http://www.SANtools.com

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Bus Tgt L  Type             Vendor   Product          Rev   SASAddress        Phy   ParentSASAddress Handle Encl Bay Port PHand. Serial               OSName           Bus->Resets  SyncLoss  InvWords DisErrors HDD->Resets SyncLoss  InvWords Diserrors

          SAS Initiator                                    500605b000074c20                             1

          SAS Initiator                                    500605b000074c21                             2

          SAS Initiator                                    500605b000074c22                             3

          SAS Initiator                                    500605b000074c23                             4

          SAS Initiator                                    500605b000074c24                             5

          SAS Initiator                                    500605b000074c25                             6

          SAS Initiator                                    500605b000074c26                             7

          SAS Initiator                                    500605b000074c27                             8

          Edge Expander                                    500a0b82e0850000         500605b000074c24   13     2        4     5

0  72  0  SAS Target       SEAGATE  ST3146855SS      MS01  5000c5000694c6ea     0   500a0b82e0850000   14     2   0    4    13  3LN23ER0             c0t72d0s2                   0         0        45        45          0         0         0         0

0  74  0  SAS Target       SEAGATE  ST3146855SS      MS01  5000c5000694bb7a     2   500a0b82e0850000   15     2   2    4    13  3LN29PAS             c0t74d0s2                   0         0      5242      5236          0         0         0         0

0  75  0  SAS Target       SEAGATE  ST3146855SS      MS01  5000c5000694c0de     3   500a0b82e0850000   16     2   3    4    13  3LN27XJ9             c0t75d0s2                   0         0         0         0          0         0         0         0

0  76  0  SAS Target       MAXTOR   ATLAS10K5_300SAS BN02  50010b900005ce5b     8   500a0b82e0850000   17     2   8    4    13  J8033Q6K             c0t76d0s2                   0         0      2602      2602          0         0         0         0

0  77  0  SAS Target       SEAGATE  ST3146855SS      MS01  5000c5000694bffe    11   500a0b82e0850000   18     2  11    4    13  3LN29QG4             c0t77d0s2                   0         4    150528    150514          0         0         0         0

0  78  0  SAS Init/Target  LSILOGIC SYM3600-SAS      0166  500a0b82e0850019    24   500a0b82e0850000   19     2  12    4    13  N/A                  ses0                        0         0         0         0

0  79  0  SAS Target       MAXTOR   ATLAS10K5_300SAS BN02  50010b9000236386     1   500605b000074c22   10     1   1    1     2  J80G2JAK             c0t79d0s2                   0         0    142776    139771          0         5      4506    394668

0  81  0  SAS Target       MAXTOR   ATLAS10K5_300SAS BN02  50010b90002362d2     0   500605b000074c21    9     1   0    0     1  J80G2H8K             c0t81d0s2                   0         0    134896    130222          0         5       498    361092

0  83  0  SATA Target      WDC WD20EADS-00R6B0       0A01  dd5c3a40bba28c7a     3   500605b000074c24   12     1   3    3     4  WD-WCAVY0302730      c0t83d0s2                   0         0      2418      2223

0  88  0  SATA Target      WDC WD20EARS-00MVWB0      AB50  365c3a44e0c3a994     2   500605b000074c23   11     1   2    2     3  WD-WMAZ20397906      c0t88d0s2                   0         0      4450      4139

 

 

Reconfiguring LSI SAS controller settings via the -zdi command

Warning: You should consult with your storage vendor and/or LSI to make sure that any changes made here will not result in data loss, O/S crashes, or instability.  If you are not sure what these settings do, then it is best that you don't change them.  These settings can provide improved performance and greater data integrity if tuned properly.

 

The session below shows some of the items that smartmon-ux can modify when the need arises.  Do so at your own risk. Changes will be made immediately, although depending on the controller model and firmware, you may have to power cycle the controller for them to take effect.

 

./smartmon-ux -zdi

SMARTMon-UX [Release 1.52, Build 29-DEC-2010] - Copyright 2001-2010 SANtools(R), Inc. http://www.SANtools.com

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

 

Discovered (1) Controllers:

 

Port #0. mpt0              RAID SAS1068E B1    MPT 105   Firmware (1.29.00)   IOC 0

x86 BIOS image's version is MPTBIOS-6.28.00.00 (2009.02.03)

FCode image's version is MPT SAS FCode Version 1.00.41 (2006.04.13)

Seg/Bus/Dev/Fun    Board Name       Board Assembly   Board Tracer

0   1   0   0     SAS3442E         L3-00151-01B     L011901606    

 

View/Modify SAS/SATA Controller Settings:

SATA Maximum Queue Depth:  [0 to 255, default is 32]

Device Missing Report Delay:  [0 to 2047, default is 0]

Device Missing I/O Delay:  [0 to 255, default is 0]

 

PhyNum  Link      Min      Max      Initiator  Target    Port

 0    Enabled     1.5      3.0    Enabled    Disabled  Auto

 1    Enabled     1.5      3.0    Enabled    Disabled  Auto

 2    Enabled     1.5      3.0    Enabled    Disabled  Auto

 3    Enabled     1.5      3.0    Enabled    Disabled  Auto

 4    Enabled     1.5      3.0    Enabled    Disabled  Auto

 5    Enabled     1.5      3.0    Enabled    Disabled  Auto

 6    Enabled     1.5      3.0    Enabled    Disabled  Auto

 7    Enabled     1.5      3.0    Enabled    Disabled  Auto

 

Select a Phy:  [0-7, 8=AllPhys, RETURN to quit] (All speeds shown in Gigabits/sec) 0

Link:  [0=Disabled, 1=Enabled, default is 1]

Min Speed:  [0=1.5 Gbps, 1=3.0 Gbps, default is 0]

Max Speed:  [0=1.5 Gbps, 1=3.0 Gbps, default is 1]

Initiator:  [0=Disabled, 1=Enabled, default is 1]

Target:  [0=Disabled, 1=Enabled, default is 0]

Port:  [0 to 7 for manual config, 8 for auto config, default is 8]

 

PhyNum  Link      Min      Max      Initiator  Target    Port

 0    Enabled     1.5      3.0    Enabled    Disabled  Auto

 1    Enabled     1.5      3.0    Enabled    Disabled  Auto

 2    Enabled     1.5      3.0    Enabled    Disabled  Auto

 3    Enabled     1.5      3.0    Enabled    Disabled  Auto

 4    Enabled     1.5      3.0    Enabled    Disabled  Auto

 5    Enabled     1.5      3.0    Enabled    Disabled  Auto

 6    Enabled     1.5      3.0    Enabled    Disabled  Auto

 7    Enabled     1.5      3.0    Enabled    Disabled  Auto

 

Select a Phy:  [0-7, 8=AllPhys, RETURN to quit] (All speeds shown in Gigabits/sec)

 

Persistence:  [0=Disabled, 1=Enabled, default is 1]

Physical mapping:  [0=None, 1=DirectAttach, 2=EnclosureSlot, default is 0]

Multi-pathing:  [0=Disabled, 1=Enabled, default is 0]

SATA Native Command Queuing:  [0=Disabled, 1=Enabled, default is 1]

SATA Write Caching:  [0=Disabled, 1=Enabled, default is 1]