LSI (MPT Internal) RAID Engines

Top Previous Next

Benefits of Directly Querying LSI RAID Controllers

•

Use the software to assess RAID health remotely, and not be dependent on a BIOS-based program, or a utility that only runs on the host console. Since the output can easily be parsed and scripted, the administrator can implement a phone-home system based on specific parameters. Obviously this can't be done from a BIOS because the host isn't even running an O/S. Limitations in LSI-supplied windows-based tools prevent you from creating customized actions based on health.

•	The software can tell you serial numbers of disk drives, the controller, and firmware/driver revisions. SMARTMon-UX frees you from having to take a system down to gather patch/BIOS/driver information

•

If you are in a high-security area, use the -zdq command as part of a polling daemon that reports that all of the disks behind a RAID controller are online and have not been taken. We have customers who have "national security" implications that use the software to make sure that nobody has stolen a disk drive. Remember if you have RAID5, then somebody could take a disk drive, and the host would run normally on the degraded LUN. Our software detects disk drive removals behind LSI-based RAID controllers.

•	Do you have newer 6Gbit SAS disks, and/or SATA drives? Is everything synced up to highest supported speed? Look at the Link Max/min rates to find out.

The results below show /etc/smartmon-ux -zd /dev/es/ses0

(You must give it the device name for something that is attached to a LSI internal RAID controller. In this case, the controller is the LSISAS3800X card, which is a JBOD controller.)

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

Port #0. /proc/mpt/ioc0 RAID SAS1068 A0 MPT 105 Firmware (1.16.00.01) IOC 0

x86 BIOS image's version: MPTBIOS-6.12.00.00 (2006.10.31)

Bus/Dev/Fun Board Name Board Assembly Board Tracer

130 3 0 SAS1068

SAS1068's phylinks are (Port 0,1,...,8): 3.0 G, 3.0 G, 3.0 G, 3.0 G, down, down, down, down

Firmware Settings

-----------------

SAS WWID: 500605b0000488c0

Multi-pathing: Disabled

SATA Native Command Queuing: Enabled

SATA Write Caching: Enabled

SATA Maximum Queue Depth: 32

Device Missing Report Delay: 0 seconds

Device Missing I/O Delay: 0 seconds

Phy Parameters for Phynum: 0 1 2 3 4 5 6 7

Link Enabled: Yes Yes Yes Yes Yes Yes Yes Yes

Link Min Rate: 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5

Link Max Rate: 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0

SSP Initiator Enabled: Yes Yes Yes Yes Yes Yes Yes Yes

SSP Target Enabled: No No No No No No No No

Port Configuration: Auto Auto Auto Auto Auto Auto Auto Auto

Target IDs per enclosure: 1

Persistent mapping: Enabled

Physical mapping type: None

Target ID 0 reserved for boot: No

Starting slot (direct attach): 0

Target IDs (physical mapping): 0

Interrupt Coalescing: Enabled, timeout is 16 us, depth is 16

Persistent Mappings

-------------------

Persistent entry 0 is valid, Bus 0 Target 0 is PhysId 5000c5000040f53d

Persistent entry 1 is valid, Bus 0 Target 1 is PhysId 0523270354666c41

Persistent entry 2 is valid, Bus 0 Target 2 is PhysId 0523270354666f3e

Persistent entry 3 is valid, Bus 0 Target 3 is PhysId 0523270354666d4b

Persistent entry 4 is valid, Bus 0 Target 4 is PhysId 0523270354666c4a

Persistent entry 5 is valid, Bus 0 Target 5 is PhysId 5000c5000694c6ea

Persistent entry 6 is valid, Bus 0 Target 6 is PhysId 5000c5000694be86

Persistent entry 7 is valid, Bus 0 Target 7 is PhysId 5000c5000694bb7a

Persistent entry 8 is valid, Bus 0 Target 8 is PhysId 5000c5000694beae

Persistent entry 9 is valid, Bus 0 Target 9 is PhysId 5000c5000694c0de

Persistent entry 10 is valid, Bus 0 Target 10 is PhysId 5000c5000694bffe

Persistent entry 11 is valid, Bus 0 Target 11 is PhysId 500a0b82e0850019

Persistent entry 12 is valid, Bus 0 Target 12 is PhysId 5000c5000694c6e9

Persistent entry 13 is valid, Bus 0 Target 13 is PhysId 5000c5000694be85

Persistent entry 14 is valid, Bus 0 Target 14 is PhysId 5000c5000694bb79

Persistent entry 15 is valid, Bus 0 Target 15 is PhysId 5000c5000694bead

Persistent entry 16 is valid, Bus 0 Target 16 is PhysId 5000c5000694c0dd

Persistent entry 17 is valid, Bus 0 Target 17 is PhysId 5000c5000694bffd

Persistent entry 18 is valid, Bus 0 Target 18 is PhysId 500a0b82e0894019

SAS1068's links are 3.0 G, 3.0 G, 3.0 G, 3.0 G, down, down, down, down

B___T___L Vendor Product Rev SASAddress PhyNum

0 12 0 SEAGATE ST3146855SS MS01 5000c5000694c6e9 0

0 13 0 SEAGATE ST3146855SS MS01 5000c5000694be85 1

0 14 0 SEAGATE ST3146855SS MS01 5000c5000694bb79 2

0 15 0 SEAGATE ST3146855SS MS01 5000c5000694bead 3

0 16 0 SEAGATE ST3146855SS MS01 5000c5000694c0dd 5

0 17 0 SEAGATE ST3146855SS MS01 5000c5000694bffd 11

0 18 0 LSILOGIC SYM3600-SAS 0166 500a0b82e0894019 24

RAID is not supported on this port

Program Ended.

The results below show /etc/smartmon-ux -zdL /dev/es/ses0

This particular controller doesn't support an event log, but the dump will still provide information about the firmware and chipset.

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

mpt0 RAID SAS1068 A0 MPT 105 Firmware 01100001 IOC 0

This controller does not support event logging

The event log is empty for the above controller, or the feature is not supported by the firmware

Program Ended.

Reporting disk drives only (the -zdq command)

The results below do an efficient scan to just report physical disks seen by the operating system, as well as disk drives that are hidden behind logical disks created by RAID firmware, using the command smartmon-ux -zdq This dump was run on a LINUX host that uses a LSI controller configured in RAID-1 mode. Note that some of the disks report a physical device (/dev/hdb, /dev/sda, /dev/sdb, /dev/sdc). Those disks are directly seen by the operating system. The HP disk at "Bus 0 Target 5" is only seen by the RAID controller and invisible to the operating system. (Note for security reasons, the serial numbers were manually changed in this document).

[root@w13 /scratch/common]# ./smartmon-ux -zdq

Discovered TSSTcorpCDW/DVD TS-L462D S/N "" on /dev/hdb (SMART unsupported)

Discovered ATA ST3500630NS S/N "9QG43RVS" on /dev/sda (Not Enabling SMART)(476940 MB)

Discovered HP DF072BAFDT S/N "BJL4P86004TB0862" at Bus 0 Target 5 (Not Enabling SMART) (70007 MB)

Discovered ATA WDC WD2500AAJS-2 S/N "WD-WMART1663509" on /dev/sdb (Not Enabling SMART)(238475 MB

Discovered LSILOGIC Logical Volume S/N "" on /dev/sdc (SMART unsupported)(69618 MB)

Here is the -zd dump from the same system, that reveals more about the configuration and how the disks are used.

[root@w13 /scratch/common]# ./smartmon-ux -zd

Discovered TSSTcorpCDW/DVD TS-L462D S/N "" on /dev/hdb (SMART unsupported)

Discovered ATA ST3500630NS S/N "9QG43RVS" on /dev/sda (Not Enabling SMART)(476940 MB)

Discovered (1) Controllers:

Port #0. /proc/mpt/ioc0 RAID SAS1068 B1 MPT 105 Firmware (1.18.00) IOC 0

x86 BIOS image's version: MPTBIOS-6.12.00.00 (2006.10.31)

Bus/Dev/Fun Board Name Board Assembly Board Tracer

130 3 0 SAS1068

SAS1068's phylinks are (Port 0,1,...,8): 1.5 G, down, 3.0 G, down, down, 3.0 G, down, down

Firmware Settings

-----------------

SAS WWID: 500d068000003505

Multi-pathing: Disabled

SATA Native Command Queuing: Enabled

SATA Write Caching: Enabled

SATA Maximum Queue Depth: 32

Device Missing Report Delay: 0 seconds

Device Missing I/O Delay: 0 seconds

Phy Parameters for Phynum: 0 1 2 3 4 5 6 7

Link Enabled: Yes Yes Yes Yes Yes Yes Yes Yes

Link Min Rate: 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5

Link Max Rate: 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0

SSP Initiator Enabled: Yes Yes Yes Yes Yes Yes Yes Yes

SSP Target Enabled: No No No No No No No No

Port Configuration: Auto Auto Auto Auto Auto Auto Auto Auto

Target IDs per enclosure: 1

Persistent mapping: Disabled

Physical mapping type: Direct Attach

Target ID 0 reserved for boot: No

Starting slot (direct attach): 0

Target IDs (physical mapping): 0

Interrupt Coalescing: Enabled, timeout is 16 us, depth is 4

Persistent Mappings

-------------------

No persistent entries found

SAS1068's phylinks are (Port 0,1,...,8): 1.5 G, down, 3.0 G, down, down, 3.0 G, down, down

Discovered ST3500630NS S/N "9QG43RVS" on RAID (Not Enabling SMART) (476940 MB)

Discovered WDC WD2500AAJS-22VTA0 S/N "WD-WMART1663590" on RAID (Not Enabling SMART) (238475 MB)

Discovered LSILOGIC Logical Volume S/N "" on RAID (Not Enabling SMART) (69618 MB)

1 volume is active, 2 physical disks are

Volume 0 is Bus 0 Target 4, Type IM (Integrated Mirroring)

Volume Name:

Volume WWID: 0a0cade5ed79d4ab

Volume State: degraded, enabled

Volume Settings: write caching disabled, auto configure

Volume draws from Hot Spare Pools: 0

Volume Size 69618 MB, Stripe Size 0 KB, 2 Members

Volume Device:

Member 1 is PhysDisk 0 at (Bus 0 Target 5)

Discovered HP DF072BAFDT S/N "BJL4P86004TB0862" at Bus 0 Target 5 (70007 MB) state=online PhysDisk=0

Discovered HP DF072BABUD S/N "J2YD2PCA" at Bus 0 Target 8 (70007 MB) state=missing, out of sync PhysDisk=1

Volume 0 State: degraded, enabled

Volume 1 State: optimal, disabled

(Additional output follows, but was truncated as it isn't relevant to the -zd command)

There are several points of interest in this dump.

•	Note that the HP Disk S/N J2YD2PCA shows state=missing. That is because this disk is no longer plugged into the system, and really is missing. The RAID controller remembers the serial number, and smartmon-ux reports it, so you can see what was manually removed).

•	Some ports are running at 1.5 Gbit/sec, others are running at 3 Gbit/sec

•	The logical device is degraded (one disk is missing from the RAID-1 mirror)

Reporting statistical totals via the -zds command)

Below shows link totals on all disk and expander ports. You would generally want to use the -zdt command to cross-reference device names and ports so you know what device is associated with each phy and expander port.

./smartmon-ux -zds

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

Port #0. mpt0 RAID SAS1068E B1 MPT 105 Firmware (1.29.00) IOC 0

x86 BIOS image's version is MPTBIOS-6.28.00.00 (2009.02.03)

FCode image's version is MPT SAS FCode Version 1.00.41 (2006.04.13)

Seg/Bus/Dev/Fun Board Name Board Assembly Board Tracer

0 1 0 0 SAS3442E L3-00151-01B L011901606

Adapter Phy 0: Link Up, No Errors

Adapter Phy 1: Link Up

Invalid words: 320

Disparity errors: 295

Loss of syncs: 0

Phys port resets: 0

Adapter Phy 2: Link Up

Invalid words: 2468

Disparity errors: 2272

Loss of syncs: 0

Phys port resets: 0

Adapter Phy 3: Link Up

Invalid words: 1776

Disparity errors: 1700

Loss of syncs: 0

Phys port resets: 0

Adapter Phy 4: Link Up, No Errors

Adapter Phy 5: Link Up, No Errors

Adapter Phy 6: Link Up, No Errors

Adapter Phy 7: Link Up, No Errors

Expander (Handle 000d) Phy 0: Link Up, No Errors

Expander (Handle 000d) Phy 1: Link Up

Invalid words: 820

Disparity errors: 820

Loss of syncs: 0

Phys port resets: 0

Expander (Handle 000d) Phy 2: Link Up, No Errors

Expander (Handle 000d) Phy 3: Link Up, No Errors

Expander (Handle 000d) Phy 4: Link Down, No Errors

Expander (Handle 000d) Phy 5: Link Down, No Errors

Expander (Handle 000d) Phy 6: Link Down, No Errors

Expander (Handle 000d) Phy 7: Link Down, No Errors

Expander (Handle 000d) Phy 8: Link Up, No Errors

Expander (Handle 000d) Phy 9: Link Down, No Errors

Expander (Handle 000d) Phy 10: Link Down, No Errors

Expander (Handle 000d) Phy 11: Link Up, No Errors

Expander (Handle 000d) Phy 12: Link Down, No Errors

Expander (Handle 000d) Phy 13: Link Down, No Errors

Expander (Handle 000d) Phy 14: Link Down, No Errors

Expander (Handle 000d) Phy 15: Link Down, No Errors

Expander (Handle 000d) Phy 16: Link Up

Invalid words: 75008

Disparity errors: 75008

Loss of syncs: 2

Phys port resets: 0

Expander (Handle 000d) Phy 17: Link Up

Invalid words: 75039

Disparity errors: 75039

Loss of syncs: 2

Phys port resets: 0

Expander (Handle 000d) Phy 18: Link Up, No Errors

Expander (Handle 000d) Phy 19: Link Up, No Errors

Expander (Handle 000d) Phy 20: Link Down, No Errors

Expander (Handle 000d) Phy 21: Link Down, No Errors

Expander (Handle 000d) Phy 22: Link Down, No Errors

Expander (Handle 000d) Phy 23: Link Down, No Errors

Expander (Handle 000d) Phy 24: Link Up, No Errors

Reporting topology details via the -zdt command)

This feature ties together SAS Expanders, physical ports, device names, as well as enumerate SAS topology. The first 4 columns of errors are reported and maintained by counters inside of the LSI controller itself. The second set of 4 columns represent counters maintained by SAS disk drives. In the report below, the WDC disks are SATA, so they do not have internal error counters. Only the 2 maxtor SAS drives show internal errors, as they are in a cheap 2-drive enclosure. Note that none of the other SAS disks show any error counters. This is because they are in a premium SAS/SATA enclosure, and the disks simply have not logged any errors yet.

./smartmon-ux -zdt

root@nex:/volumes/satamirror/Development/smartmon# ./smartmon-ux -zdt

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Bus Tgt L Type Vendor Product Rev SASAddress Phy ParentSASAddress Handle Encl Bay Port PHand. Serial OSName Bus->Resets SyncLoss InvWords DisErrors HDD->Resets SyncLoss InvWords Diserrors

SAS Initiator 500605b000074c20 1

SAS Initiator 500605b000074c21 2

SAS Initiator 500605b000074c22 3

SAS Initiator 500605b000074c23 4

SAS Initiator 500605b000074c24 5

SAS Initiator 500605b000074c25 6

SAS Initiator 500605b000074c26 7

SAS Initiator 500605b000074c27 8

Edge Expander 500a0b82e0850000 500605b000074c24 13 2 4 5

0 72 0 SAS Target SEAGATE ST3146855SS MS01 5000c5000694c6ea 0 500a0b82e0850000 14 2 0 4 13 3LN23ER0 c0t72d0s2 0 0 45 45 0 0 0 0

0 74 0 SAS Target SEAGATE ST3146855SS MS01 5000c5000694bb7a 2 500a0b82e0850000 15 2 2 4 13 3LN29PAS c0t74d0s2 0 0 5242 5236 0 0 0 0

0 75 0 SAS Target SEAGATE ST3146855SS MS01 5000c5000694c0de 3 500a0b82e0850000 16 2 3 4 13 3LN27XJ9 c0t75d0s2 0 0 0 0 0 0 0 0

0 76 0 SAS Target MAXTOR ATLAS10K5_300SAS BN02 50010b900005ce5b 8 500a0b82e0850000 17 2 8 4 13 J8033Q6K c0t76d0s2 0 0 2602 2602 0 0 0 0

0 77 0 SAS Target SEAGATE ST3146855SS MS01 5000c5000694bffe 11 500a0b82e0850000 18 2 11 4 13 3LN29QG4 c0t77d0s2 0 4 150528 150514 0 0 0 0

0 78 0 SAS Init/Target LSILOGIC SYM3600-SAS 0166 500a0b82e0850019 24 500a0b82e0850000 19 2 12 4 13 N/A ses0 0 0 0 0

0 79 0 SAS Target MAXTOR ATLAS10K5_300SAS BN02 50010b9000236386 1 500605b000074c22 10 1 1 1 2 J80G2JAK c0t79d0s2 0 0 142776 139771 0 5 4506 394668

0 81 0 SAS Target MAXTOR ATLAS10K5_300SAS BN02 50010b90002362d2 0 500605b000074c21 9 1 0 0 1 J80G2H8K c0t81d0s2 0 0 134896 130222 0 5 498 361092

0 83 0 SATA Target WDC WD20EADS-00R6B0 0A01 dd5c3a40bba28c7a 3 500605b000074c24 12 1 3 3 4 WD-WCAVY0302730 c0t83d0s2 0 0 2418 2223

0 88 0 SATA Target WDC WD20EARS-00MVWB0 AB50 365c3a44e0c3a994 2 500605b000074c23 11 1 2 2 3 WD-WMAZ20397906 c0t88d0s2 0 0 4450 4139

Reconfiguring LSI SAS controller settings via the -zdi command

Warning: You should consult with your storage vendor and/or LSI to make sure that any changes made here will not result in data loss, O/S crashes, or instability. If you are not sure what these settings do, then it is best that you don't change them. These settings can provide improved performance and greater data integrity if tuned properly.

The session below shows some of the items that smartmon-ux can modify when the need arises. Do so at your own risk. Changes will be made immediately, although depending on the controller model and firmware, you may have to power cycle the controller for them to take effect.

./smartmon-ux -zdi

Discovered LSILOGIC SYM3600-SAS S/N "0617053320" on /dev/es/ses0 [SES] (Enclosure Services)

Discovered (1) Controllers:

Port #0. mpt0 RAID SAS1068E B1 MPT 105 Firmware (1.29.00) IOC 0

x86 BIOS image's version is MPTBIOS-6.28.00.00 (2009.02.03)

FCode image's version is MPT SAS FCode Version 1.00.41 (2006.04.13)

Seg/Bus/Dev/Fun Board Name Board Assembly Board Tracer

0 1 0 0 SAS3442E L3-00151-01B L011901606

View/Modify SAS/SATA Controller Settings:

SATA Maximum Queue Depth: [0 to 255, default is 32]

Device Missing Report Delay: [0 to 2047, default is 0]

Device Missing I/O Delay: [0 to 255, default is 0]

PhyNum Link Min Max Initiator Target Port

0 Enabled 1.5 3.0 Enabled Disabled Auto

1 Enabled 1.5 3.0 Enabled Disabled Auto

2 Enabled 1.5 3.0 Enabled Disabled Auto

3 Enabled 1.5 3.0 Enabled Disabled Auto

4 Enabled 1.5 3.0 Enabled Disabled Auto

5 Enabled 1.5 3.0 Enabled Disabled Auto

6 Enabled 1.5 3.0 Enabled Disabled Auto

7 Enabled 1.5 3.0 Enabled Disabled Auto

Select a Phy: [0-7, 8=AllPhys, RETURN to quit] (All speeds shown in Gigabits/sec) 0

Link: [0=Disabled, 1=Enabled, default is 1]

Min Speed: [0=1.5 Gbps, 1=3.0 Gbps, default is 0]

Max Speed: [0=1.5 Gbps, 1=3.0 Gbps, default is 1]

Initiator: [0=Disabled, 1=Enabled, default is 1]

Target: [0=Disabled, 1=Enabled, default is 0]

Port: [0 to 7 for manual config, 8 for auto config, default is 8]

PhyNum Link Min Max Initiator Target Port

0 Enabled 1.5 3.0 Enabled Disabled Auto

1 Enabled 1.5 3.0 Enabled Disabled Auto

2 Enabled 1.5 3.0 Enabled Disabled Auto

3 Enabled 1.5 3.0 Enabled Disabled Auto

4 Enabled 1.5 3.0 Enabled Disabled Auto

5 Enabled 1.5 3.0 Enabled Disabled Auto

6 Enabled 1.5 3.0 Enabled Disabled Auto

7 Enabled 1.5 3.0 Enabled Disabled Auto

Select a Phy: [0-7, 8=AllPhys, RETURN to quit] (All speeds shown in Gigabits/sec)

Persistence: [0=Disabled, 1=Enabled, default is 1]

Physical mapping: [0=None, 1=DirectAttach, 2=EnclosureSlot, default is 0]

Multi-pathing: [0=Disabled, 1=Enabled, default is 0]

SATA Native Command Queuing: [0=Disabled, 1=Enabled, default is 1]

SATA Write Caching: [0=Disabled, 1=Enabled, default is 1]