freebsd stable - Feb 2016 - MRSAS driver/LSI MegaRaid 92XX-93XX admin question: When one of the Raid's physical drives break, how is it reported in the logs?

( ** Extremely sorry for crossposting! Was unclear where this RAID 
adapter question belongs, please clarify and I'll keep to one single 
list!
      Posted to all of stable@, scsi@ and fs@ .)


Hi,

When you run one of the MRSAS drives such as a Avatogech LSI MegaRaid 
9361 or 9266, and then eventually one of the physical RaidDrives or a 
CacheCade drives breaks, how is this reported to the FreeBSD host's 
dmesg or syslog?


I don't have the hardware in place so that I would be able to check. On 
the other hand someone among you may have extremely deep experience, in 
particular because this card is so common, so this is why I ask you 
here.

I understand that if at least one underlying copy of the data is 
accessible, the RAID card will optimize all access to that one, so when 
it comes to keeping IO working without interruption, the LSI card does a 
great job.


At some point, an SSD or HDD will break down, either completely (it 
won't connect and its SMART interface says the drive is consumed) or 
more discretely, through taking tons of time for its operations.

My best understanding is that the Raid card automatically will take 
those drives out of use, transparently. Now to the main point:


As admin, it's great to be informed when this happens i.e. an underlying 
physical Raid disk or a CacheCade disk is taken out of use or otherwise 
malfunctions.


Does the MrSas driver output this into the dmesg or syslog somehow?


Reading 
https://svnweb.freebsd.org/base/stable/10/sys/dev/mrsas/mrsas.c?revision=284267&view=markup
, the card seems to have an "event log" that the driver downloads from
the card in plaintext (??), but I don't understand from the sourcecode 
where that information is channeled. And also of course I can't see what 
that event log would contain in those cases.

(The "mfiutil" has a "show events" argument, though mfiutil
is only for
the related "mfi" driver which does not work for both 92XX and 93XX 
cards. Also in this case still I'd be interested to know how it reports 
a broken drive)

http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/3rd-party/lsi/mrsas/userguide/LSI_MR_SAS_SW_UG.pdf
on page 305, that is section "A.2 Event Messages" - I don't know
for
what LGI chip this document is, but, it does not list particular event 
message very clearly for when an individual underlying disk would have 
broken, I don't even see any event for when a hot spare would be taken 
in use!

You who have the experience, can you clarify please? Thanks :D

Tinker

(Will send any followup from now only to freebsd-scsi@ .)



Did some additional research and found that the disk failure indeed is 
reported in MRSAS' "event log".

So my final question then is, how do you extract it into userland (in 
the absence of an "mfiutil" as the MFI driver has)?



Details below. Thanks.

On 2016-02-14 19:59, Tinker wrote:
[...]
>
http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/3rd-party/lsi/mrsas/userguide/LSI_MR_SAS_SW_UG.pdf
> on page 305, that is section "A.2 Event Messages" - I don't
know for
> what LGI chip this document is, but, it does not list particular event
> message very clearly for when an individual underlying disk would have
> broken, I don't even see any event for when a hot spare would be taken
> in use!

Wait - this page:

https://www.schirmacher.de/display/Linux/Replace+failed+disk+in+MegaRAID+array

(and also 
http://serverfault.com/questions/485147/drive-is-failing-but-lsi-megaraid-controller-does-not-detect-it
)

gives an example of how the host system learns about broken disks:


Code: 0x00000051 .. Event Description: State change on VD 00/1 from 
OPTIMAL(3) to DEGRADED(2)


Code: 0x00000072 .. Event Description: State change on PD 05(e0xfc/s0) 
from ONLINE(18) to FAILED(11)

(unclean disk broken seems to be shown as:)

Code: 0x00000071 .. Event Description: Unexpected sense: PD 05(e0xfc/s0) 
Path 4433221103000000, CDB: 2e 00 3a 38 1b c7 00 00 01 00, Sense: 
b/00/00


And this version of the LSI documentation

http://hwraid.le-vert.net/raw-attachment/wiki/LSIMegaRAIDSAS/megacli_user_guide.pdf

gives a clearer definition of the physical and virtual drive states in 
"1.4.16 Physical Drive States"
and "1.4.17 Virtual Disk States" on pages 1-11 to 1-12.

So as we see, a physical drive breaking would

  * "FAILED" the physical drive

  * "DEGRADED" the Virtual Drive (that is the logical exported drive) 
(from "OPTIMAL")


So then, it was indeed the card's "event log" that contains this
info.



Last question then would only be then, *where* FreeBSD's MRSAS driver 
sends its event log?