Good Morning! Since upgrading my home server from 10.3 to 11.0-RELEASE-p1 about a week ago, I have twice had a serious problem where my LSI adapter is having errors and dropping all the drives out of my ZFS pool. Hardware: - LSI SAS3081E-R PCI-E card with the IT firmware loaded - 6x2TB WD Black drives - 1 SSD - Supermicro X10SLL-F MB (not sure that is relevant) This system has been running with this exact hardware for about a year with no problems under the 10.X versions of FreeBSD. Last weekend, I upgraded the system to 11.0-RELEASE-p1. Since then, twice, all of the drives have been marked as unavailable to ZFS after generating a stream of errors. The problems start with a number of errors like this: Oct 26 03:28:29 rivendell kernel: mpt0: request 0xfffffe0000f73058:57643 timed out for ccb 0xfffff803456ea000 (req->ccb 0xfffff803456ea000) Oct 26 03:28:29 rivendell kernel: mpt0: attempting to abort req 0xfffffe0000f73058:57643 function 0 Oct 26 03:28:29 rivendell kernel: mpt0: completing timedout/aborted req 0xfffffe0000f73058:57643 Oct 26 03:28:29 rivendell kernel: (da0:mpt0:0:10:0): READ(10). CDB: 28 00 04 c4 91 c0 00 00 08 00 Oct 26 03:28:29 rivendell kernel: (da0:mpt0:0:10:0): CAM status: CCB request terminated by the host Oct 26 03:28:29 rivendell kernel: (da0:mpt0:0:10:0): mpt0: Retrying command Oct 26 03:28:29 rivendell kernel: abort of req 0xfffffe0000f73058:0 completed Oct 26 03:28:49 rivendell kernel: mpt0: request 0xfffffe0000f6c3b0:57658 timed out for ccb 0xfffff803456ea000 (req->ccb 0xfffff803456ea000) Oct 26 03:28:49 rivendell kernel: mpt0: attempting to abort req 0xfffffe0000f6c3b0:57658 function 0 Oct 26 03:28:49 rivendell kernel: mpt0: completing timedout/aborted req 0xfffffe0000f6c3b0:57658 Oct 26 03:28:49 rivendell kernel: (da0:mpt0:0:10:0): READ(10). CDB: 28 00 04 c4 91 c0 00 00 08 00 Oct 26 03:28:49 rivendell kernel: (da0:mpt0:0:10:0): CAM status: CCB request terminated by the host Oct 26 03:28:49 rivendell kernel: (da0:mpt0:0:10:0): Retrying command Oct 26 03:28:49 rivendell kernel: mpt0: abort of req 0xfffffe0000f6c3b0:0 completed Oct 26 03:28:51 rivendell kernel: (da0:mpt0:0:10:0): READ(10). CDB: 28 00 04 c4 91 c0 00 00 08 00 Oct 26 03:28:51 rivendell kernel: (da0:mpt0:0:10:0): CAM status: SCSI Status Error Oct 26 03:28:51 rivendell kernel: (da0:mpt0:0:10:0): SCSI status: Check Condition Oct 26 03:28:51 rivendell kernel: (da0:mpt0:0:10:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) Oct 26 03:28:51 rivendell kernel: (da0:mpt0:0:10:0): Retrying command (per sense data) Also these: Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): CAM status: SCSI Status Error Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): SCSI status: Check Condition Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): Error 6, Retries exhausted Oct 26 03:29:55 rivendell kernel: (da1:mpt0:0:14:0): Invalidating pack After a bunch of rounds of the errors above, I get this: Oct 26 03:35:17 rivendell kernel: mpt0: request 0xfffffe0000f73350:62027 timed out for ccb 0xfffff800160ce000 (req->ccb 0xfffff800160ce000) Oct 26 03:35:17 rivendell kernel: mpt0: attempting to abort req 0xfffffe0000f73350:62027 function 0 Oct 26 03:35:18 rivendell kernel: mpt0: mpt_wait_req(1) timed out Oct 26 03:35:18 rivendell kernel: mpt0: mpt_recover_commands: abort timed-out. Resetting controller Oct 26 03:35:18 rivendell kernel: mpt0: mpt_cam_event: 0x0 Oct 26 03:35:18 rivendell kernel: mpt0: mpt_cam_event: 0x0 Oct 26 03:35:18 rivendell kernel: mpt0: completing timedout/aborted req 0xfffffe0000f73350:62027 After which all the drives seem to disappear and the system detaches all of them: Oct 26 03:35:33 rivendell kernel: da1 at mpt0 bus 0 scbus0 target 14 lun 0 Oct 26 03:35:33 rivendell kernel: da1: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY01559141 detached Oct 26 03:35:33 rivendell kernel: da2 at mpt0 bus 0 scbus0 target 15 lun 0 Oct 26 03:35:33 rivendell kernel: da2: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY01603430 detached Oct 26 03:35:33 rivendell kernel: da5 at mpt0 bus 0 scbus0 target 18 lun 0 Oct 26 03:35:33 rivendell kernel: da5: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY01159727 detached Oct 26 03:35:33 rivendell kernel: da6 at mpt0 bus 0 scbus0 target 19 lun 0 Oct 26 03:35:33 rivendell kernel: da6: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY02971691 detached Oct 26 03:35:33 rivendell kernel: da4 at mpt0 bus 0 scbus0 target 17 lun 0 Oct 26 03:35:33 rivendell kernel: da4: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY01470856 detached Oct 26 03:35:33 rivendell kernel: da3 at mpt0 bus 0 scbus0 target 16 lun 0 Oct 26 03:35:33 rivendell kernel: da3: <ATA WDC WD2002FAEX-0 1D05> s/n WD-WMAY01602648 detached At this point I have had to reboot the server and then all the drives magically reappear. Any help would be greatly appreciated. -Jeremy -- Jeremy Beker - @gothmog http://www.confusticate.com Condensing fact from the vapor of nuance. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2200 bytes Desc: not available URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20161030/ea870ec7/attachment.bin>