hi, if I remove the SAS cable from the JBOD (connected through a SAS switch), all disks disappears, that's fine and expected, but if I reconnect the cable, the disks are not added anymore. I've red to use "camcontrol rescan all", but nothing happens. I don't get my /dev/da0-47 devices anymore and have to reboot the whole machine. What is my mistake? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110427/c182c65a/attachment.pgp
hi, Am Mittwoch, den 27.04.2011, 14:20 +0200 schrieb Denny Schierz:> I've red to use "camcontrol rescan all", but nothing happens. I don't > get my /dev/da0-47 devices anymore and have to reboot the whole machine.small update: I've tested it again and only some disks appear again, but not all. any suggestions? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110427/ad9eb09a/attachment.pgp
On Wed, Apr 27, 2011 at 02:20:34PM +0200, Denny Schierz wrote:> hi, > > if I remove the SAS cable from the JBOD (connected through a SAS > switch), all disks disappears, that's fine and expected, but if I > reconnect the cable, the disks are not added anymore. > I've red to use "camcontrol rescan all", but nothing happens. I don't > get my /dev/da0-47 devices anymore and have to reboot the whole machine. > > What is my mistake?Does resetting the bus then rescanning improve things at all? E.g. for bus 0 (use "all" only if it's safe): camcontrol reset 0 camcontrol rescan 0 I'm doubting it will. This is the first time I've heard of something called a SAS switch (looks like a SAS expander to me), but searching the Web indicates that most of these devices have a form of firmware/administrative interface on them, which means it's highly likely that the device itself is "getting in the way". Meaning: what's to say the issue is with FreeBSD and not with the SAS switch? To me, my first choice of action would be to contact the device vendor and ask what the behaviour should be. If there's some sort of device with an ASIC in between your disks and the controller (which is the case here), that could be responsible for what you're seeing. Or, if the hot-swap backplane has something like SES (not a typo) or a QLogic GEM chip on it, that could be causing some issues. (In the case of the latter on our SCSI systems at work, we change the physical SCSI cabling in our systems to remove the GEM from the bus entirely, otherwise it does odd things like tries to renumber the SCSI IDs on devices during a failure, and more often than not locks the entire SCSI controller up). Bottom line: less actual stuff between the disk and the controller the better. :-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |
On 27.04.2011 16:39, Denny Schierz wrote:> Am Mittwoch, den 27.04.2011, 05:57 -0700 schrieb Jeremy Chadwick: >> camcontrol reset 0 > > 0:22:0 is available, 0:46:0 not: > > root@iscsihead-m:~# camcontrol reset 0:22:0 > Reset of 0:22:0 was successful > > root@iscsihead-m:~# camcontrol reset 0:46:0 > camcontrol: cam_open_btl: no passthrough device found at 0:46:0You should reset whole bus, not the specific LUN. Full reset doesn't need that passthrough device. IIRC it works via xpt0.> We bought the LSI SAS6160 switch: > > http://www.lsi.com/storage_home/products_home/sas_switch/sas6100/index.html > > use the LSI 9200-8e hostbusadapter and LSI JBODs 630j. We had a lot of > e-mail conversation with LSI and they mean, that we need the switch for > a clear failover setup. > Also a reason for the switch: increase storage with more jbods. Every > jbod has his own cable to the (later) redundant switch. Otherwise we > have to build a "bus" from JBOD to JBOD to JBOD to JBOD to host ... bad > idea ;-) > > The question is, what does the driver while FreeBSD starts?CAM exactly does full bus reset and after few seconds full rescan. What controller driver may do except it depends on it alone. -- Alexander Motin