On Sat, 3 Mar 2018, tech-lists wrote:> On 03/03/2018 00:23, Dimitry Andric wrote: >> Indeed. I have had the following for a few years now, due to USB drives >> with ZFS pools: >> >> --- /usr/src/etc/rc.d/zfs 2016-11-08 10:21:29.820131000 +0100 >> +++ /etc/rc.d/zfs 2016-11-08 12:49:52.971161000 +0100 >> @@ -25,6 +25,8 @@ >> >> zfs_start_main() >> { >> + echo "Sleeping for 10 seconds to let USB devices settle..." >> + sleep 10 >> zfs mount -va >> zfs share -a >> if [ ! -r /etc/zfs/exports ]; then >> >> For some reason, USB3 (xhci) controllers can take a very, very long time >> to correctly attach mass storage devices: I usually see many timeouts >> before they finally get detected. After that, the devices always work >> just fine, though.I have one that works for an old USB hard drive but never works for a not so old USB flash drive and a new SSD in a USB dock (just to check the SSD speed when handicapped by USB). Win7 has no problems with the xhci and USB flash drive combination, and FreeBSD has no problems with the drive on other systems.>> Whether this is due to some sort of BIOS handover trouble, or due to >> cheap and/or crappy USB-to-SATA bridges (even with brand WD and Seagate >> disks!), I have no idea. I attempted to debug it at some point, but >> a well-placed "sleep 10" was an acceptable workaround... :) > > That fixed it, thank you again :DThat won't work for the boot drive. When no boot drive is detected early enough, the kernel goes to the mountroot prompt. That seems to hold a Giant lock which inhibits further progress being made. Sometimes progress can be made by trying to mount unmountable partitions on other drives, but this usually goes too fast, especially if the USB drive often times out. Bruce
On 03/03/2018 12:56, Bruce Evans wrote:> That won't work for the boot drive.In my case the workaround is fine because it's not a boot drive -- J.
03.03.2018 19:56, Bruce Evans wrote:> On Sat, 3 Mar 2018, tech-lists wrote: > >> On 03/03/2018 00:23, Dimitry Andric wrote: >>> Indeed. I have had the following for a few years now, due to USB drives >>> with ZFS pools: >>> >>> --- /usr/src/etc/rc.d/zfs 2016-11-08 10:21:29.820131000 +0100 >>> +++ /etc/rc.d/zfs 2016-11-08 12:49:52.971161000 +0100 >>> @@ -25,6 +25,8 @@ >>> >>> zfs_start_main() >>> { >>> + echo "Sleeping for 10 seconds to let USB devices settle..." >>> + sleep 10 >>> zfs mount -va >>> zfs share -a >>> if [ ! -r /etc/zfs/exports ]; then >>> >>> For some reason, USB3 (xhci) controllers can take a very, very long time >>> to correctly attach mass storage devices: I usually see many timeouts >>> before they finally get detected. After that, the devices always work >>> just fine, though. > > I have one that works for an old USB hard drive but never works for a not > so old USB flash drive and a new SSD in a USB dock (just to check the SSD > speed when handicapped by USB). Win7 has no problems with the xhci and > USB flash drive combination, and FreeBSD has no problems with the drive > on other systems. > >>> Whether this is due to some sort of BIOS handover trouble, or due to >>> cheap and/or crappy USB-to-SATA bridges (even with brand WD and Seagate >>> disks!), I have no idea. I attempted to debug it at some point, but >>> a well-placed "sleep 10" was an acceptable workaround... :) >> >> That fixed it, thank you again :D > > That won't work for the boot drive. > > When no boot drive is detected early enough, the kernel goes to the > mountroot prompt. That seems to hold a Giant lock which inhibits > further progress being made. Sometimes progress can be made by trying > to mount unmountable partitions on other drives, but this usually goes > too fast, especially if the USB drive often times out.In fact, we have enough loader.conf quirks for that: kern.cam.boot_delay "Bus registration wait time" # miliseconds vfs.mountroot.timeout "Wait for root mount" # seconds vfs.root_mount_always_wait "Wait for root mount holds even if the root device already exists" # boolean No need in extra hacks to zfs rc.d script.
> That won't work for the boot drive. > > When no boot drive is detected early enough, the kernel goes to the > mountroot prompt.? That seems to hold a Giant lock which inhibits > further progress being made.? Sometimes progress can be made by trying > to mount unmountable partitions on other drives, but this usually goes > too fast, especially if the USB drive often times out.We have this problem in Azure with a ZFS root, was fixed by the pacth in this bug report, which actually starts off being about USB. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208882 You can then set the mountroot timeout as normal and it works. I wold really like this patch to be applied, but it seems to have languished since last summer. We use this as standard on all our cloud machines now, and it works very nicely. -pete.
On 3 Mar 2018, at 13:56, Bruce Evans <brde at optusnet.com.au> wrote:> > On Sat, 3 Mar 2018, tech-lists wrote: >> On 03/03/2018 00:23, Dimitry Andric wrote:...>>> Whether this is due to some sort of BIOS handover trouble, or due to >>> cheap and/or crappy USB-to-SATA bridges (even with brand WD and Seagate >>> disks!), I have no idea. I attempted to debug it at some point, but >>> a well-placed "sleep 10" was an acceptable workaround... :) >> >> That fixed it, thank you again :D > > That won't work for the boot drive. > > When no boot drive is detected early enough, the kernel goes to the > mountroot prompt. That seems to hold a Giant lock which inhibits > further progress being made. Sometimes progress can be made by trying > to mount unmountable partitions on other drives, but this usually goes > too fast, especially if the USB drive often times out.What I would like to know, is why our USB stack has such timeout issues at all. When I boot Linux on the same type of hardware, I never see USB timeouts. They must be doing something right, or maybe they just don't bother checking some status bits that we are very strict about? -Dimitry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 223 bytes Desc: Message signed with OpenPGP URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20180305/96728abd/attachment.sig>