zfs discuss - Mar 2009 - Mount ZFS hangs on boot

Hello,
I have an X4540 running 2008.11 snv_106
I rebooted it tonight since I had a hung iSCSI connection to the Sun
box that wouldn''t go away (couldn''t delete that particular ZFS
filesystem till the initiator drops connection)

Upon reboot, the system will hang after printing the license header. I
then rebooted again with ''-m milestone=none'' and then went to
single
user mode after that.
It appears "system/filesystem/usr:default" is the service that is
hanging during boot.
I''ve looked at ''zpool iostat'' and there is a
substantial amount of IO
happening after you initiate "system/filesystem/usr:default" but it
will it there for quite some time and not start.
I am unsure what it''s doing, zpool status shows 0 errors, all devices
normal (46 drives, in 5-6 disk RAIDZ groups).
I also loaded arcstat.pl (saw it floating around here) and its showing
~230 ops/sec, with 100% arc misses.
Whatever its doing, the load is very random I/O, and heavy, but little
progress appears to be happening.

I only have about 50 filesystems, and just a handful of snapshots for
each filesystem.

Thanks!

-- 
Brent Jones
brent at servuhome.net

On Wed, Mar 18, 2009 at 11:28 AM, Miles Nordin <carton at ivy.net>
wrote:
>>>>>> "bj" == Brent Jones <brent at
servuhome.net> writes:
>
> ? ?bj> I only have about 50 filesystems, and just a handful of
> ? ?bj> snapshots for each filesystem.
>
> there were earlier stories of people who had imports taking hours to
> complete with no feedback because ZFS was rolling forward some
> partly-completed operation interrupted by the crash, like destroying a
> snapshot or something. ?maybe you shoudl just wait.
>
Wait I did, and it did finally come up.
A partially completed operation may make sense, as when the iSCSI
target was block due to a Windows box hanging, and the connection not
letting go, a ZFS destroy on that pool never did complete.
So maybe it tried to finish that action.

A mystery for sure, but its up and working now.

-- 
Brent Jones
brent at servuhome.net