zfs discuss - Apr 2008 - zfs snapshot -r hangs

Greetings,

snv_79a
AMD 64x2 in 64 bit kernel mode.

I''m in the middle of migrating a large zfs set from a pair of 1TB
mirrors
to a 1.3TB RAIDz.

I decided to use zfs send | zfs receive, so the first order of business 
was to snap the entire source filesystem.  

# zfs snapshot -r store at now

What happened was expected, the source drives flashed and wiggled :)
What happened next was not, the destination drives (or maybe the boot 
drive, as they share one disk activity light) began flashing and wiggling, 
and have been doing so for 12 hours how.

iostat shows no activity to speak of, and no transfers at all on any of the 
disks.  ditto for zpool iostat.

all zfs commands hang, and the lack of output from truss''ing the pids 
indicate they are stuck in the kernel.  Heck, I can''t even reboot, as
that
hangs.

So what I was wondering whether there exists a dtrace recipe or some 
such that I can use to figure out where this is hung in the kernel.

Cheers!
-sam
 
 
This message posted from opensolaris.org

Sam Nicholson wrote:
> Greetings,
> 
> snv_79a
> AMD 64x2 in 64 bit kernel mode.
> 
> I''m in the middle of migrating a large zfs set from a pair of 1TB
mirrors
> to a 1.3TB RAIDz.
> 
> I decided to use zfs send | zfs receive, so the first order of business 
> was to snap the entire source filesystem.  
> 
> # zfs snapshot -r store at now
> 
> What happened was expected, the source drives flashed and wiggled :)
> What happened next was not, the destination drives (or maybe the boot 
> drive, as they share one disk activity light) began flashing and wiggling, 
> and have been doing so for 12 hours how.
> 
> iostat shows no activity to speak of, and no transfers at all on any of the
> disks.  ditto for zpool iostat.
> 
> all zfs commands hang, and the lack of output from truss''ing the
pids
> indicate they are stuck in the kernel.  Heck, I can''t even reboot,
as that
> hangs.
> 
> So what I was wondering whether there exists a dtrace recipe or some 
> such that I can use to figure out where this is hung in the kernel.
> 
> Cheers!
> -sam
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Hi
echo "::walk thread|::findstack!munges" |mdb -k > sometestfile.txt

where munges is the script I have attached ( courtesy of David Powell I believe
), ie
place munges somewhere on your path, and run above.

This text file might be large ( most likely will be, but the munges bit will
trim it down
sufficiently ), so examine it and see if there are any zfs related stuff in
there.

That might be sufficient to get an idea of where zfs is stuck, else might need
the entire
text file.
Assuming that this actually works ( seen as reboot is apparently even stuck )

Enda
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: munges
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080416/13ffec0e/attachment.ksh>