Mike Gerdts
2008-Jun-01 01:48 UTC
[zfs-discuss] panic: avl_find() succeeded inside avl_add()
I just experienced a zfs-related crash. I have filed a bug (don''t know number - grumble). I have a crash dump but little free space. If someone would like some more info from the core, please let me know in the next few days.> ::statusdebugging crash dump /pool0/vmcore.0 (64-bit) from sun operating system: 5.11 snv_76 (sun4u) panic message: avl_find() succeeded inside avl_add() dump content: kernel pages only> ::stackvpanic(128d918, 3000c1daab0, 2a101673418, 0, 3000b6a3770, 1229000) avl_add+0x38(300106ee398, 3000c1daab0, 649e74000000000, 3001b377980, 80000000000271d6, 128d800) mzap_open+0x18c(cf, 300106ee388, 3000c94caa0, 3001b377980, 300106ee370, 300106ee358) zap_lockdir+0x54(300039bce68, 26b32, 0, 0, 1, 2a1016738f8) zap_cursor_retrieve+0x40(2a1016738f0, 2a1016737d8, 0, 1, 2a1016738f0, 2) zfs_readdir+0x224(3, 2a101673aa0, 3000dfc7980, 2, 2000, 2a1016737f0) fop_readdir+0x44(3000df541c0, 2a101673aa0, 3000cb58dc8, 2a101673a9c, 2000, 111dd48) getdents64+0x90(8, 2a101673ad0, 2000, 2004, 3001e54cac8, ff0b0000) syscall_trap32+0xcc(8, ff0f4000, 2000, 2004, 0, ff0b0000) # zpool status pool0 pool: pool0 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c0t1d0s7 ONLINE 0 0 0 c0t0d0s7 ONLINE 0 0 0 errors: No known data errors # zpool get all pool0 NAME PROPERTY VALUE SOURCE pool0 size 27.2G - pool0 used 24.9G - pool0 available 2.38G - pool0 capacity 91% - pool0 altroot - default pool0 health ONLINE - pool0 guid 8395455814253440113 - pool0 version 8 default pool0 bootfs - default pool0 delegation on default pool0 autoreplace off default pool0 temporary off default -- Mike Gerdts http://mgerdts.blogspot.com/
Mike Gerdts
2008-Jun-01 02:38 UTC
[zfs-discuss] panic: avl_find() succeeded inside avl_add()
On Sat, May 31, 2008 at 8:48 PM, Mike Gerdts <mgerdts at gmail.com> wrote:> I just experienced a zfs-related crash. I have filed a bug (don''t > know number - grumble). I have a crash dump but little free space. If > someone would like some more info from the core, please let me know in > the next few days.And I am able to reproduce...>From a fresh crash:> ::statusdebugging crash dump vmcore.6 (64-bit) from sun operating system: 5.11 snv_76 (sun4u) panic message: avl_find() succeeded inside avl_add() dump content: kernel pages only> ::stackvpanic(128d918, 30011ba6638, 2a101594fb8, 0, 30011ba6868, 1229000) avl_add+0x38(30011bad320, 30011ba6638, 649e74000000000, 3000d2d8180, 80000000000271d6, 128d800) mzap_open+0x18c(cf, 30011bad310, 30011b2b480, 3000d2d8180, 30011bad2f8, 30011bad2e0) zap_lockdir+0x54(30004910c08, 26b32, 0, 0, 1, 2a1015951e8) zap_lookup+0x18(30004910c08, 26b32, 2a101595680, 8, 1, 2a1015952a8) zfs_dirent_lock+0x2f8(2a101595370, 3000b859518, 2a101595680, 2a101595378, 6, 4) zfs_dirlook+0x19c(3000b859518, 2a101595680, 2a101595678, 2a101595680, 0, 0) zfs_lookup+0x188(3000b855d00, 2a101595680, 2a101595678, 2a101595940, 0, 30004c32440) fop_lookup+0x4c(3000b855d00, 2a101595680, 2a101595678, 2a101595940, 0, 3000101fa40) lookuppnvp+0x324(2a101595940, 0, 0, 3000b855d00, 30008c61b70, 3000101fa40) lookuppnat+0x10c(3000c864600, 0, 0, 0, 2a101595ad8, 0) lookupnameat+0x5c(c461c, 0, 0, 0, 2a101595ad8, 0) cstatat_getvp+0x16c(18bd000, c461c, 1, 0, 2a101595ad8, 0) cstatat64_32+0x58(ffffffffffd19553, c461c, 1, ffbfbcc0, 1000, 0) syscall_trap32+0xcc(c461c, ffbfbcc0, c462c, 0, ff000000, 80808080)> 3000c864600::print vnode_t{ ... v_path = 0x3000c837458 "/ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix" $ ls /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix Makefile debug64/ $ find /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/.make.state.lock /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/debug64 <panic> -- Mike Gerdts http://mgerdts.blogspot.com/
Victor Latushkin
2008-Jun-01 04:23 UTC
[zfs-discuss] panic: avl_find() succeeded inside avl_add()
Mike Gerdts ?????:> On Sat, May 31, 2008 at 8:48 PM, Mike Gerdts <mgerdts at gmail.com> wrote: >> I just experienced a zfs-related crash. I have filed a bug (don''t >> know number - grumble).It''s number is 6709336. I added you second e-mail to the description. Wbr, Victor>> I have a crash dump but little free space. If >> someone would like some more info from the core, please let me know in >> the next few days. > > And I am able to reproduce... > >>From a fresh crash: > >> ::status > debugging crash dump vmcore.6 (64-bit) from sun > operating system: 5.11 snv_76 (sun4u) > panic message: avl_find() succeeded inside avl_add() > dump content: kernel pages only >> ::stack > vpanic(128d918, 30011ba6638, 2a101594fb8, 0, 30011ba6868, 1229000) > avl_add+0x38(30011bad320, 30011ba6638, 649e74000000000, 3000d2d8180, > 80000000000271d6, 128d800) > mzap_open+0x18c(cf, 30011bad310, 30011b2b480, 3000d2d8180, 30011bad2f8, > 30011bad2e0) > zap_lockdir+0x54(30004910c08, 26b32, 0, 0, 1, 2a1015951e8) > zap_lookup+0x18(30004910c08, 26b32, 2a101595680, 8, 1, 2a1015952a8) > zfs_dirent_lock+0x2f8(2a101595370, 3000b859518, 2a101595680, 2a101595378, 6, 4) > zfs_dirlook+0x19c(3000b859518, 2a101595680, 2a101595678, 2a101595680, 0, 0) > zfs_lookup+0x188(3000b855d00, 2a101595680, 2a101595678, 2a101595940, 0, > 30004c32440) > fop_lookup+0x4c(3000b855d00, 2a101595680, 2a101595678, 2a101595940, 0, > 3000101fa40) > lookuppnvp+0x324(2a101595940, 0, 0, 3000b855d00, 30008c61b70, 3000101fa40) > lookuppnat+0x10c(3000c864600, 0, 0, 0, 2a101595ad8, 0) > lookupnameat+0x5c(c461c, 0, 0, 0, 2a101595ad8, 0) > cstatat_getvp+0x16c(18bd000, c461c, 1, 0, 2a101595ad8, 0) > cstatat64_32+0x58(ffffffffffd19553, c461c, 1, ffbfbcc0, 1000, 0) > syscall_trap32+0xcc(c461c, ffbfbcc0, c462c, 0, ff000000, 80808080) > >> 3000c864600::print vnode_t > { > ... > v_path = 0x3000c837458 > "/ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix" > > $ ls /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix > Makefile debug64/ > > $ find /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/.make.state.lock > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/debug64 > <panic>
Mike Gerdts
2008-Jun-01 13:03 UTC
[zfs-discuss] panic: avl_find() succeeded inside avl_add()
On Sat, May 31, 2008 at 9:38 PM, Mike Gerdts <mgerdts at gmail.com> wrote:> $ find /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/.make.state.lock > /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/debug64 > <panic>The stack from this one is...> ::stackvpanic(128d918, 300093c3778, 2a1010c7418, 0, 300093c39a8, 1229000) avl_add+0x38(300091da548, 300093c3778, 649e74000000000, 30005f1a180, 80000000000271d6, 128d800) mzap_open+0x18c(cf, 300091da538, 300091df998, 30005f1a180, 300091da520, 300091da508) zap_lockdir+0x54(30003ac6b88, 26b32, 0, 0, 1, 2a1010c78f8) zap_cursor_retrieve+0x40(2a1010c78f0, 2a1010c77d8, 0, 1, 2a1010c78f0, 2) zfs_readdir+0x224(3, 2a1010c7aa0, 30009173308, 2, 2000, 2a1010c77f0) fop_readdir+0x44(300091fe940, 2a1010c7aa0, 30005f403b0, 2a1010c7a9c, 2000, 111dd48) getdents64+0x90(4, 2a1010c7ad0, 2000, 0, 30008245dd0, 0) syscall_trap32+0xcc(4, ff1a0000, 2000, 0, 0, 0) It tripped up on:> 300091fe940::print vnode_t v_pathv_path = 0x300082608c0 " /ws/mount/onnv-gate/usr/src/uts/sun4u/serengeti/unix/debug64" Which is a subdirectory of where it tripped up before. I am able to do "find /ws/mount -name serengeti -prune" without problems. To make it so that I can hopefully proceed with the build I have moved the directory out of the way, then did an "hg update" so that I can hopefully get the build I was trying to do to complete. -- Mike Gerdts http://mgerdts.blogspot.com/