Hi there, I?ve recently experienced two kernel panics on 8.4-RELEASE (within 2 days of each other, and both around the same time of day oddly) with ZFS. Sorry no dump available, but panic below. Any ideas where to start solving this? Will upgrading to 9 (or 10) solve it? (Sorry for any errors, hand transcribed from a screenshot) panic:solaris asset: sa.sa_magic == 0x2F505A (0x5112fb3d == 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 596 cpuid = 1 KDB: stack backtrace: #0 0xffffffffff8066cb96 at kdb_backtrace+0x66 #1 0xffffffffff8063925e at panic+0x1ce #2 0xffffffffff81292109 at assfail3+0x29 #3 0xffffffffff811157fe at zfs_space_delta_cb+0xbe #4 0xffffffffff8109ee12 at dmu_objset_userquota_get_ids+0x142 #5 0xffffffffff810a5e35 at dnode_sync+0xc5 #6 0xffffffffff8109cf3d at dmu_objset_sync_dnodes+0x5d #7 0xfffffffff8109d0c9 at dmu_objset_sync+0x169 #8 0xffffffffff810b446a at dsl_pool_sync+0xca #9 0xffffffffff810c4c3a at spa_sync+0x34a #10 0xffffffffff810d6959 at txg_sync_thread+0x139 #11 0xffffffffff8060dc7f at fork_exit+0x11f #12 0xffffffffff809a9c1e at fork_trampoline+0xe Uptime: 37d8h9m39s Cannot dump. Device not defined or unavailable Automatic reboot in 15 seconds - press a key on the console to abort (Never actually reboots) ZFS setup: [root at bellagio ~]# zpool status pool: spool state: ONLINE scan: scrub repaired 0 in 13h18m with 0 errors on Wed Mar 12 19:14:08 2014 config: NAME STATE READ WRITE CKSUM spool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 errors: No known data errors [root at bellagio ~]# zfs get all spool NAME PROPERTY VALUE SOURCE spool type filesystem - spool creation Fri Feb 25 19:45 2011 - spool used 692G - spool available 222G - spool referenced 692G - spool compressratio 1.00x - spool mounted yes - spool quota none default spool reservation none default spool recordsize 128K default spool mountpoint /var/spool/imap local spool sharenfs off default spool checksum on default spool compression off local spool atime off local spool devices on default spool exec on default spool setuid on default spool readonly off default spool jailed off default spool snapdir hidden default spool aclmode discard default spool aclinherit restricted default spool canmount on default spool xattr off temporary spool copies 1 default spool version 5 - spool utf8only off - spool normalization none - spool casesensitivity sensitive - spool vscan off default spool nbmand off default spool sharesmb off default spool refquota none default spool refreservation none default spool primarycache all default spool secondarycache all default spool usedbysnapshots 0 - spool usedbydataset 692G - spool usedbychildren 114M - spool usedbyrefreservation 0 - spool logbias latency default spool dedup off default spool mlslabel - spool sync standard default spool refcompressratio 1.00x - spool written 692G - Cheers Phil
on 11/04/2014 11:02 Phil Murray said the following:> Hi there, > > I?ve recently experienced two kernel panics on 8.4-RELEASE (within 2 days of each other, and both around the same time of day oddly) with ZFS. Sorry no dump available, but panic below. > > Any ideas where to start solving this? Will upgrading to 9 (or 10) solve it?By chance, could the system be running zfs recv at the times when the panics happened?> (Sorry for any errors, hand transcribed from a screenshot) > > panic:solaris asset: sa.sa_magic == 0x2F505A (0x5112fb3d == 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 596 > cpuid = 1 > KDB: stack backtrace: > #0 0xffffffffff8066cb96 at kdb_backtrace+0x66 > #1 0xffffffffff8063925e at panic+0x1ce > #2 0xffffffffff81292109 at assfail3+0x29 > #3 0xffffffffff811157fe at zfs_space_delta_cb+0xbe > #4 0xffffffffff8109ee12 at dmu_objset_userquota_get_ids+0x142 > #5 0xffffffffff810a5e35 at dnode_sync+0xc5 > #6 0xffffffffff8109cf3d at dmu_objset_sync_dnodes+0x5d > #7 0xfffffffff8109d0c9 at dmu_objset_sync+0x169 > #8 0xffffffffff810b446a at dsl_pool_sync+0xca > #9 0xffffffffff810c4c3a at spa_sync+0x34a > #10 0xffffffffff810d6959 at txg_sync_thread+0x139 > #11 0xffffffffff8060dc7f at fork_exit+0x11f > #12 0xffffffffff809a9c1e at fork_trampoline+0xe > Uptime: 37d8h9m39s > Cannot dump. Device not defined or unavailable > Automatic reboot in 15 seconds - press a key on the console to abort > > (Never actually reboots) > > > ZFS setup: > > [root at bellagio ~]# zpool status > pool: spool > state: ONLINE > scan: scrub repaired 0 in 13h18m with 0 errors on Wed Mar 12 19:14:08 2014 > config: > > NAME STATE READ WRITE CKSUM > spool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > > errors: No known data errors > > [root at bellagio ~]# zfs get all spool > NAME PROPERTY VALUE SOURCE > spool type filesystem - > spool creation Fri Feb 25 19:45 2011 - > spool used 692G - > spool available 222G - > spool referenced 692G - > spool compressratio 1.00x - > spool mounted yes - > spool quota none default > spool reservation none default > spool recordsize 128K default > spool mountpoint /var/spool/imap local > spool sharenfs off default > spool checksum on default > spool compression off local > spool atime off local > spool devices on default > spool exec on default > spool setuid on default > spool readonly off default > spool jailed off default > spool snapdir hidden default > spool aclmode discard default > spool aclinherit restricted default > spool canmount on default > spool xattr off temporary > spool copies 1 default > spool version 5 - > spool utf8only off - > spool normalization none - > spool casesensitivity sensitive - > spool vscan off default > spool nbmand off default > spool sharesmb off default > spool refquota none default > spool refreservation none default > spool primarycache all default > spool secondarycache all default > spool usedbysnapshots 0 - > spool usedbydataset 692G - > spool usedbychildren 114M - > spool usedbyrefreservation 0 - > spool logbias latency default > spool dedup off default > spool mlslabel - > spool sync standard default > spool refcompressratio 1.00x - > spool written 692G - > > > > Cheers > > Phil > > _______________________________________________ > freebsd-stable at freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org" >-- Andriy Gapon
Odd is that the assertion is failing only because of case sensitivity: panic:solaris asset: sa.sa_magic == 0x2F505A (0x5112fb3d == 0x2f505a) On 11 April 2014 09:02, Phil Murray <pmurray at nevada.net.nz> wrote:> Hi there, > > I've recently experienced two kernel panics on 8.4-RELEASE (within 2 days > of each other, and both around the same time of day oddly) with ZFS. Sorry > no dump available, but panic below. > > Any ideas where to start solving this? Will upgrading to 9 (or 10) solve > it? > > (Sorry for any errors, hand transcribed from a screenshot) > > panic:solaris asset: sa.sa_magic == 0x2F505A (0x5112fb3d == 0x2f505a), > file: > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, > line: 596 > cpuid = 1 > KDB: stack backtrace: > #0 0xffffffffff8066cb96 at kdb_backtrace+0x66 > #1 0xffffffffff8063925e at panic+0x1ce > #2 0xffffffffff81292109 at assfail3+0x29 > #3 0xffffffffff811157fe at zfs_space_delta_cb+0xbe > #4 0xffffffffff8109ee12 at dmu_objset_userquota_get_ids+0x142 > #5 0xffffffffff810a5e35 at dnode_sync+0xc5 > #6 0xffffffffff8109cf3d at dmu_objset_sync_dnodes+0x5d > #7 0xfffffffff8109d0c9 at dmu_objset_sync+0x169 > #8 0xffffffffff810b446a at dsl_pool_sync+0xca > #9 0xffffffffff810c4c3a at spa_sync+0x34a > #10 0xffffffffff810d6959 at txg_sync_thread+0x139 > #11 0xffffffffff8060dc7f at fork_exit+0x11f > #12 0xffffffffff809a9c1e at fork_trampoline+0xe > Uptime: 37d8h9m39s > Cannot dump. Device not defined or unavailable > Automatic reboot in 15 seconds - press a key on the console to abort > > (Never actually reboots) > > > ZFS setup: > > [root at bellagio ~]# zpool status > pool: spool > state: ONLINE > scan: scrub repaired 0 in 13h18m with 0 errors on Wed Mar 12 19:14:08 > 2014 > config: > > NAME STATE READ WRITE CKSUM > spool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > > errors: No known data errors > > [root at bellagio ~]# zfs get all spool > NAME PROPERTY VALUE SOURCE > spool type filesystem - > spool creation Fri Feb 25 19:45 2011 - > spool used 692G - > spool available 222G - > spool referenced 692G - > spool compressratio 1.00x - > spool mounted yes - > spool quota none default > spool reservation none default > spool recordsize 128K default > spool mountpoint /var/spool/imap local > spool sharenfs off default > spool checksum on default > spool compression off local > spool atime off local > spool devices on default > spool exec on default > spool setuid on default > spool readonly off default > spool jailed off default > spool snapdir hidden default > spool aclmode discard default > spool aclinherit restricted default > spool canmount on default > spool xattr off temporary > spool copies 1 default > spool version 5 - > spool utf8only off - > spool normalization none - > spool casesensitivity sensitive - > spool vscan off default > spool nbmand off default > spool sharesmb off default > spool refquota none default > spool refreservation none default > spool primarycache all default > spool secondarycache all default > spool usedbysnapshots 0 - > spool usedbydataset 692G - > spool usedbychildren 114M - > spool usedbyrefreservation 0 - > spool logbias latency default > spool dedup off default > spool mlslabel - > spool sync standard default > spool refcompressratio 1.00x - > spool written 692G - > > > > Cheers > > Phil > > _______________________________________________ > freebsd-stable at freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org" >
Phil Murray
2014-Apr-15 05:39 UTC
Re: Panic in ZFS, solaris assert: sa.sa_magic == 0x2F505A
On 11/04/2014, at 10:36 pm, Andriy Gapon <avg@FreeBSD.org> wrote:> on 11/04/2014 11:02 Phil Murray said the following: >> Hi there, >> >> I’ve recently experienced two kernel panics on 8.4-RELEASE (within 2 days of each other, and both around the same time of day oddly) with ZFS. Sorry no dump available, but panic below. >> >> Any ideas where to start solving this? Will upgrading to 9 (or 10) solve it? > > By chance, could the system be running zfs recv at the times when the panics > happened?I think it might be related to this bug reported on ZFS-on-linux when upgrading from v3 -> v5, which is exactly what I’ve done on this machine: https://github.com/zfsonlinux/zfs/issues/2025 In my case, the bogus sa.sa_magic value looks like this: panic:solaris asset: sa.sa_magic == 0x2F505A (0x5112fb3d == 0x2f505a), file: $ date -r 0x5112fb3d Thu Feb 7 13:54:21 NZDT 2013 Cheers Phil _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"