Eugene M. Zheganin
2017-Aug-03 08:54 UTC
panic: dva_get_dsize_sync(): bad DVA on 2016 11-STABLE.
Hi,
today I got the following panic on the December 2016 11-STABLE:
FreeBSD san02.playkey.net 11.0-STABLE FreeBSD 11.0-STABLE #0 r310734M:
Thu Dec 29 19:22:30 UTC 2016 emz at san02:/usr/obj/usr/src/sys/GENERIC amd64
panic: dva_get_dsize_sync(): bad DVA 4294967295:2086400
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
panic: dva_get_dsize_sync(): bad DVA 4294967295:2086400
cpuid = 2
KDB: stack backtrace:
#0 0xffffffff80b023a7 at kdb_backtrace+0x67
#1 0xffffffff80ab88e6 at vpanic+0x186
#2 0xffffffff80ab8753 at panic+0x43
#3 0xffffffff8226a148 at bp_get_dsize+0x128
#4 0xffffffff8222cc29 at dmu_tx_count_write+0x589
#5 0xffffffff8222c675 at dmu_tx_hold_write+0x35
#6 0xffffffff822c573e at zvol_strategy+0x21e
#7 0xffffffff809f6a10 at g_io_request+0x4a0
#8 0xffffffff8283a45f at ctl_be_block_dispatch_dev+0x20f
#9 0xffffffff8283bddc at ctl_be_block_worker+0x6c
#10 0xffffffff80b1484a at taskqueue_run_locked+0x14a
#11 0xffffffff80b15a38 at taskqueue_thread_loop+0xe8
#12 0xffffffff80a70785 at fork_exit+0x85
#13 0xffffffff80f55f2e at fork_trampoline+0xe
Uptime: 78d7h43m31s
My question is (since I din't find much on this) what does this "bad
DVA" mean ? I've read that this may indicate the on-disk zfs corrupton,
but I'm not suer about it. Is this fixable in any way ? Do I have to
prepare to recreate the pool (btw I have three pools) from scratch, and
how do I determine which one has the corruption.
Some [useless ?] zfs info:
# zpool status
pool: data
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
da8 ONLINE 0 0 0
da9 ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
da10 ONLINE 0 0 0
da11 ONLINE 0 0 0
da12 ONLINE 0 0 0
da13 ONLINE 0 0 0
raidz1-3 ONLINE 0 0 0
da14 ONLINE 0 0 0
da15 ONLINE 0 0 0
da16 ONLINE 0 0 0
da17 ONLINE 0 0 0
errors: No known data errors
pool: userdata
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
userdata ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gpt/userdata0 ONLINE 0 0 0
gpt/userdata1 ONLINE 0 0 0
errors: No known data errors
pool: zroot
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gpt/zroot0 ONLINE 0 0 0
gpt/zroot1 ONLINE 0 0 0
errors: No known data errors
Hardware:
# camcontrol devlist
<AHCI SGPIO Enclosure 1.00 0001> at scbus4 target 0 lun 0 (pass0,ses0)
<AHCI SGPIO Enclosure 1.00 0001> at scbus11 target 0 lun 0 (pass1,ses1)
<ATA HGST HUS722T1TAL WA05> at scbus12 target 4 lun 0 (pass2,da0)
<ATA HGST HUS722T1TAL WA05> at scbus12 target 5 lun 0 (pass3,da1)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 8 lun 0 (pass4,da2)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 9 lun 0 (pass5,da3)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 10 lun 0 (pass6,da4)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 11 lun 0 (pass7,da5)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 12 lun 0 (pass8,da6)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 13 lun 0 (pass9,da7)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 14 lun 0 (pass10,da8)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 15 lun 0 (pass11,da9)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 16 lun 0
(pass12,da10)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 17 lun 0
(pass13,da11)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 18 lun 0
(pass14,da12)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 19 lun 0
(pass15,da13)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 20 lun 0
(pass16,da14)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 21 lun 0
(pass17,da15)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 22 lun 0
(pass18,da16)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 23 lun 0
(pass19,da17)
<ATA TOSHIBA THNSN81Q 6101> at scbus12 target 24 lun 0
(pass20,da18)
<LSI SAS3x40 0601> at scbus12 target 32 lun 0
(pass21,ses2)
I also have the zdb -uuumdC for each pool, here they are in case someone
needs them:
https://enaza.ru/stub-data/zdb-uuumdC-data.txt
https://enaza.ru/stub-data/zdb-uuumdC-userdata.txt
https://enaza.ru/stub-data/zdb-uuumdC-zroot.txt
Thanks.
Eugene.