On 7/20/2016 16:32, Andriy Gapon wrote:> On 20/07/2016 17:26, Karl Denninger wrote:
>> #185 0xffffffff8222955d in zio_vdev_io_start (zio=0xfffff80108754770)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2870
>> #186 0xffffffff82225d96 in zio_execute (zio=<value optimized
out>)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1556
>> #187 0xffffffff8220c72d in vdev_queue_io_done (zio=<value optimized
out>)
>> at
>>
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:887
>> #188 0xffffffff82229719 in zio_vdev_io_done (zio=0xfffff804cfdf7770)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2895
>> #189 0xffffffff82225d96 in zio_execute (zio=<value optimized
out>)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1556
>> #190 0xffffffff8222955d in zio_vdev_io_start (zio=0xfffff804cfdf7770)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2870
>> #191 0xffffffff82225d96 in zio_execute (zio=<value optimized
out>)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1556
>> #192 0xffffffff8220c72d in vdev_queue_io_done (zio=<value optimized
out>)
>> at
>>
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:887
>> #193 0xffffffff82229719 in zio_vdev_io_done (zio=0xfffff801469a13b8)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2895
>> #194 0xffffffff82225d96 in zio_execute (zio=<value optimized
out>)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1556
>> #195 0xffffffff8222955d in zio_vdev_io_start (zio=0xfffff801469a13b8)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2870
>> #196 0xffffffff82225d96 in zio_execute (zio=<value optimized
out>)
>> at
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1556
>> #197 0xffffffff8220c72d in vdev_queue_io_done (zio=<value optimized
out>)
>> at
>>
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:887
>> #198 0xffffffff82229719 in zio_vdev_io_done (zio=0xfffff805dcd9d3b8)
> I see how this pseudo-recursion can happen in theory, but not why it has
> happened. It would interesting to see 3 or 4 of zio-s printed,
> including the top-most one at 0xfffff805dcd9d3b8.
>
> Something like:
>
> set print pretty
> p *zio
> p *zio->io_vd
> p *zio->io_vd->vdev_ops
>
> in several frames.
>
There was a clone from a previous boot environment embedded in the
snapshot list, but I don't believe it was (directly) the cause as
several backups have taken place since it was taken, and none resulted
in the panic.
I had this problem some time ago in which on the root pool/filesystem,
and ONLY on the root, I would get a panic after several days uptime
*and* the disk structure itself was ok but the incore was not. It would
manifest as a snapshot you could "cd" into but not see anything once
inside, and an attempt to destroy it would fail. If you attempted a zfs
send that touched that snapshot you got a panic.
The PR that I referenced above apparently was related, and the patch
that came out of it appears to have stopped it on 10.2, as once I put
that on I didn't have it happen again. It was consistent enough that I
actually killed automated backups on the machines involved, as it would
nail you without warning on a roughly a one-week interval.
That appears to be roughly where I am again.... about one week into
running BETA1, and "boom". But this one IS different in that this
time
an immediate repeat (which used to succeed as the disk was fine)
resulted in an immediate second panic.
I've removed the snapshots on that filesystem (on the theory that the
previous error wasn't really fixed but rather papered over) and run a
backup, which succeeded. If I get another one I'll know it's a new
problem.
Let me see what I can come up with on some frame dumps.
Here's the top one...
(kgdb) p *zio
$5 = {
io_bookmark = {
zb_objset = 0,
zb_object = 0,
zb_level = 0,
zb_blkid = 0
},
io_prop = {
zp_checksum = ZIO_CHECKSUM_INHERIT,
zp_compress = ZIO_COMPRESS_INHERIT,
zp_type = DMU_OT_NONE,
zp_level = 0 '\0',
zp_copies = 0 '\0',
zp_dedup = 0,
zp_dedup_verify = 0,
zp_nopwrite = 0
},
io_type = ZIO_TYPE_FREE,
io_child_type = ZIO_CHILD_VDEV,
io_cmd = 0,
io_priority = ZIO_PRIORITY_TRIM,
io_reexecute = 0 '\0',
io_state = 0xfffff805dcd9d405 "\001",
io_txg = 0,
io_spa = 0xfffff801c29d1000,
io_bp = 0x0,
io_bp_override = 0x0,
io_bp_copy = {
blk_dva = 0xfffff805dcd9d428,
blk_prop = 0,
blk_pad = 0xfffff805dcd9d460,
blk_phys_birth = 0,
blk_birth = 0,
blk_fill = 0,
blk_cksum = {
zc_word = 0xfffff805dcd9d488
}
},
io_parent_list = {
list_size = 48,
list_offset = 16,
list_head = {
list_next = 0xfffff8042bf35340,
list_prev = 0xfffff8042bf35340
}
},
io_child_list = {
list_size = 48,
list_offset = 32,
list_head = {
list_next = 0xfffff805dcd9d4d8,
list_prev = 0xfffff805dcd9d4d8
}
},
io_walk_link = 0x0,
io_logical = 0x0,
io_transform_stack = 0x0,
io_ready = 0,
io_physdone = 0,
io_done = 0,
io_private = 0x0,
io_prev_space_delta = 0,
io_bp_orig = {
blk_dva = 0xfffff805dcd9d528,
blk_prop = 0,
blk_pad = 0xfffff805dcd9d560,
blk_phys_birth = 0,
blk_birth = 0,
blk_fill = 0,
blk_cksum = {
zc_word = 0xfffff805dcd9d588
}
},
io_data = 0x0,
io_orig_data = 0x0,
io_size = 8192,
io_orig_size = 8192,
io_vd = 0xfffff804d8683000,
io_vsd = 0x0,
io_vsd_ops = 0x0,
io_offset = 897895682048,
io_timestamp = 941768555692856,
io_target_timestamp = 0,
io_queue_node = {
avl_child = 0xfffff805dcd9d5f8,
avl_pcb = 18446735278553267561
},
io_offset_node = {
avl_child = 0xfffff805dcd9d610,
avl_pcb = 0
},
io_flags = 789633,
io_stage = ZIO_STAGE_VDEV_IO_DONE,
io_pipeline = 3080192,
io_orig_flags = 525441,
io_orig_stage = ZIO_STAGE_OPEN,
io_orig_pipeline = 3080192,
io_error = 45,
io_child_error = 0xfffff805dcd9d644,
io_children = 0xfffff805dcd9d658,
io_child_count = 0,
io_phys_children = 0,
io_parent_count = 1,
io_stall = 0x0,
io_gang_leader = 0x0,
io_gang_tree = 0x0,
io_executor = 0xfffff80339d9d500,
io_waiter = 0x0,
io_lock = {
lock_object = {
lo_name = 0xffffffff822bf23e "zio->io_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
io_cv = {
cv_description = 0xffffffff822bf24c "zio->io_cv",
cv_waiters = 0
},
io_cksum_report = 0x0,
io_ena = 0,
io_tqent = {
tqent_task = {
ta_link = {
stqe_next = 0xfffff8016b2d0ad0
},
ta_pending = 0,
ta_priority = 0,
ta_func = 0xffffffff82185a60 <taskq_run_ent>,
ta_context = 0xfffff805dcd9d718
},
tqent_func = 0xffffffff82225b60 <zio_execute>,
tqent_arg = 0xfffff805dcd9d3b8
},
io_trim_node = {
avl_child = 0xfffff805dcd9d748,
avl_pcb = 0
},
io_trim_link = {
list_next = 0x0,
list_prev = 0x0
}
}
(kgdb) p zio->io_vd
$6 = {
vdev_id = 2,
vdev_guid = 13112130218511473524,
vdev_guid_sum = 13112130218511473524,
vdev_orig_guid = 0,
vdev_asize = 4000782221312,
vdev_min_asize = 3985729650688,
vdev_max_asize = 4000782221312,
vdev_ashift = 12,
vdev_logical_ashift = 12,
vdev_physical_ashift = 0,
vdev_state = 7,
vdev_prevstate = 0,
vdev_ops = 0xffffffff822d2200,
vdev_spa = 0xfffff801c29d1000,
vdev_tsd = 0xfffff801638b2780,
vdev_name_vp = 0x0,
vdev_devid_vp = 0x0,
vdev_top = 0xfffff803462c1800,
vdev_parent = 0xfffff803462c1800,
vdev_child = 0x0,
vdev_children = 0,
vdev_stat = {
vs_timestamp = 941737635691796,
vs_state = 0,
vs_aux = 0,
vs_alloc = 0,
vs_space = 0,
vs_dspace = 0,
vs_rsize = 0,
vs_esize = 0,
vs_ops = 0xfffff804d86830e8,
vs_bytes = 0xfffff804d8683118,
vs_read_errors = 0,
vs_write_errors = 0,
vs_checksum_errors = 0,
vs_self_healed = 0,
vs_scan_removing = 0,
vs_scan_processed = 0,
vs_configured_ashift = 0,
vs_logical_ashift = 0,
vs_physical_ashift = 0,
vs_fragmentation = 0
},
vdev_expanding = 0,
vdev_reopening = 0,
vdev_open_error = 0,
vdev_open_thread = 0x0,
vdev_crtxg = 4,
vdev_ms_array = 0,
vdev_ms_shift = 0,
vdev_ms_count = 0,
vdev_mg = 0x0,
vdev_ms = 0x0,
vdev_ms_list = {
tl_lock = {
lock_object = {
lo_name = 0xffffffff822bbd27 "tl->tl_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
tl_offset = 808,
tl_head = 0xfffff804d8683208
},
vdev_dtl_list = {
tl_lock = {
lock_object = {
lo_name = 0xffffffff822bbd27 "tl->tl_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
tl_offset = 784,
tl_head = 0xfffff804d8683250
},
vdev_txg_node = {
tn_next = 0xfffff804d8683270,
tn_member = 0xfffff804d8683290 ""
},
vdev_remove_wanted = 0,
vdev_probe_wanted = 0,
vdev_config_dirty_node = {
list_next = 0x0,
list_prev = 0x0
},
vdev_state_dirty_node = {
list_next = 0x0,
list_prev = 0x0
},
vdev_deflate_ratio = 0,
vdev_islog = 0,
vdev_removing = 0,
vdev_ishole = 0,
vdev_top_zap = 0,
vdev_dtl = 0xfffff804d86832e8,
vdev_dtl_sm = 0xfffff80455b00200,
vdev_dtl_node = {
tn_next = 0xfffff804d8683310,
tn_member = 0xfffff804d8683330 ""
},
vdev_dtl_object = 65,
vdev_psize = 4000786939904,
vdev_wholedisk = 1,
vdev_offline = 0,
vdev_faulted = 0,
vdev_degraded = 0,
vdev_removed = 0,
vdev_resilver_txg = 0,
vdev_nparity = 0,
vdev_path = 0xfffff802bd5acbe0 "/dev/label/backup42.eli",
vdev_devid = 0x0,
vdev_physpath = 0xfffff803be782ea0 "/dev/label/backup42.eli",
vdev_fru = 0x0,
vdev_not_present = 0,
vdev_unspare = 0,
vdev_nowritecache = 0,
vdev_notrim = 1,
vdev_checkremove = 0,
vdev_forcefault = 0,
vdev_splitting = 0,
vdev_delayed_close = 0,
vdev_tmpoffline = 0,
vdev_detached = 0,
vdev_cant_read = 0,
vdev_cant_write = 0,
vdev_isspare = 0,
vdev_isl2cache = 0,
vdev_queue = {
vq_vdev = 0xfffff804d8683000,
vq_class = 0xfffff804d86833e8,
vq_active_tree = {
avl_root = 0xfffff80290a71240,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 576,
avl_numnodes = 64,
avl_size = 952
},
vq_read_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 600,
avl_numnodes = 0,
avl_size = 952
},
vq_write_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 600,
avl_numnodes = 0,
avl_size = 952
},
vq_last_offset = 897918853120,
vq_io_complete_ts = 941768555692856,
vq_lock = {
lock_object = {
lo_name = 0xffffffff822bcc85 "vq->vq_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 18446735291472008448
},
vq_lastoffset = 33829150720
},
vdev_cache = {
vc_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff82208310 <vdev_cache_offset_compare>,
avl_offset = 24,
avl_numnodes = 0,
avl_size = 88
},
vc_lastused_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff82208330 <vdev_cache_lastused_compare>,
avl_offset = 48,
avl_numnodes = 0,
avl_size = 88
},
vc_lock = {
lock_object = {
lo_name = 0xffffffff822bc53d "vc->vc_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
}
}
(kgdb) p *zio->io_vd->vdev_ops
$7 = {
vdev_op_open = 0xffffffff8225c940 <vdev_geom_open>,
vdev_op_close = 0xffffffff8225d1b0 <vdev_geom_close>,
vdev_op_asize = 0xffffffff82201670 <vdev_default_asize>,
vdev_op_io_start = 0xffffffff8225d2a0 <vdev_geom_io_start>,
vdev_op_io_done = 0xffffffff8225d470 <vdev_geom_io_done>,
vdev_op_state_change = 0,
vdev_op_hold = 0xffffffff8225d480 <vdev_geom_hold>,
vdev_op_rele = 0xffffffff8225d490 <vdev_geom_rele>,
vdev_op_type = 0xffffffff822d2240 "disk",
vdev_op_leaf = 1
}
(kgdb) frame 195
#195 0xffffffff8222955d in zio_vdev_io_start (zio=0xfffff801469a13b8)
at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2870
2870 vd->vdev_ops->vdev_op_io_start(zio);
$8 = {
io_bookmark = {
zb_objset = 0,
zb_object = 0,
zb_level = 0,
zb_blkid = 0
},
io_prop = {
zp_checksum = ZIO_CHECKSUM_INHERIT,
zp_compress = ZIO_COMPRESS_INHERIT,
zp_type = DMU_OT_NONE,
zp_level = 0 '\0',
zp_copies = 0 '\0',
zp_dedup = 0,
zp_dedup_verify = 0,
zp_nopwrite = 0
},
io_type = ZIO_TYPE_FREE,
io_child_type = ZIO_CHILD_VDEV,
io_cmd = 0,
io_priority = ZIO_PRIORITY_TRIM,
io_reexecute = 0 '\0',
io_state = 0xfffff801469a1405 "\001",
io_txg = 0,
io_spa = 0xfffff801c29d1000,
io_bp = 0x0,
io_bp_override = 0x0,
io_bp_copy = {
blk_dva = 0xfffff801469a1428,
blk_prop = 0,
blk_pad = 0xfffff801469a1460,
blk_phys_birth = 0,
blk_birth = 0,
blk_fill = 0,
blk_cksum = {
zc_word = 0xfffff801469a1488
}
},
io_parent_list = {
list_size = 48,
list_offset = 16,
list_head = {
list_next = 0xfffff803c203a1c0,
list_prev = 0xfffff803c203a1c0
}
},
io_child_list = {
list_size = 48,
list_offset = 32,
list_head = {
list_next = 0xfffff801469a14d8,
list_prev = 0xfffff801469a14d8
}
},
io_walk_link = 0x0,
io_logical = 0x0,
io_transform_stack = 0x0,
io_ready = 0,
io_physdone = 0,
io_done = 0,
io_private = 0x0,
io_prev_space_delta = 0,
io_bp_orig = {
blk_dva = 0xfffff801469a1528,
blk_prop = 0,
blk_pad = 0xfffff801469a1560,
blk_phys_birth = 0,
blk_birth = 0,
blk_fill = 0,
blk_cksum = {
zc_word = 0xfffff801469a1588
}
},
io_data = 0x0,
io_orig_data = 0x0,
io_size = 4096,
io_orig_size = 4096,
io_vd = 0xfffff804d8683000,
io_vsd = 0x0,
io_vsd_ops = 0x0,
io_offset = 898323513344,
io_timestamp = 941768555692856,
io_target_timestamp = 0,
io_queue_node = {
avl_child = 0xfffff801469a15f8,
avl_pcb = 18446735288905504249
},
io_offset_node = {
avl_child = 0xfffff801469a1610,
avl_pcb = 0
},
io_flags = 789633,
io_stage = ZIO_STAGE_VDEV_IO_DONE,
io_pipeline = 3080192,
io_orig_flags = 525441,
io_orig_stage = ZIO_STAGE_OPEN,
io_orig_pipeline = 3080192,
io_error = 45,
io_child_error = 0xfffff801469a1644,
io_children = 0xfffff801469a1658,
io_child_count = 0,
io_phys_children = 0,
io_parent_count = 1,
io_stall = 0x0,
io_gang_leader = 0x0,
io_gang_tree = 0x0,
io_executor = 0xfffff80339d9d500,
io_waiter = 0x0,
io_lock = {
lock_object = {
lo_name = 0xffffffff822bf23e "zio->io_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
io_cv = {
cv_description = 0xffffffff822bf24c "zio->io_cv",
cv_waiters = 0
},
io_cksum_report = 0x0,
io_ena = 0,
io_tqent = {
tqent_task = {
ta_link = {
stqe_next = 0x0
},
ta_pending = 0,
ta_priority = 0,
ta_func = 0,
ta_context = 0x0
},
tqent_func = 0,
tqent_arg = 0x0
},
io_trim_node = {
avl_child = 0xfffff801469a1748,
avl_pcb = 0
},
io_trim_link = {
list_next = 0x0,
list_prev = 0x0
}
}
(kgdb) p *zio->io_vd
$9 = {
vdev_id = 2,
vdev_guid = 13112130218511473524,
vdev_guid_sum = 13112130218511473524,
vdev_orig_guid = 0,
vdev_asize = 4000782221312,
vdev_min_asize = 3985729650688,
vdev_max_asize = 4000782221312,
vdev_ashift = 12,
vdev_logical_ashift = 12,
vdev_physical_ashift = 0,
vdev_state = 7,
vdev_prevstate = 0,
vdev_ops = 0xffffffff822d2200,
vdev_spa = 0xfffff801c29d1000,
vdev_tsd = 0xfffff801638b2780,
vdev_name_vp = 0x0,
vdev_devid_vp = 0x0,
vdev_top = 0xfffff803462c1800,
vdev_parent = 0xfffff803462c1800,
vdev_child = 0x0,
vdev_children = 0,
vdev_stat = {
vs_timestamp = 941737635691796,
vs_state = 0,
vs_aux = 0,
vs_alloc = 0,
vs_space = 0,
vs_dspace = 0,
vs_rsize = 0,
vs_esize = 0,
vs_ops = 0xfffff804d86830e8,
vs_bytes = 0xfffff804d8683118,
vs_read_errors = 0,
vs_write_errors = 0,
vs_checksum_errors = 0,
vs_self_healed = 0,
vs_scan_removing = 0,
vs_scan_processed = 0,
vs_configured_ashift = 0,
vs_logical_ashift = 0,
vs_physical_ashift = 0,
vs_fragmentation = 0
},
vdev_expanding = 0,
vdev_reopening = 0,
vdev_open_error = 0,
vdev_open_thread = 0x0,
vdev_crtxg = 4,
vdev_ms_array = 0,
vdev_ms_shift = 0,
vdev_ms_count = 0,
vdev_mg = 0x0,
vdev_ms = 0x0,
vdev_ms_list = {
tl_lock = {
lock_object = {
lo_name = 0xffffffff822bbd27 "tl->tl_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
tl_offset = 808,
tl_head = 0xfffff804d8683208
},
vdev_dtl_list = {
tl_lock = {
lock_object = {
lo_name = 0xffffffff822bbd27 "tl->tl_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
},
tl_offset = 784,
tl_head = 0xfffff804d8683250
},
vdev_txg_node = {
tn_next = 0xfffff804d8683270,
tn_member = 0xfffff804d8683290 ""
},
vdev_remove_wanted = 0,
vdev_probe_wanted = 0,
vdev_config_dirty_node = {
list_next = 0x0,
list_prev = 0x0
},
vdev_state_dirty_node = {
list_next = 0x0,
list_prev = 0x0
},
vdev_deflate_ratio = 0,
vdev_islog = 0,
vdev_removing = 0,
vdev_ishole = 0,
vdev_top_zap = 0,
vdev_dtl = 0xfffff804d86832e8,
vdev_dtl_sm = 0xfffff80455b00200,
vdev_dtl_node = {
tn_next = 0xfffff804d8683310,
tn_member = 0xfffff804d8683330 ""
},
vdev_dtl_object = 65,
vdev_psize = 4000786939904,
vdev_wholedisk = 1,
vdev_offline = 0,
vdev_faulted = 0,
vdev_degraded = 0,
vdev_removed = 0,
vdev_resilver_txg = 0,
vdev_nparity = 0,
vdev_path = 0xfffff802bd5acbe0 "/dev/label/backup42.eli",
vdev_devid = 0x0,
vdev_physpath = 0xfffff803be782ea0 "/dev/label/backup42.eli",
vdev_fru = 0x0,
vdev_not_present = 0,
vdev_unspare = 0,
vdev_nowritecache = 0,
vdev_notrim = 1,
vdev_checkremove = 0,
vdev_forcefault = 0,
vdev_splitting = 0,
vdev_delayed_close = 0,
vdev_tmpoffline = 0,
vdev_detached = 0,
vdev_cant_read = 0,
vdev_cant_write = 0,
vdev_isspare = 0,
vdev_isl2cache = 0,
vdev_queue = {
vq_vdev = 0xfffff804d8683000,
vq_class = 0xfffff804d86833e8,
vq_active_tree = {
avl_root = 0xfffff80290a71240,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 576,
avl_numnodes = 64,
avl_size = 952
},
vq_read_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 600,
avl_numnodes = 0,
avl_size = 952
},
vq_write_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff8220b8a0 <vdev_queue_offset_compare>,
avl_offset = 600,
avl_numnodes = 0,
avl_size = 952
},
vq_last_offset = 897918853120,
vq_io_complete_ts = 941768555692856,
vq_lock = {
lock_object = {
lo_name = 0xffffffff822bcc85 "vq->vq_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 18446735291472008448
},
vq_lastoffset = 33829150720
},
vdev_cache = {
vc_offset_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff82208310 <vdev_cache_offset_compare>,
avl_offset = 24,
avl_numnodes = 0,
avl_size = 88
},
vc_lastused_tree = {
avl_root = 0x0,
avl_compar = 0xffffffff82208330 <vdev_cache_lastused_compare>,
avl_offset = 48,
avl_numnodes = 0,
avl_size = 88
},
vc_lock = {
lock_object = {
lo_name = 0xffffffff822bc53d "vc->vc_lock",
lo_flags = 40960000,
lo_data = 0,
lo_witness = 0x0
},
sx_lock = 1
}
}
(kgdb) p *zio->io_vd->vdev_ops
$10 = {
vdev_op_open = 0xffffffff8225c940 <vdev_geom_open>,
vdev_op_close = 0xffffffff8225d1b0 <vdev_geom_close>,
vdev_op_asize = 0xffffffff82201670 <vdev_default_asize>,
vdev_op_io_start = 0xffffffff8225d2a0 <vdev_geom_io_start>,
vdev_op_io_done = 0xffffffff8225d470 <vdev_geom_io_done>,
vdev_op_state_change = 0,
vdev_op_hold = 0xffffffff8225d480 <vdev_geom_hold>,
vdev_op_rele = 0xffffffff8225d490 <vdev_geom_rele>,
vdev_op_type = 0xffffffff822d2240 "disk",
vdev_op_leaf = 1
}
(kgdb)
There's two....
How many more 'ya want? As you can see they're big! :)
--
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2996 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20160720/e113438e/attachment-0001.bin>