dtrace discuss - Apr 2009 - Applications I/O

Hye,

I still try to access application side I/O size on a ZFS file system.

( example : if /var/zfs_fs is a ZFS file system
/var/zfs_fs:# dd if=/dev/urandom of=file bs=4k count=100000000000000

i try to get an information simple as
execname == "dd"
block_size = 4096

in order to trace a specific application I/O profile

I''ve tried with both "fbt::arc_access, fbt::arc_write,
fbt::arc_buf_evict, fbt::arc_change_state, fbt::zio_create, ...", but i
can''t retrieve this information.

Some times, block_size doesn''t match 4k (e.g 128k, ..., or 0 with the
zio_create probe)
Some times, execname is "sched" and hide the application name wich
initiate the I/O.

I''ve read a lot of the ZFS ans ARC source code, but can''t find
any DTrace probe that provides me those informations.
Please, need some Help

R.
-- 
This message posted from opensolaris.org

You''re tripping over the fact the these disk IOs are happening
asynchronously to the process/thread that initiated them.
The dd(1) process has long since been placed on a sleep
queue by the time you''re hitting the ARC code, which is why
execname is "sched" (the execname of PID 0  - the user
process is not running - it''s not on-cpu when the probe fires).

If you want to look at the disk IO load being generated by
an application, use the syscall provider, or the fsinfo
provider. You can''t chase IO''s and correlate them to
user processes once you get below fsinfo - they''re off
CPU and sleeping while the kernel goes off and deals with
the disk IOs.

The IO size will typically change as you move down the
stack, unless the application issues IOs that align with the
underlying file system record size, but even then things
like prefetching will result in the physical IO size not necessarily
aligning with the number of bytes the application tried to
read or write.

Starting at the bottom - if you want to monitor physical disk
IO rates, use the DTrace io provider, and/or iostat.

If you want to track IO rates through a file system,
use the DTrace fsinfo provider, and/or the fsstat command.

If you want to track disk IO''s generating by applications,
start with the syscall provider. It''s possible to connect
the dots up and down the stack, and there should be
examples of that in iotop in Brendan''s DTraceToolkit.

Thanks,
/jim


Ferrand wrote:
> Hye,
>
> I still try to access application side I/O size on a ZFS file system.
>
> ( example : if /var/zfs_fs is a ZFS file system
> /var/zfs_fs:# dd if=/dev/urandom of=file bs=4k count=100000000000000
>
> i try to get an information simple as
> execname == "dd"
> block_size = 4096
>
> in order to trace a specific application I/O profile
>
> I''ve tried with both "fbt::arc_access, fbt::arc_write,
fbt::arc_buf_evict, fbt::arc_change_state, fbt::zio_create, ...", but i
can''t retrieve this information.
>
> Some times, block_size doesn''t match 4k (e.g 128k, ..., or 0 with
the zio_create probe)
> Some times, execname is "sched" and hide the application name
wich initiate the I/O.
>
> I''ve read a lot of the ZFS ans ARC source code, but can''t
find any DTrace probe that provides me those informations.
> Please, need some Help
>
> R.
>

>
> provider. You can''t chase IO''s and correlate them to
> user processes once you get below fsinfo - they''re off
> CPU and sleeping while the kernel goes off and deals with
> the disk IOs.
I need to make this clearer. With DTrace, you can correlate
disk IO load to applications, it''s just not as straight-forward
as tracking "execname" in the io provider, or somewhere
else down in the bowels of the kernel''s IO stack, because
of the aforementioned asynchronous nature of disk IO.

Thanks,
/jim

Thanks,

I already noticed the asynchronous effect on tracing applications I/O.
I''ve already make an efficient D script to trace syscalls write, read
and seek in order to trace each application I/O.

What i wanted to do here is observing those I/O from the Arc''s point on
view.

Thanks for your answer, that''s true that asynchronous write force us to
use only syscalls to trace specific application I/O.

Thanks for your answer(s)
R.
-- 
This message posted from opensolaris.org