Trond Norbye
2006-Jul-20 07:58 UTC
[zfs-discuss] How can I watch IO operations with dtrace on zfs?
I have been using iosoop script (see http://www.opensolaris.org/os/community/dtrace/scripts/) written by Brendan Gregg to look at the IO operations of my application. When I was running my test-program on a UFS filesystem I could see both read and write operations like: UID PID D BLOCK SIZE COMM PATHNAME 203803 4436 R 6016592 16384 diskio <none> 203803 4436 W 3448432 16384 diskio <none> 203803 4436 R 5510832 16384 diskio <none> 203803 4436 W 4652608 16384 diskio <none> But when I''m running my program on a ZFS file-system, I can only see the read io transfers: UID PID D BLOCK SIZE COMM PATHNAME 203803 4445 R 2647296 131072 diskio <none> 203803 4445 R 1351168 131072 diskio <none> So how can I get the same information from a ZFS file-system? The reason I''m asking is because I''m trying to see how IO operations behave on the different filesystems. Any help / pointers would be appreciated. Trond This message posted from opensolaris.org
Brendan Gregg
2006-Jul-20 08:52 UTC
[zfs-discuss] How can I watch IO operations with dtrace on zfs?
G''Day Trond, On Thu, 20 Jul 2006, Trond Norbye wrote:> I have been using iosoop script (see > http://www.opensolaris.org/os/community/dtrace/scripts/) written by > Brendan Gregg to look at the IO operations of my application. When I was > running my test-program on a UFS filesystem I could see both read and > write operations like:The most up to date version would be in the DTraceToolkit, but that shouldn''t cause a problem here.> UID PID D BLOCK SIZE COMM PATHNAME > 203803 4436 R 6016592 16384 diskio <none> > 203803 4436 W 3448432 16384 diskio <none> > 203803 4436 R 5510832 16384 diskio <none> > 203803 4436 W 4652608 16384 diskio <none> > > But when I''m running my program on a ZFS file-system, I can only see > the read io transfers: > > UID PID D BLOCK SIZE COMM PATHNAME > 203803 4445 R 2647296 131072 diskio <none> > 203803 4445 R 1351168 131072 diskio <none> > > So how can I get the same information from a ZFS file-system?Firstly, we need to bear in mind that iosnoop matches on disk I/O, not application level I/O. Many writes will be cached in memory and flushed sometime later to disk, asynchronously. That''s when iosnoop picks it up. Were you matching on a process name or PID? no worries, Brendan
Matthew Ahrens
2006-Jul-20 18:28 UTC
[zfs-discuss] How can I watch IO operations with dtrace on zfs?
On Thu, Jul 20, 2006 at 12:58:31AM -0700, Trond Norbye wrote:> I have been using iosoop script (see > http://www.opensolaris.org/os/community/dtrace/scripts/) written by > Brendan Gregg to look at the IO operations of my application....> So how can I get the same information from a ZFS file-system?As you can see, ZFS is not yet fully integrated with the dtrace i/o provider. With ZFS, writes are (typically) deferred, so it is nontrivial to assign each write i/o to a particular application. If you are familiar with dtrace, you can use fbt to look at the zio_done() function, eg. with something like this: zio_done:entry /args[0]->io_type == 1 && args[0]->io_bp != NULL/ { @bytes["read", args[0]->io_bookmark.zb_objset, args[0]->io_bookmark.zb_object, args[0]->io_bookmark.zb_level, args[0]->io_bookmark.zb_blkid != 0] /* sum(args[0]->io_size); */ count(); } zio_done:entry /args[0]->io_type == 2/ { @bytes["write", args[0]->io_bookmark.zb_objset, args[0]->io_bookmark.zb_object, args[0]->io_bookmark.zb_level, args[0]->io_bookmark.zb_blkid != 0] /* sum(args[0]->io_size); */ count(); } END { printf("r/w objset object level blk>0 i/os\n"); printa("%5s %4d %7d %d %d %@d\n", @bytes); printf("r/w objset object level blk>0 i/os\n"); } --matt