dtrace discuss - Oct 2009 - dtrace provider for page faults

Can someone suggest a DTrace provider for tracking virtual memory  
pageins and pageouts in a certain memory range?

The memory range would be one established by mmap and I''m tracking  
memory access patterns within Firefox. Various dynamic libraries that  
Firefox uses are mmap-ed by the dynamic linker and I capture the  
address and range of each mapping.

What I would like to do now is see what order the pages in each  
library range are being pulled in from disk. Eventually, I''d like to  
optimize the order of symbols in each shared library to make access  
sequential as possible.

Any suggestions on how to track page faults in a memory range?

	Thanks, Joel

---
firefox for android!
http://wagerlabs.com

Too bad vminfo doesn''t provide addresses but this may serve as a 
starting point...

#pragma D option quiet

/*
 * For i86pc systems!
 */

#include <sys/types.h>
#include <ia32/sys/trap.h>
#include <ia32/sys/psw.h>

:::trap:entry
/pid == $target && USERMODE(args[0]->r_cs) != 0 &&
args[0]->r_trapno ==
T_PGFLT/
{
     printf("0x%p\n", args[1]);
}

dtrace -C -s trap.d -c "some command and args"

Jim
----
Joel Reymont wrote:
> Can someone suggest a DTrace provider for tracking virtual memory 
> pageins and pageouts in a certain memory range?
>
> The memory range would be one established by mmap and I''m tracking
> memory access patterns within Firefox. Various dynamic libraries that 
> Firefox uses are mmap-ed by the dynamic linker and I capture the 
> address and range of each mapping.
>
> What I would like to do now is see what order the pages in each 
> library range are being pulled in from disk. Eventually, I''d like
to
> optimize the order of symbols in each shared library to make access 
> sequential as possible.
>
> Any suggestions on how to track page faults in a memory range?
>
>     Thanks, Joel
>
> ---
> firefox for android!
> http://wagerlabs.com
>
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
>

James Litchfield wrote:
> Too bad vminfo doesn''t provide addresses but this may serve as a 
> starting point...
>
> #pragma D option quiet
>
> /*
> * For i86pc systems!
> */
>
> #include <sys/types.h>
> #include <ia32/sys/trap.h>
> #include <ia32/sys/psw.h>
>
> :::trap:entry
> /pid == $target && USERMODE(args[0]->r_cs) != 0 &&
args[0]->r_trapno
> == T_PGFLT/
> {
>     printf("0x%p\n", args[1]);
> }
>
> dtrace -C -s trap.d -c "some command and args"
>
> Jim
> ----
> Joel Reymont wrote:
>> Can someone suggest a DTrace provider for tracking virtual memory 
>> pageins and pageouts in a certain memory range?
>>
>> The memory range would be one established by mmap and I''m
tracking
>> memory access patterns within Firefox. Various dynamic libraries that 
>> Firefox uses are mmap-ed by the dynamic linker and I capture the 
>> address and range of each mapping.
>>
>> What I would like to do now is see what order the pages in each 
>> library range are being pulled in from disk. Eventually, I''d
like to
>> optimize the order of symbols in each shared library to make access 
>> sequential as possible.
>>
>> Any suggestions on how to track page faults in a memory range?
>>     
Tracking page faults  is pretty straight forward.  Tracking pageins and 
pageouts is more
problematic.  You can use the fbt provider to track page faults.  This 
should work on
both x86 and sparc.

pagefaultrange.d

#!/usr/sbin/dtrace -s

pagefault:entry
/execname == "firefox-bin" &&
$1 <= arg0 && arg0 <= $2/
{
    printf("firefox fault on addr %a, fault type = %x\n", arg0, arg1);
}

To run:

# ./pagefaultrange.d 0 0xffffffff

This will track pagefaults in all addresses in (32-bit) firefox-bin.
If arg1 is 0, the fault address is not valid (and a page may need to be 
allocated
or reclaimed), and if arg1 is 1, it is a protection (copy-on-write, 
perhaps) fault.

The problem with tracking pageins is that a pagefault may result in
more than 1 page being paged in.  The problem with tracking pageouts
is that there is no direct link from the page being paged out back to the
process(es) using the page.  You would have to keep track of the pages as
they are brought in to see when any of those pages are paged out.  (Note
that this does not mean it can''t be done, but it is more work).

max

>>
>

Hi Joel,

I did not notice you are running macos.  Disregard my script as it is 
for solaris.
Sorry, I don''t know anything about macos.

max

Joel Reymont wrote:
> I need to identify page-ins by library so the following script is what 
> I use right now.
>
> The problem is that Snow Leopard, and I''m sure Leopard too, cache 
> things in the Unified Buffer Cache (UBC). The purge command does not 
> evict certain portions of the cache (anonymously mmaped pages?) so the 
> code stays cached. I do see page-ins after a reboot, though.
>
> Note that the dynamic linker entry point for libraries is 
> dyld??loadPhase*, although Firefox uses dlopen for some libraries.
>
> ---%<---
>
> typedef struct nameidata* nameidata_t; /* sys/namei.h */
>
> pid$target::dlopen:entry
> /arg0/
> {
>   self->ppath = arg0;
>   self->dylib = 1;
> }
>
> pid$target::dyld??loadPhase5*:entry,
> pid$target::dyld??loadPhase4*:entry
> /!self->dylib && arg0/
> {
>   self->path = copyinstr(arg0);
>   self->func = probefunc;
>   self->dylib = 1;
> }
>
> pid$target::dlopen:return
> {
>   self->dylib = 0;
> }
>
> pid$target::dyld??loadPhase5*:return,
> pid$target::dyld??loadPhase4*:return
> /self->func != 0 && self->func == probefunc/
> {
>   self->dylib = 0;
> }
>
> /* file name memory should be wired in by now */
>
> pid$target::open:entry
> /self->dylib && self->ppath && self->path == 0/
> {
>   self->path = copyinstr(self->ppath);
> }
>
> fbt::vn_open_auth:entry
> {
>   self->ndp = (nameidata_t)arg0;
> }
>
> /* wait to make sure ndp and vnode are fully populated */
>
> fbt::vn_open_auth:return
> /self->path != 0/
> {
>   self->curpath = stringof((self->ndp)->ni_pathbuf);
> }
>
> /* make sure we are opening the same file */
>
> fbt::vn_open_auth:return
> /self->curpath != 0 && self->path == self->curpath/
> {
>   this->vp = (vnode_t)(self->ndp)->ni_vp;
>   this->lib = stringof((this->vp)->v_name);
>   self->lib[this->lib] = self->path;
>   self->vnode[this->lib] = this->vp;
>   /*
>   printf("match = %s, lib = %s, vnode: %x\n",
>     self->curpath, this->lib, (long)this->vp);
>   */
> }
>
> fbt::vn_open_auth:return
> {
>   self->path = 0;
>   self->ppath = 0;
>   self->curpath = 0;
>   self->ndp = 0;
>   self->func = 0;
>   self->dylib = 0;
> }
>
> fbt::vnode_pagein:entry
> {
>   self->v_name = stringof(((vnode_t)arg0)->v_name);
> }
>
> /* vnode pointers should match but v_name seems more secure */
>
> fbt::vnode_pagein:entry
> /self->lib[self->v_name] != 0/
> {
>   printf("vnode_pagein: %s, offset: %u, size: %u\n",
>     self->v_name, arg3, arg4);
> }
>
>
> ---
> firefox for android!
> http://wagerlabs.com
>
>

I need to identify page-ins by library so the following script is what  
I use right now.

The problem is that Snow Leopard, and I''m sure Leopard too, cache  
things in the Unified Buffer Cache (UBC). The purge command does not  
evict certain portions of the cache (anonymously mmaped pages?) so the  
code stays cached. I do see page-ins after a reboot, though.

Note that the dynamic linker entry point for libraries is dyld?? 
loadPhase*, although Firefox uses dlopen for some libraries.

---%<---

typedef struct nameidata* nameidata_t; /* sys/namei.h */

pid$target::dlopen:entry
/arg0/
{
   self->ppath = arg0;
   self->dylib = 1;
}

pid$target::dyld??loadPhase5*:entry,
pid$target::dyld??loadPhase4*:entry
/!self->dylib && arg0/
{
   self->path = copyinstr(arg0);
   self->func = probefunc;
   self->dylib = 1;
}

pid$target::dlopen:return
{
   self->dylib = 0;
}

pid$target::dyld??loadPhase5*:return,
pid$target::dyld??loadPhase4*:return
/self->func != 0 && self->func == probefunc/
{
   self->dylib = 0;
}

/* file name memory should be wired in by now */

pid$target::open:entry
/self->dylib && self->ppath && self->path == 0/
{
   self->path = copyinstr(self->ppath);
}

fbt::vn_open_auth:entry
{
   self->ndp = (nameidata_t)arg0;
}

/* wait to make sure ndp and vnode are fully populated */

fbt::vn_open_auth:return
/self->path != 0/
{
   self->curpath = stringof((self->ndp)->ni_pathbuf);
}

/* make sure we are opening the same file */

fbt::vn_open_auth:return
/self->curpath != 0 && self->path == self->curpath/
{
   this->vp = (vnode_t)(self->ndp)->ni_vp;
   this->lib = stringof((this->vp)->v_name);
   self->lib[this->lib] = self->path;
   self->vnode[this->lib] = this->vp;
   /*
   printf("match = %s, lib = %s, vnode: %x\n",
     self->curpath, this->lib, (long)this->vp);
   */
}

fbt::vn_open_auth:return
{
   self->path = 0;
   self->ppath = 0;
   self->curpath = 0;
   self->ndp = 0;
   self->func = 0;
   self->dylib = 0;
}

fbt::vnode_pagein:entry
{
   self->v_name = stringof(((vnode_t)arg0)->v_name);
}

/* vnode pointers should match but v_name seems more secure */

fbt::vnode_pagein:entry
/self->lib[self->v_name] != 0/
{
   printf("vnode_pagein: %s, offset: %u, size: %u\n",
     self->v_name, arg3, arg4);
}


---
firefox for android!
http://wagerlabs.com