dtrace discuss - Jan 2006 - Measuring migrations with DTrace

Howdy,

I have been trying to digest the contents of cpu.c, and was curious which 
probes would be best suited to watching thread migrations? To see how 
stuff moves about (or doesn''t) a system, I want to conjure up a script 
that produces output similar to the following:

$ migrate.d

sshd  (pid: 234 tid: 0) migrated from CPU0 to CPU1
httpd (pid: 456 tid: 1) migrated from CPU1 to CPU0

Also -- are the sysinfo::swtch:pswitch and sysinfo::preempt:inv_swtch 
probes the best way to find voluntary and involuntary context switches 
(such as those reported by mpstat)?

Thank you,
- Ryan 
--
UNIX Administrator
http://daemons.net/~matty

G''Day Matty,

On Wed, 11 Jan 2006, Matty wrote:
> Howdy,
>
> I have been trying to digest the contents of cpu.c, and was curious which
> probes would be best suited to watching thread migrations? To see how
> stuff moves about (or doesn''t) a system, I want to conjure up a
script
> that produces output similar to the following:
>
> $ migrate.d
>
> sshd  (pid: 234 tid: 0) migrated from CPU0 to CPU1
> httpd (pid: 456 tid: 1) migrated from CPU1 to CPU0
check out /usr/demo/dtrace/whosteal.d, it''s based on dequeue and
probably does what you want.
> Also -- are the sysinfo::swtch:pswitch and sysinfo::preempt:inv_swtch
> probes the best way to find voluntary and involuntary context switches
> (such as those reported by mpstat)?
AFAIK, yes. Remember sysinfo:::pswitch is both voluntary and involuntary,
and sysinfo:::inv_swtch is just involuntary. Too many text books get that
wrong (the man page is right. :)

check out disp.c if you havent already...

no worries,

Brendan

[Sydney, Australia]

> 
> Howdy,
> 
> I have been trying to digest the contents of cpu.c, and was curious which 
> probes would be best suited to watching thread migrations? To see how 
> stuff moves about (or doesn''t) a system, I want to conjure up a
script
> that produces output similar to the following:
> 
> $ migrate.d
> 
> sshd  (pid: 234 tid: 0) migrated from CPU0 to CPU1
> httpd (pid: 456 tid: 1) migrated from CPU1 to CPU0
You can use sched::resume:on-cpu probe. The curthread->t_cpu->cpu_id
contains
the CPU that the thread was using before and curcpu->cpu_id is the new CPU.

- Alexander Kolbasov

On Thu, Jan 12, 2006 at 01:36:26PM -0800, Alexander Kolbasov
wrote:
> > 
> > Howdy,
> > 
> > I have been trying to digest the contents of cpu.c, and was curious
which
> > probes would be best suited to watching thread migrations? To see how 
> > stuff moves about (or doesn''t) a system, I want to conjure up
a script
> > that produces output similar to the following:
> > 
> > $ migrate.d
> > 
> > sshd  (pid: 234 tid: 0) migrated from CPU0 to CPU1
> > httpd (pid: 456 tid: 1) migrated from CPU1 to CPU0
> 
> You can use sched::resume:on-cpu probe. The curthread->t_cpu->cpu_id
contains
> the CPU that the thread was using before and curcpu->cpu_id is the new
CPU.
You shouldn''t specify :resume: in the probe; the function name is not a
stable interface.  In addition, curcpu->cpu_id expands to
curthread->t_cpu->cpu_id, and is always the *new* cpu.

The stable way to do this is to watch both on-cpu and off-cpu events:

sched:::off-cpu
{
	self->known = 1;
	self->prev_cpu = cpu;
}

sched:::on-cpu
/self->known && self->prev_cpu != cpu/
{
	printf("... migrated from CPU%d to CPU%d\n", ..., self->prev_cpu,
cpu);
}

sched:::on-cpu
{
	self->prev_cpu = 0;
	self->known = 0;
}

Cheers,
- jonathan
> 
> - Alexander Kolbasov
> 
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
-- 
Jonathan Adams, Solaris Kernel Development

> On Thu, Jan 12, 2006 at 01:36:26PM -0800, Alexander Kolbasov wrote:
> > > 
> > > Howdy,
> > > 
> > > I have been trying to digest the contents of cpu.c, and was
curious which
> > > probes would be best suited to watching thread migrations? To see
how
> > > stuff moves about (or doesn''t) a system, I want to
conjure up a script
> > > that produces output similar to the following:
> > > 
> > > $ migrate.d
> > > 
> > > sshd  (pid: 234 tid: 0) migrated from CPU0 to CPU1
> > > httpd (pid: 456 tid: 1) migrated from CPU1 to CPU0
> > 
> > You can use sched::resume:on-cpu probe. The
curthread->t_cpu->cpu_id contains
> > the CPU that the thread was using before and curcpu->cpu_id is the
new CPU.
> 
> You shouldn''t specify :resume: in the probe; the function name is
not a
> stable interface.  In addition, curcpu->cpu_id expands to
> curthread->t_cpu->cpu_id, and is always the *new* cpu.
Right, I should have checked that.
> The stable way to do this is to watch both on-cpu and off-cpu events:
> 
> sched:::off-cpu
> {
> 	self->known = 1;
> 	self->prev_cpu = cpu;
> }
> 
> sched:::on-cpu
> /self->known && self->prev_cpu != cpu/
> {
> 	printf("... migrated from CPU%d to CPU%d\n", ...,
self->prev_cpu, cpu);
> }
> 
> sched:::on-cpu
> {
> 	self->prev_cpu = 0;
> 	self->known = 0;
> }
Can we just watch on-cpu event:

sched:::on-cpu
/self->prev_cpu && self->prev_cpu != cpu/
{
	printf("... migrated from CPU%d to CPU%d\n", ..., self->prev_cpu,
cpu)
}

sched:::on-cpu
{
	self->prev_cpu = cpu;	
}
> 
> Cheers,
> - jonathan
>
> > - Alexander Kolbasov
> >