dtrace discuss - Aug 2005 - tracing across door calls

Anyone have any examples of using Dtrace to track interaction across door calls?

I managed to find the perfect combination of load and configuration with the
native ldap client on Solaris to bring all name service calls to their knees
(lucky me :)).  The most noticable symptoms when this would happen was that su
to any user would hang seemingly indefinately, cron stopped working, and later
clusters starting going haywire.

Since most of the systems using it are Solaris 8 at the moment, it was not fun
to track down all the interactions that contributed to causing the problem (I
actually found

I thought it would make a good real world (for us at least) example of the
benefits of dtrace by contrasting how we had to do it with how it could have
been done.
This message posted from opensolaris.org

On Thu, Aug 18, 2005 at 08:39:27PM -0700, Jason King
wrote:
> Anyone have any examples of using Dtrace to track interaction across
> door calls?
>
> I managed to find the perfect combination of load and configuration
> with the native ldap client on Solaris to bring all name service
> calls to their knees (lucky me :)). The most noticable symptoms
> when this would happen was that su to any user would hang seemingly
> indefinately, cron stopped working, and later clusters starting going
> haywire.
>
> Since most of the systems using it are Solaris 8 at the moment, it was
> not fun to track down all the interactions that contributed to causing
> the problem (I actually found
>
> I thought it would make a good real world (for us at least) example of
> the benefits of dtrace by contrasting how we had to do it with how it
> could have been done.
Here''s a script I put together a while ago; it relies on implementation
details, but in a minimal fashion.  It should continue working.

It just sums up time spent processing door requests by client+server execname,
but it wouldn''t be hard to make it do more.

Cheers,
- jonathan

-- 
Jonathan Adams, Solaris Kernel Development
-------------- next part --------------
#!/usr/sbin/dtrace -qs

fbt::door_call:entry
{
	self->door_call = 1;
}

fbt::shuttle_resume:entry
/self->door_call/
{
	self->target = args[0];
	source[args[0]] = execname;
}

fbt::door_return:return
/source[curthread] != 0/
{
	/*
	 * at this point, we are in the server thread, right before it
	 * returns to userland.
	 */
	self->start = vtimestamp;
}

fbt::door_return:entry
/source[curthread] != 0/
{
	/*
	 * The server is done with it''s processing, and is ready to send back
	 * the results.  Clean up stuff here.
	 */
	@a[source[curthread], execname] = sum(vtimestamp - self->start);
	self->start = 0;
}

fbt::door_call:return
/self->door_call/
{
	/*
	 * We''re now back in the client, returning to userland.  Note that
	 * if the child recieves a signal, the server thread could still be
	 * processing the request.   errno would be EINTR in that case.
	 */
	self->door_call = 0;
	source[self->target] = 0;
	self->target = 0;
}

END
{
	printf("%-16s %-16s %16s\n", "CLIENT", "SERVER",
"NANOSECS");
	printa("%-16s %-16s %@16d\n", @a);
}