Howdy, I have been working on an Apache DTrace module for the past few weeks, and just released version 0.2a. This module integrates Dtrace probes into Apache through the Apache hook framework. These probes can be used to observe and correlate a wide-variety of application and system behavior, which I have found useful for correlating network/virtual memory events with requests erved by Apache. I have written a few scripts to show some of the things I have been able to do with mod_Dtrace, and have attached [1] some screenshots to this e-mail. If your interested in using mod_dtrace, you can grab the source code and shell scripts from my web site: http://daemons.net/~matty This is still a work in progress, and I would love to get feedback (e.g., script enhancements, code enhancements, feature requests, recommendations etc.) from the folks on the list. I would like to thank Clay McClure, Brendan Gregg, Adam Leventhal, Bryan Cantrill, Sean McGrath and Mike Shapiro for all of their help! Thanks a ton guys!!!!!! You rock!!!!!!!!!!!!! Thanks, - Ryan -- UNIX Administrator http://daemons.net/~matty [1] viewreqs.pl allows you to view requests by HTTP method type: $ viewreqs.pl -d 5 Timestamp GETs POSTs HEADs TRACEs 02:05:59 0 0 0 0 02:06:04 12 0 0 0 02:06:09 15 0 0 0 02:06:14 59 0 0 0 02:06:19 68 0 0 0 02:06:24 70 0 0 0 viewreqsbyproc.d allows you to view requests by httpd process id: $ viewreqsbyproc.d Execname Process Requests Processed httpd 19103 16 httpd 19152 17 httpd 19101 17 httpd 19102 19 httpd 19104 20 httpd 19105 20 viewbrowsers.pl allows you to view requests by browser type: $ viewbrowsers.pl Sampling browser data for 60 seconds... Browser GETs POSTs HEADs TRACEs Firefox 34 0 0 0 Safari 20 0 0 0 Wget 7 0 0 0 viewserver.pl allows you to view requests by virtual server: $ viewserver.pl -d 60 Sampling virtual server data for 60 seconds... Server GETs POSTs HEADs TRACEs www.foo.com 101 0 0 0 www.bar.com 23 0 0 0 responsecodes.d allows you to view response codes aggregated by the number of times they have been used in an HTTP response: $ responsecodes.d Sampling response code data (Ctrl-C to stop): Response Code Count 404 4 200 22 responsecodes.pl allows you to view response codes along with the URI and file they are associated with: $ responsecodes.pl Response Code URI File Name 200 / /tmp/apache/htdocs/index.html.var 304 /apache_pb.gif /tmp/apache/htdocs/apache_pb.gif 200 / /tmp/apache/htdocs/index.html.var 200 /apache_pb.gif /tmp/apache/htdocs/apache_pb.gif 404 /favicon.ico /tmp/apache/htdocs/favicon.ico responsecodes.pl also allows you to filter by response codes (e.g., 404 document not found): $ responsecodes.pl -c 404 Response Code URI File Name 404 /favicon.ico /tmp/apache/htdocs/favicon.ico 404 /favicon.ico /tmp/apache/htdocs/favicon.ico 404 /favicon.ico /tmp/apache/htdocs/favicon.ico 404 /favicon.ico /tmp/apache/htdocs/favicon.ico redirects.pl allows you to view HTTP 301/302 redirects along with the URI and URL they apply to: $ redirects.pl Response Code URI Requested Redirect Sent To Client 301 /yikes http://192.168.1.3:8080/yikes2 301 /foo http://192.168.1.3:8080/bar 301 /bling http://192.168.1.3:8080/crunk requestsize.d allows you to view request sizes in a qunatized distribution: $ requestsize.d dtrace: script ''requestsize.d'' matched 1 probe httpd 5549 value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@ 43 1 | 0 2 | 0 4 | 0 8 | 0 16 | 0 32 | 0 64 | 0 128 | 0 256 |@@@@ 8 512 | 0 1024 | 0 2048 |@@@@@@@@@@@@ 22 4096 | 0 dapachetop allows you to view requests in a "top"-like display: $ dapachetop 10 2005 Nov 28 02:01:00, load: 0.00, Requests: 4, Bytes: 8 Kb Requests : GET: 4 POST: 0 HEAD: 0 TRACE: 0 Response Codes : 1XXL 0 2XX: 4 3XX: 0 4XX: 0 Requests Request/Sec Bytes Sent URI 2 .20 4 Kb /foo 1 .10 2 kb /bar 1 .10 2 kb /bling