Howdy, Is there a method to get the number of elements in an aggregation? Are the results stored in an aggregation guaranteed to be in any type of order? Thanks for any insight, - Ryan -- UNIX Administrator http://daemons.net/~matty
On Mon, Oct 31, 2005 at 03:28:41PM -0500, Matty wrote:> > Howdy, > > Is there a method to get the number of elements in an aggregation?In probe context, no; that information only exists in the userland address space of the dtrace(1M) process after it has snapshotted and processed the aggregation data.> Are the results stored in an aggregation guaranteed to be in any type > of order?They are sorted by the userland process before display, by the aggregation value. Bryan is currently working on some new options to control the sorting of aggregation output; I''ll let him elaborate. Cheers, - jonathan -- Jonathan Adams, Solaris Kernel Development
> > Are the results stored in an aggregation guaranteed to be in any type > > of order? > > They are sorted by the userland process before display, by the aggregation > value. Bryan is currently working on some new options to control > the sorting of aggregation output; I''ll let him elaborate.Currently, the order is guaranteed to be in value order, with ties broken by key order. In my current wad to add multiple aggregation support to printa(), I''m also adding two new boolean options: aggsortkey => denotes that aggregations should be sorted in key order, with ties broken by value order aggsortrev => denotes that aggregations should be reverse sorted And: aggsortpos => when multiple aggregations are being printed, the position of the aggregation that should act as the primary sort key aggsortkeypos => when multiple keys are present, the position of the key that should act as the primary sort key So, for example (and with the caveat that it uses functionality that only exists in my workspace at present) take the following script: ---8<--- rwsummary.d ---8<--- #pragma D option quiet syscall::read:return /errno == 0/ { @reads[execname, pid] = count(); @rbytes[execname, pid] = sum(arg1); } syscall::write:entry { @writes[execname, pid] = count(); @wbytes[execname, pid] = sum(arg2); } END { normalize(@wbytes, 1024); normalize(@rbytes, 1024); printf("%20s %7s %7s %7s %7s %7s\n", "PROGRAM", "PID", "READS", "KB", "WRITES", "KB"); printa("%20s %7d %@7d %@6dK %@7d %@6dK\n", @reads, @rbytes, @writes, @wbytes); } ---8<--- rwsummary.d ---8<--- Just running this might yield output like: PROGRAM PID READS KB WRITES KB dtrace 152294 0 0K 1 0K svcadm 152296 0 0K 31 0K svcs 152295 1 0K 165 8K sh 152280 4 0K 3 0K utmpd 100392 4 2K 0 0K in.telnetd 152277 4 8K 4 0K init 1 6 3K 2 0K svc.configd 100006 58 58K 54 46K If I ran the same script with "aggsortrev" set (either via -x, "#pragma D option" or the setopt() function), I would see the reverse-sorted output: PROGRAM PID READS KB WRITES KB svc.configd 100006 58 58K 54 46K init 1 6 3K 2 0K in.telnetd 152277 4 8K 4 0K utmpd 100392 4 2K 0 0K sh 152280 4 0K 3 0K svcs 152295 1 0K 165 8K svcadm 152296 0 0K 31 0K dtrace 152294 0 0K 1 0K If I set "aggsortpos" to "2", the output would be sorted based on the number of writes: PROGRAM PID READS KB WRITES KB utmpd 100392 4 2K 0 0K dtrace 152294 0 0K 1 0K init 1 6 3K 2 0K sh 152280 4 0K 3 0K in.telnetd 152277 4 8K 4 0K svcadm 152296 0 0K 31 0K svc.configd 100006 58 58K 54 46K svcs 152295 1 0K 165 8K (The positions number from 0: @reads is position 0, @rbytes is position 1, @writes is position 2 and @wbytes is position 3.) If I set "aggsortkey", the aggregation will be sorted in key order: PROGRAM PID READS KB WRITES KB dtrace 152294 0 0K 1 0K in.telnetd 152277 4 8K 4 0K init 1 6 3K 2 0K sh 152280 4 0K 3 0K svc.configd 100006 58 58K 54 46K svcadm 152296 0 0K 31 0K svcs 152295 1 0K 165 8K utmpd 100392 4 2K 0 0K And similarly, if I both set "aggsortkey" and set "aggsortkeypos" to "1": PROGRAM PID READS KB WRITES KB init 1 6 3K 2 0K svc.configd 100006 58 58K 54 46K utmpd 100392 4 2K 0 0K in.telnetd 152277 4 8K 4 0K sh 152280 4 0K 3 0K dtrace 152294 0 0K 1 0K svcs 152295 1 0K 165 8K svcadm 152296 0 0K 31 0K Finally, these options can be set in combination (for example, you could set "aggsortrev" and "aggsortkey", etc.) and they can be set dynamically via setopt(): setopt("aggsortrev"); printa(@foo); /* print in reverse-sorted order */ setopt("aggsortrev", "false"); printa(@foo); /* print in sorted order */ This wad will be in code review soon (hopefully, tomorrow), and will be in Solaris after testing is completed and in OpenSolaris and Solaris Express shortly thereafter... - Bryan -------------------------------------------------------------------------- Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc
Hi Bryan, This is sweet :-) I am waiting for something like this for a while. Now we can have our own version of prstat equivalent :-) I guess the important thing would be to have key signature same for your printa statement. What will happen if the key signature differs? Thanks. Regards, Jignesh Bryan Cantrill wrote:>>>Are the results stored in an aggregation guaranteed to be in any type >>>of order? >>> >>> >>They are sorted by the userland process before display, by the aggregation >>value. Bryan is currently working on some new options to control >>the sorting of aggregation output; I''ll let him elaborate. >> >> > >Currently, the order is guaranteed to be in value order, with ties broken >by key order. In my current wad to add multiple aggregation support to >printa(), I''m also adding two new boolean options: > > aggsortkey => denotes that aggregations should be sorted in key > order, with ties broken by value order > > aggsortrev => denotes that aggregations should be reverse sorted > >And: > > aggsortpos => when multiple aggregations are being printed, the > position of the aggregation that should act as the > primary sort key > > aggsortkeypos => when multiple keys are present, the position of the > key that should act as the primary sort key > >So, for example (and with the caveat that it uses functionality that >only exists in my workspace at present) take the following script: > >---8<--- rwsummary.d ---8<--- > >#pragma D option quiet > >syscall::read:return >/errno == 0/ >{ > @reads[execname, pid] = count(); > @rbytes[execname, pid] = sum(arg1); >} > >syscall::write:entry >{ > @writes[execname, pid] = count(); > @wbytes[execname, pid] = sum(arg2); >} > >END >{ > normalize(@wbytes, 1024); > normalize(@rbytes, 1024); > > printf("%20s %7s %7s %7s %7s %7s\n", "PROGRAM", "PID", > "READS", "KB", "WRITES", "KB"); > printa("%20s %7d %@7d %@6dK %@7d %@6dK\n", @reads, @rbytes, > @writes, @wbytes); >} > >---8<--- rwsummary.d ---8<--- > >Just running this might yield output like: > > PROGRAM PID READS KB WRITES KB > dtrace 152294 0 0K 1 0K > svcadm 152296 0 0K 31 0K > svcs 152295 1 0K 165 8K > sh 152280 4 0K 3 0K > utmpd 100392 4 2K 0 0K > in.telnetd 152277 4 8K 4 0K > init 1 6 3K 2 0K > svc.configd 100006 58 58K 54 46K > >If I ran the same script with "aggsortrev" set (either via -x, "#pragma >D option" or the setopt() function), I would see the reverse-sorted output: > > PROGRAM PID READS KB WRITES KB > svc.configd 100006 58 58K 54 46K > init 1 6 3K 2 0K > in.telnetd 152277 4 8K 4 0K > utmpd 100392 4 2K 0 0K > sh 152280 4 0K 3 0K > svcs 152295 1 0K 165 8K > svcadm 152296 0 0K 31 0K > dtrace 152294 0 0K 1 0K > >If I set "aggsortpos" to "2", the output would be sorted based on the >number of writes: > > PROGRAM PID READS KB WRITES KB > utmpd 100392 4 2K 0 0K > dtrace 152294 0 0K 1 0K > init 1 6 3K 2 0K > sh 152280 4 0K 3 0K > in.telnetd 152277 4 8K 4 0K > svcadm 152296 0 0K 31 0K > svc.configd 100006 58 58K 54 46K > svcs 152295 1 0K 165 8K > >(The positions number from 0: @reads is position 0, @rbytes is position 1, >@writes is position 2 and @wbytes is position 3.) If I set "aggsortkey", >the aggregation will be sorted in key order: > > PROGRAM PID READS KB WRITES KB > dtrace 152294 0 0K 1 0K > in.telnetd 152277 4 8K 4 0K > init 1 6 3K 2 0K > sh 152280 4 0K 3 0K > svc.configd 100006 58 58K 54 46K > svcadm 152296 0 0K 31 0K > svcs 152295 1 0K 165 8K > utmpd 100392 4 2K 0 0K > >And similarly, if I both set "aggsortkey" and set "aggsortkeypos" to "1": > > PROGRAM PID READS KB WRITES KB > init 1 6 3K 2 0K > svc.configd 100006 58 58K 54 46K > utmpd 100392 4 2K 0 0K > in.telnetd 152277 4 8K 4 0K > sh 152280 4 0K 3 0K > dtrace 152294 0 0K 1 0K > svcs 152295 1 0K 165 8K > svcadm 152296 0 0K 31 0K > >Finally, these options can be set in combination (for example, you >could set "aggsortrev" and "aggsortkey", etc.) and they can be set >dynamically via setopt(): > > setopt("aggsortrev"); > printa(@foo); /* print in reverse-sorted order */ > > setopt("aggsortrev", "false"); > printa(@foo); /* print in sorted order */ > >This wad will be in code review soon (hopefully, tomorrow), and will >be in Solaris after testing is completed and in OpenSolaris and Solaris >Express shortly thereafter... > > - Bryan > >-------------------------------------------------------------------------- >Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc >_______________________________________________ >dtrace-discuss mailing list >dtrace-discuss at opensolaris.org > >
> This is sweet :-) I am waiting for something like this for a while.Yes, it''s the kind of thing that we''ve been meaning to do for a while (the RFE is a million years old, after all), but the implementation was just hard enough to put off. For whatever it''s worth, it was Dragan''s post ("Can I use printa() for printing multiple aggregations?") that pushed us (me) over the edge in terms of doing this now instead of later. Which actually brings me to an important point: if you have things that are annoying you about DTrace, please don''t be afraid to pipe up. Even if we can''t (or don''t) get to it immediately, your voice is important in terms of helping us prioritize. And of course, if we''re really dragging our feet on something, you''re always welcome to take a whack at implementing it yourself... ;)> Now > we can have our own version of prstat equivalent :-)Yes -- very cool!> I guess the important thing would be to have key signature same for your > printa statement. What will happen if the key signature differs?You''ll get an error message very similar to the one that you get if you attempt to use the same aggregation with two different key signatures. e.g.: ---8<--- badsig.d ---8<--- BEGIN { @foo["bar", "baz", 1] = count(); @bar["bippity", "boppity", "boo"] = count(); printa(@foo, @bar); } ---8<--- badsig.d ---8<--- # dtrace -s ./badsig.d dtrace: failed to compile script ./badsig.d: line 2: printa( ): @bar[ ] argument #3 is incompatible with @foo: foo argument #3: int bar argument #3: string # Hopefully the least surprising behavior... - Bryan -------------------------------------------------------------------------- Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc
On Wed, Nov 02, 2005 at 09:28:30AM -0800, Bryan Cantrill wrote:> # dtrace -s ./badsig.d > dtrace: failed to compile script ./badsig.d: line 2: printa( ): @bar[ ] argument #3 is incompatible with @foo: > foo argument #3: int > bar argument #3: string > # > > Hopefully the least surprising behavior...Are the D expressions on which aggregations are parameterized really ''arguments'' or are they ''keys''? The chapter on Aggregations describes them as ''keys''. Adam -- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl
On Wed, Nov 02, 2005 at 11:36:47AM -0800, Adam Leventhal wrote:> On Wed, Nov 02, 2005 at 09:28:30AM -0800, Bryan Cantrill wrote: > > # dtrace -s ./badsig.d > > dtrace: failed to compile script ./badsig.d: line 2: printa( ): @bar[ ] argument #3 is incompatible with @foo: > > foo argument #3: int > > bar argument #3: string > > # > > > > Hopefully the least surprising behavior... > > Are the D expressions on which aggregations are parameterized really > ''arguments'' or are they ''keys''? The chapter on Aggregations describes them > as ''keys''.Yeah, they''re "keys" -- but the current error message for aggregations reports it as an "argument" on a mismatch, and I thought I would keep it consistent. Of course, as Emerson said, "a foolish consistency is the hobgoblin of little minds", so perhaps I should be changing the error message for aggregations instead of emulating it. I''ll look into it... - Bryan -------------------------------------------------------------------------- Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc
I come from the userland usage of DTrace :-) Currently there is no simple way of getting easy acess to most popular metrics through dtrace. (Maybe a poll can be taken on what are the most popular info that users seek in particular providers) Like for pid provider, I want to get very easy access to information leading to %usr, %sys, %idle, Read iops/sec, Write iops/sec, VM usage watermark. This can cut down diagnostic time significantly since this allows to understand how a particular process impacts the system. I always miss scripts when I am on a new system and typing them up again takes lot of time (and typos ...still more .. forgetting data structures.. .still more..) Something around this thought process will simplify DTrace scripts greatly to get more users hooked up to it. Regards, Jignesh Bryan Cantrill wrote:>>This is sweet :-) I am waiting for something like this for a while. >> >> > >Yes, it''s the kind of thing that we''ve been meaning to do for a while (the >RFE is a million years old, after all), but the implementation was just >hard enough to put off. For whatever it''s worth, it was Dragan''s post >("Can I use printa() for printing multiple aggregations?") that pushed us >(me) over the edge in terms of doing this now instead of later. Which >actually brings me to an important point: if you have things that are >annoying you about DTrace, please don''t be afraid to pipe up. Even if >we can''t (or don''t) get to it immediately, your voice is important in >terms of helping us prioritize. And of course, if we''re really dragging >our feet on something, you''re always welcome to take a whack at >implementing it yourself... ;) > > > >>Now >>we can have our own version of prstat equivalent :-) >> >> > >Yes -- very cool! > > > >>I guess the important thing would be to have key signature same for your >>printa statement. What will happen if the key signature differs? >> >> > >You''ll get an error message very similar to the one that you get if you >attempt to use the same aggregation with two different key signatures. >e.g.: > >---8<--- badsig.d ---8<--- > >BEGIN >{ > @foo["bar", "baz", 1] = count(); > @bar["bippity", "boppity", "boo"] = count(); > printa(@foo, @bar); >} > >---8<--- badsig.d ---8<--- > ># dtrace -s ./badsig.d >dtrace: failed to compile script ./badsig.d: line 2: printa( ): @bar[ ] argument #3 is incompatible with @foo: > foo argument #3: int > bar argument #3: string ># > >Hopefully the least surprising behavior... > > - Bryan > >-------------------------------------------------------------------------- >Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc > >
On 11/2/05, Jignesh K. Shah <J.K.Shah at sun.com> wrote:> I come from the userland usage of DTrace :-) > > Currently there is no simple way of getting easy acess to most popular > metrics through dtrace. > (Maybe a poll can be taken on what are the most popular info that users > seek in particular providers) > > Like for pid provider, I want to get very easy access to information > leading to %usr, %sys, %idle, Read iops/sec, Write iops/sec, VM usage > watermark. This can cut down diagnostic time significantly since this > allows to understand how a particular process impacts the system. > > I always miss scripts when I am on a new system and typing them up again > takes lot of time (and typos ...still more .. forgetting data > structures.. .still more..) >Brendan G. took care of writing a bunch of scripts that help the common man (non kernel guru) access to the power of dtrace. Check out his toolkit, others have gone as far as to create shell aliases to grab the latest version and intstall. http://users.tpg.com.au/adsln4yb/dtrace.html#DTraceToolkit James Dickens uadmin.blogspot.com> Something around this thought process will simplify DTrace scripts > greatly to get more users hooked up to it. > > > Regards, > Jignesh > > > > > > Bryan Cantrill wrote: > > >>This is sweet :-) I am waiting for something like this for a while. > >> > >> > > > >Yes, it''s the kind of thing that we''ve been meaning to do for a while (the > >RFE is a million years old, after all), but the implementation was just > >hard enough to put off. For whatever it''s worth, it was Dragan''s post > >("Can I use printa() for printing multiple aggregations?") that pushed us > >(me) over the edge in terms of doing this now instead of later. Which > >actually brings me to an important point: if you have things that are > >annoying you about DTrace, please don''t be afraid to pipe up. Even if > >we can''t (or don''t) get to it immediately, your voice is important in > >terms of helping us prioritize. And of course, if we''re really dragging > >our feet on something, you''re always welcome to take a whack at > >implementing it yourself... ;) > > > > > > > >>Now > >>we can have our own version of prstat equivalent :-) > >> > >> > > > >Yes -- very cool! > > > > > > > >>I guess the important thing would be to have key signature same for your > >>printa statement. What will happen if the key signature differs? > >> > >> > > > >You''ll get an error message very similar to the one that you get if you > >attempt to use the same aggregation with two different key signatures. > >e.g.: > > > >---8<--- badsig.d ---8<--- > > > >BEGIN > >{ > > @foo["bar", "baz", 1] = count(); > > @bar["bippity", "boppity", "boo"] = count(); > > printa(@foo, @bar); > >} > > > >---8<--- badsig.d ---8<--- > > > ># dtrace -s ./badsig.d > >dtrace: failed to compile script ./badsig.d: line 2: printa( ): @bar[ ] argument #3 is incompatible with @foo: > > foo argument #3: int > > bar argument #3: string > ># > > > >Hopefully the least surprising behavior... > > > > - Bryan > > > >-------------------------------------------------------------------------- > >Bryan Cantrill, Solaris Kernel Development. http://blogs.sun.com/bmc > > > > > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org >
On Wed, 2 Nov 2005, James Dickens wrote:> Brendan G. took care of writing a bunch of scripts that help the > common man (non kernel guru) access to the power of dtrace. Check out > his toolkit, others have gone as far as to create shell aliases to > grab the latest version and intstall. > > http://users.tpg.com.au/adsln4yb/dtrace.html#DTraceToolkitBrendan''s toolkit is really useful, and I was so taken by it that I decided to write an article on the toolkit for SysAdmin magazine: http://www.samag.com/documents/s=9915/sam0512a/0512a.htm The article shows how to download and install the toolkit, and covers numerous I/O related scripts (iopattern, rwsnoop, iosnoop, seeksize, bitesize, etc.). Brendan was nice enough to proofread the article, so hopefully the material will be accurate and useful. Hope folks are having a nice day, - Ryan -- UNIX Administrator http://daemons.net/~matty
Sorry, my post has nothing to do with your quesiton, but I just visited your
website - very interesting and worth to check it - and saw the logrotate.sh
script.
Your script logrotate.sh http://daemons.net/~matty/code/logrotate does starting
and stopping the syslogd at the end.
But if you execute
# zero out the logfile ( NEW LOGFILE )
cat /dev/null > ${i}
instead of
cp /dev/null ${i}
then you don''t have to restart the syslogd because the file ${i} will
have the same inode an it is still open for syslogd.
What I don''t know if, when using this method, some syslog messages
could be dropped when a message is just sent during modifying the ${i} file. But
I guess that syslogd got a buffer, large enough to store a few messages in case
of an I/O wait.
Regards
Marcel
This message posted from opensolaris.org
Hi, getting back to a very old question:> > Is there a method to get the number of elements in > an aggregation?> In probe context, no; that information only exists > in the userland address space of the dtrace(1M) > process after it has snapshotted and processed the > aggregation data.OK, so no access in probe context. But what about dtrace:::END? Although being called a probe, I guess it would be the place where the information required could be available (and accessed safely). Is that correct? Would it make sense to request that DTrace provides means to access the number of elements of an aggregation in dtrace:::END? It really would ease postprocessing of collected data. Cheers, Manfred -- This message posted from opensolaris.org