Hello, I''n updating Devilator, the performance data collector for Orca and FreeBSD to include ZFS monitoring. So far I am graphing the ARC and L2ARC size, L2ARC writes and reads, and several hit/misses data pairs. Any suggestions to improve it? What other variables can be interesting? An example of the current state of the program is here: http://devilator.frobula.com Thanks, Borja.
On Mon, Feb 11, 2013 at 9:53 AM, Borja Marcos <borjam at sarenet.es> wrote:> > Hello, > > I''n updating Devilator, the performance data collector for Orca and > FreeBSD to include ZFS monitoring. So far I am graphing the ARC and L2ARC > size, L2ARC writes and reads, and several hit/misses data pairs. > > Any suggestions to improve it? What other variables can be interesting? > > An example of the current state of the program is here: > > http://devilator.frobula.com > > Thanks, > > > > > > Borja. >The zpool iostat output has all sorts of statistics I think would be useful/interesting to record over time. --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130211/93ce6735/attachment.html>
On Feb 11, 2013, at 4:56 PM, Tim Cook wrote:> The zpool iostat output has all sorts of statistics I think would be useful/interesting to record over time.Yes, thanks :) I think I will add them, I just started with the esoteric ones. Anyway, still there''s no better way to read it than running zpool iostat and parsing the output, right? Borja.
On 02/11/2013 04:53 PM, Borja Marcos wrote:> > Hello, > > I''n updating Devilator, the performance data collector for Orca and FreeBSD to include ZFS monitoring. So far I am graphing the ARC and L2ARC size, L2ARC writes and reads, and several hit/misses data pairs. > > Any suggestions to improve it? What other variables can be interesting? > > An example of the current state of the program is here: > > http://devilator.frobula.comHi Borja, I''ve got one thing up for review in Illumos for upstreaming: #3137 L2ARC Compression. This adds another kstat called l2_asize, which tells you how big the L2ARC actually is, taking into account compression, e.g. # kstat -n arcstats 5 | egrep ''(\<l2_size\>|\<l2_asize\>)'' l2_asize 25708032 l2_size 29117952 You can use this to track L2ARC compression efficiency, etc. If the kstat is not present, then L2ARC compression isn''t available. Anyway, just a quick thought. -- Saso
On 2013-02-11 17:14, Borja Marcos wrote:> > On Feb 11, 2013, at 4:56 PM, Tim Cook wrote: > >> The zpool iostat output has all sorts of statistics I think would be useful/interesting to record over time. > > > Yes, thanks :) I think I will add them, I just started with the esoteric ones. > > Anyway, still there''s no better way to read it than running zpool iostat and parsing the output, right?I believe, in this case you''d have to run it as a continuous process and parse the outputs after the first one (overall uptime stat, IIRC). Also note that on problems with ZFS engine itself, "zpool" may lock up and thus halt your program - so have it ready to abort an outstanding statistics read after a timeout and perhaps log an error. And if pools are imported-exported during work, the "zpool iostat" output changes dynamically, so you basically need to parse its text structure every time. The "zpool iostat -v" might be even more interesting though, as it lets you see per-vdev statistics and perhaps notice imbalances, etc... All that said, I don''t know if this data isn''t also available as some set of kstats - that would probably be a lot better for your cause. Inspect the "zpool" source to see where it gets its numbers from... and perhaps make and RTI relevant kstats, if they aren''t yet there ;) On the other hand, I am not certain how Solaris-based kstats interact or correspond to structures in FreeBSD (or Linux for that matter)?.. HTH, //Jim Klimov
On Mon, Feb 11, 2013 at 05:39:27PM +0100, Jim Klimov wrote:> On 2013-02-11 17:14, Borja Marcos wrote: > > > > On Feb 11, 2013, at 4:56 PM, Tim Cook wrote: > > > >> The zpool iostat output has all sorts of statistics I think would be useful/interesting to record over time. > > > > > > Yes, thanks :) I think I will add them, I just started with the esoteric ones. > > > > Anyway, still there''s no better way to read it than running zpool iostat and parsing the output, right? > > > I believe, in this case you''d have to run it as a continuous process > and parse the outputs after the first one (overall uptime stat, IIRC). > Also note that on problems with ZFS engine itself, "zpool" may lock up > and thus halt your program - so have it ready to abort an outstanding > statistics read after a timeout and perhaps log an error. > > And if pools are imported-exported during work, the "zpool iostat" > output changes dynamically, so you basically need to parse its text > structure every time. > > The "zpool iostat -v" might be even more interesting though, as it lets > you see per-vdev statistics and perhaps notice imbalances, etc... > > All that said, I don''t know if this data isn''t also available as some > set of kstats - that would probably be a lot better for your cause. > Inspect the "zpool" source to see where it gets its numbers from... > and perhaps make and RTI relevant kstats, if they aren''t yet there ;) > > On the other hand, I am not certain how Solaris-based kstats interact > or correspond to structures in FreeBSD (or Linux for that matter)?..I made kstat data available on FreeBSD via ''kstat'' sysctl tree: # sysctl kstat -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130212/9fa61964/attachment.bin>
On Feb 12, 2013, at 11:25 AM, Pawel Jakub Dawidek wrote:> I made kstat data available on FreeBSD via ''kstat'' sysctl tree:Yes, I am using the data. I wasn''t sure about how getting something meaningful from it, but I''ve found the arcstats.pl script and I am using it as a model. Suggestions will be always welcome, though :) (the sample pages I put on devilator.froblua.com aren''t using the better organized graphs, though, it''s just a crude parameter dump) Borja.