Hi All, We have a new and improved version of LMT released on Sourceforge. For those who have never used it, LMT is the Lustre Monitoring Tool developed at LLNL. It provides realtime monitoring of a Lustre filesystem (or multiple filesystems). It also graphs data over time for a set of attributes specified by the user. Please feel free to try it out. Below are the release notes. http://sourceforge.net/projects/lmt =======================================================================Release Notes for LMT 2.6.0 07 Apr 2008 ======================================================================= * This version of LMT 2 has been tested to work properly with Lustre 1.6.x It is no longer necessary to keep the old lustre *.xml configuration and there are no more xwatch-lustre.conf or lmt.conf files to set up. * LMT now uses Cerebro to transport the data collected from the Lustre filesystem being monitored. For more info see: http://sourceforge.net/projects/cerebro * Building the rpms requires ibm-java since this is what we use in Chaos 4. This can be changed in the specfile to require Sun Java if that is preferred (the main issue is that we need a full fledged JVM that supports swing). * There is now a dependency on mysql to store the collected data. Since the resolution of the collection is ~5 seconds this can quickly consume a lot of space. The clients and servers are completely decoupled. * Please refer to the documentation for instructions on how to install and configure LMT 2.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi All, We have a new and improved version of LMT released on Sourceforge. For those who have never used it, LMT is the Lustre Monitoring Tool developed at LLNL. It provides realtime monitoring of a Lustre filesystem (or multiple filesystems). It also graphs data over time for a set of attributes specified by the user. Please feel free to try it out. Below are the release notes. (please forgive this if it is a repost as it looks like lustre-discuss may have finally come back up) http://sourceforge.net/projects/lmt =======================================================================Release Notes for LMT 2.6.0 07 Apr 2008 ======================================================================= * This version of LMT 2 has been tested to work properly with Lustre 1.6.x It is no longer necessary to keep the old lustre *.xml configuration and there are no more xwatch-lustre.conf or lmt.conf files to set up. * LMT now uses Cerebro to transport the data collected from the Lustre filesystem being monitored. For more info see: http://sourceforge.net/projects/cerebro * Building the rpms requires ibm-java since this is what we use in Chaos 4. This can be changed in the specfile to require Sun Java if that is preferred (the main issue is that we need a full fledged JVM that supports swing). * There is now a dependency on mysql to store the collected data. Since the resolution of the collection is ~5 seconds this can quickly consume a lot of space. The clients and servers are completely decoupled. * Please refer to the documentation for instructions on how to install and configure LMT 2. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 NwuPz3GIhOU0zAeZDrueQhA=QWFx -----END PGP SIGNATURE-----
Are there any screenshots/FAQs? I looked at the package, and the depandancy list is long. As most of my work is remote, does the GUI run remotely over an SSH tunnel? Are there any similar console/text-based utilities w/ a shorter dependency list and lighter weight for remote access? Thanks, Chris On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > > Hi All, > We have a new and improved version of LMT released > on Sourceforge. For those who have never used it, LMT is the Lustre > Monitoring Tool developed at LLNL. It provides realtime monitoring of a > Lustre filesystem (or multiple filesystems). It also graphs data over time > for a set of attributes specified by the user. Please feel free to try it > out. > > Below are the release notes. > (please forgive this if it is a repost as it looks like lustre-discuss > may have finally come back up) > > > http://sourceforge.net/projects/lmt > > =======================================================================> Release Notes for LMT 2.6.0 07 Apr 2008 > =======================================================================> > * This version of LMT 2 has been tested to work properly with Lustre 1.6.x > It is no longer necessary to keep the old lustre *.xml configuration and > there are no more xwatch-lustre.conf or lmt.conf files to set up. > > * LMT now uses Cerebro to transport the data collected from the Lustre > filesystem being monitored. For more info see: > http://sourceforge.net/projects/cerebro > > * Building the rpms requires ibm-java since this is what we use in Chaos 4. > This can be changed in the specfile to require Sun Java if that is > preferred (the main issue is that we need a full fledged JVM that > supports swing). > > * There is now a dependency on mysql to store the collected data. Since the > resolution of the collection is ~5 seconds this can quickly consume a lot > of space. The clients and servers are completely decoupled. > > * Please refer to the documentation for instructions on how to install and > configure LMT 2. > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org > > iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 > NwuPz3GIhOU0zAeZDrueQhA> =QWFx > -----END PGP SIGNATURE----- > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
Hi Chris, I think Herb is reporting for jury duty today so I''ll speak up and let him follow up later if I missed anything. There are text clients (ltop and lstat). I haven''t tried running the GUI remotely and tunneling the X session, or running the client locally and tunneling mysql protocol, but I don''t know of anything architectural that would prevent either from working. We''ll get some screenshots out there. Herb was reticent to publish LMT without good documentation, but I encouraged him because there were a couple of people anxious to try it out. So the lack of screenshots/ documentation is my bad - we''ll try to fix that as time permits. Regarding the dependencies: it''s always a tradeoff between reinventing the wheel and adding dependencies. Sigh. In this case, we leveraged MySQL for the database of historical data; we managed to get a graphics group here which is a Java shop to code the clients, so they interface directly to MySQL using the Java MySQL bindings; and we leveraged the Cerebro multicast based monitoring tool for the data collection because it introduced no new dependencies on lustre servers at our site (since we already use it to monitor other things), and is very lightweight. I think this is a good architecture: the clients and data collection parts are independent with interfaces defined by the database schema, so they could be independently replaced, for example if someone wanted to write text clients in C/curses/MySQL, or replace the backend with some other data collection infrastructure that they are already using or feel is superior. Jim On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote:> Are there any screenshots/FAQs? > > I looked at the package, and the depandancy list is long. > > As most of my work is remote, does the GUI run remotely over an SSH tunnel? > > Are there any similar console/text-based utilities w/ a shorter > dependency list and lighter weight for remote access? > > Thanks, > > Chris > On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > > > Hi All, > > We have a new and improved version of LMT released > > on Sourceforge. For those who have never used it, LMT is the Lustre > > Monitoring Tool developed at LLNL. It provides realtime monitoring of a > > Lustre filesystem (or multiple filesystems). It also graphs data over time > > for a set of attributes specified by the user. Please feel free to try it > > out. > > > > Below are the release notes. > > (please forgive this if it is a repost as it looks like lustre-discuss > > may have finally come back up) > > > > > > http://sourceforge.net/projects/lmt > > > > =======================================================================> > Release Notes for LMT 2.6.0 07 Apr 2008 > > =======================================================================> > > > * This version of LMT 2 has been tested to work properly with Lustre 1.6.x > > It is no longer necessary to keep the old lustre *.xml configuration and > > there are no more xwatch-lustre.conf or lmt.conf files to set up. > > > > * LMT now uses Cerebro to transport the data collected from the Lustre > > filesystem being monitored. For more info see: > > http://sourceforge.net/projects/cerebro > > > > * Building the rpms requires ibm-java since this is what we use in Chaos 4. > > This can be changed in the specfile to require Sun Java if that is > > preferred (the main issue is that we need a full fledged JVM that > > supports swing). > > > > * There is now a dependency on mysql to store the collected data. Since the > > resolution of the collection is ~5 seconds this can quickly consume a lot > > of space. The clients and servers are completely decoupled. > > > > * Please refer to the documentation for instructions on how to install and > > configure LMT 2. > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG v1.4.7 (GNU/Linux) > > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org > > > > iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 > > NwuPz3GIhOU0zAeZDrueQhA> > =QWFx > > -----END PGP SIGNATURE----- > > > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
Hi Jim, Out of curiosity (and bear with me if this is a stupid question), do you know if anyone''s integrated LMT with Ganglia? The clusters we have that use Lustre are all ROCKS-based, which uses Ganglia for monitoring ... I''m not familiar with Cerebro, so I''m not sure how you''re using it, but it would be nice to be able to integrate LMT''s statistics into an existing monitoring solution rather than implementing a second one. Other than that, I''m looking forward to screenshots -- this sounds like a nice add-on. cheers, Klaus On 4/9/08 8:59 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone tablets:> Hi Chris, > > I think Herb is reporting for jury duty today so I''ll speak up and let him > follow up later if I missed anything. > > There are text clients (ltop and lstat). I haven''t tried running > the GUI remotely and tunneling the X session, or running the client > locally and tunneling mysql protocol, but I don''t know of anything > architectural that would prevent either from working. > > We''ll get some screenshots out there. Herb was reticent to publish > LMT without good documentation, but I encouraged him because there were > a couple of people anxious to try it out. So the lack of screenshots/ > documentation is my bad - we''ll try to fix that as time permits. > > Regarding the dependencies: it''s always a tradeoff between reinventing the > wheel and adding dependencies. Sigh. In this case, we leveraged MySQL for > the database of historical data; we managed to get a graphics group here > which is a Java shop to code the clients, so they interface directly to > MySQL using the Java MySQL bindings; and we leveraged the Cerebro multicast > based monitoring tool for the data collection because it introduced no > new dependencies on lustre servers at our site (since we already use it > to monitor other things), and is very lightweight. > > I think this is a good architecture: the clients and data collection > parts are independent with interfaces defined by the database schema, > so they could be independently replaced, for example if someone wanted to > write text clients in C/curses/MySQL, or replace the backend with some > other data collection infrastructure that they are already using or feel > is superior. > > Jim > > On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote: >> Are there any screenshots/FAQs? >> >> I looked at the package, and the depandancy list is long. >> >> As most of my work is remote, does the GUI run remotely over an SSH tunnel? >> >> Are there any similar console/text-based utilities w/ a shorter >> dependency list and lighter weight for remote access? >> >> Thanks, >> >> Chris >> On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA512 >>> >>> >>> Hi All, >>> We have a new and improved version of LMT released >>> on Sourceforge. For those who have never used it, LMT is the Lustre >>> Monitoring Tool developed at LLNL. It provides realtime monitoring of a >>> Lustre filesystem (or multiple filesystems). It also graphs data over time >>> for a set of attributes specified by the user. Please feel free to try it >>> out. >>> >>> Below are the release notes. >>> (please forgive this if it is a repost as it looks like lustre-discuss >>> may have finally come back up) >>> >>> >>> http://sourceforge.net/projects/lmt >>> >>> =======================================================================>>> Release Notes for LMT 2.6.0 07 Apr 2008 >>> =======================================================================>>> >>> * This version of LMT 2 has been tested to work properly with Lustre 1.6.x >>> It is no longer necessary to keep the old lustre *.xml configuration and >>> there are no more xwatch-lustre.conf or lmt.conf files to set up. >>> >>> * LMT now uses Cerebro to transport the data collected from the Lustre >>> filesystem being monitored. For more info see: >>> http://sourceforge.net/projects/cerebro >>> >>> * Building the rpms requires ibm-java since this is what we use in Chaos 4. >>> This can be changed in the specfile to require Sun Java if that is >>> preferred (the main issue is that we need a full fledged JVM that >>> supports swing). >>> >>> * There is now a dependency on mysql to store the collected data. Since the >>> resolution of the collection is ~5 seconds this can quickly consume a lot >>> of space. The clients and servers are completely decoupled. >>> >>> * Please refer to the documentation for instructions on how to install and >>> configure LMT 2. >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1.4.7 (GNU/Linux) >>> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org >>> >>> iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 >>> NwuPz3GIhOU0zAeZDrueQhA>>> =QWFx >>> -----END PGP SIGNATURE----- >>> >>> >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
Hi Klaus, Nobody''s done that yet that I know of, but it would seem doable. Cerebro is similar to ganglia in that it uses multicast with listeners and talkers. You would need machinery to get lustre proc values into ganglia on the lustre servers (the talkers), and then from ganglia to the mysql database (the listener). Jim On Wed, Apr 09, 2008 at 11:02:49AM -0700, Klaus Steden wrote:> > Hi Jim, > > Out of curiosity (and bear with me if this is a stupid question), do you > know if anyone''s integrated LMT with Ganglia? The clusters we have that use > Lustre are all ROCKS-based, which uses Ganglia for monitoring ... I''m not > familiar with Cerebro, so I''m not sure how you''re using it, but it would be > nice to be able to integrate LMT''s statistics into an existing monitoring > solution rather than implementing a second one. > > Other than that, I''m looking forward to screenshots -- this sounds like a > nice add-on. > > cheers, > Klaus > > On 4/9/08 8:59 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone > tablets: > > > Hi Chris, > > > > I think Herb is reporting for jury duty today so I''ll speak up and let him > > follow up later if I missed anything. > > > > There are text clients (ltop and lstat). I haven''t tried running > > the GUI remotely and tunneling the X session, or running the client > > locally and tunneling mysql protocol, but I don''t know of anything > > architectural that would prevent either from working. > > > > We''ll get some screenshots out there. Herb was reticent to publish > > LMT without good documentation, but I encouraged him because there were > > a couple of people anxious to try it out. So the lack of screenshots/ > > documentation is my bad - we''ll try to fix that as time permits. > > > > Regarding the dependencies: it''s always a tradeoff between reinventing the > > wheel and adding dependencies. Sigh. In this case, we leveraged MySQL for > > the database of historical data; we managed to get a graphics group here > > which is a Java shop to code the clients, so they interface directly to > > MySQL using the Java MySQL bindings; and we leveraged the Cerebro multicast > > based monitoring tool for the data collection because it introduced no > > new dependencies on lustre servers at our site (since we already use it > > to monitor other things), and is very lightweight. > > > > I think this is a good architecture: the clients and data collection > > parts are independent with interfaces defined by the database schema, > > so they could be independently replaced, for example if someone wanted to > > write text clients in C/curses/MySQL, or replace the backend with some > > other data collection infrastructure that they are already using or feel > > is superior. > > > > Jim > > > > On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote: > >> Are there any screenshots/FAQs? > >> > >> I looked at the package, and the depandancy list is long. > >> > >> As most of my work is remote, does the GUI run remotely over an SSH tunnel? > >> > >> Are there any similar console/text-based utilities w/ a shorter > >> dependency list and lighter weight for remote access? > >> > >> Thanks, > >> > >> Chris > >> On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: > >>> -----BEGIN PGP SIGNED MESSAGE----- > >>> Hash: SHA512 > >>> > >>> > >>> Hi All, > >>> We have a new and improved version of LMT released > >>> on Sourceforge. For those who have never used it, LMT is the Lustre > >>> Monitoring Tool developed at LLNL. It provides realtime monitoring of a > >>> Lustre filesystem (or multiple filesystems). It also graphs data over time > >>> for a set of attributes specified by the user. Please feel free to try it > >>> out. > >>> > >>> Below are the release notes. > >>> (please forgive this if it is a repost as it looks like lustre-discuss > >>> may have finally come back up) > >>> > >>> > >>> http://sourceforge.net/projects/lmt > >>> > >>> =======================================================================> >>> Release Notes for LMT 2.6.0 07 Apr 2008 > >>> =======================================================================> >>> > >>> * This version of LMT 2 has been tested to work properly with Lustre 1.6.x > >>> It is no longer necessary to keep the old lustre *.xml configuration and > >>> there are no more xwatch-lustre.conf or lmt.conf files to set up. > >>> > >>> * LMT now uses Cerebro to transport the data collected from the Lustre > >>> filesystem being monitored. For more info see: > >>> http://sourceforge.net/projects/cerebro > >>> > >>> * Building the rpms requires ibm-java since this is what we use in Chaos 4. > >>> This can be changed in the specfile to require Sun Java if that is > >>> preferred (the main issue is that we need a full fledged JVM that > >>> supports swing). > >>> > >>> * There is now a dependency on mysql to store the collected data. Since the > >>> resolution of the collection is ~5 seconds this can quickly consume a lot > >>> of space. The clients and servers are completely decoupled. > >>> > >>> * Please refer to the documentation for instructions on how to install and > >>> configure LMT 2. > >>> -----BEGIN PGP SIGNATURE----- > >>> Version: GnuPG v1.4.7 (GNU/Linux) > >>> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org > >>> > >>> iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 > >>> NwuPz3GIhOU0zAeZDrueQhA> >>> =QWFx > >>> -----END PGP SIGNATURE----- > >>> > >>> > >>> _______________________________________________ > >>> Lustre-discuss mailing list > >>> Lustre-discuss at lists.lustre.org > >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss > >>> > >> _______________________________________________ > >> Lustre-discuss mailing list > >> Lustre-discuss at lists.lustre.org > >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss
Hi Jim, Hmmm, good to know ... does LMT support shell call-outs? If so, it would be easy enough to have it populate a Ganglia receptor using gmetric (using the older pre-3.1.x framework) ... not sure about recording from Ganglia to mysql, although it would record historical data using rrdtool. Klaus On 4/9/08 11:32 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone tablets:> Hi Klaus, > > Nobody''s done that yet that I know of, but it would seem doable. > > Cerebro is similar to ganglia in that it uses multicast with > listeners and talkers. You would need machinery to get lustre proc > values into ganglia on the lustre servers (the talkers), and then from > ganglia to the mysql database (the listener). > > Jim >
If the goal is to get lustre data into ganglia and use existing mechanisms to look at the data, LMT is probably not the right path. But if you wanted to set up LMT and just replace the back-end data collection mechanism, that is probably doable. I''m afraid my knowledge of ganglia falls short here but I would assume you could craft a shell script that feeds lustre /proc values to gmetric or write some sort of "native" ganglia module to do the job more efficiently. Jim On Wed, Apr 09, 2008 at 11:57:01AM -0700, Klaus Steden wrote:> > Hi Jim, > > Hmmm, good to know ... does LMT support shell call-outs? If so, it would be > easy enough to have it populate a Ganglia receptor using gmetric (using the > older pre-3.1.x framework) ... not sure about recording from Ganglia to > mysql, although it would record historical data using rrdtool. > > Klaus > > On 4/9/08 11:32 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone > tablets: > > > Hi Klaus, > > > > Nobody''s done that yet that I know of, but it would seem doable. > > > > Cerebro is similar to ganglia in that it uses multicast with > > listeners and talkers. You would need machinery to get lustre proc > > values into ganglia on the lustre servers (the talkers), and then from > > ganglia to the mysql database (the listener). > > > > Jim > >
> Hi Chris, > > I think Herb is reporting for jury duty today so I''ll speak up and let him > follow up later if I missed anything. > > There are text clients (ltop and lstat). I haven''t tried running > the GUI remotely and tunneling the X session, or running the client > locally and tunneling mysql protocol, but I don''t know of anything > architectural that would prevent either from working. >Hi Chris, As Jim stated you should have no problem running the GUI over an SSH tunnel. The performance may suffer though depending on your network setup. If you prefer could either of the text-based clients ltop or lstat.> We''ll get some screenshots out there. Herb was reticent to publish > LMT without good documentation, but I encouraged him because there were > a couple of people anxious to try it out. So the lack of screenshots/ > documentation is my bad - we''ll try to fix that as time permits. >I actually updated the documentation on the site. It is not great, but should help you get started. There is no FAQ yet though. I will probably build one up as people encounter issues.> Regarding the dependencies: it''s always a tradeoff between reinventing the > wheel and adding dependencies. Sigh. In this case, we leveraged MySQL > for > the database of historical data; we managed to get a graphics group here > which is a Java shop to code the clients, so they interface directly to > MySQL using the Java MySQL bindings; and we leveraged the Cerebro > multicast > based monitoring tool for the data collection because it introduced no > new dependencies on lustre servers at our site (since we already use it > to monitor other things), and is very lightweight. > > I think this is a good architecture: the clients and data collection > parts are independent with interfaces defined by the database schema, > so they could be independently replaced, for example if someone wanted to > write text clients in C/curses/MySQL, or replace the backend with some > other data collection infrastructure that they are already using or feel > is superior. > > Jim >> On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote: >> Are there any screenshots/FAQs? >> >> I looked at the package, and the depandancy list is long. >> >> As most of my work is remote, does the GUI run remotely over an SSH >> tunnel? >> >> Are there any similar console/text-based utilities w/ a shorter >> dependency list and lighter weight for remote access? >> >> Thanks, >> >> Chris >> On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: >> > -----BEGIN PGP SIGNED MESSAGE----- >> > Hash: SHA512 >> > >> > >> > Hi All, >> > We have a new and improved version of LMT released >> > on Sourceforge. For those who have never used it, LMT is the Lustre >> > Monitoring Tool developed at LLNL. It provides realtime monitoring >> of a >> > Lustre filesystem (or multiple filesystems). It also graphs data >> over time >> > for a set of attributes specified by the user. Please feel free to >> try it >> > out. >> > >> > Below are the release notes. >> > (please forgive this if it is a repost as it looks like >> lustre-discuss >> > may have finally come back up) >> > >> > >> > http://sourceforge.net/projects/lmt >> > >> > =======================================================================>> > Release Notes for LMT 2.6.0 07 Apr >> 2008 >> > =======================================================================>> > >> > * This version of LMT 2 has been tested to work properly with Lustre >> 1.6.x >> > It is no longer necessary to keep the old lustre *.xml configuration >> and >> > there are no more xwatch-lustre.conf or lmt.conf files to set up. >> > >> > * LMT now uses Cerebro to transport the data collected from the >> Lustre >> > filesystem being monitored. For more info see: >> > http://sourceforge.net/projects/cerebro >> > >> > * Building the rpms requires ibm-java since this is what we use in >> Chaos 4. >> > This can be changed in the specfile to require Sun Java if that is >> > preferred (the main issue is that we need a full fledged JVM that >> > supports swing). >> > >> > * There is now a dependency on mysql to store the collected data. >> Since the >> > resolution of the collection is ~5 seconds this can quickly consume >> a lot >> > of space. The clients and servers are completely decoupled. >> > >> > * Please refer to the documentation for instructions on how to >> install and >> > configure LMT 2. >> > -----BEGIN PGP SIGNATURE----- >> > Version: GnuPG v1.4.7 (GNU/Linux) >> > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org >> > >> > iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 >> > NwuPz3GIhOU0zAeZDrueQhA>> > =QWFx >> > -----END PGP SIGNATURE----- >> > >> > >> > _______________________________________________ >> > Lustre-discuss mailing list >> > Lustre-discuss at lists.lustre.org >> > http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
> > Hi Jim, > > Out of curiosity (and bear with me if this is a stupid question), do you > know if anyone''s integrated LMT with Ganglia? The clusters we have that > use > Lustre are all ROCKS-based, which uses Ganglia for monitoring ... I''m not > familiar with Cerebro, so I''m not sure how you''re using it, but it would > be > nice to be able to integrate LMT''s statistics into an existing monitoring > solution rather than implementing a second one. > > Other than that, I''m looking forward to screenshots -- this sounds like a > nice add-on. > > cheers, > Klaus >Klaus, No I don''t know of anyone who has integrated LMT with ganglia. The way that LMT is currently designed we have the LMT client(s) and LMT server(s) (actually just cerebro modules) completely decoupled. To integrate LMT into Ganglia you would need to ganglia modules to collect the data and put it into the database. You would not need to modify the clients at all. At LLNL we use Cerebro instead of Ganglia, so for us it made sense to use our existing monitoring solution. I do not kow much about Ganglia, but from what I understand, Cerebro was inspired by Ganglia, so hopefully it would not be too difficult to write the Ganglia modules for it. I would be happy to give you information about the database schema if you are up for it...=)> On 4/9/08 8:59 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone > tablets: > >> Hi Chris, >> >> I think Herb is reporting for jury duty today so I''ll speak up and let >> him >> follow up later if I missed anything. >> >> There are text clients (ltop and lstat). I haven''t tried running >> the GUI remotely and tunneling the X session, or running the client >> locally and tunneling mysql protocol, but I don''t know of anything >> architectural that would prevent either from working. >> >> We''ll get some screenshots out there. Herb was reticent to publish >> LMT without good documentation, but I encouraged him because there were >> a couple of people anxious to try it out. So the lack of screenshots/ >> documentation is my bad - we''ll try to fix that as time permits. >> >> Regarding the dependencies: it''s always a tradeoff between reinventing >> the >> wheel and adding dependencies. Sigh. In this case, we leveraged MySQL >> for >> the database of historical data; we managed to get a graphics group here >> which is a Java shop to code the clients, so they interface directly to >> MySQL using the Java MySQL bindings; and we leveraged the Cerebro >> multicast >> based monitoring tool for the data collection because it introduced no >> new dependencies on lustre servers at our site (since we already use it >> to monitor other things), and is very lightweight. >> >> I think this is a good architecture: the clients and data collection >> parts are independent with interfaces defined by the database schema, >> so they could be independently replaced, for example if someone wanted >> to >> write text clients in C/curses/MySQL, or replace the backend with some >> other data collection infrastructure that they are already using or feel >> is superior. >> >> Jim >> >> On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote: >>> Are there any screenshots/FAQs? >>> >>> I looked at the package, and the depandancy list is long. >>> >>> As most of my work is remote, does the GUI run remotely over an SSH >>> tunnel? >>> >>> Are there any similar console/text-based utilities w/ a shorter >>> dependency list and lighter weight for remote access? >>> >>> Thanks, >>> >>> Chris >>> On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA512 >>>> >>>> >>>> Hi All, >>>> We have a new and improved version of LMT released >>>> on Sourceforge. For those who have never used it, LMT is the Lustre >>>> Monitoring Tool developed at LLNL. It provides realtime monitoring >>>> of a >>>> Lustre filesystem (or multiple filesystems). It also graphs data >>>> over time >>>> for a set of attributes specified by the user. Please feel free to >>>> try it >>>> out. >>>> >>>> Below are the release notes. >>>> (please forgive this if it is a repost as it looks like >>>> lustre-discuss >>>> may have finally come back up) >>>> >>>> >>>> http://sourceforge.net/projects/lmt >>>> >>>> =======================================================================>>>> Release Notes for LMT 2.6.0 07 Apr >>>> 2008 >>>> =======================================================================>>>> >>>> * This version of LMT 2 has been tested to work properly with Lustre >>>> 1.6.x >>>> It is no longer necessary to keep the old lustre *.xml configuration >>>> and >>>> there are no more xwatch-lustre.conf or lmt.conf files to set up. >>>> >>>> * LMT now uses Cerebro to transport the data collected from the >>>> Lustre >>>> filesystem being monitored. For more info see: >>>> http://sourceforge.net/projects/cerebro >>>> >>>> * Building the rpms requires ibm-java since this is what we use in >>>> Chaos 4. >>>> This can be changed in the specfile to require Sun Java if that is >>>> preferred (the main issue is that we need a full fledged JVM that >>>> supports swing). >>>> >>>> * There is now a dependency on mysql to store the collected data. >>>> Since the >>>> resolution of the collection is ~5 seconds this can quickly consume >>>> a lot >>>> of space. The clients and servers are completely decoupled. >>>> >>>> * Please refer to the documentation for instructions on how to >>>> install and >>>> configure LMT 2. >>>> -----BEGIN PGP SIGNATURE----- >>>> Version: GnuPG v1.4.7 (GNU/Linux) >>>> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org >>>> >>>> iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 >>>> NwuPz3GIhOU0zAeZDrueQhA>>>> =QWFx >>>> -----END PGP SIGNATURE----- >>>> >>>> >>>> _______________________________________________ >>>> Lustre-discuss mailing list >>>> Lustre-discuss at lists.lustre.org >>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>>> >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
Klaus, As the LMT backend is currently a set of Cerebro modules, it does not have to ability to support callouts on its own. Cerebro supports the callouts that you would be looking for with cerebro-stat. As Jim mentioned you need the listener (monitor module), to populate the mysql database if you want to use the LMT clients since they contact the databse for their information. -Herb> > Hi Jim, > > Hmmm, good to know ... does LMT support shell call-outs? If so, it would > be > easy enough to have it populate a Ganglia receptor using gmetric (using > the > older pre-3.1.x framework) ... not sure about recording from Ganglia to > mysql, although it would record historical data using rrdtool. > > Klaus > > On 4/9/08 11:32 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone > tablets: > >> Hi Klaus, >> >> Nobody''s done that yet that I know of, but it would seem doable. >> >> Cerebro is similar to ganglia in that it uses multicast with >> listeners and talkers. You would need machinery to get lustre proc >> values into ganglia on the lustre servers (the talkers), and then from >> ganglia to the mysql database (the listener). >> >> Jim >> > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
> Are there any screenshots/FAQs? >I just replaced the older LMT screenshots that were on sourceforge with a few new ones. The filesystem view looks largely the same as it did in the older versions (although now there is also on OSS panel in addition to the OST panel). There are also now plot windows (that currently only work properly for OSTs and routers). These can plot history or if you want it can plot real-time data as well.> I looked at the package, and the depandancy list is long. > > As most of my work is remote, does the GUI run remotely over an SSH > tunnel? > > Are there any similar console/text-based utilities w/ a shorter > dependency list and lighter weight for remote access? > > Thanks, > > Chris > On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> >> Hi All, >> We have a new and improved version of LMT released >> on Sourceforge. For those who have never used it, LMT is the Lustre >> Monitoring Tool developed at LLNL. It provides realtime monitoring of >> a >> Lustre filesystem (or multiple filesystems). It also graphs data over >> time >> for a set of attributes specified by the user. Please feel free to try >> it >> out. >> >> Below are the release notes. >> (please forgive this if it is a repost as it looks like lustre-discuss >> may have finally come back up) >> >> >> http://sourceforge.net/projects/lmt >> >> =======================================================================>> Release Notes for LMT 2.6.0 07 Apr >> 2008 >> =======================================================================>> >> * This version of LMT 2 has been tested to work properly with Lustre >> 1.6.x >> It is no longer necessary to keep the old lustre *.xml configuration >> and >> there are no more xwatch-lustre.conf or lmt.conf files to set up. >> >> * LMT now uses Cerebro to transport the data collected from the Lustre >> filesystem being monitored. For more info see: >> http://sourceforge.net/projects/cerebro >> >> * Building the rpms requires ibm-java since this is what we use in >> Chaos 4. >> This can be changed in the specfile to require Sun Java if that is >> preferred (the main issue is that we need a full fledged JVM that >> supports swing). >> >> * There is now a dependency on mysql to store the collected data. Since >> the >> resolution of the collection is ~5 seconds this can quickly consume a >> lot >> of space. The clients and servers are completely decoupled. >> >> * Please refer to the documentation for instructions on how to install >> and >> configure LMT 2. >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.7 (GNU/Linux) >> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org >> >> iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5 >> NwuPz3GIhOU0zAeZDrueQhA>> =QWFx >> -----END PGP SIGNATURE----- >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >