Hi Gluster experts, I have a three brick setup that I'm monitoring with Gluster, which throws a warning when 2 bricks are detected and critical when 1 or less are detected. Every now and again, seemingly randomly, I will get a critical warning on one of the three servers, saying that no bricks are found. This is incorrect as when you log onto the server and check gluster vol status all three bricks are online and healthy. After a few minutes Nagios returns to reporting a healthy volume. Does anyone else monitor their gluster volume with Nagios, and see random critical alerts? Cheers, Gary -- GARY ARMSTRONG Systems Engineer Tibus T: +44 (0)28 9033 1122 F: +44 (0)28 9042 4709 E: garmstrong at tibus.com W: www.tibus.com Follow us on Twitter @tibus Tibus is a trading name of The Internet Business Ltd, a company limited by share capital and registered in Northern Ireland, NI31235. Tibus is a part of UTV Media Plc. This e-mail and any attachment may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorised to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message.
Hello,
I am monitoring with nagios (Centreon to be exact), but not yet with script
provided by gluster dev.
I currently monitor through nrpe:
- Gluster brick:
- Disk space
- glusterd daemon (present and #)
- glusterfs daemon (present and #)
- glusterfsd daemon (present and #)
- Gluster client:
- checking if all mount points are present using this script
<https://github.com/echocat/nagios-plugin-check_mountpoints>
Monitoring do same things all the time. If it's reporting issue, but
resolving quick, you may have a repeating short issue.
Best shot in this case is to manually execute nagios script inside a loop,
like every 10s, to cach it by yourself.
If you wait for nagios alert to log on and try, short issue may already be
gone.
Cordialement,
Mathieu CHATEAU
http://www.lotp.fr
2015-10-20 13:23 GMT+02:00 Gary Armstrong <garmstrong at tibus.com>:
> Hi Gluster experts,
>
> I have a three brick setup that I'm monitoring with Gluster, which
throws
> a warning when 2 bricks are detected and critical when 1 or less are
> detected.
>
> Every now and again, seemingly randomly, I will get a critical warning on
> one of the three servers, saying that no bricks are found. This is
> incorrect as when you log onto the server and check gluster vol status all
> three bricks are online and healthy. After a few minutes Nagios returns to
> reporting a healthy volume.
>
> Does anyone else monitor their gluster volume with Nagios, and see random
> critical alerts?
>
> Cheers,
> Gary
>
> --
> GARY ARMSTRONG
> Systems Engineer
> Tibus
>
> T: +44 (0)28 9033 1122
> F: +44 (0)28 9042 4709
> E: garmstrong at tibus.com
> W: www.tibus.com
>
> Follow us on Twitter @tibus
>
> Tibus is a trading name of The Internet Business Ltd, a company limited by
> share capital and registered in Northern Ireland, NI31235. Tibus is a part
> of UTV Media Plc.
>
>
>
> This e-mail and any attachment may contain confidential and privileged
> information for the sole use of the intended recipient. Any review, use,
> distribution or disclosure by others is strictly prohibited. If you are not
> the intended recipient (or authorised to receive for the recipient), please
> contact the sender by reply e-mail and delete all copies of this message.
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151020/b259e45f/attachment.html>
On Tuesday 20 October 2015 04:53 PM, Gary Armstrong wrote:> Hi Gluster experts, > > I have a three brick setup that I'm monitoring with Gluster, which > throws a warning when 2 bricks are detected and critical when 1 or less > are detected. > > Every now and again, seemingly randomly, I will get a critical warning > on one of the three servers, saying that no bricks are found. This is > incorrect as when you log onto the server and check gluster vol status > all three bricks are online and healthy. After a few minutes Nagios > returns to reporting a healthy volume. > > Does anyone else monitor their gluster volume with Nagios, and see > random critical alerts? >I am wondering if it is due to a transient failure with "gluster volume status". Do you observer errors related to volume status in glusterd/cli logs? Regards, Vijay
On 10/20/2015 04:53 PM, Gary Armstrong wrote:> Hi Gluster experts, > > I have a three brick setup that I'm monitoring with Gluster, which > throws a warning when 2 bricks are detected and critical when 1 or > less are detected. > > Every now and again, seemingly randomly, I will get a critical warning > on one of the three servers, saying that no bricks are found. This is > incorrect as when you log onto the server and check gluster vol status > all three bricks are online and healthy. After a few minutes Nagios > returns to reporting a healthy volume. > > Does anyone else monitor their gluster volume with Nagios, and see > random critical alerts? >We have never experienced this issue in our test setup. Can u post the nagios log from /var/log/nagios/nagios.log. Regards, Ramesh> Cheers, > Gary >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151021/b1c944aa/attachment.html>