Can you give some more details about the event. Are the open fds files or
directories? are they deleted? Are you accessing the filesystem via NFS or
FUSE?
Thanks,
Avati
On Tue, Aug 2, 2011 at 5:33 PM, Dave Bour
<dcbour at desktopsolutioncenter.ca>wrote:
> I've got a 2 peer replicated setup with 2 bricks on each system.
>
> Starting sometime overnight, one system ground to a halt, to the point a
> local login was not possible.
> Restarted.
> A couple hours later I tried to log in via SSH, too many open files. A
> quick 'lsof" showed thousands of open files on glusterfs.
>
> I just restarted the server again...and appears to be doing the same
> thing. Addition of nearly 2500 files in 4 minutes, not 5 minutes after
> rebooting.
>
> for d in {1..1000};do a=`date`;b=`lsof|grep gluster|wc -l`;echo $a "
"
> $b;sleep 60;done
> Tue Aug 2 07:56:37 EDT 2011 2924
> Tue Aug 2 07:57:37 EDT 2011 3568
> Tue Aug 2 07:58:38 EDT 2011 4154
> Tue Aug 2 07:59:38 EDT 2011 4718
> Tue Aug 2 08:00:38 EDT 2011 5376
>
> It's peer is more what's expected:
> for d in {1..1000};do a=`date`;b=`lsof|grep gluster|wc -l`;echo $a "
"
> $b;sleep 60;done
> Tue Aug 2 07:57:18 EDT 2011 193
> Tue Aug 2 07:58:18 EDT 2011 192
> Tue Aug 2 07:59:18 EDT 2011 193
> Tue Aug 2 08:00:19 EDT 2011 193
> Tue Aug 2 08:01:19 EDT 2011 191
>
> Its running 3.2.2
>
> Any suggestions to what's happening?
> Where to start to troubleshoot.
>
> D.
>
>
>
>
> --
> Dave Bour
> Senior Consultant
> Desktop Solution Center
> 905.381.0077 x501
> dcbour at desktopsolutioncenter.ca
>
> Linked In: http://ca.linkedin.com/in/dcbour
> Facebook: http://www.facebook.com/dcbour
>
>
> Providing Complete IT Peace of Mind
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110802/be30e2fe/attachment.html>