Gluster users - Aug 2015 - gluster small file performance

> Note: The log files attached have the "No data available"
messages parsed out
> to reduce the file size. There were an enormous amount of these. One of my
> colleagues submitted something to the message board about these errors in
> 3.7.3.
> > [2015-08-17 17:03:37.270219] W [fuse-bridge.c:1230:fuse_err_cbk]
> > 0-glusterfs-fuse: 6643: REMOVEXATTR()
> > /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data
available)
> 
> > [2015-08-17 17:03:37.271004] W [fuse-bridge.c:1230:fuse_err_cbk]
> > 0-glusterfs-fuse: 6646: REMOVEXATTR()
> > /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data
available)
> 
> > [2015-08-17 17:03:37.271663] W [fuse-bridge.c:1230:fuse_err_cbk]
> > 0-glusterfs-fuse: 6648: REMOVEXATTR()
> > /boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data
available)
> 
> > [2015-08-17 17:03:37.274273] W [fuse-bridge.c:1230:fuse_err_cbk]
> > 0-glusterfs-fuse: 6662: REMOVEXATTR()
> > /boost_1_57_0/boost/accumulators/accumulators_fwd.hpp => -1 (No
data
> > available)
> 
I can't help but wonder how much these are affecting your performance.
That's a lot of extra messages, and even more effort to log the failures.
When I run your tests myself, I don't see any of these and I don't see a
performance drop-off either. Maybe something ACL- or SELinux-related? It would
be extra-helpful to get a stack trace for just one of these, to see where
they're coming from.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150818/d94b2ea6/attachment.html>

I changed the logging to "error" to get rid of these messages as I was
wondering if this was part of the problem.  It didn't change the 
performance.
Also, I get these same errors both before and after the reboot.  I only 
see the slowdown after the reboot.

I have SELinux disabled.  Not sure about ACL.  Don't think I can turn 
ACL off on XFS.

I am happy to post results of strace.  Do I just do 'strace tar -xPf 
boost.tar >& strace.log'?

David


------ Original Message ------
From: "Jeff Darcy" <jdarcy at redhat.com>
To: "David Robinson" <drobinson at corvidtec.com>
Cc: gluster-users at gluster.org; "Gluster Devel" 
<gluster-devel at gluster.org>
Sent: 8/18/2015 4:49:40 PM
Subject: Re: [Gluster-users] gluster small file performance
>>Note: The log files attached have the "No data available"
messages
>>parsed out to reduce the file size.  There were an enormous amount of 
>>these.  One of my colleagues submitted something to the message board 
>>about these errors in 3.7.3.
>>>[2015-08-17 17:03:37.270219] W [fuse-bridge.c:1230:fuse_err_cbk] 
>>>0-glusterfs-fuse: 6643: REMOVEXATTR() 
>>>/boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data 
>>>available)
>>>[2015-08-17 17:03:37.271004] W [fuse-bridge.c:1230:fuse_err_cbk] 
>>>0-glusterfs-fuse: 6646: REMOVEXATTR() 
>>>/boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data 
>>>available)
>>>[2015-08-17 17:03:37.271663] W [fuse-bridge.c:1230:fuse_err_cbk] 
>>>0-glusterfs-fuse: 6648: REMOVEXATTR() 
>>>/boost_1_57_0/boost/accumulators/accumulators.hpp => -1 (No data 
>>>available)
>>>[2015-08-17 17:03:37.274273] W [fuse-bridge.c:1230:fuse_err_cbk] 
>>>0-glusterfs-fuse: 6662: REMOVEXATTR() 
>>>/boost_1_57_0/boost/accumulators/accumulators_fwd.hpp => -1 (No
data
>>>available)
>
>I can't help but wonder how much these are affecting your performance.  
>That's a lot of extra messages, and even more effort to log the 
>failures.  When I run your tests myself, I don't see any of these and I 
>don't see a performance drop-off either.  Maybe something ACL- or 
>SELinux-related?  It would be extra-helpful to get a stack trace for 
>just one of these, to see where they're coming from.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150818/34d5ba9b/attachment.html>