thr3ads.net - Gluster users - [Gluster-users] logfile flooding [Nov 2014]

If this information is useful, please help other people find it:
Share via:
Bernhard Glomm
2014-Nov-06 09:32 UTC
[Gluster-users] logfile flooding

Hi all,

I like to report an "kind of glitchy" behaviour of glusterfs I
experienced today,
and would ask for a work around for it.


I'm running?
glusterfs 3.4.5 built on Aug ?6 2014 19:15:07
from
deb http://ppa.launchpad.net/semiosis/ubuntu-glusterfs-3.4/ubuntu trusty main
in replica?2 mode on two up-to-date ubuntu 14.04 servers (<ip_srv_1>
and?<ip_srv_2>)
A client, also, ubuntu 14.04, same gluster version, fuse mounts
a volume like this:


/bin/mount -t glusterfs <ip_srv_1>:/<volume_name> <mountpoint>


I had several time stopped the client (unmounted the volume)
updated (and rebooted) first server, than the otherone, remounted the volume
in the client and?all went nice and smooth as expected.


This time I indeed for the first time unmounted the volume in the client
updated/rebooted?<ip_srv_2>, mounted the volume from?<ip_srv_2>
in the client and then updated rebooted?<ip_srv_1>.


That should work, right? and it does. All data on the volume seems to be okay
as far as I can tell.


BUT:
The logfiles on the client and server get flooded like this:


server:
/var/log/glusterfs/bricks/<volume_name>brick.log:

[2014-11-06 06:18:41.417901] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.418003] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280274: REMOVEXATTR
/<path>/.<file>.qjStjE (852ed4f2-c821-4a08-9254-11ec987d1233) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.423443] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.423497] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280285: REMOVEXATTR
/<path>/.<file>.qjStjE (852ed4f2-c821-4a08-9254-11ec987d1233) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.426623] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.426675] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280294: REMOVEXATTR
/<path>/.<file>.qjStjE (852ed4f2-c821-4a08-9254-11ec987d1233) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.429518] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.429594] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280302: REMOVEXATTR
/<path>/.<file>.qjStjE (852ed4f2-c821-4a08-9254-11ec987d1233) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.436899] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.436921] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280328: REMOVEXATTR
/<path>/.<file>.zLuCBE (357ba81d-a774-462f-9e49-e3d646f3afcd) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.453168] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.453255] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280354: REMOVEXATTR
/<path>/.<file>.zLuCBE (357ba81d-a774-462f-9e49-e3d646f3afcd) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.456518] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.456569] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280363: REMOVEXATTR
/<path>/.<file>.zLuCBE (357ba81d-a774-462f-9e49-e3d646f3afcd) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.459100] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.459148] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280372: REMOVEXATTR
/<path>/.<file>.zLuCBE (357ba81d-a774-462f-9e49-e3d646f3afcd) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.691399] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.691442] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280570: REMOVEXATTR
/<path>/.<file>.Kb2vBF (69d2a022-0d12-45dd-ad5b-24c206471e8f) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.707157] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.707181] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280603: REMOVEXATTR
/<path>/.<file>.Kb2vBF (69d2a022-0d12-45dd-ad5b-24c206471e8f) of key
security.ima ==> (No data available)
[2014-11-06 06:18:41.708946] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:41.708972] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33280611: REMOVEXATTR
/<path>/.<file>.Kb2vBF (69d2a022-0d12-45dd-ad5b-24c206471e8f) of key
security.ima ==> (No data available)
[2014-11-06 06:18:43.826544] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:43.826587] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33282135: REMOVEXATTR
/<path>/.<file>.rJfnCM (4dfb3cf7-80d8-4278-b3e0-5b726fa9cf0d) of key
security.ima ==> (No data available)
[2014-11-06 06:18:43.832881] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:43.832933] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33282153: REMOVEXATTR
/<path>/.<file>.rJfnCM (4dfb3cf7-80d8-4278-b3e0-5b726fa9cf0d) of key
security.ima ==> (No data available)
[2014-11-06 06:18:43.835039] E [marker.c:2140:marker_removexattr_cbk]
0-gldata_<volume_name>-marker: No data available occurred while creating
symlinks
[2014-11-06 06:18:43.835104] I [server-rpc-fops.c:745:server_removexattr_cbk]
0-gldata_<volume_name>-server: 33282162: REMOVEXATTR
/<path>/.<file>.rJfnCM (4dfb3cf7-80d8-4278-b3e0-5b726fa9cf0d) of key
security.ima ==> (No data available)6


client:
/var/log/glusterfs/srv-<volume_name>.log:

[2014-11-06 06:18:41.418521] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.418694] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.419664] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178626: REMOVEXATTR() /<path>/.<file>.qjStjE
=> -1 (No data available)
[2014-11-06 06:18:41.423948] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.423989] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.424837] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178635: REMOVEXATTR() /<path>/.<file>.qjStjE
=> -1 (No data available)
[2014-11-06 06:18:41.427270] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.427539] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.428103] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178641: REMOVEXATTR() /<path>/.<file>.qjStjE
=> -1 (No data available)
[2014-11-06 06:18:41.429955] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.430082] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.430441] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178645: REMOVEXATTR() /<path>/.<file>.qjStjE
=> -1 (No data available)
[2014-11-06 06:18:41.437450] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.437500] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.437854] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178663: REMOVEXATTR() /<path>/.<file>.zLuCBE
=> -1 (No data available)
[2014-11-06 06:18:41.453801] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.453830] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.454373] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178688: REMOVEXATTR() /<path>/.<file>.zLuCBE
=> -1 (No data available)
[2014-11-06 06:18:41.457040] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.457115] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.457673] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178693: REMOVEXATTR() /<path>/.<file>.zLuCBE
=> -1 (No data available)
[2014-11-06 06:18:41.459532] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.459647] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.460125] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178699: REMOVEXATTR() /<path>/.<file>.zLuCBE
=> -1 (No data available)
[2014-11-06 06:18:41.691852] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.692116] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.692711] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178894: REMOVEXATTR() /<path>/.<file>.Kb2vBF
=> -1 (No data available)
[2014-11-06 06:18:41.707673] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.707716] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.708004] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178923: REMOVEXATTR() /<path>/.<file>.Kb2vBF
=> -1 (No data available)
[2014-11-06 06:18:41.709422] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-1: remote operation failed: No data
available
[2014-11-06 06:18:41.709468] W
[client-rpc-fops.c:1232:client3_3_removexattr_cbk]
0-gldata_<volume_name>-client-0: remote operation failed: No data
available
[2014-11-06 06:18:41.709783] W [fuse-bridge.c:1172:fuse_err_cbk]
0-glusterfs-fuse: 37178929: REMOVEXATTR() /<path>/.<file>.Kb2vBF
=> -1 (No data available)


As I said above, the data itself seems to be okay,
but how can I stop the log message flood?

TIA

Bernhard

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20141106/f329fe7f/attachment.html>
Gluster users - Nov 2014 - logfile flooding

[Gluster-users] logfile flooding
