thr3ads.net - Gluster users - [Gluster-users] error msg in the glustershd.log [Aug 2017]

If this information is useful, please help other people find it:
Share via:

Amudhan P

2017-Aug-29 11:17 UTC

[Gluster-users] error msg in the glustershd.log

Hi ,

I need some clarification for below error msg in the glustershd.log file.
What is this msg? Why is this showing up?. currently using glusterfs 3.10.1

when ever I start write process to volume (volume mounted thru fuse) I am
seeing this kind of error and glustershd process consumes some percentage
of cpu until write process completes.

[2017-08-28 10:01:13.030710] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769963520-1769947136, mode:
100755-100755)
[2017-08-28 10:01:13.030752] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.031127] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769947136-1769963520, mode:
100755-100755)
The message "N [MSGID: 122029] [ec-generic.c:684:ec_combine_lookup]
0-glustervol-disperse-109: Mismatching iatt in answers of
'GF_FOP_LOOKUP'"
repeated 3 times between [2017-08-28 10:01:13.030752] and [2017-08-28
10:01:13.031215]
[2017-08-28 10:01:13.032033] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode:
100755-100755)
[2017-08-28 10:01:13.032425] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.032746] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode:
100755-100755)
[2017-08-28 10:01:13.032797] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.032983] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode:
100755-100755)
[2017-08-28 10:01:13.033047] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.033099] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to
combine iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 1769979904-1769996288, mode:
100755-100755)
[2017-08-28 10:01:13.033176] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'


[2017-08-29 11:02:37.054929] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to
combine iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode:
100755-100755)
[2017-08-29 11:02:37.054981] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055014] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to
combine iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode:
100755-100755)
[2017-08-29 11:02:37.055072] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055132] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to
combine iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode:
100755-100755)
[2017-08-29 11:02:37.055155] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055189] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to
combine iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode:
100755-100755)
[2017-08-29 11:02:37.055206] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055237] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to
combine iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1,
uid: 0-0, gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode:
100755-100755)
[2017-08-29 11:02:37.055255] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching
iatt in answers of 'GF_FOP_LOOKUP'


regards,
Amudhan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170829/1f6d3829/attachment.html>

Ashish Pandey

2017-Aug-29 11:33 UTC

head link

[Gluster-users] error msg in the glustershd.log

Whenever we do some fop on EC volume on a file, we check the xattr also to see
if the file is healthy or not. If not, we trigger heal.
lookup is the fop for which we don't take inodelk lock so it is possible
that the xattr which we get for lookup fop are different for some bricks.
This difference is not reliable but still we are triggering heal and that is why
you are seeing these messages.

We have fixed it in latest release https://review.gluster.org/16468 
Now, we check if fop actually needs heal or not. 

---- 
Ashish 



From: "Amudhan P" <amudhan83 at gmail.com> 
To: "Gluster Users" <gluster-users at gluster.org> 
Sent: Tuesday, August 29, 2017 4:47:10 PM 
Subject: [Gluster-users] error msg in the glustershd.log 

Hi , 

I need some clarification for below error msg in the glustershd.log file. What
is this msg? Why is this showing up?. currently using glusterfs 3.10.1

when ever I start write process to volume (volume mounted thru fuse) I am seeing
this kind of error and glustershd process consumes some percentage of cpu until
write process completes.

[2017-08-28 10:01:13.030710] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769963520-1769947136, mode: 100755-100755)
[2017-08-28 10:01:13.030752] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.031127] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769947136-1769963520, mode: 100755-100755)
The message "N [MSGID: 122029] [ec-generic.c:684:ec_combine_lookup]
0-glustervol-disperse-109: Mismatching iatt in answers of
'GF_FOP_LOOKUP'" repeated 3 times between [2017-08-28
10:01:13.030752] and [2017-08-28 10:01:13.031215]
[2017-08-28 10:01:13.032033] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
[2017-08-28 10:01:13.032425] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.032746] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
[2017-08-28 10:01:13.032797] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.032983] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
[2017-08-28 10:01:13.033047] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-28 10:01:13.033099] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-109: Failed to combine
iatt (inode: 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 1769979904-1769996288, mode: 100755-100755)
[2017-08-28 10:01:13.033176] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'


[2017-08-29 11:02:37.054929] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to combine
iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
[2017-08-29 11:02:37.054981] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055014] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to combine
iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
[2017-08-29 11:02:37.055072] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055132] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to combine
iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
[2017-08-29 11:02:37.055155] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055189] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to combine
iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
[2017-08-29 11:02:37.055206] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'
[2017-08-29 11:02:37.055237] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine] 0-glustervol-disperse-116: Failed to combine
iatt (inode: 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0,
gid: 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
[2017-08-29 11:02:37.055255] N [MSGID: 122029]
[ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116: Mismatching iatt
in answers of 'GF_FOP_LOOKUP'


regards, 
Amudhan 

_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://lists.gluster.org/mailman/listinfo/gluster-users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170829/940fc3a1/attachment.html>

Amudhan P

2017-Aug-29 13:08 UTC

head link

[Gluster-users] error msg in the glustershd.log

I am using 3.10.1 from which version this update is available.


On Tue, Aug 29, 2017 at 5:03 PM, Ashish Pandey <aspandey at redhat.com>
wrote:
>
> Whenever we do some fop on EC volume on a file, we check the xattr also to
> see if the file is healthy or not. If not, we trigger heal.
> lookup is the fop for which we don't take inodelk lock so it is
possible
> that the xattr which we get for lookup fop are different for some bricks.
> This difference is not reliable but still we are triggering heal and that
> is why you are seeing these messages.
>
> We have fixed it in latest release https://review.gluster.org/16468
> Now, we check if fop actually needs heal or not.
>
> ----
> Ashish
>
>
>
> *From: *"Amudhan P" <amudhan83 at gmail.com>
> *To: *"Gluster Users" <gluster-users at gluster.org>
> *Sent: *Tuesday, August 29, 2017 4:47:10 PM
> *Subject: *[Gluster-users] error msg in the glustershd.log
>
>
> Hi ,
>
> I need some clarification for below error msg in the glustershd.log file.
> What is this msg? Why is this showing up?. currently using glusterfs 3.10.1
>
> when ever I start write process to volume (volume mounted thru fuse) I am
> seeing this kind of error and glustershd process consumes some percentage
> of cpu until write process completes.
>
> [2017-08-28 10:01:13.030710] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769963520-1769947136, mode: 100755-100755)
> [2017-08-28 10:01:13.030752] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-28 10:01:13.031127] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769947136-1769963520, mode: 100755-100755)
> The message "N [MSGID: 122029] [ec-generic.c:684:ec_combine_lookup]
> 0-glustervol-disperse-109: Mismatching iatt in answers of
'GF_FOP_LOOKUP'"
> repeated 3 times between [2017-08-28 10:01:13.030752] and [2017-08-28
> 10:01:13.031215]
> [2017-08-28 10:01:13.032033] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
> [2017-08-28 10:01:13.032425] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-28 10:01:13.032746] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
> [2017-08-28 10:01:13.032797] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-28 10:01:13.032983] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769996288-1769979904, mode: 100755-100755)
> [2017-08-28 10:01:13.033047] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-28 10:01:13.033099] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-109: Failed to combine iatt (inode:
> 11548094941524765708-11548094941524765708, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 1769979904-1769996288, mode: 100755-100755)
> [2017-08-28 10:01:13.033176] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-109:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
>
>
> [2017-08-29 11:02:37.054929] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-116: Failed to combine iatt (inode:
> 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
> [2017-08-29 11:02:37.054981] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-29 11:02:37.055014] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-116: Failed to combine iatt (inode:
> 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
> [2017-08-29 11:02:37.055072] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-29 11:02:37.055132] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-116: Failed to combine iatt (inode:
> 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
> [2017-08-29 11:02:37.055155] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-29 11:02:37.055189] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-116: Failed to combine iatt (inode:
> 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
> [2017-08-29 11:02:37.055206] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
> [2017-08-29 11:02:37.055237] W [MSGID: 122006]
[ec-combine.c:191:ec_iatt_combine]
> 0-glustervol-disperse-116: Failed to combine iatt (inode:
> 11207221728611356412-11207221728611356412, links: 1-1, uid: 0-0, gid:
> 0-0, rdev: 0-0, size: 314900480-314867712, mode: 100755-100755)
> [2017-08-29 11:02:37.055255] N [MSGID: 122029]
> [ec-generic.c:684:ec_combine_lookup] 0-glustervol-disperse-116:
> Mismatching iatt in answers of 'GF_FOP_LOOKUP'
>
>
> regards,
> Amudhan
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170829/7bcb4078/attachment.html>

Maybe Matching Threads

Search for more maybe matching threads

Gluster users - Aug 2017 - error msg in the glustershd.log

[Gluster-users] error msg in the glustershd.log

[Gluster-users] error msg in the glustershd.log

[Gluster-users] error msg in the glustershd.log

Maybe Matching Threads