Artem Russakovskii
2019-Jan-30 20:43 UTC
[Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument]
Also, not sure if related or not, but I got a ton of these "Failed to dispatch handler" in my logs as well. Many people have been commenting about this issue here https://bugzilla.redhat.com/show_bug.cgi?id=1651246. ==> mnt-SITE_data1.log <=> [2019-01-30 20:38:20.783713] W [dict.c:761:dict_ref]> (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) > [0x7fd966fcd329] > -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) > [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) > [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument] > ==> mnt-SITE_data3.log <=> The message "E [MSGID: 101191] > [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch > handler" repeated 413 times between [2019-01-30 20:36:23.881090] and > [2019-01-30 20:38:20.015593] > The message "I [MSGID: 108031] [afr-common.c:2543:afr_local_discovery_cbk] > 2-SITE_data3-replicate-0: selecting local read_child SITE_data3-client-0" > repeated 42 times between [2019-01-30 20:36:23.290287] and [2019-01-30 > 20:38:20.280306] > ==> mnt-SITE_data1.log <=> The message "I [MSGID: 108031] [afr-common.c:2543:afr_local_discovery_cbk] > 2-SITE_data1-replicate-0: selecting local read_child SITE_data1-client-0" > repeated 50 times between [2019-01-30 20:36:22.247367] and [2019-01-30 > 20:38:19.459789] > The message "E [MSGID: 101191] > [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch > handler" repeated 2654 times between [2019-01-30 20:36:22.667327] and > [2019-01-30 20:38:20.546355] > [2019-01-30 20:38:21.492319] I [MSGID: 108031] > [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data1-replicate-0: > selecting local read_child SITE_data1-client-0 > ==> mnt-SITE_data3.log <=> [2019-01-30 20:38:22.349689] I [MSGID: 108031] > [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data3-replicate-0: > selecting local read_child SITE_data3-client-0 > ==> mnt-SITE_data1.log <=> [2019-01-30 20:38:22.762941] E [MSGID: 101191] > [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch > handlerI'm hoping raising the issue here on the mailing list may bring some additional eyeballs and get them both fixed. Thanks. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Wed, Jan 30, 2019 at 12:26 PM Artem Russakovskii <archon810 at gmail.com> wrote:> I found a similar issue here: > https://bugzilla.redhat.com/show_bug.cgi?id=1313567. There's a comment > from 3 days ago from someone else with 5.3 who started seeing the spam. > > Here's the command that repeats over and over: > [2019-01-30 20:23:24.481581] W [dict.c:761:dict_ref] > (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) > [0x7fd966fcd329] > -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) > [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) > [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument] > > Is there any fix for this issue? > > Thanks. > > Sincerely, > Artem > > -- > Founder, Android Police <http://www.androidpolice.com>, APK Mirror > <http://www.apkmirror.com/>, Illogical Robot LLC > beerpla.net | +ArtemRussakovskii > <https://plus.google.com/+ArtemRussakovskii> | @ArtemR > <http://twitter.com/ArtemR> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190130/38fdabf3/attachment.html>
Raghavendra Gowdappa
2019-Jan-31 02:37 UTC
[Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument]
On Thu, Jan 31, 2019 at 2:14 AM Artem Russakovskii <archon810 at gmail.com> wrote:> Also, not sure if related or not, but I got a ton of these "Failed to > dispatch handler" in my logs as well. Many people have been commenting > about this issue here https://bugzilla.redhat.com/show_bug.cgi?id=1651246. >https://review.gluster.org/#/c/glusterfs/+/22046/ addresses this.> ==> mnt-SITE_data1.log <=>> [2019-01-30 20:38:20.783713] W [dict.c:761:dict_ref] >> (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) >> [0x7fd966fcd329] >> -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) >> [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) >> [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument] >> ==> mnt-SITE_data3.log <=>> The message "E [MSGID: 101191] >> [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch >> handler" repeated 413 times between [2019-01-30 20:36:23.881090] and >> [2019-01-30 20:38:20.015593] >> The message "I [MSGID: 108031] >> [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data3-replicate-0: >> selecting local read_child SITE_data3-client-0" repeated 42 times between >> [2019-01-30 20:36:23.290287] and [2019-01-30 20:38:20.280306] >> ==> mnt-SITE_data1.log <=>> The message "I [MSGID: 108031] >> [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data1-replicate-0: >> selecting local read_child SITE_data1-client-0" repeated 50 times between >> [2019-01-30 20:36:22.247367] and [2019-01-30 20:38:19.459789] >> The message "E [MSGID: 101191] >> [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch >> handler" repeated 2654 times between [2019-01-30 20:36:22.667327] and >> [2019-01-30 20:38:20.546355] >> [2019-01-30 20:38:21.492319] I [MSGID: 108031] >> [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data1-replicate-0: >> selecting local read_child SITE_data1-client-0 >> ==> mnt-SITE_data3.log <=>> [2019-01-30 20:38:22.349689] I [MSGID: 108031] >> [afr-common.c:2543:afr_local_discovery_cbk] 2-SITE_data3-replicate-0: >> selecting local read_child SITE_data3-client-0 >> ==> mnt-SITE_data1.log <=>> [2019-01-30 20:38:22.762941] E [MSGID: 101191] >> [event-epoll.c:671:event_dispatch_epoll_worker] 2-epoll: Failed to dispatch >> handler > > > I'm hoping raising the issue here on the mailing list may bring some > additional eyeballs and get them both fixed. > > Thanks. > > Sincerely, > Artem > > -- > Founder, Android Police <http://www.androidpolice.com>, APK Mirror > <http://www.apkmirror.com/>, Illogical Robot LLC > beerpla.net | +ArtemRussakovskii > <https://plus.google.com/+ArtemRussakovskii> | @ArtemR > <http://twitter.com/ArtemR> > > > On Wed, Jan 30, 2019 at 12:26 PM Artem Russakovskii <archon810 at gmail.com> > wrote: > >> I found a similar issue here: >> https://bugzilla.redhat.com/show_bug.cgi?id=1313567. There's a comment >> from 3 days ago from someone else with 5.3 who started seeing the spam. >> >> Here's the command that repeats over and over: >> [2019-01-30 20:23:24.481581] W [dict.c:761:dict_ref] >> (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) >> [0x7fd966fcd329] >> -->/usr/lib64/glusterfs/5.3/xlator/performance/io-cache.so(+0xaaf5) >> [0x7fd9671deaf5] -->/usr/lib64/libglusterfs.so.0(dict_ref+0x58) >> [0x7fd9731ea218] ) 2-dict: dict is NULL [Invalid argument] >> >+Milind Changire <mchangir at redhat.com> Can you check why this message is logged and send a fix?>> Is there any fix for this issue? >> >> Thanks. >> >> Sincerely, >> Artem >> >> -- >> Founder, Android Police <http://www.androidpolice.com>, APK Mirror >> <http://www.apkmirror.com/>, Illogical Robot LLC >> beerpla.net | +ArtemRussakovskii >> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR >> <http://twitter.com/ArtemR> >> > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190131/5502a3a6/attachment.html>