dovecot - Aug 2005 - Re: Dovecot inotify problem

Hi Reuben,

I hope you don't mind me copying the dovecot list again.
> dovecot: Aug 21 04:08:22 Error: IMAP(reuben): inotify_rm_watch() failed: 
> Invalid argument
Odd.
> This is with a 2.6.13-rc6-mm1 release.  I've posted to LKML because
there is
> also a kernel stack produced at one point which looks inotify related (but
the
> messages above are contant).
> See http://www.ussg.iu.edu/hypermail/linux/kernel/0508.2/1180.html and the 
> followup from akpm.
Thanks for the link. I don't think akpm is right, unless something is
really odd here. Could you add debugging output for me please?

Maybe something like (this is not a patch you can apply, look at line
164 in src/lib/ioloop-notify-inotify.c)

        ctx->inotify_fd = inotify_init();
+	i_warning("inotify_init returned fd %d", ctx->inotify_fd);
        if (ctx->inotify_fd == -1) {

and line 106:
                                            IN_CLOSE | IN_MODIFY);
+	i_warning("inotify_add_watch returned %d", watchdescriptor);
        if (watchdescriptor < 0) {
and line 145:
        }
+	inotify_rm_watch("removing wd %d from inotify fd %d",
io->notify_context, ctx->inotify_fd);
        if (inotify_rm_watch(ctx->inotify_fd, io->notify_context) < 0)

Or something similar.

If that doesn't help and output looks consistent (we're not removing a
watch descriptor from a wrong fd etc.), can you try removing the
fd_close_on_exec call please (line 171)? I don't know if that could
cause trouble (I don't even think that the imap binary ever forks) but
you never know.

johannes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050822/c932b936/attachment.bin>

Hi again Johannes,

On 22/08/2005 7:04 p.m., Johannes Berg wrote:
> Hi Reuben,
> 
> I hope you don't mind me copying the dovecot list again.
Nope that's fine...
>> dovecot: Aug 21 04:08:22 Error: IMAP(reuben): inotify_rm_watch()
failed:
>> Invalid argument
> 
> Odd.
> 
>> This is with a 2.6.13-rc6-mm1 release.  I've posted to LKML because
there is
>> also a kernel stack produced at one point which looks inotify related
(but the
>> messages above are contant).
>> See http://www.ussg.iu.edu/hypermail/linux/kernel/0508.2/1180.html and
the
>> followup from akpm.
> 
> Thanks for the link. I don't think akpm is right, unless something is
> really odd here. Could you add debugging output for me please?
> 
> Maybe something like (this is not a patch you can apply, look at line
> 164 in src/lib/ioloop-notify-inotify.c)
> 
>         ctx->inotify_fd = inotify_init();
> +	i_warning("inotify_init returned fd %d", ctx->inotify_fd);
>         if (ctx->inotify_fd == -1) {
> 
> and line 106:
>                                             IN_CLOSE | IN_MODIFY);
> +	i_warning("inotify_add_watch returned %d", watchdescriptor);
>         if (watchdescriptor < 0) {
> and line 145:
>         }
> +	inotify_rm_watch("removing wd %d from inotify fd %d",
io->notify_context, ctx->inotify_fd);
>         if (inotify_rm_watch(ctx->inotify_fd, io->notify_context)
< 0)
So it looks like this?

         for (io_p = &ioloop->notifys; *io_p != NULL; io_p =
&(*io_p)->next) {
                 if (*io_p == io) {
                         *io_p = io->next;
                         break;
                 }
         }

         inotify_rm_watch("removing wd %d from inotify fd %d", 
io->notify_context, ctx->inotify_fd);
         if (inotify_rm_watch(ctx->inotify_fd, io->notify_context) < 0)
                 i_error("inotify_rm_watch() failed: %m");

         p_free(ioloop->pool, io);


It barfs with gcc:

[root at tornado dovecot]# make
make  all-recursive
make[1]: Entering directory `/usr/src/dovecot/dovecot'
Making all in src
make[2]: Entering directory `/usr/src/dovecot/dovecot/src'
Making all in lib
make[3]: Entering directory `/usr/src/dovecot/dovecot/src/lib'
if gcc -DHAVE_CONFIG_H -I. -I. -I../..     -std=gnu99 -g -O2 -Wall -W 
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts 
-Wformat=2 -Wbad-function-cast -I/usr/kerberos/include    -MT 
ioloop-notify-inotify.o -MD -MP -MF ".deps/ioloop-notify-inotify.Tpo"
-c -o
ioloop-notify-inotify.o ioloop-notify-inotify.c; \
then mv -f ".deps/ioloop-notify-inotify.Tpo"
".deps/ioloop-notify-inotify.Po";
else rm -f ".deps/ioloop-notify-inotify.Tpo"; exit 1; fi
ioloop-notify-inotify.c: In function 'io_loop_notify_remove':
ioloop-notify-inotify.c:146: warning: passing argument 1 of
'inotify_rm_watch'
makes integer from pointer without a cast
ioloop-notify-inotify.c:146: error: too many arguments to function 
'inotify_rm_watch'
make[3]: *** [ioloop-notify-inotify.o] Error 1
make[3]: Leaving directory `/usr/src/dovecot/dovecot/src/lib'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/usr/src/dovecot/dovecot/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/src/dovecot/dovecot'
make: *** [all] Error 2
[root at tornado dovecot]


reuben

Hi again Johannes,

On 22/08/2005 7:04 p.m., Johannes Berg wrote:
> Maybe something like (this is not a patch you can apply, look at line
> 164 in src/lib/ioloop-notify-inotify.c)
> 
>         ctx->inotify_fd = inotify_init();
> +	i_warning("inotify_init returned fd %d", ctx->inotify_fd);
>         if (ctx->inotify_fd == -1) {
> 
> and line 106:
>                                             IN_CLOSE | IN_MODIFY);
> +	i_warning("inotify_add_watch returned %d", watchdescriptor);
>         if (watchdescriptor < 0) {
> and line 145:
>         }
> +	inotify_rm_watch("removing wd %d from inotify fd %d",
io->notify_context, ctx->inotify_fd);
>         if (inotify_rm_watch(ctx->inotify_fd, io->notify_context)
< 0)
> 
> Or something similar.
> 
> If that doesn't help and output looks consistent (we're not
removing a
> watch descriptor from a wrong fd etc.), can you try removing the
> fd_close_on_exec call please (line 171)? I don't know if that could
> cause trouble (I don't even think that the imap binary ever forks) but
> you never know.
Output from 3/4 hrs of logging is up at http://www.reub.net/dovecot/.  It's
100k, I figure probably a bit big to post to the list.

I'll try removing the fd_close_on_exec call now see what happens, and put
the
output of that up there too in about 10 mins.

Reuben