> On 1 Jul, 2016, at 10:52, Timo Sirainen <tss at iki.fi> wrote: > > On 01 Jul 2016, at 19:39, Adam Weinberger <adamw at adamw.org> wrote: >> >> Hi, >> >> 2.2.25 crashes on FreeBSD with a kqueue-related message. I see references to something similar (http://www.dovecot.org/list/dovecot/2012-February.txt) from a couple years ago. >> >> I get: >> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor >> >> It's not dumping core, and I get the message even with "protocols =" >> >> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? > > gdb backtrace: http://dovecot.org/bugreport.html#coredumps >Fuller logs attached. Thanks for your help, Timo. # Adam Okay. Here's my steps: 1) Install 2.2.5 into fresh i386 FreeBSD 10.3-RELEASE-p5 jail. 2) Built with the following settings: root at apnoea-builder:/usr/ports/mail/dovecot2 # make -V CONFIGURE_ARGS --localstatedir=/var --with-statedir=/var/db/dovecot --without-shadow --with-docs --with-ioloop=kqueue --with-notify=kqueue --without-libwrap --with-ssl=openssl --without-vpopmail --without-gssapi --without-cdb --without-ldap --without-mysql --without-pgsql --without-sqlite --without-icu --without-lucene --without-solr --without-textcat --prefix=/usr/local ${_LATE_CONFIGURE_ARGS} 3) Copied files into place and made following changes: root at apnoea-builder:/usr/ports/mail/dovecot2 # dovecot -n # 2.2.25 (7be1766): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.3-RELEASE-p5 i386 auth_debug = yes auth_verbose = yes log_path = /var/log/dovelog mail_debug = yes namespace inbox { inbox = yes location mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix } passdb { driver = pam } ssl = no userdb { driver = passwd } 4. Started dovecot root at apnoea-builder:/usr/ports/mail/dovecot2 # /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf Last died with error (see error log for more information): kevent(EV_ADD, READ, 57) failed: Bad file descriptor root at apnoea-builder:~ # tail /var/log/dovelog Jul 02 00:15:56 master: Info: Dovecot v2.2.25 (7be1766) starting up for imap, pop3, lmtp Jul 02 00:15:56 master: Panic: kevent(EV_ADD, READ, 57) failed: Bad file descriptor 5. Backtrace root at apnoea-builder:~ # gdb /usr/local/sbin/dovecot /dovecot.core This GDB was configured as "i386-marcel-freebsd"... Core was generated by `dovecot'. Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x28295237 in thr_kill () from /lib/libc.so.7 (gdb) bt #0 0x28295237 in thr_kill () from /lib/libc.so.7 #1 0x2829522a in raise () from /lib/libc.so.7 #2 0x28295176 in abort () from /lib/libc.so.7 #3 0x2815aea2 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 #4 0x2815b686 in i_syslog_fatal_handler (ctx=0xffffaaf0, format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m", args=0xffffab24 "8") at failures.c:418 #5 0x0804dfd1 in master_fatal_callback (ctx=0xffffaaf0, format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m", args=0xffffab24 "8") at main.c:167 #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890 (gdb) bt full #0 0x28295237 in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x2829522a in raise () from /lib/libc.so.7 No symbol table info available. #2 0x28295176 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x2815aea2 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 backtrace = 0x0 #4 0x2815b686 in i_syslog_fatal_handler (ctx=0xffffaaf0, format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m", args=0xffffab24 "8") at failures.c:418 status = 0 #5 0x0804dfd1 in master_fatal_callback (ctx=0xffffaaf0, format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m", args=0xffffab24 "8") at main.c:167 path = 0x28803810 "master-fatal.lastlog" str = 0x28803860 "kevent(EV_ADD, READ, 56) failed: Bad file descriptor" args2 = 0xffffab24 "8" pid = 94804 fd = -1 #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = 0xffffab24 "8" #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 ctx = (struct ioloop_handler_context *) 0x288131d0 ev = {ident = 56, filter = -1, flags = 1, fflags = 0, data = 0, udata = 0x288843a0} #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 io = (struct io_file *) 0x288843a0 #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 io = (struct io_file *) 0xffffac80 #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 h = (struct signal_handler *) 0x2805e079 #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 No locals. #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890 _data_stack_cur_id = 2 set = (struct master_settings *) 0x2881d098 error = 0x282f42cf "\215e?[]?", '\220' <repeats 11 times>, "U\211?SWV\203?\020\211M??" doveconf_arg = 0x0 orig_info_callback = (failure_callback_t *) 0x2815b800 <i_syslog_error_handler> orig_debug_callback = (failure_callback_t *) 0x2815b800 <i_syslog_error_handler> foreground = false ask_key_pass = false i = 3 c = -1 doubleopts = <value optimized out>
> On 1 Jul, 2016, at 18:30, Adam Weinberger <adamw at adamw.org> wrote: > >> On 1 Jul, 2016, at 10:52, Timo Sirainen <tss at iki.fi> wrote: >> >> On 01 Jul 2016, at 19:39, Adam Weinberger <adamw at adamw.org> wrote: >>> >>> Hi, >>> >>> 2.2.25 crashes on FreeBSD with a kqueue-related message. I see references to something similar (http://www.dovecot.org/list/dovecot/2012-February.txt) from a couple years ago. >>> >>> I get: >>> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor >>> >>> It's not dumping core, and I get the message even with "protocols =" >>> >>> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? >> >> gdb backtrace: http://dovecot.org/bugreport.html#coredumps >> > > Fuller logs attached. Thanks for your help, Timo. > > # Adam > > > Okay. Here's my steps: > 1) Install 2.2.5 into fresh i386 FreeBSD 10.3-RELEASE-p5 jail. > > 2) Built with the following settings: > root at apnoea-builder:/usr/ports/mail/dovecot2 # make -V CONFIGURE_ARGS > --localstatedir=/var --with-statedir=/var/db/dovecot --without-shadow --with-docs --with-ioloop=kqueue --with-notify=kqueue --without-libwrap --with-ssl=openssl --without-vpopmail --without-gssapi --without-cdb --without-ldap --without-mysql --without-pgsql --without-sqlite --without-icu --without-lucene --without-solr --without-textcat --prefix=/usr/local ${_LATE_CONFIGURE_ARGS}Just a follow-up: it runs without dumping core if I build with "--with-ioloop=poll --with-notify=none". # Adam -- Adam Weinberger adamw at adamw.org http://www.adamw.org
On 02 Jul 2016, at 03:30, Adam Weinberger <adamw at adamw.org> wrote:> >>> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor >>> >>> It's not dumping core, and I get the message even with "protocols =" >>> >>> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? >> >> gdb backtrace: http://dovecot.org/bugreport.html#coredumps >> > > Fuller logs attached. Thanks for your help, Timo. > > #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 > #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 > #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 > #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 > #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 > #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 > #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890Weird. I haven't touched anything even close to that. Also kqueue works on my OSX. The only thing that comes to my mind is https://github.com/dovecot/core/commit/fde7b8a03bf91cfa5bb7ca3e84545386243fa0d2 Does it happen to work if you use the attached patch? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 6057 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20160703/920929ac/attachment-0001.obj>
On 16-07-03 03:30:36, Timo Sirainen wrote:> On 02 Jul 2016, at 03:30, Adam Weinberger <adamw at adamw.org> wrote: > > > >>> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor > >>> > >>> It's not dumping core, and I get the message even with "protocols =" > >>> > >>> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? > >> > >> gdb backtrace: http://dovecot.org/bugreport.html#coredumps > >> > > > > Fuller logs attached. Thanks for your help, Timo. > > > > #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 > > #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 > > #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 > > #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 > > #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 > > #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 > > #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890 > > Weird. I haven't touched anything even close to that. Also kqueue works on my OSX. The only thing that comes to my mind is https://github.com/dovecot/core/commit/fde7b8a03bf91cfa5bb7ca3e84545386243fa0d2 > > Does it happen to work if you use the attached patch? >Looks like its an issue on OpenBSD -current as well. This GDB was configured as "i386-unknown-openbsd6.0"... Core was generated by `dovecot'. Program terminated with signal 6, Aborted. Loaded symbols for /home/edgar/sbin/dovecot Reading symbols from /home/edgar/lib/dovecot/libdovecot.so.0.0...done. Loaded symbols for /home/edgar/lib/dovecot/libdovecot.so.0.0 Reading symbols from /usr/lib/libc.so.88.0...done. Loaded symbols for /usr/lib/libc.so.88.0 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so #0 0x0d8ca24f in thrkill () at <stdin>:2 2 <stdin>: No such file or directory. in <stdin> (gdb) bt #0 0x0d8ca24f in thrkill () at <stdin>:2 #1 0x0d8cd691 in *_libc_raise (s=6) at /usr/src/lib/libc/gen/raise.c:37 #2 0x0d8cb8fc in *_libc_abort () at /usr/src/lib/libc/stdlib/abort.c:52 #3 0x092d5a2f in default_fatal_finish (type=Variable "type" is not available. ) at failures.c:201 #4 0x092d5ae4 in i_syslog_fatal_handler (ctx=0xcf7dcdcc, format=0x2925f1d4 "kevent(EV_ADD, READ, %d) failed: %m", args=0xcf7dcdf4 "<") at failures.c:418 #5 0x17bd6f68 in master_fatal_callback (ctx=0xcf7dcdcc, format=0x2925f1d4 "kevent(EV_ADD, READ, %d) failed: %m", args=0xcf7dcdf4 "<") at main.c:167 #6 0x092d4ea6 in i_panic (format=Could not find the frame base for "i_panic". ) at failures.c:275 #7 0x092de2b9 in io_loop_handle_add (io=Variable "io" is not available. ) at ioloop-kqueue.c:67 #8 0x092df3e2 in io_add_file (fd=60, condition=IO_READ, source_linenum=244, callback=0x92efe80 <signal_read>, context=0x0) at ioloop.c:59 #9 0x092df491 in io_add (fd=60, condition=IO_READ, source_linenum=244, callback=0x92efe80 <signal_read>, context=0x0) at ioloop.c:81 #10 0x092f0283 in lib_signals_set_handler (signo=1, flags=3, handler=0x17bd5d30 <sig_settings_reload>, context=0x0) at lib-signals.c:243 #11 0x17bd58fb in main_init (set=0x86b34098) at main.c:518 #12 0x17bd6878 in main (argc=Cannot access memory at address 0x0 ) at main.c:890 Current language: auto; currently asm (gdb) bt full #0 0x0d8ca24f in thrkill () at <stdin>:2 No locals. #1 0x0d8cd691 in *_libc_raise (s=6) at /usr/src/lib/libc/gen/raise.c:37 No locals. #2 0x0d8cb8fc in *_libc_abort () at /usr/src/lib/libc/stdlib/abort.c:52 mask = 4294967263 sa = {__sigaction_u = {__sa_handler = 0xcf7dcd38, __sa_sigaction = 0xcf7dcd38}, sa_mask = 690456752, sa_flags = 1} #3 0x092d5a2f in default_fatal_finish (type=Variable "type" is not available. ) at failures.c:201 backtrace = Variable "backtrace" is not available. (gdb) I'm happy to test patches, etc. -- Edgar Pettijohn
On 16-07-03 03:30:36, Timo Sirainen wrote:> On 02 Jul 2016, at 03:30, Adam Weinberger <adamw at adamw.org> wrote: > > > >>> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor > >>> > >>> It's not dumping core, and I get the message even with "protocols =" > >>> > >>> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? > >> > >> gdb backtrace: http://dovecot.org/bugreport.html#coredumps > >> > > > > Fuller logs attached. Thanks for your help, Timo. > > > > #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 > > #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 > > #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 > > #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 > > #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 > > #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 > > #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890 > > Weird. I haven't touched anything even close to that. Also kqueue works on my OSX. The only thing that comes to my mind is https://github.com/dovecot/core/commit/fde7b8a03bf91cfa5bb7ca3e84545386243fa0d2 > > Does it happen to work if you use the attached patch? >The patch doesn't apply cleanly for me on OpenBSD -current. Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |commit 3fea4d5988de365503df44a9b067e3b181cac65c |Author: Timo Sirainen <timo.sirainen at dovecot.fi> |Date: Thu Jun 30 14:08:24 2016 +0300 | | master: Stopping didn't close dead-pipes early enough. | | This caused stopping to be too slow. Broken by 0153cf542. | |diff --git a/src/master/service-monitor.c b/src/master/service-monitor.c |index 19655c2..44cdb26 100644 |--- a/src/master/service-monitor.c |+++ b/src/master/service-monitor.c -------------------------- Patching file service-monitor.c using Plan A... Hunk #1 succeeded at 513 (offset 8 lines). Hunk #2 failed at 538. Hunk #3 succeeded at 595 with fuzz 2 (offset 8 lines). 1 out of 3 hunks failed--saving rejects to service-monitor.c.rej Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |commit fde7b8a03bf91cfa5bb7ca3e84545386243fa0d2 |Author: Timo Sirainen <timo.sirainen at dovecot.fi> |Date: Tue Jun 28 23:38:11 2016 +0300 | | master: Added support for stopping specific services. | | We need to have a per-service fd for detecting a dead master. | |diff --git a/src/master/service-monitor.c b/src/master/service-monitor.c |index bf4e931..fb4f64f 100644 |--- a/src/master/service-monitor.c |+++ b/src/master/service-monitor.c -------------------------- Patching file service-monitor.c using Plan A... Reversed (or previously applied) patch detected! Assume -R? [y] If I select [y] and continue to build I get lots of build errors. -- Edgar Pettijohn
On 16-07-03 03:30:36, Timo Sirainen wrote:> On 02 Jul 2016, at 03:30, Adam Weinberger <adamw at adamw.org> wrote: > > > >>> Jul 1 10:07:27 imap dovecot: master: Panic: kevent(EV_ADD, READ, 54) failed: Bad file descriptor > >>> > >>> It's not dumping core, and I get the message even with "protocols =" > >>> > >>> Downgrading back to 2.2.24 fixes it. What info would be helpful for me to provide here? > >> > >> gdb backtrace: http://dovecot.org/bugreport.html#coredumps > >> > > > > Fuller logs attached. Thanks for your help, Timo. > > > > #6 0x2815b23a in i_panic (format=0x281ccf7a "kevent(EV_ADD, READ, %d) failed: %m") at failures.c:275 > > #7 0x28185e10 in io_loop_handle_add (io=0x288843a0) at ioloop-kqueue.c:67 > > #8 0x281815a8 in io_add_file (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:59 > > #9 0x281813a6 in io_add (fd=56, condition=IO_READ, source_linenum=244, callback=0x2818a7d0 <signal_read>, context=0x0) at ioloop.c:81 > > #10 0x2818a666 in lib_signals_set_handler (signo=1, flags=3, handler=0x804e6c0 <sig_settings_reload>, context=0x0) at lib-signals.c:243 > > #11 0x0804e129 in main_init (set=0x2881d098) at main.c:518 > > #12 0x0804d562 in main (argc=3, argv=0xffffae08) at main.c:890 > > Weird. I haven't touched anything even close to that. Also kqueue works on my OSX. The only thing that comes to my mind is https://github.com/dovecot/core/commit/fde7b8a03bf91cfa5bb7ca3e84545386243fa0d2 > > Does it happen to work if you use the attached patch? >Still trying to learn git, but here is what happened when I tried applying this diff. Sun Jul 03 10:12:32 ~/dovecot/core $ git reset --hard 3fea4d5988de365503df44a9b067e3b181cac65c HEAD is now at 3fea4d5 master: Stopping didn't close dead-pipes early enough. Sun Jul 03 10:12:37 ~/dovecot/core $ git apply diff.patch error: patch failed: src/master/service-monitor.c:522 error: src/master/service-monitor.c: patch does not apply error: patch failed: src/master/service-monitor.c:452 error: src/master/service-monitor.c: patch does not apply error: patch failed: src/master/service-process.c:133 error: src/master/service-process.c: patch does not apply error: patch failed: src/master/service.c:283 error: src/master/service.c: patch does not apply error: patch failed: src/master/service.h:85 error: src/master/service.h: patch does not apply Same thing happened before the reset --hard. -- Edgar Pettijohn