Hi. I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial. Thanks. Laeeth Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques ts for laeeth at laeeth.com Mar 12 20:48:39 indexer-worker(laeeth at laeeth.com): Fatal: master: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth at laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth at laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): assertion failed: (callback != NULL) Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7fc47d63fc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7fc47d63fd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc47d5f02e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87bd0) [0x7fc47d650bd0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7fc47d65183e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7fc47d63db14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7fc47d614448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7fc47d60c876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7fc47c733159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7fc47c72f637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7fc47cf5dd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x614) [0x7fc47cf5e3c4] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7fc47cf6314e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7fc47d8ef429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fc47d65155c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7fc47d65249b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fc47d6515c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fc47d651648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fc47d5f5703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 15 reque sts for rosie at kaleidicassociates.com Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24022 killed with signal 6 (core dumps disabled) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Warning: I/O leak: 0x7f22c4d4dcf0 (line 127, fd 23) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Panic: file ioloop.c: line 829 (io_loop_move_io): assertion failed: ((old_io->condition & IO_NOTIFY) == 0) Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f22c4d7dc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f22c4d7dd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f22c4d2e2e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x888ba) [0x7f22c4d8f8ba] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f22c4d7bb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f22c4d52448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7f22c4d4a876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7f22c3e71159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7f22c3e6d637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7f22c469bd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x123) [0x7f22c469bed3] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7f22c46a114e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7f22c502d429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io _loop_call_io+0x4c) [0x7f22c4d8f55c] -> /usr/lib64/dovecot/libdovecot.s o.0(io_loop_handler_run_internal+0xbb) [0x7f22c4d9049b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f22c4d8f5c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f22c4d8f648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f22c4d33703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 6 requests for laeeth at kaleidicassociates.com Mar 12 20:49:17 indexer-worker(laeeth at kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24020 killed with signal 6 (core dumps disabled) Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Warning: I/O leak: 0x7f0d5aa00cf0 (line 127, fd 23) Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Panic: file ioloop-iolist.c: line 22: unreached Mar 12 20:49:26 indexer-worker(laeeth at laeeth.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f0d5aa30c40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f0d5aa30d1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0d5a9e12e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x88a0d) [0x7f0d5aa42a0d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f0d5aa4319b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87b61) [0x7f0d5aa41b61] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f0d5aa4283e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f0d5aa2eb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f0d5aa0
On 12 Mar 2015, at 22:55, Laeeth Isharc <laeeth at laeeth.com> wrote:> > Hi. > > I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial.dovecot.org isn't slow as far as I know, although I've noticed some network connectivity problems to some places .. It's in Azure, so I guess somehow related to that..> Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques > ts for laeeth at laeeth.com > Mar 12 20:48:39 indexer-worker(laeeth at laeeth.com): Fatal: master: service(indexe > r-worker): child 24003 killed with signal 6 (core dumps disabled) > Mar 12 20:49:01 dsync-local(laeeth at laeeth.com): Error: Couldn't lock /home/mail/ > laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds > Mar 12 20:49:16 indexer-worker(laeeth at laeeth.com): Error: fts_tika: PUT http://l > ocalhost:9997/tika/ failed: 500 Server Error > Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Warning: I/O leak: > 0x7fc47d60fcf0 (line 127, fd 25) > Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Panic: file ioloop > .c: line 39 (io_add_file): assertion failed: (callback != NULL)I tried this a few times, but I wasn't able to cause the crash. Since you're already pulling from mercurial, can you find which mercurial commit caused it to start breaking?