thr3ads.net - dovecot - bug in indexer/indexer-worker [Jun 2015]

If this information is useful, please help other people find it:
Share via:

Infoomatic

2015-Jun-16 12:07 UTC

bug in indexer/indexer-worker

Hi,

I have already mentioned this in
http://www.dovecot.org/pipermail/dovecot/2014-November/098592.html

I could reproduce the errors above in a self-compiled v2.2.18 and the prebuilt
packages from xi.rename-it.nl (in addition to version 2.2.15 mentioned in the
link)

The problem occurs when enabling fts_tika in the plugins (tried tika 1.6, 1.7
and 1.8). I tried to move a folder of my mailbox with about 2000 mails to my
server (no users, modern hardware).
At some point I get an error and from this time on dovecot keeps repeating the
last lines with every mail that comes in ... see [1]
I then get kernel syslog messages like in [2]. when I try to index all users
(had another testuser) mailboxes with "doveadm index" I get the errors
of [3].

The mailfolder does contain mails with attachments of all sorts, videos, office
documents etc. When disabling fts_tika it works. dovecot -n output is [4]

Could someone of the devs please have a look?

best regards,
infoomatic


[1] Then comes the panic which produces loads of errormessages below the
"Raw backtrace" part:
Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f40b55c39d0 (line 411, fd 20)
Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: Timeout leak:
0x7f40b55c3780 (line 325)
Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Panic: file ioloop.c:
line 39 (io_add_file): assertion failed: (callback != NULL)
Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Error: Raw backtrace:
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x78320) [0x7f40b55f4320] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x783fe) [0x7f40b55f43fe] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f40b55a3b25] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x897d0) [0x7f40b56057d0] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f40b560643e]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14)
[0x7f40b55f21d4] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28)
[0x7f40b55c8148] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x44536)
[0x7f40b55c0536] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x49)
[0x7f40b108d469] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x3ac9) [0x7f40b1089ac9]
-> /opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x3b6d)
[0x7f40b1089b6d] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x86ec)
[0x7f40b495c6ec] ->
/opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_deinit+0x23)
[0x7f40b495c723] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xe5cc)
[0x7f40b49625cc] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xf451)
[0x7f40b4963451] -> /opt/dovecot/lib/dovecot/lib10_quota_plugin.so(+0xb454)
[0x7f40b4b97454] -> /opt/dovecot/lib/dovecot/lib01_acl_plugin.so(+0xcfea)
[0x7f40b4daafea] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x41)
[0x7f40b58a64f1] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0xe)
[0x7f40b58a658e] -> dovecot/indexer-worker() [0x40250c] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f40b560615c]
->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb)
[0x7f40b56070ab] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9)
[0x7f40b56061c9] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f40b5606248]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f40b55a9093] -> dovecot/indexer-worker(main+0xe3) [0x401f03] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f40b51d8ec5] ->
dovecot/indexer-worker() [0x401fc2]
Jun 16 10:11:08 testhost dovecot: indexer: Error: Indexer worker disconnected,
discarding 2 requests for user1
Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2066 killed with signal 6 (core dumped)
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f25bc1689d0 (line 120, fd 23)
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: Timeout leak:
0x7f25bc168780 (line 325)
Jun 16 10:11:09 testhost dovecot: indexer: Error: Indexer worker disconnected,
discarding 2 requests for user1
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2124 killed with signal 11 (core dumped)
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f6cabb149d0 (line 120, fd 23)
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: Timeout leak:
0x7f6cabb14780 (line 325)
Jun 16 10:11:09 testhost dovecot: indexer: Error: Indexer worker disconnected,
discarding 2 requests for user1
Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2125 killed with signal 11 (core dumped)

[2] kernel syslog:
[Tue Jun 16 09:51:15 2015] indexer-worker[7592]: segfault at 8 ip
00007f066e8c8c27 sp 00007fff36a038b0 error 4 in
libdovecot.so.0.0.0[7f066e832000+e7000]
[Tue Jun 16 09:51:16 2015] indexer-worker[7594]: segfault at 8 ip
00007f702671ac27 sp 00007ffc6e134840 error 4 in
libdovecot.so.0.0.0[7f7026684000+e7000]
[Tue Jun 16 09:51:16 2015] indexer-worker[7596]: segfault at 20 ip
00007fb8a5af2c47 sp 00007fff0a344780 error 4 in
libdovecot.so.0.0.0[7fb8a5a69000+e7000]
[Tue Jun 16 09:51:17 2015] indexer-worker[7598]: segfault at 20 ip
00007f5834b6ac47 sp 00007ffd54f35c00 error 4 in
libdovecot.so.0.0.0[7f5834ae1000+e7000]
[Tue Jun 16 09:51:18 2015] indexer-worker[7600]: segfault at 20 ip
00007f5b2f6fac47 sp 00007fff1a697690 error 4 in
libdovecot.so.0.0.0[7f5b2f671000+e7000]
[Tue Jun 16 09:51:19 2015] indexer-worker[7602]: segfault at 20 ip
00007fd6084ddc47 sp 00007ffdb4495af0 error 4 in
libdovecot.so.0.0.0[7fd608454000+e7000]
[Tue Jun 16 09:51:20 2015] indexer-worker[7604]: segfault at 20 ip
00007f7049789c47 sp 00007ffe71e4bdd0 error 4 in
libdovecot.so.0.0.0[7f7049700000+e7000]
[Tue Jun 16 09:51:20 2015] indexer-worker[7606]: segfault at 20 ip
00007f771d54dc47 sp 00007ffe01c6ff80 error 4 in
libdovecot.so.0.0.0[7f771d4c4000+e7000]
[Tue Jun 16 09:51:21 2015] indexer-worker[7608]: segfault at 20 ip
00007fc19d482c47 sp 00007ffc9a0b0280 error 4 in
libdovecot.so.0.0.0[7fc19d3f9000+e7000]
[Tue Jun 16 09:51:22 2015] indexer-worker[7610]: segfault at 29 ip
00007fa14df25c2b sp 00007ffeb246e5f0 error 4 in
libdovecot.so.0.0.0[7fa14de8f000+e7000]
[Tue Jun 16 09:51:23 2015] indexer-worker[7612]: segfault at 8 ip
00007fe8e41f2c27 sp 00007ffc700e2170 error 4 in
libdovecot.so.0.0.0[7fe8e415c000+e7000]
[Tue Jun 16 09:51:24 2015] indexer-worker[7614]: segfault at 8 ip
00007ff9a0c76c27 sp 00007fffecd44c90 error 4 in
libdovecot.so.0.0.0[7ff9a0be0000+e7000]
[Tue Jun 16 09:51:25 2015] indexer-worker[7616]: segfault at 8 ip
00007fd27867dc27 sp 00007ffedd8407f0 error 4 in
libdovecot.so.0.0.0[7fd2785e7000+e7000]
[Tue Jun 16 09:51:25 2015] indexer-worker[7618]: segfault at 8 ip
00007f94f8753c27 sp 00007fff56cc5ea0 error 4 in
libdovecot.so.0.0.0[7f94f86bd000+e7000]

[3] errors from "doveadm index":
/opt/dovecot/bin/doveadm index -A '*'
doveadm(user1): Warning: I/O leak: 0x7f0e072519d0 (line 120, fd 27)
doveadm(user1): Warning: Timeout leak: 0x7f0e07251780 (line 325)
Segmentation fault (core dumped)

/opt/dovecot/bin/doveadm index -u user1 '*'
doveadm(user1): Warning: I/O leak: 0x7f99b4fba9d0 (line 411, fd 14)
doveadm(user1): Warning: Timeout leak: 0x7f99b4fba780 (line 325)
doveadm(user1): Fatal: pool_system_realloc(17179869184): Out of memory
doveadm(user1): Error: Raw backtrace:
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x78320) [0x7f99b4feb320] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a)
[0x7f99b4feb38a] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f99b4f9ac7f] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x8f3a3)
[0x7f99b50023a3] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x7472d)
[0x7f99b4fe772d] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x5f)
[0x7f99b4fe7f5f] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x24)
[0x7f99b4ffdd94] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x89761)
[0x7f99b4ffc761] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f99b4ffd43e]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14)
[0x7f99b4fe91d4] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28)
[0x7f99b4fbf148] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x44536)
[0x7f99b4fb7536] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x49)
[0x7f99b0c9d469] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x373f) [0x7f99b0c9973f]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x91e8) [0x7f99b43541e8]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x95f0) [0x7f99b43545f0]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x714)
[0x7f99b4354d34] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xf17e)
[0x7f99b435a17e] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x19)
[0x7f99b5294e89] -> /opt/dovecot/bin/doveadm() [0x423211] ->
/opt/dovecot/bin/doveadm() [0x41ecc6] ->
/opt/dovecot/bin/doveadm(doveadm_mail_try_run+0x27f) [0x41fb8f] ->
/opt/dovecot/bin/doveadm(main+0x38f) [0x412a9f] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f99b4bcfec5] ->
/opt/dovecot/bin/doveadm() [0x412b99]

[4] dovecot -n
# 2.2.18: /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.8 (0c4ae064f307+)
# OS: Linux 3.13.0-52-generic x86_64 Ubuntu 14.04.2 LTS ext4
auth_username_format = %n
auth_verbose = yes
disable_plaintext_auth = no
imap_idle_notify_interval = 5 mins
mail_gid = vmail
mail_home = /var/lib/vmail/home/%d/%u
mail_location = mdbox:/var/lib/vmail/mail/%d/%u
mail_plugins = fts fts_solr zlib acl quota zlib fts fts_solr acl
mail_uid = vmail
mailbox_idle_check_interval = 1 mins
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
imapflags notify
mdbox_rotate_size = 32 M
namespace {
  list = children
  location = mdbox:/var/lib/vmail/mail/%d/%%u
  prefix = shared/%%u/
  separator = /
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  acl = vfile
  acl_defaults_from_inbox = yes
  acl_shared_dict = file:/var/lib/vmail/shared-mailboxes
  fts = solr
  fts_autoindex = yes
  fts_solr = url=http://localhost:8983/solr/ break-imap-search
  fts_tika = http://localhost:8081/tika
  quota = dict:Quota::file:%h/dovecot-quota
  quota_rule = *:storage=1G
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +notify +imapflags
  zlib_save = gz
  zlib_save_level = 9
}
protocols = imap sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
  unix_listener auth-userdb {
    group = vmail
    user = vmail
  }
}
service imap-login {
  inet_listener imaps {
    port = 0
  }
}
service managesieve-login {
  inet_listener sieve_deprecated {
    port = 2000
  }
}
service pop3-login {
  inet_listener pop3 {
    port = 0
  }
  inet_listener pop3s {
    port = 0
  }
}
ssl = required
ssl_cert = </etc/ssl/testhost/testhost.crt
ssl_cipher_list =
EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
ssl_key = </etc/ssl/testhost/testhost.key
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
protocol lda {
  mail_plugins = fts fts_solr zlib sieve acl
}
protocol imap {
  mail_plugins = fts fts_solr zlib acl quota imap_acl imap_quota
}
protocol sieve {
  mail_plugins = fts fts_solr
}

Infoomatic

2015-Jun-16 13:42 UTC

head link

Aw: bug in indexer/indexer-worker

I guess this is the same issue:
http://www.dovecot.org/list/dovecot/2015-June/101029.html

> Gesendet: Dienstag, 16. Juni 2015 um 14:07 Uhr
> Von: Infoomatic <infoomatic at gmx.at>
> An: "Dovecot ML" <dovecot at dovecot.org>
> Betreff: bug in indexer/indexer-worker
>
> Hi,
> 
> I have already mentioned this in
http://www.dovecot.org/pipermail/dovecot/2014-November/098592.html
> 
> I could reproduce the errors above in a self-compiled v2.2.18 and the
prebuilt packages from xi.rename-it.nl (in addition to version 2.2.15 mentioned
in the link)
> 
> The problem occurs when enabling fts_tika in the plugins (tried tika 1.6,
1.7 and 1.8). I tried to move a folder of my mailbox with about 2000 mails to my
server (no users, modern hardware).
> At some point I get an error and from this time on dovecot keeps repeating
the last lines with every mail that comes in ... see [1]
> I then get kernel syslog messages like in [2]. when I try to index all
users (had another testuser) mailboxes with "doveadm index" I get the
errors of [3].
> 
> The mailfolder does contain mails with attachments of all sorts, videos,
office documents etc. When disabling fts_tika it works. dovecot -n output is [4]
> 
> Could someone of the devs please have a look?
> 
> best regards,
> infoomatic
> 
> 
> [1] Then comes the panic which produces loads of errormessages below the
"Raw backtrace" part:
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f40b55c39d0 (line 411, fd 20)
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: Timeout
leak: 0x7f40b55c3780 (line 325)
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Panic: file
ioloop.c: line 39 (io_add_file): assertion failed: (callback != NULL)
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Error: Raw
backtrace: /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x78320) [0x7f40b55f4320]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x783fe) [0x7f40b55f43fe] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f40b55a3b25] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x897d0) [0x7f40b56057d0] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f40b560643e]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14)
[0x7f40b55f21d4] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28)
[0x7f40b55c8148] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x44536)
[0x7f40b55c0536] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x49)
[0x7f40b108d469] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x3ac9) [0x7f40b1089ac9]
-> /opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x3b6d)
[0x7f40b1089b6d] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x86ec)
[0x7f40b495c6ec] ->
/opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_deinit+0x23)
[0x7f40b495c723] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xe5cc)
[0x7f40b49625cc] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xf451)
[0x7f40b4963451] -> /opt/dovecot/lib/dovecot/lib10_quota_plugin.so(+0xb454)
[0x7f40b4b97454] -> /opt/dovecot/lib/dovecot/lib01_acl_plugin.so(+0xcfea)
[0x7f40b4daafea] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x41)
[0x7f40b58a64f1] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit+0xe)
[0x7f40b58a658e] -> dovecot/indexer-worker() [0x40250c] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f40b560615c]
->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb)
[0x7f40b56070ab] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9)
[0x7f40b56061c9] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f40b5606248]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f40b55a9093] -> dovecot/indexer-worker(main+0xe3) [0x401f03] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f40b51d8ec5] ->
dovecot/indexer-worker() [0x401fc2]
> Jun 16 10:11:08 testhost dovecot: indexer: Error: Indexer worker
disconnected, discarding 2 requests for user1
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2066 killed with signal 6 (core dumped)
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f25bc1689d0 (line 120, fd 23)
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: Timeout
leak: 0x7f25bc168780 (line 325)
> Jun 16 10:11:09 testhost dovecot: indexer: Error: Indexer worker
disconnected, discarding 2 requests for user1
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2124 killed with signal 11 (core dumped)
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f6cabb149d0 (line 120, fd 23)
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Warning: Timeout
leak: 0x7f6cabb14780 (line 325)
> Jun 16 10:11:09 testhost dovecot: indexer: Error: Indexer worker
disconnected, discarding 2 requests for user1
> Jun 16 10:11:09 testhost dovecot: indexer-worker(user1): Fatal: master:
service(indexer-worker): child 2125 killed with signal 11 (core dumped)
> 
> [2] kernel syslog:
> [Tue Jun 16 09:51:15 2015] indexer-worker[7592]: segfault at 8 ip
00007f066e8c8c27 sp 00007fff36a038b0 error 4 in
libdovecot.so.0.0.0[7f066e832000+e7000]
> [Tue Jun 16 09:51:16 2015] indexer-worker[7594]: segfault at 8 ip
00007f702671ac27 sp 00007ffc6e134840 error 4 in
libdovecot.so.0.0.0[7f7026684000+e7000]
> [Tue Jun 16 09:51:16 2015] indexer-worker[7596]: segfault at 20 ip
00007fb8a5af2c47 sp 00007fff0a344780 error 4 in
libdovecot.so.0.0.0[7fb8a5a69000+e7000]
> [Tue Jun 16 09:51:17 2015] indexer-worker[7598]: segfault at 20 ip
00007f5834b6ac47 sp 00007ffd54f35c00 error 4 in
libdovecot.so.0.0.0[7f5834ae1000+e7000]
> [Tue Jun 16 09:51:18 2015] indexer-worker[7600]: segfault at 20 ip
00007f5b2f6fac47 sp 00007fff1a697690 error 4 in
libdovecot.so.0.0.0[7f5b2f671000+e7000]
> [Tue Jun 16 09:51:19 2015] indexer-worker[7602]: segfault at 20 ip
00007fd6084ddc47 sp 00007ffdb4495af0 error 4 in
libdovecot.so.0.0.0[7fd608454000+e7000]
> [Tue Jun 16 09:51:20 2015] indexer-worker[7604]: segfault at 20 ip
00007f7049789c47 sp 00007ffe71e4bdd0 error 4 in
libdovecot.so.0.0.0[7f7049700000+e7000]
> [Tue Jun 16 09:51:20 2015] indexer-worker[7606]: segfault at 20 ip
00007f771d54dc47 sp 00007ffe01c6ff80 error 4 in
libdovecot.so.0.0.0[7f771d4c4000+e7000]
> [Tue Jun 16 09:51:21 2015] indexer-worker[7608]: segfault at 20 ip
00007fc19d482c47 sp 00007ffc9a0b0280 error 4 in
libdovecot.so.0.0.0[7fc19d3f9000+e7000]
> [Tue Jun 16 09:51:22 2015] indexer-worker[7610]: segfault at 29 ip
00007fa14df25c2b sp 00007ffeb246e5f0 error 4 in
libdovecot.so.0.0.0[7fa14de8f000+e7000]
> [Tue Jun 16 09:51:23 2015] indexer-worker[7612]: segfault at 8 ip
00007fe8e41f2c27 sp 00007ffc700e2170 error 4 in
libdovecot.so.0.0.0[7fe8e415c000+e7000]
> [Tue Jun 16 09:51:24 2015] indexer-worker[7614]: segfault at 8 ip
00007ff9a0c76c27 sp 00007fffecd44c90 error 4 in
libdovecot.so.0.0.0[7ff9a0be0000+e7000]
> [Tue Jun 16 09:51:25 2015] indexer-worker[7616]: segfault at 8 ip
00007fd27867dc27 sp 00007ffedd8407f0 error 4 in
libdovecot.so.0.0.0[7fd2785e7000+e7000]
> [Tue Jun 16 09:51:25 2015] indexer-worker[7618]: segfault at 8 ip
00007f94f8753c27 sp 00007fff56cc5ea0 error 4 in
libdovecot.so.0.0.0[7f94f86bd000+e7000]
> 
> [3] errors from "doveadm index":
> /opt/dovecot/bin/doveadm index -A '*'
> doveadm(user1): Warning: I/O leak: 0x7f0e072519d0 (line 120, fd 27)
> doveadm(user1): Warning: Timeout leak: 0x7f0e07251780 (line 325)
> Segmentation fault (core dumped)
> 
> /opt/dovecot/bin/doveadm index -u user1 '*'
> doveadm(user1): Warning: I/O leak: 0x7f99b4fba9d0 (line 411, fd 14)
> doveadm(user1): Warning: Timeout leak: 0x7f99b4fba780 (line 325)
> doveadm(user1): Fatal: pool_system_realloc(17179869184): Out of memory
> doveadm(user1): Error: Raw backtrace:
/opt/dovecot/lib/dovecot/libdovecot.so.0(+0x78320) [0x7f99b4feb320] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a)
[0x7f99b4feb38a] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f99b4f9ac7f] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x8f3a3)
[0x7f99b50023a3] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x7472d)
[0x7f99b4fe772d] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x5f)
[0x7f99b4fe7f5f] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x24)
[0x7f99b4ffdd94] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x89761)
[0x7f99b4ffc761] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f99b4ffd43e]
-> /opt/dovecot/lib/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14)
[0x7f99b4fe91d4] ->
/opt/dovecot/lib/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28)
[0x7f99b4fbf148] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0x44536)
[0x7f99b4fb7536] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x49)
[0x7f99b0c9d469] ->
/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so(+0x373f) [0x7f99b0c9973f]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x91e8) [0x7f99b43541e8]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x95f0) [0x7f99b43545f0]
-> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x714)
[0x7f99b4354d34] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xf17e)
[0x7f99b435a17e] ->
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x19)
[0x7f99b5294e89] -> /opt/dovecot/bin/doveadm() [0x423211] ->
/opt/dovecot/bin/doveadm() [0x41ecc6] ->
/opt/dovecot/bin/doveadm(doveadm_mail_try_run+0x27f) [0x41fb8f] ->
/opt/dovecot/bin/doveadm(main+0x38f) [0x412a9f] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f99b4bcfec5] ->
/opt/dovecot/bin/doveadm() [0x412b99]
> 
> [4] dovecot -n
> # 2.2.18: /opt/dovecot/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.8 (0c4ae064f307+)
> # OS: Linux 3.13.0-52-generic x86_64 Ubuntu 14.04.2 LTS ext4
> auth_username_format = %n
> auth_verbose = yes
> disable_plaintext_auth = no
> imap_idle_notify_interval = 5 mins
> mail_gid = vmail
> mail_home = /var/lib/vmail/home/%d/%u
> mail_location = mdbox:/var/lib/vmail/mail/%d/%u
> mail_plugins = fts fts_solr zlib acl quota zlib fts fts_solr acl
> mail_uid = vmail
> mailbox_idle_check_interval = 1 mins
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
imapflags notify
> mdbox_rotate_size = 32 M
> namespace {
>   list = children
>   location = mdbox:/var/lib/vmail/mail/%d/%%u
>   prefix = shared/%%u/
>   separator = /
>   subscriptions = no
>   type = shared
> }
> namespace inbox {
>   inbox = yes
>   location = 
>   mailbox Drafts {
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     special_use = \Junk
>   }
>   mailbox Sent {
>     special_use = \Sent
>   }
>   mailbox "Sent Messages" {
>     special_use = \Sent
>   }
>   mailbox Trash {
>     special_use = \Trash
>   }
>   prefix =
>   separator = /
>   type = private
> }
> passdb {
>   args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> plugin {
>   acl = vfile
>   acl_defaults_from_inbox = yes
>   acl_shared_dict = file:/var/lib/vmail/shared-mailboxes
>   fts = solr
>   fts_autoindex = yes
>   fts_solr = url=http://localhost:8983/solr/ break-imap-search
>   fts_tika = http://localhost:8081/tika
>   quota = dict:Quota::file:%h/dovecot-quota
>   quota_rule = *:storage=1G
>   quota_warning = storage=95%% quota-warning 95 %u
>   quota_warning2 = storage=80%% quota-warning 80 %u
>   sieve = ~/.dovecot.sieve
>   sieve_dir = ~/sieve
>   sieve_extensions = +notify +imapflags
>   zlib_save = gz
>   zlib_save_level = 9
> }
> protocols = imap sieve
> service auth {
>   unix_listener /var/spool/postfix/private/auth {
>     group = postfix
>     mode = 0666
>     user = postfix
>   }
>   unix_listener auth-userdb {
>     group = vmail
>     user = vmail
>   }
> }
> service imap-login {
>   inet_listener imaps {
>     port = 0
>   }
> }
> service managesieve-login {
>   inet_listener sieve_deprecated {
>     port = 2000
>   }
> }
> service pop3-login {
>   inet_listener pop3 {
>     port = 0
>   }
>   inet_listener pop3s {
>     port = 0
>   }
> }
> ssl = required
> ssl_cert = </etc/ssl/testhost/testhost.crt
> ssl_cipher_list =
EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> ssl_key = </etc/ssl/testhost/testhost.key
> ssl_prefer_server_ciphers = yes
> ssl_protocols = !SSLv2 !SSLv3
> userdb {
>   args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> protocol lda {
>   mail_plugins = fts fts_solr zlib sieve acl
> }
> protocol imap {
>   mail_plugins = fts fts_solr zlib acl quota imap_acl imap_quota
> }
> protocol sieve {
>   mail_plugins = fts fts_solr
> }
>

Timo Sirainen

2015-Jun-17 10:13 UTC

head link

bug in indexer/indexer-worker

On 16 Jun 2015, at 15:07, Infoomatic <infoomatic at gmx.at> wrote:
> [1] Then comes the panic which produces loads of errormessages below the
"Raw backtrace" part:
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: I/O leak:
0x7f40b55c39d0 (line 411, fd 20)
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: Timeout
leak: 0x7f40b55c3780 (line 325)
> Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Panic: file
ioloop.c: line 39 (io_add_file): assertion failed: (callback != NULL)
Should be fixed by http://hg.dovecot.org/dovecot-2.2/rev/3ae8cf3f8320

Infoo Matic

2015-Jun-17 22:19 UTC

head link

Re: bug in indexer/indexer-worker

On Wednesday, June 17, 2015 12:13 CEST, Timo Sirainen <tss at iki.fi>
wrote:
 > On 16 Jun 2015, at 15:07, Infoomatic <infoomatic at gmx.at> wrote:
> 
> > [1] Then comes the panic which produces loads of errormessages below
the "Raw backtrace" part:
> > Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning: I/O
leak: 0x7f40b55c39d0 (line 411, fd 20)
> > Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Warning:
Timeout leak: 0x7f40b55c3780 (line 325)
> > Jun 16 10:11:08 testhost dovecot: indexer-worker(user1): Panic: file
ioloop.c: line 39 (io_add_file): assertion failed: (callback != NULL)
> 
> Should be fixed by http://hg.dovecot.org/dovecot-2.2/rev/3ae8cf3f8320
wow, that was fast! Great, works on my tests! Thanks a lot!

Possibly Parallel Threads

Search for more seemingly similar threads

dovecot - Jun 2015 - bug in indexer/indexer-worker

bug in indexer/indexer-worker

Aw: bug in indexer/indexer-worker

bug in indexer/indexer-worker

Re: bug in indexer/indexer-worker

Possibly Parallel Threads