We're running 2.2.24 with fts_lucene and I wonder why indexer-working
can be out-of-memory even with
vsz_limit=750 MB
?!
Peer
May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal:
pool_system_realloc(134217728): Out of memory
May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error:
Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7)
[0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851)
[0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405)
[0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2)
[0x7f37027b36e2] -> /usr/lib/dovecot/libdovecot.so.0(+0xa5813)
[0x7f37027b3813] -> /usr/lib/dovecot/libdovecot.so.0(+0xa59e3)
[0x7f37027b39e3] -> /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x37)
[0x7f37027b3cf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10b7a6)
[0x7f3702b617a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10c0c3)
[0x7f3702b620c3] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x34d)
[0x7f3702b625e5]
-> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10fb49) [0x7f3702b65b49]
-> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x38)
[0x7f3702b66262] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x5ba)
[0x7f3702b669b2] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0xbf)
[0x7f3702b59eb7] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_refresh+0x1d)
[0x7f3702b79e97] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x11cc33)
[0x7f3702b72c33] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x127)
[0x7f3702b72dba] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit+0x1f)
[0x7f3702b72c91] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_header+0x68)
[0x7f37014ebfd0] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_last_uid+0x3b)
[0x7f37014ec00d]
-> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4093)
[0x7f3700686093] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_get_last_uid+0x81)
[0x7f37014eb3e5] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11b66) [0x7f37014f3b66]
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11c40)
[0x7f37014f3c40] ->
/usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xea0d) [0x7f3701936a0d]
-> /usr/lib/dovecot/modules/lib01_acl_plugin.so(+0x11f19)
[0x7f3701b50f19] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x75)
[0x7f3702aa2db4]
May 15 12:14:51 dobby4 dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user at example.com
May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com):
Fatal: master: service(indexer-worker): child 179030 returned error 83
(Out of memory (service indexer-worker { vsz_limit=750 MB }, you may
need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)
--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42
Fax: 030 / 405051-19
Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin
aki.tuomi at dovecot.fi
2016-May-15 14:28 UTC
indexer-worker out of memory with vsz_limit=750 MB?!
> On May 15, 2016 at 2:48 PM Peer Heinlein <p.heinlein at heinlein-support.de> wrote: > > > > We're running 2.2.24 with fts_lucene and I wonder why indexer-working > can be out-of-memory even with > > vsz_limit=750 MB > > ?! > > Peer > > > May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: > pool_system_realloc(134217728): Out of memory > May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error: > Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) > [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) > [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) > [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) > [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2)Any chance for you to get a corefile and run gdb command 'bt full' on it? Aki
> On 15 May 2016, at 17:28, aki.tuomi at dovecot.fi wrote: > >> >> On May 15, 2016 at 2:48 PM Peer Heinlein <p.heinlein at heinlein-support.de> wrote: >> >> >> >> We're running 2.2.24 with fts_lucene and I wonder why indexer-working >> can be out-of-memory even with >> >> vsz_limit=750 MB >> >> ?! >> >> Peer >> >> >> May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: >> pool_system_realloc(134217728): Out of memory >> May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error: >> Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) >> [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) >> [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) >> [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) >> [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2) > > Any chance for you to get a corefile and run gdb command 'bt full' on it?Setting CORE_OUTOFMEM=1 environment will produce core dump on out of memory (a huge one of course).