[Dovecot] Possible mdbox corruption

Daniel L. Miller
Thu, 05 Jan 2012 00:24:40 -0000

I thought I had cleared out the corruption I had before - perhaps I was
mistaken.  What steps should I take to help locate these issues?
Currently using 2.1rc1.  I see the following errors in my logs,
including out of memory and message size issues (at 15:30):

Jan  4 05:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f6e17cbfd0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f6e17cbfd56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f6e17c98d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f6e17cd0310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f6e17cbc965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f6e17cbd0ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f6e164b7292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f6e164b7a97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f6e166c4abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f6e166c5561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f6e166ca630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f6e17ccc0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f6e17ccd17f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f6e17ccc098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f6e17cb9123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f6e1791cd8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 05:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 05:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 10896 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 06:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 06:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f0ec1a57d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f0ec1a57d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f0ec1a30d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f0ec1a68310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f0ec1a54965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f0ec1a550ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f0ec024f292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f0ec024fa97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f0ec045cabc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f0ec045d561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f0ec0462630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f0ec1a640f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f0ec1a6517f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f0ec1a64098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f0ec1a51123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f0ec16b4d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 06:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 06:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 11941 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 07:17:18 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 07:17:18 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7faed4e56d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7faed4e56d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7faed4e2fd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7faed4e67310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7faed4e53965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7faed4e540ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7faed364e292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7faed364ea97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7faed385babc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7faed385c561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7faed3861630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7faed4e630f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7faed4e6417f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7faed4e63098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7faed4e50123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7faed4ab3d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 07:17:18 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 07:17:18 bubba dovecot: master: Error: service(indexer-worker):
child 13299 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 08:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 08:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7ffd84382d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7ffd84382d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7ffd8435bd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7ffd84393310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7ffd8437f965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7ffd843800ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7ffd82b7a292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7ffd82b7aa97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7ffd82d87abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7ffd82d88561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7ffd82d8d630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7ffd8438f0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7ffd8439017f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7ffd8438f098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7ffd8437c123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7ffd83fdfd8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 08:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 08:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 14413 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 09:17:19 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 09:17:19 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7fb701bf5d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7fb701bf5d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7fb701bced08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7fb701c06310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7fb701bf2965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7fb701bf30ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7fb7003ed292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7fb7003eda97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7fb7005faabc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7fb7005fb561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7fb700600630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7fb701c020f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7fb701c0317f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7fb701c02098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7fb701bef123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7fb701852d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 09:17:19 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 09:17:19 bubba dovecot: master: Error: service(indexer-worker):
child 15486 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 10:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 10:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f8dc590ed0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f8dc590ed56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f8dc58e7d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f8dc591f310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f8dc590b965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f8dc590c0ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f8dc4106292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f8dc4106a97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f8dc4313abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f8dc4314561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f8dc4319630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7510]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f8dc591b0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f8dc591c17f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f8dc591b098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f8dc5908123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7510](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f8dc556bd8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7510]() [0x401d19]
Jan  4 10:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 10:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 16472 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 11:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 11:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7ff619c1dd0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7ff619c1dd56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7ff619bf6d08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7ff619c2e310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7ff619c1a965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7ff619c1b0ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7ff618415292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7ff618415a97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7ff618622abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7ff618623561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7ff618628630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7514]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7514]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7ff619c2a0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7ff619c2b17f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7ff619c2a098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7ff619c17123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7514](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7ff61987ad8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7514]() [0x401d19]
Jan  4 11:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 11:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 17522 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 12:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 12:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7ffd988c1d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7ffd988c1d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7ffd9889ad08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7ffd988d2310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7ffd988be965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7ffd988bf0ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7ffd970b9292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7ffd970b9a97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7ffd972c6abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7ffd972c7561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7ffd972cc630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7515]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7515]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7ffd988ce0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7ffd988cf17f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7ffd988ce098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7ffd988bb123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7515](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7ffd9851ed8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7515]() [0x401d19]
Jan  4 12:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 12:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 18498 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 13:17:16 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 13:17:16 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f51b0163d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f51b0163d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f51b013cd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f51b0174310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f51b0160965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f51b01610ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f51ae95b292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f51ae95ba97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f51aeb68abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f51aeb69561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f51aeb6e630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7515]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7515]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f51b01700f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f51b017117f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f51b0170098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f51b015d123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7515](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f51afdc0d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7515]() [0x401d19]
Jan  4 13:17:16 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 13:17:16 bubba dovecot: master: Error: service(indexer-worker):
child 19550 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 14:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 14:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f423b546d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f423b546d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f423b51fd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f423b557310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f423b543965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f423b5440ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f4239d3e292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f4239d3ea97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f4239f4babc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f4239f4c561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f4239f51630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7516]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7516]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f423b5530f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f423b55417f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f423b553098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f423b540123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7516](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f423b1a3d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7516]() [0x401d19]
Jan  4 14:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 20638 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 14:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 15:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 15:17:17 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7f3ab5e51d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7f3ab5e51d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f3ab5e2ad08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7f3ab5e62310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7f3ab5e4e965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7f3ab5e4f0ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7f3ab4649292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7f3ab4649a97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7f3ab4856abc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7f3ab4857561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7f3ab485c630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7517]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7517]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f3ab5e5e0f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7f3ab5e5f17f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f3ab5e5e098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f3ab5e4b123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7517](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7f3ab5aaed8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7517]() [0x401d19]
Jan  4 15:17:17 bubba dovecot: master: Error: service(indexer-worker):
child 21821 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 15:17:17 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com
Jan  4 15:30:48 bubba dovecot: imap(user2 at domain.com): Error: Cached
message size smaller than expected (822 < 1493)
Jan  4 15:30:48 bubba dovecot: imap(user2 at domain.com): Error: Corrupted
index cache file
/var/mail/amfes.com/lmiller/mdbox/mailboxes/Sent/dbox-Mails/dovecot.index.cache:
Broken physical size for mail UID 1786
Jan  4 15:30:48 bubba dovecot: imap(user2 at domain.com): Error:
read(/var/mail/amfes.com/lmiller/mdbox/storage/m.208) failed:
Input/output error (FETCH for mailbox Sent UID 1786)
Jan  4 16:17:20 bubba dovecot: indexer-worker(user1 at domain.com): Fatal:
pool_system_realloc(134217728): Out of memory
Jan  4 16:17:20 bubba dovecot: indexer-worker(user1 at domain.com): Error:
Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed0a)
[0x7ffc91276d0a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3ed56)
[0x7ffc91276d56] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7ffc9124fd08] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4f310)
[0x7ffc91287310] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3b965)
[0x7ffc91273965] ->
/usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x7c)
[0x7ffc912740ec] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3292)
[0x7ffc8fa6e292] ->
/usr/local/lib/dovecot/lib21_fts_solr_plugin.so(+0x3a97)
[0x7ffc8fa6ea97] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_update_set_build_key+0x2c)
[0x7ffc8fc7babc] ->
/usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x2d1)
[0x7ffc8fc7c561] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xc630)
[0x7ffc8fc81630] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7517]() [0x40245f] -> dovecot/indexer-worker [user1 at domain.com Sent
- 5500/7517]() [0x4027dd] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7ffc912830f6] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f)
[0x7ffc9128417f] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7ffc91283098] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7ffc91270123] -> dovecot/indexer-worker [user1 at domain.com Sent -
5500/7517](main+0x109) [0x401f29] ->
/lib/libc.so.6(__libc_start_main+0xfe) [0x7ffc90ed3d8e] ->
dovecot/indexer-worker [user1 at domain.com Sent - 5500/7517]() [0x401d19]
Jan  4 16:17:20 bubba dovecot: master: Error: service(indexer-worker):
child 22927 returned error 83 (Out of memory (vsz_limit=256 MB, you may
need to increase it))
Jan  4 16:17:20 bubba dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for user1 at domain.com

--
Daniel