dovecot - Jan 2009 - Out of memory

Sorry for previous post with incorrect referrals/thread.

For large mailboxes (64k + messages)  using maildir I got  random "out
of memory" errors.


[mail addresses hidden]

dovecot: Dec 05 18:25:27 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:25:27 Error: child 14142 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:27:29 Error: IMAP(****@****): file maildir-uidlist.c: line
1117 (maildir_uidlist_sync_deinit): assertion failed: (ctx->locked)
dovecot: Dec 05 18:27:29 Error: IMAP(****@****): Raw backtrace: imap
[0x5555555bd6fe] -> imap [0x5555555bd43e] -> imap [0x55555557dd57] ->
imap [0x55555557c026] -> imap(maildir_storage_sync_force+0x43)
[0x55555557c2b3] -> imap(maildir_storage_sync_init+0xc2) [0x55555557c3e2]
-> imap(imap_sync_nonselected+0xf) [0x55555557533f] ->
imap(_cmd_select_full+0xc5) [0x55555556da05] -> imap(cmd_select+0xb)
[0x55555556db9b] -> imap [0x55555556eee7] -> imap [0x55555556ef79] ->
imap(_client_input+0x6f) [0x55555556f61f] -> imap(io_loop_handler_run+0x108)
[0x5555555c2f38] -> imap(io_loop_run+0x18) [0x5555555c20f8] ->
imap(main+0x41b) [0x5555555770fb] -> /lib64/libc.so.6(__libc_start_main+0xf4)
[0x2b02fcb30b54] -> imap [0x55555556a6e9]
dovecot: Dec 05 18:27:29 Error: child 14144 (imap) killed with signal 6
dovecot: Dec 05 18:27:37 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:27:37 Error: child 14211 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:29:41 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:29:41 Error: child 14215 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:31:46 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:31:46 Error: child 14248 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:33:50 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:33:50 Error: child 14345 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:35:55 Error: IMAP(****@****): block_alloc(): Out of memory
dovecot: Dec 05 18:35:55 Error: child 14406 (imap) returned error 83 (Out of
memory)
dovecot: Dec 05 18:37:59 Error: IMAP(****@****): block_alloc(): Out of memory


dovecot 1.0.17
mail_prcosess_size = 256

ulimit -a :
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 73728
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) 6959460
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 73728
virtual memory          (kbytes, -v) 9910880
file locks                      (-x) unlimited

cat /proc/meminfo :
MemTotal:      8187540 kB
MemFree:        353060 kB
Buffers:        737968 kB
Cached:        5094700 kB
SwapCached:         20 kB
Active:        4255100 kB
Inactive:      2787980 kB
SwapTotal:     4200988 kB
SwapFree:      4192956 kB
Dirty:           20184 kB
Writeback:         124 kB
AnonPages:     1165756 kB
Mapped:          60472 kB
Slab:           686100 kB
SReclaimable:   591232 kB
SUnreclaim:      94868 kB
PageTables:      31408 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   8294756 kB
Committed_AS:  3417584 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     91040 kB
VmallocChunk: 34359645611 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB



What I need to configure to increase available resources? What else I
can done?

Uldis

On Tue, 2009-01-13 at 20:12 +0200, Uldis Pakuls wrote:
> Sorry for previous post with incorrect referrals/thread.
> 
> For large mailboxes (64k + messages)  using maildir I got  random "out
> of memory" errors.
> 
> 
> [mail addresses hidden]
> 
> dovecot: Dec 05 18:25:27 Error: IMAP(****@****): block_alloc(): Out of
memory
> dovecot: Dec 05 18:25:27 Error: child 14142 (imap) returned error 83 (Out
of memory)
Increase mail_process_size setting in dovecot.conf or just set it to 0
to disable the limits. Although only 64k messages shouldn't normally use
up the default 256 MB limit.. How large is the dovecot.index.cache file
for those mailboxes? mmaping that file is usually what takes most of the
memory.
> dovecot: Dec 05 18:27:29 Error: IMAP(****@****): file maildir-uidlist.c:
line 1117 (maildir_uidlist_sync_deinit): assertion failed: (ctx->locked)
This is probably unrelated, but shouldn't happen either. Do you see them
often? Or does it always happen among these out of memory errors?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090113/e2023e4e/attachment-0002.bin>

Timo Sirainen wrote:
> How large is the dovecot.index.cache file
> for those mailboxes? mmaping that file is usually what takes most of the
> memory.
>
>   
~ 10-15 MB
mail file size 2-8k
>> dovecot: Dec 05 18:27:29 Error: IMAP(****@****): file
maildir-uidlist.c: line 1117 (maildir_uidlist_sync_deinit): assertion failed:
(ctx->locked)
>>     
>
> This is probably unrelated, but shouldn't happen either. Do you see
them
> often? Or does it always happen among these out of memory errors?
>   
It always happen among these errors. Always this error.

On Tue, 2009-01-13 at 20:12 +0200, Uldis Pakuls wrote:
> dovecot 1.0.17
There is no such version. What version are you really using?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090114/f5537e74/attachment-0002.bin>

Timo Sirainen wrote:
> On Tue, 2009-01-13 at 20:12 +0200, Uldis Pakuls wrote:
>   
>> dovecot 1.0.17
>>     
>
> There is no such version. What version are you really using?
>
>   
Sorry - mistyped this 1.0.15. Complied from source: 
http://www.dovecot.org/releases/1.0/dovecot-1.0.15.tar.gz