dovecot - Jan 2006 - dovecot memory consumption beta vs. old "stable" series

Hi!

I just wanted to know if anyone can give a definitive answer if dovecot 
beta's memory requirements are much better than in the old pre-alpha 
"stable" series?

I'm using a rather old stable.20050712 release on an also rather old 
Debian Sarge box to manage my personal mails. I'm pretty satisfied with 
this dovecot release but it's memory consumption when deleting messages 
causes major troubles on this box - more than 500 MB of virtual memory 
allocated to the "imap" process is quite a lot and often causes random
processes to fall victim to the OOM killer. :-(

Adding even more swap space also is no viable option. (The system has 64 
MB RAM and currently about 650 MB swap space.)

So, would it be worth a try to install dovecot beta2 or something when 
it's released, or won't it help in this regard? I won't want to
sacrificy
the otherwise well working installation, if dovecot beta2 isn't 
guaranteed to help I'll continue to monitor available memory and swap 
space while cleaning up my spams...

Greetings,

  Gunter

-- 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ http://aachen.uni-dsl.de/   -   Der direkte Draht in's Hochschulnetz! +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
"What do people like to drink here, then?"
The landlord looked sideways at his customers, a clever trick given that 
they were directly in front of him.        -- Mort goes out for a drink  
         (Terry Pratchett, Mort)
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+                   PGP-verschl?sselte Mails bevorzugt!                 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060120/607f361d/attachment.pgp

On Fri, 2006-01-20 at 14:01 +0100, Gunter Ohrner wrote:
> I just wanted to know if anyone can give a definitive answer if dovecot 
> beta's memory requirements are much better than in the old pre-alpha 
> "stable" series?
Probably somewhat similar.
> I'm using a rather old stable.20050712 release on an also rather old 
> Debian Sarge box to manage my personal mails. I'm pretty satisfied with
> this dovecot release but it's memory consumption when deleting messages
> causes major troubles on this box - more than 500 MB of virtual memory 
> allocated to the "imap" process is quite a lot and often causes
random
> processes to fall victim to the OOM killer. :-(
Umm. If it's allocating 500MB when deleting messages there's something
seriously wrong. I haven't heard this happening with any Dovecot version
before.

With "deleting" do you mean marking messages as deleted, or the
expunge
operation? With mbox or maildir? Does it matter how many messages you're
deleting? How large the mailbox itself is?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060122/a4ecea9a/attachment.pgp

Whoops, sorry for the late answer, I somehow missed your response!

Am Sonntag, 22. Januar 2006 13:33, schrieb Timo
Sirainen:
> > dovecot beta's memory requirements are much better than in the old
> Probably somewhat similar.
Mh, ok.
> > messages causes major troubles on this box - more than 500 MB of
> > virtual memory allocated to the "imap" process is quite a
lot and
> > often causes random processes to fall victim to the OOM killer. :-(
> Umm. If it's allocating 500MB when deleting messages there's
something
> seriously wrong. I haven't heard this happening with any Dovecot
> version before.
Ok. Well, it does and it's easily reproduceable with this stable 
release. :-)

I want to upgrade dovecot to a recent beta. If the problem also happens 
with this version, I could try to help to track it down. Or should I 
first do some tests with the currently installed release?
> With "deleting" do you mean marking messages as deleted, or the
expunge
> operation?
It seems to be the marking. It happens if I "delete" messages from
kMail,
instead of moving them to trash. I'm not sure what happens under the hood 
if I do, but the message files seem to stay there, at least until the 
imap process exits cleanly.
> With mbox or maildir? 
maildir
> Does it matter how many messages you're deleting?
Yes, that's the reason. I run into it if I clean up my Spam which I do in 
about 2 week intervals. I just tested it, deleting (marking) 6000 to 7000 
files caused dovecot to grow to more than 200 MB:

 9253 gunter     9   0  203m  14m  12m S  0.0 23.0   6:40.52 imap

I noticed that the REAL memory consumption starts if dovecot gets OOM 
killed in the first try and I retry after artificially increasing the 
swap space size. If repeating the same operation after this kill, dovecot 
easily eats twice as much VIRT memory than it would have before. (This 
"would have" is extrapolated from experience in this case, of course.)

Maybe it's somehow index related, if the killed process leaves stale  or 
corrupt index files?
> How large the mailbox itself is? 
Currently about 17000 files. (My Spam since the beginning of January.)

Thanks,

  Gunter

-- 
*** Powered by AudioScrobbler --> http://www.last.fm/user/Interneci/ ***
00:00 | Lullacry - Don't Touch the Flame
23:57 | Paradise Lost - Self Obsessed
23:52 | Therion - Secret of the Runes
23:47 | Within Temptation - See Who I Am
*** PGP-Verschl?sselung bei eMails erw?nscht :-) *** PGP: 0x1128F25F ***
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20060214/359b67b9/attachment.bin>