I've noticed that my dovecot.index.cache file has got very large (24MB, which is more than 1/3 of my INBOX size). I've tried deleting (well, renaming) the dovecot.index.* files, and re-opening my INBOX, and the new dovecot.index.cache is only 90KB so far. I'm using dovecot-1.0-stable and mbox format. My mail_cache_fields and mail_never_cache_fields are set to the default. Looking back in the list archives, I found a message from Timo that suggested the cache entries are created for each message when the indexes are created, rather than when each message is accessed, so it ought not to grow? What happens when a message is deleted? I "strings/grep" ed through the dovecot.index.cache file and it looks like it's not caching any headers it shouldn't (like Received, X-Spam-XYZ etc.), but they seem to occur multiple times for each message. None of my other dovecot testers seem to have huge cache files (yet!). Anybody else seen this? Best Wishes, Chris -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin at reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On Tuesday 14 June 2005 15:35, Chris Wakelin wrote:> I've noticed that my dovecot.index.cache file has got very large > (24MB, which is more than 1/3 of my INBOX size). I've tried deleting > (well, renaming) the dovecot.index.* files, and re-opening my INBOX, > and the new dovecot.index.cache is only 90KB so far. > > I'm using dovecot-1.0-stable and mbox format. My mail_cache_fields > and mail_never_cache_fields are set to the default. > > Looking back in the list archives, I found a message from Timo that > suggested the cache entries are created for each message when the > indexes are created, rather than when each message is accessed, so it > ought not to grow? What happens when a message is deleted? > > I "strings/grep" ed through the dovecot.index.cache file and it looks > like it's not caching any headers it shouldn't (like Received, > X-Spam-XYZ etc.), but they seem to occur multiple times for each > message. > > None of my other dovecot testers seem to have huge cache files > (yet!). > > Anybody else seen this?A sample from some mailboxes here. Dovecot 0.99.14 and maildirs. Username Mailbox Size Index Size Messages A 41M 5M 7467 B 115M 24M 12221 C 303M 40M 63067 D 1.7G 8.5M 10459 E 505M 2M 1193 F 109M 25M 30594 G 763M 1.6M 2723> Best Wishes, > Chris-- Dominic GoodforBusiness.co.uk I.T. Services for SMEs in the UK.
On 14.6.2005, at 17:35, Chris Wakelin wrote:> I've noticed that my dovecot.index.cache file has got very large > (24MB, which is more than 1/3 of my INBOX size). I've tried deleting > (well, renaming) the dovecot.index.* files, and re-opening my INBOX, > and the new dovecot.index.cache is only 90KB so far.Fixed in CVS. The mails weren't being marked as deleted in cache file because of a simple mistake ("!=" -> "=="). So after the change the files still won't get compressed immediately, but it'll work in future. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20050616/7a1b709b/attachment-0001.bin>
Timo Sirainen wrote:> On 14.6.2005, at 17:35, Chris Wakelin wrote: > >> I've noticed that my dovecot.index.cache file has got very large >> (24MB, which is more than 1/3 of my INBOX size). I've tried deleting >> (well, renaming) the dovecot.index.* files, and re-opening my INBOX, >> and the new dovecot.index.cache is only 90KB so far. > > > Fixed in CVS. The mails weren't being marked as deleted in cache file > because of a simple mistake ("!=" -> "=="). So after the change the > files still won't get compressed immediately, but it'll work in future.Thanks! I've applied the fix (I think!) to dovecot-stable-20050617 and am now using it. My dovecot.index.cache is 5MB or so at the moment, and still seems to be growing, even when I delete lots of old messages. Presumably there needs to be 20% "deleted space" before anything will happen. Does it ever rebuild the cache from scratch in normal operation? What happens if a non-dovecot process deletes messages, do they ever get removed from the cache? Lastly, do you have a tool or utility to analyse the cache files so I can see how many messages/headers and how much "deleted space" there is? "strings" isn't too reliable an analysis method ;) Best Wishes, Chris -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin at reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094