Rob Redpath
2013-Jan-29 19:58 UTC
[Dovecot] Broken physical size caching in Dovecot 2.1.10
Hi, I'm encountering two, probably related, errors in my Dovecot 2.1.10 install. The issue relates to compressed email stored in Maildir format directories. In some cases, a mailbox will become inaccessible, and the following will be logged from a doveadm fetch:- doveadm(info at example.com): Error: Corrupted index cache file /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken physical size for mail UID 2777 doveadm(info at example.com): Error: Cached message size smaller than expected (18996 < 64624) doveadm(info at example.com): Error: Maildir filename has wrong S value, renamed the file from /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z to /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z Interestingly, the file mentioned in the error ( /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't exist. The uncompressed size of the file is 64624 bytes, the compressed size is 18996. My attempts to debug the issue myself didn't get very far - the function that appears to return the wrong value is in /src/lib-storage/index/maildir/maildir-mail.c in the 2.1.13 source; it's do_fix_size() and appears to not consider whether a file is compressed or not when reporting it's size for S= value purposes. Can anyone help? I'm happy to provide parts of dovecot -n output if required, but I'd rather not post configuration in a public forum if it's not required. -- Rob Redpath Systems Administrator Heart Internet Ltd E: rob.redpath at heartinternet.co.uk DDI: 0115 845 6643 www.heartinternet.co.uk<http://www.heartinternet.co.uk> ****************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. Heart Internet Ltd accepts no responsibility for information, errors or omissions in this email. ******************************************************************
Timo Sirainen
2013-Feb-05 02:06 UTC
[Dovecot] Broken physical size caching in Dovecot 2.1.10
On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote:> I'm encountering two, probably related, errors in my Dovecot 2.1.10 > install. The issue relates to compressed email stored in Maildir format > directories. > > In some cases, a mailbox will become inaccessible, and the following > will be logged from a doveadm fetch:- > > doveadm(info at example.com): Error: Corrupted index cache file > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken > physical size for mail UID 2777 > doveadm(info at example.com): Error: Cached message size smaller than > expected (18996 < 64624) > doveadm(info at example.com): Error: Maildir filename has wrong S value, > renamed the file from > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > to > /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z > > Interestingly, the file mentioned in the error ( > /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't > exist. > > The uncompressed size of the file is 64624 bytes, the compressed size is > 18996.Dovecot's zlib plugin requires that the maildir filenames have ,S=uncompressed size. Otherwise you get those errors. http://dovecot.org/tools/maildir-size-check.sh can be used to fix the filenames.
Steffen Kaiser
2013-Mar-06 16:59 UTC
[Dovecot] Bug in maildir-size-fix.pl (was Broken physical size caching in Dovecot 2.1.10)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 6 Mar 2013, Steffen Kaiser wrote: Sorry for the noise, you cannot upgrade V1 uidlist files to V3 ones on the fly with this script, hence, this match:> chomp $_; > - if (/^(\d+) ([^:]*)?:(.*)$/) { > + if (/^(\d+) ([^:]*:)?(.*)$/) { > my ($uid, $extra, $fname) = ($1, $2, $3);breaks IMAP UIDs afterwards, so the patch for maildir-size-fix.pl is nonsense. I wonder why it worked in first tests ... . - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUTd2Cl3r2wJMiz2NAQI69Qf+MpXpakEW02Yo2vAq+Z/aScpWTlurZBAD heBYWemoCeQ5liZYhPC/JbVlhhg6EH39WxvUtN8tMl3Cr7YZHIbZsguij3jR/+al +mYQ1BFpkSX0TUP8xTda82J2ShKEsEDMQLlaayXRZjod87LV6UcKlwRG4Gqr7Eho l382HgHb6XOiYe5bVyVXiwAvolHm1C9OmV50/cgLtrkWC+fKXbU0iP92HbLIobZI 2qkuL3yzLDdyGRJCFJhIlaGOyg/KS9hSZrckMX/giEVQlr2RYqpzc7cT0R38A6Bp QvbV8Lw4VuESLtRO+NtZQ+bV4mhuiwLKVNpb4XVp33ohKcSUBT5xXA==y99O -----END PGP SIGNATURE-----