Hi
I'm migrating some mailboxes from an old dovecot system where mailboxes are
stored maildir uncompressed. I enabled zlib on the new system and migrated
an account:
doveadm -o mail_fsync=never backup -R -u tonyg imapc:
when the account connects to the new system after a while i'm seeing the
following error and the mailbox is no longer functioning:
Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error:
read(/srv/vmail/532868/tonyg/mdbox/storage/m.1): FETCH BODY[] for mailbox
INBOX UID 61 got too little data: 4106341 vs 6054268
Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error:
unlink(/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache)
failed: No such file or directory (in mail-cache.c:28)
Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error: Corrupted index cache
file
/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache:
Broken virtual size for mail UID 61
Jan 15 13:58:06 S611 dovecot: imap(tonyg): FETCH failed in=181 out=4109653
Jan 15 13:58:06 S611 dovecot: auth-worker(6130): ldap(tonyg,144.132.1.90):
unknown user
Jan 15 13:58:06 S611 dovecot: imap-login: Login: user=<tonyg>,
method=PLAIN, rip=144.132.1.90, lip=203.134.11.127, mpid=6746, secured,
session=<7qqWklYpnGjLhgHK>
Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error:
read(/srv/vmail/532868/tonyg/mdbox/storage/m.1): FETCH BODY[] for mailbox
INBOX UID 61 got too little data: 4106341 vs 6054268
Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error:
unlink(/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache)
failed: No such file or directory (in mail-cache.c:28)
Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error: Corrupted index cache
file
/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache:
Broken virtual size for mail UID 61
Jan 15 13:58:07 S611 dovecot: imap(tonyg): FETCH failed in=181 out=4109653
If i migrate the account without zlib everthing works fine and the error
can't be reproduced.
I have tried force-resync and index commands to no avail.
Any ideas what's going on here?
doveconf -n:
[root at S611 532868]# doveconf -n
# 2.2.21 (5345f22): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.11 (6ddf62b87f60+)
# OS: Linux 3.10.0-327.4.4.el7.x86_64 x86_64 CentOS Linux release 7.2.1511
(Core)
auth_debug = yes
auth_verbose = yes
imapc_features = rfc822.size fetch-headers
imapc_host = removed
imapc_master_user = removed
imapc_password = # hidden, use -P to show it
imapc_user = %u
lmtp_proxy = yes
login_trusted_networks = 192.168.10.0/24
mail_debug = yes
mail_gid = vmail
mail_location = mdbox:%h/mdbox:ALT=/altstorage%h/mdbox
mail_plugins = quota zlib
mail_prefetch_count = 50
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date
index ihave duplicate mime foreverypart
mdbox_rotate_size = 10 M
namespace inbox {
inbox = yes
location mailbox "Deleted Messages" {
special_use = \Trash
}
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix }
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
passdb {
args = /etc/dovecot/dovecot-ldap-m2core.conf.ext
driver = ldap
}
plugin {
quota = dict:User quota::file:%h/dovecot-quota
sieve = file:~/sieve;active=~/.dovecot.sieve
zlib_save = gz
zlib_save_level = 6
}
pop3c_host = <removed>
pop3c_password = # hidden, use -P to show it
service lmtp {
inet_listener lmtp {
address = <removed>
port = 24
}
}
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
verbose_proctitle = yes
protocol imap {
mail_plugins = quota imap_quota imap_zlib
}
figured out my issue
I changed:
protocol imap {
mail_plugins = quota imap_quota imap_zlib
}
to this:
protocol imap {
mail_plugins = $mail_plugins imap_quota imap_zlib
}
No longer getting the error after migration with zlib enabled.. could there
be any kind of better warning or error handling around this? (if this is
the actual issue)
On Fri, Jan 15, 2016 at 2:25 PM Leon Kyneur <leon at dexterous.org> wrote:
> Hi
>
> I'm migrating some mailboxes from an old dovecot system where mailboxes
> are stored maildir uncompressed. I enabled zlib on the new system and
> migrated an account:
> doveadm -o mail_fsync=never backup -R -u tonyg imapc:
>
> when the account connects to the new system after a while i'm seeing
the
> following error and the mailbox is no longer functioning:
>
> Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error:
> read(/srv/vmail/532868/tonyg/mdbox/storage/m.1): FETCH BODY[] for mailbox
> INBOX UID 61 got too little data: 4106341 vs 6054268
> Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error:
>
unlink(/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache)
> failed: No such file or directory (in mail-cache.c:28)
> Jan 15 13:58:06 S611 dovecot: imap(tonyg): Error: Corrupted index cache
> file
>
/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache:
> Broken virtual size for mail UID 61
> Jan 15 13:58:06 S611 dovecot: imap(tonyg): FETCH failed in=181 out=4109653
> Jan 15 13:58:06 S611 dovecot: auth-worker(6130): ldap(tonyg,144.132.1.90):
> unknown user
> Jan 15 13:58:06 S611 dovecot: imap-login: Login: user=<tonyg>,
> method=PLAIN, rip=144.132.1.90, lip=203.134.11.127, mpid=6746, secured,
> session=<7qqWklYpnGjLhgHK>
> Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error:
> read(/srv/vmail/532868/tonyg/mdbox/storage/m.1): FETCH BODY[] for mailbox
> INBOX UID 61 got too little data: 4106341 vs 6054268
> Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error:
>
unlink(/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache)
> failed: No such file or directory (in mail-cache.c:28)
> Jan 15 13:58:07 S611 dovecot: imap(tonyg): Error: Corrupted index cache
> file
>
/srv/vmail/532868/tonyg/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache:
> Broken virtual size for mail UID 61
> Jan 15 13:58:07 S611 dovecot: imap(tonyg): FETCH failed in=181 out=4109653
>
> If i migrate the account without zlib everthing works fine and the error
> can't be reproduced.
>
> I have tried force-resync and index commands to no avail.
>
> Any ideas what's going on here?
>
> doveconf -n:
>
> [root at S611 532868]# doveconf -n
>
>
> # 2.2.21 (5345f22): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.11 (6ddf62b87f60+)
> # OS: Linux 3.10.0-327.4.4.el7.x86_64 x86_64 CentOS Linux release 7.2.1511
> (Core)
> auth_debug = yes
> auth_verbose = yes
> imapc_features = rfc822.size fetch-headers
> imapc_host = removed
> imapc_master_user = removed
> imapc_password = # hidden, use -P to show it
> imapc_user = %u
> lmtp_proxy = yes
> login_trusted_networks = 192.168.10.0/24
> mail_debug = yes
> mail_gid = vmail
> mail_location = mdbox:%h/mdbox:ALT=/altstorage%h/mdbox
> mail_plugins = quota zlib
> mail_prefetch_count = 50
> mail_uid = vmail
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
> copy include variables body enotify environment mailbox date
> index ihave duplicate mime foreverypart
> mdbox_rotate_size = 10 M
> namespace inbox {
> inbox = yes
> location > mailbox "Deleted Messages" {
> special_use = \Trash
> }
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix > }
> passdb {
> args = /etc/dovecot/dovecot-ldap.conf.ext
> driver = ldap
> }
> passdb {
> args = /etc/dovecot/dovecot-ldap-m2core.conf.ext
> driver = ldap
> }
> plugin {
> quota = dict:User quota::file:%h/dovecot-quota
> sieve = file:~/sieve;active=~/.dovecot.sieve
> zlib_save = gz
> zlib_save_level = 6
> }
> pop3c_host = <removed>
> pop3c_password = # hidden, use -P to show it
> service lmtp {
> inet_listener lmtp {
> address = <removed>
> port = 24
> }
> }
> ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
> ssl_key = </etc/pki/dovecot/private/dovecot.pem
> verbose_proctitle = yes
> protocol imap {
> mail_plugins = quota imap_quota imap_zlib
> }
>
>
>
>
On 1/14/16 8:51 PM, Leon Kyneur wrote:> figured out my issue > > I changed: > protocol imap { > mail_plugins = quota imap_quota imap_zlib > } > > to this: > protocol imap { > mail_plugins = $mail_plugins imap_quota imap_zlib > } >Right. This is because it needs the "zlib" plugin enabled, and in your original configuration, the zlib plugin was enabled systemwide:> mail_plugins = quota zlib... but these lines completely replaced the "mail_plugins" setting for imap, and didn't include zlib:> protocol imap { > mail_plugins = quota imap_quota imap_zlib > }Your fixed configuration *adds* additional plugins to the systemwide mail_plugins setting rather than replacing them, so it correctly uses zlib for IMAP. It would also work if you used: protocol imap { mail_plugins = quota zlib imap_quota imap_zlib }> No longer getting the error after migration with zlib enabled.. couldthere> be any kind of better warning or error handling around this? (if this is > the actual issue)Well, the documentation at <http://wiki.dovecot.org/Plugins/Zlib> does suggest doing it with "mail_plugins = $mail_plugins zlib", and it gives those errors in the log if you forget it. There's an argument to be made for "Dovecot should *always* try to uncompress 'obviously' compressed files", that apparently opens up security issues, exposing Dovecot to bugs in the system's compression libraries even if the admin intentionally doesn't use compression. -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/