dovecot - Apr 2015 - Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads

-------- Original Message --------
Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries,
duplicated messages, excessive full re-downloads
From: Timo Sirainen <tss at iki.fi>
To: David Gessel <gessel at blackrosetech.com>
Date: Thu Apr 16 2015 21:31:30 GMT+0300 (Arabic Standard Time)
> On 16 Apr 2015, at 04:45, David Gessel <gessel at blackrosetech.com>
wrote:
>>
>> My Dovecot 2.2.16 server is interacting poorly with a variety of mail
clients.  The symptoms include disappearing messages, mismatched summaries,
duplicated messages, and excessive full re-downloads.
> 
> What were you running before Dovecot v2.2.16? An earlier Dovecot? A whole
different system?
> 
>> dovecot.log shows runs of this error:
>> Apr 15 09:02:37 imap(user at domain.com): Warning: Maildir
/mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old
uid=306286, file=1429102024.M826331P1156.host,S=1678,W=1724:2,S)
>>
>> or this error:
>> Apr 15 10:04:00 indexer-worker(user at domain.com): Warning: Maildir
/mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old
uid=306803, file=1428785463.M645546P27215.host,S=5389,W=5442:2,S)
> ..
>> The first entry in dovecot.log.0 is at Apr 15 10:00:12, and the last
entry at Apr 15 12:00:13; Dovecot is logging 4,490 expunged message reappeared
messages per hour.  Note the server has 4 accounts set up and very limited
traffic, so 4,490 per hour would seem like a lot of anything to be logged.
> 
> These really shouldn't be happening..
> 
One possibility is that there is a scrub/verify routine running that is checking
the actual size vs. reported size of messages, and perhaps that routine
doesn't know about ZFS compression:

zroot/ezjail                             used           117G      -
zroot/ezjail                             compressratio  1.25x     -
zroot/ezjail                             compression    lz4       local
zroot/ezjail                             logicalused    137G      -

and is seeing something anomalous and trying to "fix" that, triggering
a rescan.  A long shot and seems unlikely, but I have no other ideas.

I also note from the documentation on the wiki:

"Because Dovecot uses its own non-standard locking (dovecot-uidlist.lock
dotlock file), other MUAs accessing the maildir don't support it. This means
that if another MUA is updating messages' flags or expunging messages,
Dovecot might temporarily lose some message. After the next sync when it finds
it again, an error message may be written to log and the message will receive a
new UID. "

...which are the right symptoms, but seems unlikely to be the right cause.

And that:

"Maildir format is somewhat compatible with MH format. This is sometimes a
problem when people configure their procmail to deliver mails to Maildir/new.
This makes procmail create the messages in MH format, which basically means that
the file is called msg.inode_number. While this appears to work first, after
expunging messages from the maildir the inodes are freed and will be reused
later. This means that another file with the same name may come to the maildir,
which makes Dovecot think that an expunged file reappeared into the mailbox and
an error is logged. "

...which is exactly the error, but the 

And since I'm using Dovecot LMTP/LDA, it seems odd that errors would be
generated.

My main.cf config includes:

virtual_transport = dovecot
dovecot_destination_recipient_limit = 1
home_mailbox = Maildir/
mail_spool_directory = /mail
mailbox_command = /usr/local/libexec/dovecot/deliver
mailbox_transport = dovecot

On 18 Apr 2015, at 15:02, David Gessel <gessel at blackrosetech.com>
wrote:
> 
>>> dovecot.log shows runs of this error:
>>> Apr 15 09:02:37 imap(user at domain.com): Warning: Maildir
/mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old
uid=306286, file=1429102024.M826331P1156.host,S=1678,W=1724:2,S)
>>> 
>>> or this error:
>>> Apr 15 10:04:00 indexer-worker(user at domain.com): Warning:
Maildir /mail/domain.com/user//.Sent: Expunged message reappeared, giving a new
UID (old uid=306803, file=1428785463.M645546P27215.host,S=5389,W=5442:2,S)
> 
> One possibility is that there is a scrub/verify routine running that is
checking the actual size vs. reported size of messages, and perhaps that routine
doesn't know about ZFS compression:
> 
> zroot/ezjail                             used           117G      -
> zroot/ezjail                             compressratio  1.25x     -
> zroot/ezjail                             compression    lz4       local
> zroot/ezjail                             logicalused    137G      -
> 
> and is seeing something anomalous and trying to "fix" that,
triggering a rescan.  A long shot and seems unlikely, but I have no other ideas.
No. My best guess is that (your) ZFS+FreeBSD is simply not behaving the way
Dovecot wants filesystems to behave (readdir() sometimes skipping over files). I
don't have any other suggestion than trying a different filesystem or OS. I
haven't heard ZFS having these problems before though.

-------- Original Message --------
Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries,
duplicated messages, excessive full re-downloads
From: Timo Sirainen <tss at iki.fi>
To: David Gessel <gessel at blackrosetech.com>
Date: Sat Apr 18 2015 15:48:28 GMT+0300 (Arabic Standard Time)
> No. My best guess is that (your) ZFS+FreeBSD is simply not behaving the way
Dovecot wants filesystems to behave (readdir() sometimes skipping over files). I
don't have any other suggestion than trying a different filesystem or OS. I
haven't heard ZFS having these problems before though.

Or migrating to sdbox.  Absent any further suggestions, that will be my next
step.  My commitment to Maildir is far lower than my commitment to FreeBSD/ZFS. 
It is just a historical preference coming from a decade+ of courier IMAP.