dovecot - Oct 2010 - Zero-sized Maildir files and index corruption on over-quota

We are using Postfix as an MTA delivering via Dovecot's LDA (with sieve). We
also use Dovecot as a POP/IMAP server. Mail storage is Maildir on NFS, indexes
are stored locally. Quotas are FS quotas enforced by the NFS server. The Dovecot
version is 1.2.11.

Recently, for one user being over quota, some attempts to deliver mail bounced
as expected while some others resulted in zero-sized Maildir files. I can verify
zero-sized files in cur, while I cannot tell whether they were already
zero-sized in new. However, since nobody except Dovecot is accessing the mail
storage, it's either Dovecot's LDA or Dovecot's IMAP producing the
zero-sized files.

Logs show successful delivery (deliver: stored mail into mailbox
'INBOX') but ,,Corrupted index cache file
/var/db/dovecot/indexes/XXX/.INBOX/dovecot.index.cache: Corrupted virtual size
for uid=YYY: 0 != NNNN'' alerts on IMAP access.

It looks like all mails which correctly got bounced are >64KB while those
resulting in zero-sized files are <64KB (there is one >32KB). My first
guess would be that maybe for large files, write() fails, while for small ones,
close() fails or there is some buffering involved for large files.

Edgar Fu? wrote:
> We are using Postfix as an MTA delivering via Dovecot's LDA (with
sieve). We also use Dovecot as a POP/IMAP server. Mail storage is Maildir on
NFS, indexes are stored locally. Quotas are FS quotas enforced by the NFS
server. The Dovecot version is 1.2.11.
> 
> Recently, for one user being over quota, some attempts to deliver mail
bounced as expected while some others resulted in zero-sized Maildir files. I
can verify zero-sized files in cur, while I cannot tell whether they were
already zero-sized in new. However, since nobody except Dovecot is accessing the
mail storage, it's either Dovecot's LDA or Dovecot's IMAP producing
the zero-sized files.
> 
> Logs show successful delivery (deliver: stored mail into mailbox
'INBOX') but ,,Corrupted index cache file
/var/db/dovecot/indexes/XXX/.INBOX/dovecot.index.cache: Corrupted virtual size
for uid=YYY: 0 != NNNN'' alerts on IMAP access.
> 
> It looks like all mails which correctly got bounced are >64KB while
those resulting in zero-sized files are <64KB (there is one >32KB). My
first guess would be that maybe for large files, write() fails, while for small
ones, close() fails or there is some buffering involved for large files.
> 
Are your NFS Maildirs mounted on more than one server? Check the list 
archive for "nfs director". There has been a good bit of discussion 
about this recently. Cor Bosman explain the situation nicely here:
http://www.mail-archive.com/dovecot at dovecot.org/msg31713.html

-- 
-Eric 'shubes'

On Tue, 2010-10-05 at 15:29 +0200, Edgar Fu? wrote:
> It looks like all mails which correctly got bounced are >64KB while
> those resulting in zero-sized files are <64KB (there is one >32KB).
My
> first guess would be that maybe for large files, write() fails, while
> for small ones, close() fails or there is some buffering involved for
> large files.
Even if close() fails, the file is still in tmp/ and Dovecot aborts the
save. The only reason I can think of is that you've done async NFS mount
and the failed write() is noticed way too late by the kernel.

On Fri, 2010-10-15 at 18:03 +0200, Edgar Fu? wrote:
> > It would log an error and delete the tmp file.
> Excellent. Does this also apply if sieve is involved?
Yes. Sieve uses the same functions for saving mails as the rest of
Dovecot.