dovecot - Feb 2008 - constant mailbox rebuilding with dovecot 1.0.10

I've recently upgraded to 1.0.10 in hopes that this problem would go
away but no luck.  I've got some users at my company with large
inboxes on maildir (say 22,000 messages).  Their mailboxes are hosted
on nfs and dovecot is configured to do dotlocking.  Still I see
messages like this continually in the logs (sometimes every hour or
so):

server dovecot: IMAP(user): dotlock /home/user/Maildir/dovecot.index.log.lock
was immediately recreated under us
server dovecot: IMAP(user): file_dotlock_create() failed with file
/home/user/Maildir/dovecot.index.log: No such file or directory
server dovecot: IMAP(user): Our dotlock file
/home/user/Maildir/dovecot-uidlist.lock was overridden (kept it 0 secs)
server dovecot: IMAP(user): Duplicate file in uidlist file
/home/user/Maildir/dovecot-uidlist: 1162416499.010493.mbox:2,Sa
server dovecot: IMAP(user): Maildir /home/user/Maildir sync: UID inserted in the
middle of mailbox (48573 > 1, file = 1162416499.010493.mbox:2,Sa)

or this:

server dovecot: IMAP(user): dotlock /home/user/Maildir/dovecot.index.log.lock
was immediately recreated under us
server dovecot: IMAP(user): file_dotlock_create() failed with file
/home/user/Maildir/dovecot.index.log: No such file or directory
server dovecot: IMAP(user): Duplicate file in uidlist file
/home/user/Maildir/dovecot-uidlist:
1167674195.83666_2.tandoori.schrodinger.com:2,Sklwv
server dovecot: IMAP(user): Maildir /home/user/Maildir sync: UID inserted in the
middle of mailbox (34969 > 1, file = 1167674195.83666_2.server.com:2,Sklwv)
server dovecot: IMAP(user): Corrupted transaction log file
/home/user/Maildir/dovecot.index.log.2: invalid indexid (1204046496 !=
1204055346)


it seems that every time this sort of thing happens the mail client
(thunderbird) sits and rebuilds it's mailbox info for as long as 20-30
minutes, during which time it is unusable.  I suspect this happens
with other users as well but since they have smaller inboxes they
don't notice the long delays.

Also this is not linked to version 1.0.10, I've been seeing on and off
with the previous version (1.0rc17).

Here's the 'dovecot -n' output.  I should add that the default is
for
users to use mbox but some users (like the above one) use maildir via
a separate dovecot.passwd file.  Server is FreeBSD 4.7.

# 1.0.10: /usr/local/etc/dovecot.conf
base_dir: /var/run/dovecot/
protocols: imaps
ssl_cert_file: /usr/local/openssl/certs/server.crt
ssl_key_file: /usr/local/openssl/private/server.key
login_dir: /var/run/dovecot//login
login_executable: /usr/local/libexec/dovecot/imap-login
login_greeting_capability: yes
verbose_proctitle: yes
first_valid_uid: 700
last_valid_uid: 9999
first_valid_gid: 100
last_valid_gid: 200
mail_extra_groups: mail
default_mail_env: mbox:%h/mail:INBOX=/var/mail/%u
mail_location: mbox:%h/mail:INBOX=/var/mail/%u
mmap_disable: yes
lock_method: dotlock
mbox_read_locks: dotlock
mbox_write_locks: dotlock
imap_client_workarounds: delay-newmail outlook-idle netscape-eoh
tb-extra-mailbox-sep
auth default:
  passdb:
    driver: pam
  passdb:
    driver: passwd-file
    args: /usr/local/etc/dovecot.passwd
  passdb:
    driver: passwd
  userdb:
    driver: passwd-file
    args: /usr/local/etc/dovecot.passwd
  userdb:
    driver: passwd


-- 
Phil Hollenback
hollenba at schrodinger.com
Schrodinger, Inc.

On Tue, 2008-02-26 at 15:56 -0500, Philip J. Hollenback
wrote:
> I've recently upgraded to 1.0.10 in hopes that this problem would go
> away but no luck.  I've got some users at my company with large
> inboxes on maildir (say 22,000 messages).  Their mailboxes are hosted
> on nfs and dovecot is configured to do dotlocking.
Are there multiple Dovecot servers accessing the mailboxes? Can the same
user have different connections to different servers at the same time?

This sounds like NFS caching problems that are described in
http://wiki.dovecot.org/NFS

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080226/0f1ef723/attachment-0002.bin>

On Tue, 2008-02-26 at 15:56 -0500, Philip J. Hollenback
wrote:
> server dovecot: IMAP(user): Our dotlock file
/home/user/Maildir/dovecot-uidlist.lock was overridden (kept it 0 secs)
..
> server dovecot: IMAP(user): dotlock
/home/user/Maildir/dovecot.index.log.lock was immediately recreated under us
These could also mean that your clocks aren't synchronized.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080226/aff7846d/attachment-0002.bin>

On 02/26/08, Timo Sirainen wrote:
> On Tue, 2008-02-26 at 15:56 -0500, Philip J. Hollenback wrote:
> > I've recently upgraded to 1.0.10 in hopes that this problem would
go
> > away but no luck.  I've got some users at my company with large
> > inboxes on maildir (say 22,000 messages).  Their mailboxes are hosted
> > on nfs and dovecot is configured to do dotlocking.
> 
> Are there multiple Dovecot servers accessing the mailboxes? Can the same
> user have different connections to different servers at the same time?
> 
> This sounds like NFS caching problems that are described in
> http://wiki.dovecot.org/NFS
The real problem seemed to be when users run multiple imap clients and
they stomp on each others' index files on the nfs filesystem.

I believe I have this fixed.  I should clarify that there is only
one dovecot server which in turn nfs-mounts the mailbox filesystem from
a netapp.  I realized after reading all the dovecot nfs documentation
that I could point the INDEX variable to a local filesystem on the
dovecot server.  I did that with the following dovecot.conf
default_mail_env:

default_mail_env = mbox:%h/mail:INBOX=/var/mail/%u:INDEX=/opt/dcache/%u

because the default at my company is mbox.  Then for each maildir user I
set a dovecot.passwd entry as follows:

user:XXXXX:501:100::/home/user:/bin/false::Maildir:/home/%u/Maildir:INDEX=/opt/dcache/%u

finally I set my dovecot.conf locking/nfs options as follows:

mmap_disable=yes
dotlock_use_excl = no
lock_method = fcntl
mbox_read_locks = dotlock
mbox_write_locks = dotlock

since I'm using dovecot 1.0 I believe this is the correct way to do
dotlocking on the mbox files and fcntl locking on the index files (which
remember are on the local filesystem under /opt/dcache).

Since I've made this change I've seen no more corrupted index files.

P.

-- 
Phil Hollenback
hollenba at schrodinger.com
Schrodinger, Inc.