dovecot - Dec 2005 - Dovecot strange behaviors on Solaris

Folks -

    I've been using Dovecot for some time now, but have only recently
upgraded
to 1.0alpha4.  I'm struggling to get my configuration right and I'm
seeing
some truly bizarre issues.

My Setup: Solaris 2.8 / Thunderbird / Procmail / Dovecot 1.0alpha4

My mail setup is only for me, so Dovecot runs at an odd port and under my 
userID.  All my mail is in my home directory, which is on NFS (NetApp).  The 
main mail receiver is a different machine which mounts my home directory.

Typical entry from .procmailrc:

:0
* ^(From|To|CC):.*@.*\<si2.org
Maildir/.SI2/

Highlights from dovecot.conf:

#default_mail_env = maildir:%h/.mail/Maildir:INDEX=/var/tmp/mario/dovecot
default_mail_env = maildir:%h/.mail/Maildir:INDEX=MEMORY

mmap_disable = yes

#lock_method = flock
lock_method = fcntl
#lock_method = dotlock

protocol imap {
   listen = *:2143
}

auth_worker_max_count = 0

auth default {
   mechanisms = plain
   userdb passwd {
   }
}

When I run as above (INDEX=MEMORY), I have access to all my mail, but I'm 
seeing literally thousands of .nfs files created all throughout my 
.mail/Maildir directories, and Thunderbird periodically locks up.  As the 
filesystem fills with useless .nfs files (I deleted 10000 from ONE mailbox 
directory this morning), everything gets really, really slow, and I suspect 
this contributes to Thunder bird's problem.

But, if I run with the INDEX on a local disk, many mailboxes will not load 
(the most critical ones, of course...), but also some do.  My inbox will list 
messages, but cannot display any.  Others won't even list messages.

I'm at a total loss how to proceed, here.  I've tried everything I can
think
of to debug what's going on, but have gotten nowhere (well, I did discover
the
whole lock_method=fcntl trick).

I've tried enabling rawlog, but am just not familiar enough with IMAP to get
help from those logs.

Here's what's happening in a typical mailbox:

% pwd
/home/mario/.mail/Maildir/.SI2
% ls -al
total 544
drwx--S---   5 rvvk40   lowv       425984 Dec  2 14:42 ./
drwx--S---  64 rvvk40   lowv        16384 Dec  2 14:43 ../
-rw-------   1 rvvk40   lowv           15 Nov 16 07:16 .customflags
-rw-------   1 rvvk40   lowv           24 Dec  2 14:38 .nfs00986
-rw-------   1 rvvk40   lowv           24 Dec  2 14:40 .nfs08986
-rw-------   1 rvvk40   lowv          312 Dec  2 14:11 .nfs16686
-rw-------   1 rvvk40   lowv           24 Dec  2 14:32 .nfs1A886

-- 40+ deleted here for brevity --

-rw-------   1 rvvk40   lowv           24 Dec  2 14:39 .nfsE3986
-rw-------   1 rvvk40   lowv          312 Dec  2 14:38 .nfsEF886
-rw-------   1 rvvk40   lowv           24 Dec  2 14:26 .nfsF3886
-rw-------   1 rvvk40   lowv          312 Dec  2 14:32 .nfsF9886
-rw-------   1 rvvk40   lowv          312 Dec  2 14:17 .nfsFF686
drwx--S---   2 rvvk40   lowv         4096 Dec  2 07:58 cur/
drwx--S---   2 rvvk40   lowv         4096 Dec  2 07:58 new/
drwx--S---   2 rvvk40   lowv         4096 Dec  2 07:58 tmp/
% pwd
/home/mario/.mail/Maildir/.SI2

				Help!

					Mario

On Fri, Dec 02, 2005 at 03:04:17PM -0700, Mario Nigrovic-rvvk40 wrote:
> Typical entry from .procmailrc:
> :0
> * ^(From|To|CC):.*@.*\<si2.org
> Maildir/.SI2/
The :0 means procmail won't be using locking for this mailbox (which is
perfectly fine, for Maildir), however:
> #lock_method = flock
> lock_method = fcntl
Here you tell dovecot to use fcntl for locking. fcntl over NFS might work
(depending on the client and server, and IIRC NetApp at least tries to do it
right) but it has some fundamental flaws (NFS is stateless, locking
obviously is not) so I would highly recommend you switch to dotlock,
instead. In any case, it's the most reliable lock method over NFS. Since
procmail won't be locking, you're only guarding against multiple
instances
of dovecot, which should all be using dotlock.

(Procmail can also use dotlocking if you want, but it uses a compile-time
test to see which lock method it should use.)
> #lock_method = dotlock
> 
> protocol imap {
>   listen = *:2143
> }
> When I run as above (INDEX=MEMORY), I have access to all my mail, but
I'm
> seeing literally thousands of .nfs files created all throughout my 
> .mail/Maildir directories, and Thunderbird periodically locks up.  As the 
> filesystem fills with useless .nfs files (I deleted 10000 from ONE mailbox 
> directory this morning), everything gets really, really slow, and I suspect
> this contributes to Thunder bird's problem.
.nfs* files are a hack used to deal with state (open files, quite likely
your lockfiles) over the stateless NFS protocol.
> But, if I run with the INDEX on a local disk, many mailboxes will not load 
> (the most critical ones, of course...), but also some do.  My inbox will 
> list messages, but cannot display any.  Others won't even list
messages.
I'm not sure why a local index wouldn't work. It worked fine for us for
a
while, until we had too many users to be able to deal with the occasional
index corruption. You may also want to place the control files on a local
disk, by the way, since only dovecot uses them. Unless you have multiple
servers that run dovecot, of course. Using local controlfiles may also
relieve your locking woes, without switching locking mechanism (although
dotlock also works fine for local filesystems.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!