dovecot - Apr 2005 - Setting location for indexes

We have NFS-mounted user directories which cause significant problems with
stale file handles:

dovecot: IMAP(uid): fstat() failed with file
/home/uid/mail/.imap/INBOX/dovecot.index.log: Stale NFS file handle

(a) why can't it just close the file and re-open ?

(b) is there a way to specify with the 'default_mail_env' (or in some
other way)
the location of **only** the indexes (and let dovecot autodetect all else?).

I have already tried 'mmap_disable = yes', but that does not solve the
problem,
and if I set 'default_mail_env = INDEX=/var/spool/mail/INDEX/%u', I get
errors
like:

IMAP(uid): Failed to create storage with data: INDEX=/var/spool/mail/INDEX/uid

I don't want to specify mbox:~/mail/ or mbox:~/Mail/, because some users
have
one and some the other.

On Wed, Apr 27, 2005 at 09:47:25AM +0200, dovecot.7.signal11 at spamgourmet.com
wrote:
> We have NFS-mounted user directories which cause significant problems with
> stale file handles:
> dovecot: IMAP(uid): fstat() failed with file
/home/uid/mail/.imap/INBOX/dovecot.index.log: Stale NFS file handle
> (a) why can't it just close the file and re-open ?
NFS has issues with files that are open on one client, and get deleted on
another. UNIX systems require that files don't disappear while they are in
use, but NFS is stateless and it simply can't know when a file is in use.
The result is the 'stale NFS handle' kludge, which also prevents a
simple
close-and-open. Don't use index files on NFS disks.
> (b) is there a way to specify with the 'default_mail_env' (or in
some other way)
> the location of **only** the indexes (and let dovecot autodetect all
else?).
Haven't figured that one out myself either :)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!

dovecot.7.signal11 at spamgourmet.com wrote:
>We have NFS-mounted user directories which cause significant problems with
>stale file handles:
>
>dovecot: IMAP(uid): fstat() failed with file
/home/uid/mail/.imap/INBOX/dovecot.index.log: Stale NFS file handle
>
>(a) why can't it just close the file and re-open ?
>
The Stale NFS file handle error is a fatal error on the system side.  
Closing and re-opening isn't a guaranteed solution.

Actually, the real question, in my opinion, would be what is causing the 
system to get a stale NFS file handle?  IF you are going to store and 
access mail from NFS filesystems, the filesystem needs to be extremely 
stable and accessible..

-Rich

> > dovecot: IMAP(uid): fstat() failed with file
/home/uid/mail/.imap/INBOX/dovecot.index.log: Stale NFS file handle
> 
> > (a) why can't it just close the file and re-open ?
> 
> NFS has issues with files that are open on one client, and get deleted on
> another. UNIX systems require that files don't disappear while they are
in
> use, but NFS is stateless and it simply can't know when a file is in
use.
> The result is the 'stale NFS handle' kludge, which also prevents a
simple
> close-and-open. Don't use index files on NFS disks.
First, the file did not disappear - the NFS is flaky under high load, that's
all. Closing and re-opening the file would probably fix the issue, at least
after the traffic spike.

Second, these are files that are intended for dovecots internal use only. If the
NFS filehandle goes stale, it's anyway highly unlikely that someone else has
deleted them, and dovecot should know that - IMHO it should at least try to
recover.

Third, I'm perfectly willing to put the index files on a local disk, but I
need
dovecot to autodetect the rest of the mail environment, unless I want to define
it completely for each user individually, and apparently dovecot does not let
me do that(?).
>From the (few) answers I got so far, it seems that it is in fact impossible
to specify 'default_mail_env = INDEX=/var/spool/mail/INDEX/%u', so would
anybody know where in the sources the root directory for the indexes is set,
such that I can patch the code?

On Wed, 2005-04-27 at 10:56 +0200, Thomas Wouters wrote:
> On Wed, Apr 27, 2005 at 09:47:25AM +0200, dovecot.7.signal11 at
spamgourmet.com wrote:
> 
> > We have NFS-mounted user directories which cause significant problems
with
> > stale file handles:
> 
> > dovecot: IMAP(uid): fstat() failed with file
/home/uid/mail/.imap/INBOX/dovecot.index.log: Stale NFS file handle
> 
> > (a) why can't it just close the file and re-open ?
> 
> NFS has issues with files that are open on one client, and get deleted on
> another. UNIX systems require that files don't disappear while they are
in
> use, but NFS is stateless and it simply can't know when a file is in
use.
> The result is the 'stale NFS handle' kludge, which also prevents a
simple
> close-and-open. Don't use index files on NFS disks.
This is actually a TODO item. Dovecot should be able to handle all
ESTALE errors, but I just haven't implemented them. With
dovecot.index.log handling code the ESTALE is an expected error with NFS
because Dovecot rotates the log file continuously overwriting the
existing one.

Actually the new code in CVS nowadays rotates .log file to .log.2 file,
which should make this problem much more rare.

"First, the file did not disappear - the NFS is flaky under high load,
that's all. Closing and re-opening the file would probably fix the
issue, at least after the traffic spike."

Can this really happen with high NFS load, or is it just that Dovecot
had more time to rotate its log files? I'm not sure if I can reliably
fix the ESTALE problems if they happen randomly with the file actually
being lost.
> > (b) is there a way to specify with the 'default_mail_env' (or
in some other way)
> > the location of **only** the indexes (and let dovecot autodetect all
else?).
> 
> Haven't figured that one out myself either :)
Guess not. New configuration file code has split them to different
settings though, that'll fix it. Now if I only could get all that code
to work so I could commit it.. :)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050508/713b2455/attachment-0001.bin>