dovecot - Nov 2009 - problem with uidlist

In both 1.2.6 and 1.2.7  (probably also before that, but dont have logs) im
seeing quite a few of these:

Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2650:
1257768394._smtp.mxdrop238.41874,S=17304:2,Sad (uid 34270 -> 37166)

I never really looked at it because it doesnt seem to be causing many problems,
but i just noticed something that probably hints at the cause..

Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2650:
1257768394._smtp.mxdrop238.41874,S=17304:2,Sad (uid 34270 -> 37166)
Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2651:
1257769385._smtp.mxdrop155.53889,S=305996:2,Sad (uid 34271 -> 37167)
Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2652:
1257769984._smtp.mxdrop107.23815,S=320781:2,Sabd (uid 34272 -> 37168)
Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2670:
1257858287._smtp.mxdrop131.43563,S=26827:2,Sad (uid 34311 -> 37186)
Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2671: 1257858303._smtp.mxdrop238.88513,S=2143:2,Sad
(uid 34312 -> 37187)
Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2672: 1257858868._smtp.mxdrop217.21298,S=8548:2,Sad
(uid 34314 -> 37188)
Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 994: 1244800736._smtp.mxdrop237.59318,S=4573:2,Sad
(uid 32467 -> 35510)
Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 995:
1244807927._smtp.mxdrop215.53275,S=439887:2,Sad (uid 32468 -> 35511)
Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 996:
1244828590._smtp.mxdrop111.81795,S=3606:2,RSabd (uid 32469 -> 35512)

This customer seems to be using our IMAP service from 2 machines at the same
time. Possibly a webmail service and a desktop imap client. It may be that both
machines are trying to manipulate the uidlist at the same time, causing who
knows that kind of corruption. This could maybe explain some of the coredumps im
seeing in 1.2.7?

This is on Debian/Lenny, using NFS spool and NFS control/indexes. 

Timo, you asked in private mail a few days ago if we're sure the filenames
are forever unique. Im sure. I double checked it in the source code. The above
filenames are timestamp_hostname.pid, and that seems pretty unique right?

Regards,

Cor

On Tue, 2009-11-17 at 12:55 +0100, Cor Bosman wrote:
> Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx):
/var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist:
Duplicate file entry at line 2650:
1257768394._smtp.mxdrop238.41874,S=17304:2,Sad (uid 34270 -> 37166)
> 
> I never really looked at it because it doesnt seem to be causing many
problems, but i just noticed something that probably hints at the cause..
I suppose this is more or less the same problem as with the other two
NFS unhappy people in "dovecot-1.2.8 imap crash (with backtrace)"
thread, although you're only getting this error without the
assert-crashes or "expunged message reappeared" errors?

I should probably try again stress testing on NFS with imaptest. Maybe
these happen constantly when imaptest is run on two machines at the same
time?..

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20091229/f58ff885/attachment-0002.bin>

On 12/29/2009 06:40 PM, Timo Sirainen wrote
> I should probably try again stress testing on NFS with imaptest. Maybe
> these happen constantly when imaptest is run on two machines at the same
> time?..
>
>    
That sounds like a good test to me. Our environment (which seems to trip 
many of these issues up) is 10 dedicated IMAP/POP servers, 2 frontend 
*nix boxes running pine, and four webmail machines using imapproxy to 
connect to the IMAP servers... all using the same NFS backend. More than 
likely it's an multi-server access NFS issue.

-- 
===============================David Halik
System Administrator
OIT-CSS Rutgers University
dhalik at jla.rutgers.edu
================================