dovecot - Jul 2009 - Clients and dovecot-uidlist

Hello,

I'm trying to figure out what exactly (and why) are the consequences of a
lost
or removal of the dovecot-uidlist file on an IMAP client (Thunderbird for
instance).

I took for granted that it was that the client would download all messages
again since it might be confused by some UID changes.  But I don't really
see
why (on the IMAP protocol level) and I don't know how to reproduce it.

My understanding is that 

  . dovecot-uidlist is not recreated until some client selects the mailbox

  . the client does something like :

     x UID fetch 1:* (FLAGS)

     to begin with anyway.

So I guess the problem does not occur when no new messages arrive on the mailbox
since the same UIDs would be assigned to the same messages ?

But why would that change ? In which situation ?

But can you show me some IMAP protocol command exchange that would illustrate 

  . when such a problem can occur
  . how the client would detect that there has been some UID corruption
  . how it would fix it 

Thanks

--
Thomas Hummel

On Tue, Jul 28, 2009 at 07:30:27PM +0200, Thomas Hummel wrote:
> I took for granted that it was that the client would download all messages
> again since it might be confused by some UID changes. 
Is it correct ? Timo ?
>  But I don't really see
> why (on the IMAP protocol level) and I don't know how to reproduce it.
-- 
Thomas Hummel 	    | Institut Pasteur
<hummel at pasteur.fr> | P?le informatique - syst?mes et r?seau

On Tue, 2009-07-28 at 19:30 +0200, Thomas Hummel wrote:
> Hello,
> 
> I'm trying to figure out what exactly (and why) are the consequences of
a lost
> or removal of the dovecot-uidlist file on an IMAP client (Thunderbird for
> instance).
Just dovecot-uidlist, or also dovecot.index*? If you delete both,
UIDVALIDITY is changed. If you delete only dovecot-uidlist, it'll
probably preserve UIDVALIDITY and just give new UIDs to messages
(because next_uid is also stored in index file).
> I took for granted that it was that the client would download all messages
> again since it might be confused by some UID changes.  But I don't
really see
> why (on the IMAP protocol level) and I don't know how to reproduce it.
> 
> My understanding is that 
> 
>   . dovecot-uidlist is not recreated until some client selects the mailbox
> 
>   . the client does something like :
> 
>      x UID fetch 1:* (FLAGS)
> 
>      to begin with anyway.
> 
> So I guess the problem does not occur when no new messages arrive on the
mailbox
> since the same UIDs would be assigned to the same messages ?
Well, Dovecot reads the uidlist file lazily. If the index files still
exist and you don't try to open any mail or there are no new mails, then
uidlist's existence isn't checked and UIDs won't change.
> But can you show me some IMAP protocol command exchange that would
illustrate
> 
>   . when such a problem can occur
>   . how the client would detect that there has been some UID corruption
>   . how it would fix it 
Either the client notices that UIDVALIDITY changes on SELECT:

* OK [UIDVALIDITY 1248669921] UIDs valid

Or the UIDs change.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090730/3f9244d0/attachment-0002.bin>

On Thu, Jul 30, 2009 at 01:21:40PM -0400, Timo Sirainen wrote:
> Just dovecot-uidlist, or also dovecot.index*? 
Just dovecot-uidlist.
> If you delete both, UIDVALIDITY is changed. If you delete only
> dovecot-uidlist, it'll probably preserve UIDVALIDITY and just give new
UIDs
> to messages (because next_uid is also stored in index file).
I don't get that : do indexes store UIDVALIDITY ? nextUID isn't enough
to
guaranty mailbox instance uniqueness, right ?

And, same as in my previous answer : why would they get new UIDs ? As they will
be "sorted" (maybe because of INTERNAL DATE) in the same order ? When
I erase dovecot-uidlist manually, it gets
recreated with the same UIDs (but I didn't delete indexes though)...
> Well, Dovecot reads the uidlist file lazily. If the index files still
> exist and you don't try to open any mail or there are no new mails,
then
> uidlist's existence isn't checked and UIDs won't change.
Same as above : I cannot get them to change in a test situation where I delete
dovecot-uidlist, send a new mail and check mail again.

Thanks

-- 
Thomas Hummel 	    | Institut Pasteur
<hummel at pasteur.fr> | P?le informatique - syst?mes et r?seau