dovecot - Oct 2009 - dovecot 1.0.15 upgrading to dovecot 1.1.x or 1.2.x, and POP3 UIDL issue

Hi,


   We're upgrading a cluster of servers from v1.0.15 to
v1.1.x or v1.2.x. It appears that the UIDL generation mechanism
has changed, and thus we'll be getting POP3 dups with users
that leave mail on server (which is a nasty practice, I know).

   From checking the sources, and performing a number of controlled
tests in various scenarios, here is the outcome:

upgrade from 1.0.15 to 1.1.x (latest), UIDLs change
(no change to the mail backend, that is, mbox)

However, the interesting part is that if for example
I read a mailbox for the first time with the 1.1.x version,
and then I migrate the mailbox to a different server, running
1.1.x or 1.2.x, with Maildir or mailbox the UIDL does *NOT* change.
We do not have X-UIDL headers in the mailboxes.

I did every possible permutation of these tests. It appears that
the way UIDLs are created in 1.0.x is different than what is used
in 1.1.x and 1.2.x. (I am not talking about the format, we're using
the same format in both, that is, UW compatible).

Since we're a largish site (university), and we have some >60k accounts,
even the smallest percentage of users who use POP3 and "leave mail on
server" is a large number of angry staff members landing on our 
helpdesk, which is the reason why we're not migrating.

Has anyone ever ported the UIDL generation algorithm from 1.0.x to 
1.1.x/1.2.x to maintain compatibility ?  Is there some other option
that I am missing ?


--Ariel

Ariel Biener wrote:
> 
> Hi,
I think I found the problem eventually:

In the dovecot.conf example file, the section explaining
the various UIDL formats is as follows:

   # If you want UIDL compatibility with other POP3 servers, use:
   #  UW's ipop3d         : %08Xv%08Xu
   #  Courier             : %f or %v-%u (both might be used simultaneosly)
   #  Cyrus (<= 2.1.3)    : %u
   #  Cyrus (>= 2.1.4)    : %v.%u
   #  Dovecot v0.99.x     : %v.%u
   #  tpop3d              : %Mf
   #
   # Note that Outlook 2003 seems to have problems with %v.%u format 
which was
   # Dovecot's default, so if you're building a new server it would be a
good
   # idea to change this. %08Xu%08Xv should be pretty fail-safe.
   #
   #pop3_uidl_format = %08Xu%08Xv


Notice that the UW format at the top is: %08Xv%08Xu , while at the 
bottom it is %08Xu%08Xv. So, in 1.0.15 we had it as the top would have 
it, while at the 1.1.x/1.2.x instances we just uncommented the bottom.

I must have looked at these files 1000 times.


--Ariel
> 
> 
>   We're upgrading a cluster of servers from v1.0.15 to
> v1.1.x or v1.2.x. It appears that the UIDL generation mechanism
> has changed, and thus we'll be getting POP3 dups with users
> that leave mail on server (which is a nasty practice, I know).
> 
>   From checking the sources, and performing a number of controlled
> tests in various scenarios, here is the outcome:
> 
> upgrade from 1.0.15 to 1.1.x (latest), UIDLs change
> (no change to the mail backend, that is, mbox)
> 
> However, the interesting part is that if for example
> I read a mailbox for the first time with the 1.1.x version,
> and then I migrate the mailbox to a different server, running
> 1.1.x or 1.2.x, with Maildir or mailbox the UIDL does *NOT* change.
> We do not have X-UIDL headers in the mailboxes.
> 
> I did every possible permutation of these tests. It appears that
> the way UIDLs are created in 1.0.x is different than what is used
> in 1.1.x and 1.2.x. (I am not talking about the format, we're using
> the same format in both, that is, UW compatible).
> 
> Since we're a largish site (university), and we have some >60k
accounts,
> even the smallest percentage of users who use POP3 and "leave mail on
> server" is a large number of angry staff members landing on our 
> helpdesk, which is the reason why we're not migrating.
> 
> Has anyone ever ported the UIDL generation algorithm from 1.0.x to 
> 1.1.x/1.2.x to maintain compatibility ?  Is there some other option
> that I am missing ?
> 
> 
> --Ariel
> 
> 
>