dovecot - Jul 2011 - Converting from qpopper mbox to dovecot maildir

We have an older mail server using qpopper and the mbox format which we
need to update to dovecot and maildir format.  I have read through the docs
on migrating from mbox to maildir, as well as a few nuggets on how to
migrate from qpopper to dovecot, and I was wondering if I could get some
suggestions on best practices for this specific migration.  Would the built
in dovecot conversion plugin be a viable method to migrate users?  We will
be migrating to dovecot 2.0.12 from qpopper 4.0.5.

Thanks.

On Tue, 5 Jul 2011, list at airstreamcomm.net wrote:
> We have an older mail server using qpopper and the mbox format which we
> need to update to dovecot and maildir format.  I have read through the docs
> on migrating from mbox to maildir, as well as a few nuggets on how to
> migrate from qpopper to dovecot, and I was wondering if I could get some
> suggestions on best practices for this specific migration.  Would the built
> in dovecot conversion plugin be a viable method to migrate users?  We will
> be migrating to dovecot 2.0.12 from qpopper 4.0.5.
I didn't do anything special other than to use

 	pop3_reuse_xuidl = yes

so that clients don't re-download all their messages.

Joseph Tam <jtam.home at gmail.com>

Dimos Alevizos <dalevizo at otenet.gr> wrote:
> We recently completed the migration in our company from qpopper/mbox to
> dovecot/mbox using only pop3_reuse_xuidl = yes and while everything went
much
> easier than we feared we came across an interesting problem that you might
want
> to pay attention to.
> 
> We use dovecot to provide both pop3 and imap (directly and via webmail)
access
> to our client's mailboxes and we noticed that when a client moves a
mail to a
> different folder and then back into INBOX (say he deleted it by mistake and
> moves it back), dovecot inserts the same X-UIDL into it's cache, thus
creating
> duplicate entries.
> 
> When a pop3 client sees this (at least outlook in our case) it think's
that
> there's something wrong with the server's X-UIDL handling and
re-downloads the
> duplicate messages. EVERY time the user checks his mails.
I haven't heard any user complaint about this, but
perhaps nobody tried this before.

I recreated this situation (copy old qpopper message INBOX -> tmpbox ->
INBOX) and produced two messages with identical X-UIDL headers, but
according to the Wikipedia entry for POP3:

 	Comparison with IMAP

 	Clients that leave mail on servers generally use the UIDL command
 	to get the current association of message-numbers to message
 	identified by its unique identifier. The unique identifier
 	is arbitrary, and might be repeated if the mailbox contains
 	identical messages.

This is echoed in RFC1939 (page 11-12)

 	While it is generally preferable for server implementations
 	to store arbitrarily assigned unique-ids in the maildrop,
 	this specification is intended to permit unique-ids to be
 	calculated as a hash of the message.  Clients should be able
 	to handle a situation where two identical copies of a
 	message in a maildrop have the same unique-id.

So POP3 UIDLs are not guaranteed to be unique, especially if the messages
are the same.  Perhaps some mail readers are making unwarranted assumptions
about the uniqueness of UIDLs:

 
http://www.eggheadcafe.com/software/aspnet/33485042/duplicate-messageiduidlleave-on-serverrepeated-downloads-of-em.aspx
 	https://www.ritlabs.com/bt/view.php?id=3599

Joseph Tam <jtam.home at gmail.com>