Dovecot UIDs and POP.

Plutocrat
Sat, 21 Sep 2019 04:38:39 -0000

Hi,

I recently performed a mail server migration for a client under fairly serious
time constraints (ahem, yes one of those jobs). I would normally use imapsync to
get all the mail copied to the new server, but under the circumstances, I had to
write a script to rsync each of the mailboxes over ssh, from one server to
another.

This all worked fine, except for the fact that if a client was using a POP email
client, when they connected to the new server, they re-downloaded all their
mail, creating duplicates of everything in their Outlook client. Of course they
weren't happy about this. IMAP email clients were OK. No duplicates, and
everything was fine.

So while I was migrating the mail, I did try for a while to understand the
format of the UID files, but failed to do so in the available time, so the
client just had to deal with duplicate emails. But now the smoke has cleared,
I'd like to understand the problem a little better, and I was hoping someone
on this forum could explain it to me, and the changes I'd need to make to
the files so that the POP client DIDN'T download the duplicate emails.

Here's what I understand. The list of dovecot files in an account looks like
this:
-rw-r-----   1 admin mail    0 Jul 11  2018 dovecot-acl-list
-rw-r-----   1 admin mail  14K Sep 20 08:36 dovecot.index
-rw-r-----   1 admin mail 216K Sep 20 17:24 dovecot.index.cache
-rw-r-----   1 admin mail  29K Sep 20 17:24 dovecot.index.log
-rw-r-----   1 admin mail  388 Dec 17  2017 dovecot.list.index
-rw-r-----   1 admin mail 1.6K Jul 11  2018 dovecot.list.index.log
-rw-r-----   1 admin mail   24 Jul 11  2018 dovecot.mailbox.log
-rw-r-----   1 admin mail   53 Sep 12 09:42 dovecot-quota
-rw-r-----   1 admin mail  52K Sep 20 17:24 dovecot-uidlist
-rw-r-----   1 admin mail    8 Jul 11  2018 dovecot-uidvalidity
-rw-r-----   1 admin mail    0 Jul  4  2016 dovecot-uidvalidity.5779bbeb

It seems that the dovecot-uidlist is an index of all the mail files in the
account. As far as I can gather, this should be the same on both servers after
I've rsynced them over ssh: The files and the list of them in
dovecot-uidlist should correspond.
So what is it that the POP client looks at and decides to re-download all the
emails?
And how could I have stopped it from doing that, and get it to realize that the
list of emails it holds internally is the same as the list on the server? What
would I need to change in the dovecot files?

Thanks for any insight into the process.

As a secondary question -- and perhaps I should put this in a separate message
-- I did notice that "doveadm sync" would apparently have helped me
with this, but I wasn't able to get that to work either. I believe it was
something to do with the fact that all the mailboxes were under the same linux
user account on the target server, and I couldn't figure out all the paths
and permissions in time. Would 'doveadm sync' have fixed all the UID and
duplicate POP email issues?

P.