dovecot - Jan 2014 - dsync mbox to maildir migration does not delete, deleted e-mails

Hello,

I am planning to migrate all users from mbox to maildir.

I am trying to do it with minimum downtime. (~100GB data)

All users are currently using POP3.

Dovecot version is 2.2.10 (latest).
Command used is:
dsync -u username mirror maildir:~/Maildir


Process I plan is: (omitting steps related to sendmail / procmail)


1) keep dovecot running (with mail_location as mbox)
2) dsync for all users (this may take 3-4 hours or more)

3) (downtime starts) stop dovecot and sendmail (to stop new e-mails)

4) block pop3, imap ports on firewall (so users can not connect)
5) start dovecot (still with mbox)

6) dsync again to sync e-mails arrived between step 2 and 3

7) dsync again (just to make sure!)
8) (downtime ends) restart dovecot (with mail_location as maildir)


Now, here is my problem.

Lets say there is user joe, who has 50 NEW e-mails in mbox (INBOX).


Step 2 perfectly syncs his 50 e-mails to 'new' folder of maildir.


Now in the mean time, before step 3, he connected via POP3 and
downloaded and deleted 50 e-mails.

Now when we reach step 6 (re-sync), what I expected was dsync will
detect that 50 e-mails are deleted and it will delete 50 e-mails from
'new' directory of maildir.


But that is not happening. 50 e-mails are still there.

I fear that these e-mails will be re-downloaded on his Outlook once
I switch dovecot to maildir.

This will happen for each and every user which will cause huge mess.


So how to tell, dsync to delete non-existent e-mails which are no more
there in mbox (INBOX)?

Thanks in advance,

Regards,

A M

I think I have found solution.

CONVERTING section of dsync man page suggests to use 'mirror'
command.


But I think, right command is 'backup' and not 'mirror'.

With 'backup' as command, I found that it copies only new e-mails and
also deletes DELETED messages (which is exactly what I wanted)

'mirror' is for 2 way syncing but for CONVERTING, in most cases,
one needs 1 way syncing (backup)


Possibly the dsync man page also needs this change.


Hope it helps others.

Regards,

A M



________________________________
 From: A M <ammdispose-dovecot at yahoo.com>
To: "dovecot at dovecot.org" <dovecot at dovecot.org> 
Sent: Wednesday, 15 January 2014 10:38 AM
Subject: [Dovecot] dsync mbox to maildir migration does not delete,	deleted
e-mails
 

Hello,

I am planning to migrate all users from mbox to maildir.

I am trying to do it with minimum downtime. (~100GB data)

All users are currently using POP3.

Dovecot version is 2.2.10 (latest).
Command used is:
dsync -u username mirror maildir:~/Maildir


Process I plan is: (omitting steps related to sendmail / procmail)


1) keep dovecot running (with mail_location as mbox)
2) dsync for all users (this may take 3-4 hours or more)

3) (downtime starts) stop dovecot and sendmail (to stop new e-mails)

4) block pop3, imap ports on firewall (so users can not connect)
5) start dovecot (still with mbox)

6) dsync again to sync e-mails arrived between step 2 and 3

7) dsync again (just to make sure!)
8) (downtime ends) restart dovecot (with mail_location as maildir)


Now, here is my problem.

Lets say there is user joe, who has 50 NEW e-mails in mbox (INBOX).


Step 2 perfectly syncs his 50 e-mails to 'new' folder of maildir.


Now in the mean time, before step 3, he connected via POP3 and
downloaded and deleted 50 e-mails.

Now when we reach step 6 (re-sync), what I expected was dsync will
detect that 50 e-mails are deleted and it will delete 50 e-mails from
'new' directory of maildir.


But that is not happening. 50 e-mails are still there.

I fear that these e-mails will be re-downloaded on his Outlook once
I switch dovecot to maildir.

This will happen for each and every user which will cause huge mess.


So how to tell, dsync to delete non-existent e-mails which are no more
there in mbox (INBOX)?

Thanks in advance,

Regards,

A M