I'm not sure what you are attempting to do here. It looks like you just ran a doveadm backup and the process completed for userx with a warning that the remote system (your nfs mount) lost a particular mailbox (possibly your indexes changed or a mailbox was deleted). From the logs you pasted it appears the process completed normally. I personally do not use dovecot's backup or replication processes. If I needed to I would use its replication process to sync active data between multiple systems, but I have no need for this as of yet. Personally I chose to create offsite backups using rsync a long time ago, as rebuilding a mailbox (reindexing) is very simple. Try running??????????? doveadm mailbox status -u userx guid '*'??? as the mailbox administrator and see if you can find that GUID, 7e05c335174bf1608f0a02004eac7fb4. Also, see if the backup you've written to nfs has the GUID. On 2/27/22 23:33, Stephane Magnier wrote:> > Well no ..I thought that dsync was for synchro " realtime for 2 > different places ? > > Having no 2 machines in parallel ( Just a single machine ) , I thought > that a backup at regular interval? would be enough ? > > So, a simple backup should be done by dsync finally ? > > Do you recommend finally NOT to do a backup ( Doveadm backup ) but a > replication process ? ( https://wiki.dovecot.org/Replication ) ? > > On 2/28/22 06:24, Ben Burk wrote: >> >> Did you try running dsync? >> >> On 2/27/22 23:15, Stephane Magnier wrote: >>> >>> HI, >>> >>> Any idea ? Any clue ? >>> >>> On 2/25/22 21:50, Stephane Magnier wrote: >>>> >>>> Hi >>>> >>>> I've recently tried to use the Dovecadm backup to backup the >>>> emails..? with the following syntax >>>> >>>> doveadm -Dv backup -u userx maildir:/mnt/nfs-backup/userx >>>> >>>> Sounds to be OK with few emails... Some of them got a lot of emails >>>> and one f them got an error and stop ! >>>> >>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>> type=expunge GUID=1725fa475d774ee19cb98dfb6737b4f1 UID=24891 >>>> hdr_hash= result=GUIDs match >>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>> type=expunge GUID=916ed110b4b1522868be6194f1ae36ff UID=24892 >>>> hdr_hash= result=GUIDs match >>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>> type=expunge GUID=eb8d75c530a7b02fc26b494d9006c91b UID=24893 >>>> hdr_hash= result=GUIDs match >>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>> type=expunge GUID=aee9155875c34861fd6500f1f2f51a26 UID=24894 >>>> hdr_hash= result=GUIDs match >>>> dsync(userx): Debug: brain S: Import Trash: Last common >>>> UID=24894. Delayed expunges>>>> dsync(userx): Debug: brain S: Import Trash: Saved UIDs: >>>> dsync(userx): Debug: brain S: Import Trash: Finish update: >>>> min_next_uid=24895 min_first_recent_uid=24895 >>>> min_highest_modseq=35344 min_highest_pvt_modseq=0 >>>> dsync(userx): Debug: >>>> /mnt/nfs-backup/userx/.Trash/dovecot.index.cache: Compressed, >>>> file_seq changed 1645803588 -> 1645803589, size=32, max_uid=24894 >>>> dsync(userx): Warning: Mailbox changes caused a desync. You may >>>> want to run dsync again: Remote lost mailbox GUID >>>> 7e05c335174bf1608f0a02004eac7fb4 (maybe it was just deleted?) >>>> dsync(userx): Debug: auth-master: conn >>>> unix:/run/dovecot/auth-userdb: Disconnected: Connection closed >>>> (fd=10) >>>> >>>> >>>> I empty the trash... exactly the same problem... >>>> >>>> Any idea why this ?? >>>> >>>> Thanks and regards, >>>>-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20220228/042057db/attachment.htm>
I've seen in a previous post, that the fact to do an RSYNC might break the? index.. So, I've heard that this is not recommended. that's the reason why I decided to find a way to do a "clean" backup and be able to come back online if needed. So, do you use an RSYNC and in case of restoring the mailbox, do you do a? simple *doveadm* *index* *-u* *USERx* *INBOX *And that's it ? works fine ?**2) I will try the Backup , or Sync.. locally.. Effectively, I don't know where the problem comes from..I have effectively an NFS Mount for the mailboxes and a VM for the Email server and that could be another another point of failure :-*( *Thanks for sharing.. I am a bit in a rush... I realized that my backup maybe not correct.. and I prefer not to discover it while running into trouble..** On 2/28/22 19:03, Ben Burk wrote:> > I'm not sure what you are attempting to do here. It looks like you > just ran a doveadm backup and the process completed for userx with a > warning that the remote system (your nfs mount) lost a particular > mailbox (possibly your indexes changed or a mailbox was deleted). From > the logs you pasted it appears the process completed normally. > > I personally do not use dovecot's backup or replication processes. If > I needed to I would use its replication process to sync active data > between multiple systems, but I have no need for this as of yet. > Personally I chose to create offsite backups using rsync a long time > ago, as rebuilding a mailbox (reindexing) is very simple. > > Try running??????????? doveadm mailbox status -u userx guid '*'??? as > the mailbox administrator and see if you can find that GUID, > 7e05c335174bf1608f0a02004eac7fb4. Also, see if the backup you've > written to nfs has the GUID. > > > On 2/27/22 23:33, Stephane Magnier wrote: >> >> Well no ..I thought that dsync was for synchro " realtime for 2 >> different places ? >> >> Having no 2 machines in parallel ( Just a single machine ) , I >> thought that a backup at regular interval? would be enough ? >> >> So, a simple backup should be done by dsync finally ? >> >> Do you recommend finally NOT to do a backup ( Doveadm backup ) but a >> replication process ? ( https://wiki.dovecot.org/Replication ) ? >> >> On 2/28/22 06:24, Ben Burk wrote: >>> >>> Did you try running dsync? >>> >>> On 2/27/22 23:15, Stephane Magnier wrote: >>>> >>>> HI, >>>> >>>> Any idea ? Any clue ? >>>> >>>> On 2/25/22 21:50, Stephane Magnier wrote: >>>>> >>>>> Hi >>>>> >>>>> I've recently tried to use the Dovecadm backup to backup the >>>>> emails..? with the following syntax >>>>> >>>>> doveadm -Dv backup -u userx maildir:/mnt/nfs-backup/userx >>>>> >>>>> Sounds to be OK with few emails... Some of them got a lot of >>>>> emails and one f them got an error and stop ! >>>>> >>>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>>> type=expunge GUID=1725fa475d774ee19cb98dfb6737b4f1 UID=24891 >>>>> hdr_hash= result=GUIDs match >>>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>>> type=expunge GUID=916ed110b4b1522868be6194f1ae36ff UID=24892 >>>>> hdr_hash= result=GUIDs match >>>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>>> type=expunge GUID=eb8d75c530a7b02fc26b494d9006c91b UID=24893 >>>>> hdr_hash= result=GUIDs match >>>>> dsync(userx): Debug: brain S: Import Trash: Import change >>>>> type=expunge GUID=aee9155875c34861fd6500f1f2f51a26 UID=24894 >>>>> hdr_hash= result=GUIDs match >>>>> dsync(userx): Debug: brain S: Import Trash: Last common >>>>> UID=24894. Delayed expunges>>>>> dsync(userx): Debug: brain S: Import Trash: Saved UIDs: >>>>> dsync(userx): Debug: brain S: Import Trash: Finish update: >>>>> min_next_uid=24895 min_first_recent_uid=24895 >>>>> min_highest_modseq=35344 min_highest_pvt_modseq=0 >>>>> dsync(userx): Debug: >>>>> /mnt/nfs-backup/userx/.Trash/dovecot.index.cache: Compressed, >>>>> file_seq changed 1645803588 -> 1645803589, size=32, max_uid=24894 >>>>> dsync(userx): Warning: Mailbox changes caused a desync. You >>>>> may want to run dsync again: Remote lost mailbox GUID >>>>> 7e05c335174bf1608f0a02004eac7fb4 (maybe it was just deleted?) >>>>> dsync(userx): Debug: auth-master: conn >>>>> unix:/run/dovecot/auth-userdb: Disconnected: Connection closed >>>>> (fd=10) >>>>> >>>>> >>>>> I empty the trash... exactly the same problem... >>>>> >>>>> Any idea why this ?? >>>>> >>>>> Thanks and regards, >>>>>-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20220228/4bb45c25/attachment.htm>
Hi Stephen, Does it work if you delete the Maildir/ on the destination first and then initiate the backup? I ran into this problem recently - see my post on 21 Feb.? Looks very similar if not the same root cause. On 1/03/2022 6:46 am, Stephane Magnier wrote:> > I've seen in a previous post, that the fact to do an RSYNC might break > the? index.. So, I've heard that this is not recommended. that's the > reason why I decided to find a way to do a "clean" backup and be able > to come back online if needed. > > So, do you use an RSYNC and in case of restoring the mailbox, do you > do a? simple > > *doveadm* *index* *-u* *USERx* *INBOX *-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20220301/dae560ee/attachment.htm>
>>>>> "Stephane" == Stephane Magnier <steph.mag220 at netcourrier.com> writes:Sorry, I deleted your most recent email post before I could reply. But why don't you just do 'imapsync' instead from your production dovecot box to some other backup system? Otherwise I'd probably work to setup dovecot's own replication but only have it go one way. For example, I've got a VPS out in the cloud for my email, and I should probably back it up to my home system using replication, but it would be strictly primary->secondary. I wouldn't be trying to run two primaries replicating between each other. Imapsync would be an improvement over rsync because it works within dovecot, so you'd get a more consistent view, but maybe not quite as upto date. But how important is your email if you worry about losing 20 minutes worth of it? If it's that critical, then you should be investing in a more robust setup. John Stephane> I've seen in a previous post, that the fact to do an RSYNC Stephane> might break the? index.. So, I've heard that this is not Stephane> recommended. that's the reason why I decided to find a way Stephane> to do a "clean" backup and be able to come back online if Stephane> needed. Stephane> So, do you use an RSYNC and in case of restoring the mailbox, do you do a? simple Stephane> doveadm index -u USERx INBOX Stephane> And that's it ? works fine ? Stephane> 2) I will try the Backup , or Sync.. locally.. Effectively, I don't know where the problem comes from..I have effectively an NFS Mount Stephane> for the mailboxes and a VM for the Email server and that could be another another point of failure :-( Stephane> Thanks for sharing.. I am a bit in a rush... I realized that my backup maybe not correct.. and I prefer not to discover it while running into trouble.. Stephane> On 2/28/22 19:03, Ben Burk wrote: Stephane> I'm not sure what you are attempting to do here. It looks like you just ran a doveadm backup Stephane> and the process completed for userx with a warning that the remote system (your nfs mount) Stephane> lost a particular mailbox (possibly your indexes changed or a mailbox was deleted). From the Stephane> logs you pasted it appears the process completed normally. Stephane> I personally do not use dovecot's backup or replication processes. If I needed to I would use Stephane> its replication process to sync active data between multiple systems, but I have no need for Stephane> this as of yet. Personally I chose to create offsite backups using rsync a long time ago, as Stephane> rebuilding a mailbox (reindexing) is very simple. Stephane> Try running??????????? doveadm mailbox status -u userx guid '*'??? as the mailbox Stephane> administrator and see if you can find that GUID, 7e05c335174bf1608f0a02004eac7fb4. Also, see Stephane> if the backup you've written to nfs has the GUID. Stephane> On 2/27/22 23:33, Stephane Magnier wrote: Stephane> Well no ..I thought that dsync was for synchro " realtime for 2 different places ? Stephane> Having no 2 machines in parallel ( Just a single machine ) , I thought that a backup at Stephane> regular interval? would be enough ? Stephane> So, a simple backup should be done by dsync finally ? Stephane> Do you recommend finally NOT to do a backup ( Doveadm backup ) but a replication process ? Stephane> ( https://wiki.dovecot.org/Replication? ) ? Stephane> On 2/28/22 06:24, Ben Burk wrote: Stephane> Did you try running dsync? Stephane> On 2/27/22 23:15, Stephane Magnier wrote: Stephane> HI, Stephane> Any idea ? Any clue ? Stephane> On 2/25/22 21:50, Stephane Magnier wrote: Stephane> Hi Stephane> I've recently tried to use the Dovecadm backup to backup the emails..? with Stephane> the following syntax Stephane> doveadm -Dv backup -u userx maildir:/mnt/nfs-backup/userx Stephane> Sounds to be OK with few emails... Some of them got a lot of emails and one f Stephane> them got an error and stop ! Stephane> dsync(userx): Debug: brain S: Import Trash: Import change type=expunge Stephane> GUID=1725fa475d774ee19cb98dfb6737b4f1 UID=24891 hdr_hash= result=GUIDs Stephane> match Stephane> dsync(userx): Debug: brain S: Import Trash: Import change type=expunge Stephane> GUID=916ed110b4b1522868be6194f1ae36ff UID=24892 hdr_hash= result=GUIDs Stephane> match Stephane> dsync(userx): Debug: brain S: Import Trash: Import change type=expunge Stephane> GUID=eb8d75c530a7b02fc26b494d9006c91b UID=24893 hdr_hash= result=GUIDs Stephane> match Stephane> dsync(userx): Debug: brain S: Import Trash: Import change type=expunge Stephane> GUID=aee9155875c34861fd6500f1f2f51a26 UID=24894 hdr_hash= result=GUIDs Stephane> match Stephane> dsync(userx): Debug: brain S: Import Trash: Last common UID=24894. Delayed Stephane> expungesStephane> dsync(userx): Debug: brain S: Import Trash: Saved UIDs: Stephane> dsync(userx): Debug: brain S: Import Trash: Finish update: min_next_uidStephane> 24895 min_first_recent_uid=24895 min_highest_modseq=35344 Stephane> min_highest_pvt_modseq=0 Stephane> dsync(userx): Debug: /mnt/nfs-backup/userx/.Trash/dovecot.index.cache: Stephane> Compressed, file_seq changed 1645803588 -> 1645803589, size=32, max_uidStephane> 24894 Stephane> dsync(userx): Warning: Mailbox changes caused a desync. You may want to Stephane> run dsync again: Remote lost mailbox GUID 7e05c335174bf1608f0a02004eac7fb4 Stephane> (maybe it was just deleted?) Stephane> dsync(userx): Debug: auth-master: conn unix:/run/dovecot/auth-userdb: Stephane> Disconnected: Connection closed (fd=10) Stephane> I empty the trash... exactly the same problem... Stephane> Any idea why this ?? Stephane> Thanks and regards,