Up until now I've been backing up my IMAP server by taking an atomic snapshot of the entire VM, so I've not really had to worry about it. I'm moving to hosting where I have neither VM snapshots nor a filesystem that can take snapshots, so I need to think about consistent backups a bit more. Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup? (I could use doveadm backup if I were backing up to local storage I controlled, but I'm pushing it to offsite buckets so I'd prefer to avoid backing up the entire thing locally, then copying that offsite). Cheers, Steve
Been rsyncing my whole /var/vmail subdir (virtual users) for a while. So far so
good.
Yassine.
On Sunday, April 8, 2018, 11:56:29 PM GMT+1, Steve Atkins <steve at
blighty.com> wrote:
Up until now I've been backing up my IMAP server by taking an atomic
snapshot of the entire VM, so I've not really had to worry about it. I'm
moving to hosting where I have neither VM snapshots nor a filesystem that can
take snapshots, so I need to think about consistent backups a bit more.
Will a simple backup of all the files of an multi-dbox give me a consistent
state when I recover, or do I need to do something (e.g. lock writes to the
mailbox) while I run a backup?
(I could use doveadm backup if I were backing up to local storage I controlled,
but I'm pushing it to offsite buckets so I'd prefer to avoid backing up
the entire thing locally, then copying that offsite).
Cheers,
? Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20180409/a27154a9/attachment-0001.html>
Am 2018-04-09 00:55, schrieb Steve Atkins:> Will a simple backup of all the files of an multi-dbox give me a > consistent state when I recover, or do I need to do something (e.g. > lock writes to the mailbox) while I run a backup?If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names. If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely. But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later... -- Greetings, Marc
Ah, I didn't know there was a diff. b/w maildir and mdbox in terms of
storage. I should have mentionned I am using Maildir.
Yassine.
On Monday, April 9, 2018, 1:35:15 PM GMT+1, Marc St?rmer <mail at
marc-stuermer.de> wrote:
Am 2018-04-09 00:55, schrieb Steve Atkins:
> Will a simple backup of all the files of an multi-dbox give me a
> consistent state when I recover, or do I need to do something (e.g.
> lock writes to the mailbox) while I run a backup?
If you are doing this backup with a fully operational mail storage: no,
it won't. The reason is quite simple: mdbox saves the data of an email
in several files, so there's a good chance that those files might
slightly change during backup, while with Maildir all necessary data for
the restore is stored in the file names.
If you want to do a consistent backup, you do need file system snapshots
and should make sure that this snapshot is consistent or to turn off the
mail store during backup completely.
But just doing rsync on the fly as with Maildir is just begging on your
side to run into problems sooner or later...
--
Greetings,
Marc
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20180409/d858ac61/attachment.html>
Le 4/9/2018 ? 2:34 PM, Marc St?rmer a ?crit?:> Am 2018-04-09 00:55, schrieb Steve Atkins: > >> Will a simple backup of all the files of an multi-dbox give me a >> consistent state when I recover, or do I need to do something (e.g. >> lock writes to the mailbox) while I run a backup? > > If you are doing this backup with a fully operational mail storage: > no, it won't. The reason is quite simple: mdbox saves the data of an > email in several files, so there's a good chance that those files > might slightly change during backup, while with Maildir all necessary > data for the restore is stored in the file names. > > If you want to do a consistent backup, you do need file system > snapshots and should make sure that this snapshot is consistent or to > turn off the mail store during backup completely. > > But just doing rsync on the fly as with Maildir is just begging on > your side to run into problems sooner or later... >Personally, I always do a doveadm backup -u <users> to a different location (in mdbox format). Then do a git -A, git commit, git repack *THEN* rsync the resulting .git directories to an off-site location (thus only sending the difference between one bakup and the next - in packed format). The advantage is I can retrieve the status of any mailbox at any point in time (right now I have a year long of hour per hour backup of my servers mailboxes with a volume that is less than twice the size of the entire mailbox sizes). Granted, it's a low volume, low trafic server, but it works ! --Ivan (PS : Sorry Mark for the duplicate as I answered to you directly instead of the list) -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20180409/4222d83b/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3970 bytes Desc: Signature cryptographique S/MIME URL: <https://dovecot.org/pipermail/dovecot/attachments/20180409/4222d83b/attachment-0001.p7s>
> On Apr 9, 2018, at 5:34 AM, Marc St?rmer <mail at marc-stuermer.de> wrote: > > Am 2018-04-09 00:55, schrieb Steve Atkins: > >> Will a simple backup of all the files of an multi-dbox give me a >> consistent state when I recover, or do I need to do something (e.g. >> lock writes to the mailbox) while I run a backup? > > If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names. > > If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completelyThanks. Time to read up on snapshots! Cheers, Steve