I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use. I have run years without any backup, but would really like to have something in place.? I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere.? Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup. But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql Are there good tools that nicely does this?? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync? And stopping services itself is thought provoking.? What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart. Just scary stuff and, in part, why I have never tackled this in the past. thanks for all feedback
On 10/02/19 4:50 PM, Robert Moskowitz via dovecot wrote:> I have been thinking, and reading, on how to back up my mailserver. I > have not found any approach that seems ready to use. > > > I have run years without any backup, but would really like to have > something in place.? I figure I can attach a USB drive and backup to > that, then from there rsync to something elsewhere.? Further if that USB > drive is a full mailserver image, I actually have a 'hot backup' where I > only have to put the backup drive into a system and boot up at the last > backup. > > But this means properly copying all of /home/vmail and probably > /home/sieve plus the /var/lib/mysql > > Are there good tools that nicely does this?? Or do I choose a time late > at night (only I am sometimes in non-US timezones) to shut down all > services and just use rsync? > > And stopping services itself is thought provoking.? What if Dovecot, > amavis, mysql, or whatelse is in the middle of writing out a mail file > what happens to that file and restart.I use dirvish, which is based on rsync, and which only syncs changed files but uses hardlinks to keep storage and bandwidth down - it leaves me with a full copy of the tree for each date, with links between. I run it on a machine at home. I don't ususally worry about stopping services; maildir should be all right. For databases, I use the backup tools provided to back up the running database, then use dirvish on the backups (and skip the database files themselves). I do do it at night, to save load on the server (it's an LXC container on a VPS). I believe rdiffbackup does something similar to dirvish. Richard
Hello Robert,
----- Nachricht von Robert Moskowitz via dovecot <dovecot at dovecot.org>
---------
Datum: Sat, 9 Feb 2019 22:50:24 -0500
Von: Robert Moskowitz via dovecot <dovecot at dovecot.org>
Antwort an: Robert Moskowitz <rgm at htt-consult.com>, Dovecot Mailing
List <dovecot at dovecot.org>
Betreff: How to backup maildir
An: Dovecot Mailing List <dovecot at dovecot.org>
> I have been thinking, and reading, on how to back up my mailserver.
> I have not found any approach that seems ready to use.
>
>
> I have run years without any backup, but would really like to have
> something in place.
you're a really lucky guy! - I've been struck in the past for such
carelessness on the one or other machine with dataloss ;-)
> ? I figure I can attach a USB drive and backup to that, then from
> there rsync to something elsewhere.? Further if that USB drive is a
> full mailserver image, I actually have a 'hot backup' where I only
> have to put the backup drive into a system and boot up at the last
> backup.
>
> But this means properly copying all of /home/vmail and probably
> /home/sieve plus the /var/lib/mysql
>
Are you aware of the dovecot command "dsync"? (man dsync or
https://wiki.dovecot.org/Tools/Doveadm/Sync)
This could be an approach of using dsync:
dsync backup -o plugin/quota= -f -u $user backup
maildir:/mnt/USB/dovecot-backup/Maildir/$user/mail
> Are there good tools that nicely does this?? Or do I choose a time
> late at night (only I am sometimes in non-US timezones) to shut down
> all services and just use rsync?
>
> And stopping services itself is thought provoking.? What if Dovecot,
> amavis, mysql, or whatelse is in the middle of writing out a mail
> file what happens to that file and restart.
>
> Just scary stuff and, in part, why I have never tackled this in the past.
>
> thanks for all feedback
----- Ende der Nachricht von Robert Moskowitz via dovecot
<dovecot at dovecot.org> -----
It really depends on how important your data is to you ... But you
should really think about a general backup-strategy!
"Mr. Google" can help you to get some ideas how YOUR backup-strategy
could look like...
Also there is much input for backing up dovecot with it's different
mail storage flavours.
- But you have to invest some effort on your own, to search, read,
evaluate and finally choose what's fitting into YOUR setup!
But as an starting point:
I'm using a for _MY setup_ modified and adopted version of Klaus
Tachtler's dovecot-backup script:
https://github.com/tachtler/dovecot-backup/blob/master/dovecot_backup.sh
... mixed it with Borg Backup:
https://www.borgbackup.org/
... some further encryption, cloud storage and ... and ... and other stuff.
But as above mentioned:
YOU have to think about the grade of your paranoia level, how
importand the data is to you in case of an data loss, time and money
you are willing to invest and build upon this YOUR PERSONAL backup
strategy.
- Sadly there is no one-size-fits-all!
Last famous words:
I've looked at your vita and was wondering about your post - you were
writing RFCs, but have no clue about backing up your mail-data???
Strange ...
Nevermind! Just my 2ct - hope this helps
Christoph.
--
Christoph Haas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-keys
Size: 3178 bytes
Desc: ?ffentlicher PGP-Schl?ssel
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20190210/62d6d994/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 821 bytes
Desc: Digitale PGP-Signatur
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20190210/62d6d994/attachment-0001.sig>
On 2/10/19 2:24 AM, Christoph Haas via dovecot wrote:> Hello Robert, > > ----- Nachricht von Robert Moskowitz via dovecot <dovecot at dovecot.org> > --------- > ???? Datum: Sat, 9 Feb 2019 22:50:24 -0500 > ?????? Von: Robert Moskowitz via dovecot <dovecot at dovecot.org> > Antwort an: Robert Moskowitz <rgm at htt-consult.com>, Dovecot Mailing > List <dovecot at dovecot.org> > ?? Betreff: How to backup maildir > ??????? An: Dovecot Mailing List <dovecot at dovecot.org> > > >> I have been thinking, and reading, on how to back up my mailserver. I >> have not found any approach that seems ready to use. >> >> >> I have run years without any backup, but would really like to have >> something in place. > > you're a really lucky guy! - I've been struck in the past for such > carelessness on the one or other machine with dataloss ;-)Absolute laziness.? No real excuse.?? Also all users were POPing until 2 years ago.? Finally got everyone on IMAP just in the last year.? So a server loss would have been an inconvenience.? For myself, almost nothing in the IMAP store, everything in local folders that I have a separate backup procedure.> >> ? I figure I can attach a USB drive and backup to that, then from >> there rsync to something elsewhere. Further if that USB drive is a >> full mailserver image, I actually have a 'hot backup' where I only >> have to put the backup drive into a system and boot up at the last >> backup. >> >> But this means properly copying all of /home/vmail and probably >> /home/sieve plus the /var/lib/mysql >> > > Are you aware of the dovecot command "dsync"? (man dsync or > https://wiki.dovecot.org/Tools/Doveadm/Sync) > This could be an approach of using dsync: > dsync backup -o plugin/quota= -f -u $user backup > maildir:/mnt/USB/dovecot-backup/Maildir/$user/mailMy search foo is weak.? This is a long documented fact.? I did spend a number of hours searching and reading before opening my mouth here, fully expecting to put more than my toes in. I will read up on dsync.> >> Are there good tools that nicely does this?? Or do I choose a time >> late at night (only I am sometimes in non-US timezones) to shut down >> all services and just use rsync? >> >> And stopping services itself is thought provoking.? What if Dovecot, >> amavis, mysql, or whatelse is in the middle of writing out a mail >> file what happens to that file and restart. >> >> Just scary stuff and, in part, why I have never tackled this in the >> past. >> >> thanks for all feedback > > > ----- Ende der Nachricht von Robert Moskowitz via dovecot > <dovecot at dovecot.org> ----- > > It really depends on how important your data is to you ... But you > should really think about a general backup-strategy! > "Mr. Google" can help you to get some ideas how YOUR backup-strategy > could look like... > Also there is much input for backing up dovecot with it's different > mail storage flavours. > - But you have to invest some effort on your own, to search, read, > evaluate and finally choose what's fitting into YOUR setup!I have been and have been searching.? Some hits, but so far nothing was hitting the spot.? But, I will blame my dyslexia that my search foo is weak.> > But as an starting point: > I'm using a for _MY setup_ modified and adopted version of Klaus > Tachtler's dovecot-backup script: > https://github.com/tachtler/dovecot-backup/blob/master/dovecot_backup.sh > > ... mixed it with Borg Backup: > https://www.borgbackup.org/ > > ... some further encryption, cloud storage and ... and ... and other > stuff.I will check both of these.> > > But as above mentioned: > YOU have to think about the grade of your paranoia level, how > importand the data is to you in case of an data loss, time and money > you are willing to invest and build upon this YOUR PERSONAL backup > strategy. > - Sadly there is no one-size-fits-all!If there was, we would not be here, I suspect.> > Last famous words: > I've looked at your vita and was wondering about your post - you were > writing RFCs, but have no clue about backing up your mail-data??? > Strange ...MY mail is well backed up.? Locally so that I can read on a plane and the like.? I have been running one flavor or another of my own mail server since '95.? I switched to dovecot 6 years ago. My home file server is backed up 4 ways around.? And I have lost file servers and file server drives over the years.? Upgrading my file server is the next project. But I write RFCs.? I have been using geany for XML for a few years. I 'code' in English.? I have not written computer code since probably the late 80s.? I never coded in C, but I did use B for a while!? These days I can write simple scripts when forced to.? :) My 'spare' time these days is working with armv7 boards.? I have been using Cubietech since '13, recently got an Odroid HC1 and that is what this server will be.? For the most part you will find me on the Centos/Fedora lists and their arm lists.? Occasionally I will put on my Kevlar suit and ask something basic on a product list. But as a result of doing this, recently there is a new SElinux policy for permitting Dovecot to access Mysql.? It has already been patched into Fedora 28 and up.? Sometimes it pays to ask for help.> > > Nevermind! Just my 2ct - hope this helps > Christoph.Euro ct?? :) Better than US ct!
Hey, A few years back, I was also suffered for how to backup Maildir data files and then, I came to know about the Maildir Converter <https://www.cubexsoft.com/maildir/> solution. It has transferred my all Maildir files to desktop without showing any single error. You can also try this same solution. -- Sent from: http://dovecot.2317879.n4.nabble.com/