dovecot - Aug 2015 - backing up email / saving maildir on external hard drives

Hello,
Yesterday I tried to back up a 40GB maildir .
I tried to move the maildir from home to external HDD but failed. Decided
then to compress it(which took several hours). Now changing the disk format
from FAT to exFAT to allow the transfer for the large compressed file.

How does one back up emails on a external drive?
Some advice would be greatly appreciated.

Am 08. August 2015 06:56:55 MESZ, schrieb Kevin Laurie <superinterstellar at
gmail.com>:
>Hello,
>Yesterday I tried to back up a 40GB maildir .
>I tried to move the maildir from home to external HDD but failed.
>Decided
>then to compress it(which took several hours). Now changing the disk
>format
>from FAT to exFAT to allow the transfer for the large compressed file.
>
>How does one back up emails on a external drive?
>Some advice would be greatly appreciated.
i backup my maildir with rsync to my home nas.

i assume you are using linux or similar, so when backing up to an external hdd
i'd also use rsync to copy over the maildir as is to the external hdd but
use a different filesystem n that hdd like ext4 or xfs or anything else than
fat/vfat/exfat.

when something goes wrong you can just copy the files back.

what was the error message you got when you tried to copy to the ext. hdd?

regards
- c

Dear Christian,
Thanks for your feedback.
The HDD will not accept larger than 4GB (as its in FAT format). Its a new
external HDD. Thinking of the best format(that would work with Mac , Win
and Linux) .seems like a challenge.
What's your view on NTFS? And why not exFAT?

Thanks
Kevin

On Saturday, August 8, 2015, Christian Kivalo <ml+dovecot at valo.at>
wrote:
>
>
> Am 08. August 2015 06:56:55 MESZ, schrieb Kevin Laurie <
> superinterstellar at gmail.com <javascript:;>>:
> >Hello,
> >Yesterday I tried to back up a 40GB maildir .
> >I tried to move the maildir from home to external HDD but failed.
> >Decided
> >then to compress it(which took several hours). Now changing the disk
> >format
> >from FAT to exFAT to allow the transfer for the large compressed file.
> >
> >How does one back up emails on a external drive?
> >Some advice would be greatly appreciated.
>
> i backup my maildir with rsync to my home nas.
>
> i assume you are using linux or similar, so when backing up to an external
> hdd i'd also use rsync to copy over the maildir as is to the external
hdd
> but use a different filesystem n that hdd like ext4 or xfs or anything else
> than fat/vfat/exfat.
>
> when something goes wrong you can just copy the files back.
>
> what was the error message you got when you tried to copy to the ext. hdd?
>
> regards
> - c
>

On Sat, 8 Aug 2015 10:26:55 +0530
Kevin Laurie <superinterstellar at gmail.com> wrote:
> Hello,
> Yesterday I tried to back up a 40GB maildir .
> I tried to move the maildir from home to external HDD but failed.
If you tried to *move* it it's an archive, not a backup. If you tried
to *copy* it, with the intent of keeping the original on the original
hard disk and using it further, and keeping today's copy on some other
media, *that's* a backup. I'm not trying to be pedantic, but there are
many distinctions between the two. Archives must be re-transferred
frequently: Backups merely need to be redone at intervals.
> Decided then to compress it(which took several hours). Now changing
> the disk format from FAT to exFAT to allow the transfer for the large
> compressed file.
Reading https://en.wikipedia.org/wiki/File_Allocation_Table#exFAT , I
personally wouldn't use exFAT. Regular FAT32 has a max filesize of
2GB-1, which is 50 times the size of your whole uncompressed maildir.
> 
> How does one back up emails on a external drive?
> Some advice would be greatly appreciated.
Check this out:

===============================================slitt at mydesq2:~$ df -h
~/mail/Maildir
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb8       116G   11G  100G  10% /home/slitt/mail/Maildir
slitt at mydesq2:~$ 
===============================================
I don't have 40 GB, but * have 11, which is less than an order of
magnitude away. I just back up this puppy to my backup server with my
normal rsync based backup procedures, which you can read about here:

* http://www.troubleshooters.com/lpm/200609/200609.htm

* http://www.troubleshooters.com/linux/blu-ray-backup.htm

* http://www.troubleshooters.com/lpm/201408/201408.htm

The stuff about Blu-Ray is important only if you back up to blu-ray. I
like to keep some backups on write-once media, because kept in the
shade at reasonable temperatures and humidities, it tends to last
longer. And spinning disks that spend the majority of their time not
spinning tend to have problems.

If this is a *backup*, I'd leave it uncompressed so you can take
incremental backups regularly. If it's an *archive*, meaning that the
data is immediately removed from your computer after copy, compression
might be in order, but you should make two copies and test them both
thoroughly before deleting the original, and you should test them every
couple months and if either goes bad, copy the other one to something
good. Archives are a PITA. For 40GB in these days of $150 2TB drives,
I'd keep the data intact, back it up, and when you outgrow your hard
drive, just get a bigger one.

In other parts of this thread you ask how to separate backups from
different accounts from different computers. As far as accounts, I
think that Maildir directory structures would take care of that. As far
as different machines, just put the hostname at the front of each
destination directory.



SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21

Dear Steve,
Very valuable info. Appreciate it and will be careful when using terms.
Actually I think I should just use rsync without compressing. The reason
why I started compressing was because the GUI gave some errors when I was
trying to copy then files.

I'll just rsync the data from my laptop HDD to my external drive(without
compressing)

Thanks
Kevin

On Sunday, August 9, 2015, Steve Litt <slitt at troubleshooters.com>
wrote:
> On Sat, 8 Aug 2015 10:26:55 +0530
> Kevin Laurie <superinterstellar at gmail.com <javascript:;>>
wrote:
>
> > Hello,
> > Yesterday I tried to back up a 40GB maildir .
> > I tried to move the maildir from home to external HDD but failed.
>
> If you tried to *move* it it's an archive, not a backup. If you tried
> to *copy* it, with the intent of keeping the original on the original
> hard disk and using it further, and keeping today's copy on some other
> media, *that's* a backup. I'm not trying to be pedantic, but there
are
> many distinctions between the two. Archives must be re-transferred
> frequently: Backups merely need to be redone at intervals.
>
> > Decided then to compress it(which took several hours). Now changing
> > the disk format from FAT to exFAT to allow the transfer for the large
> > compressed file.
>
> Reading https://en.wikipedia.org/wiki/File_Allocation_Table#exFAT , I
> personally wouldn't use exFAT. Regular FAT32 has a max filesize of
> 2GB-1, which is 50 times the size of your whole uncompressed maildir.
>
> >
> > How does one back up emails on a external drive?
> > Some advice would be greatly appreciated.
>
> Check this out:
>
> ===============================================> slitt at mydesq2:~$ df
-h ~/mail/Maildir
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/sdb8       116G   11G  100G  10% /home/slitt/mail/Maildir
> slitt at mydesq2:~$
> ===============================================>
> I don't have 40 GB, but * have 11, which is less than an order of
> magnitude away. I just back up this puppy to my backup server with my
> normal rsync based backup procedures, which you can read about here:
>
> * http://www.troubleshooters.com/lpm/200609/200609.htm
>
> * http://www.troubleshooters.com/linux/blu-ray-backup.htm
>
> * http://www.troubleshooters.com/lpm/201408/201408.htm
>
> The stuff about Blu-Ray is important only if you back up to blu-ray. I
> like to keep some backups on write-once media, because kept in the
> shade at reasonable temperatures and humidities, it tends to last
> longer. And spinning disks that spend the majority of their time not
> spinning tend to have problems.
>
> If this is a *backup*, I'd leave it uncompressed so you can take
> incremental backups regularly. If it's an *archive*, meaning that the
> data is immediately removed from your computer after copy, compression
> might be in order, but you should make two copies and test them both
> thoroughly before deleting the original, and you should test them every
> couple months and if either goes bad, copy the other one to something
> good. Archives are a PITA. For 40GB in these days of $150 2TB drives,
> I'd keep the data intact, back it up, and when you outgrow your hard
> drive, just get a bigger one.
>
> In other parts of this thread you ask how to separate backups from
> different accounts from different computers. As far as accounts, I
> think that Maildir directory structures would take care of that. As far
> as different machines, just put the hostname at the front of each
> destination directory.
>
>
>
> SteveT
>
> Steve Litt
> July 2015 featured book: Rapid Learning for the 21st Century
> http://www.troubleshooters.com/rl21
>