Il 01/07/20 17:13, Leroy Tennison ha scritto:> I realize this shouldn't happen, the file is a tgz and isn't being modified while being transmitted. This has happened maybe three times this year and unfortunately I've just had to deal with it rather than invest the time to do the research. > > > Harriscomputer > > Leroy Tennison > Network Information/Cyber Security SpHi Leroy, I think that in my case I could not use a tgz archive. I'm speaking about full backups that reach 600/700GiB, compressing them and then rsync them could take so much time that it will be useless.
I setup drbd to replicate a ~50TB backuppc hive to the DR copy, an identical box in a different DC on the same campus, with approximately gigE speeds, and ran this for a year or two. It worked well enough but required babysitting from time to time. Both nodes were mdraid lvm logical volumes formatted as a single huge xfs on centos. I never automated the failover as it never failed, and as a dev/test backup, 8 hour response seemed adequate On Thu, Jul 2, 2020, 1:22 AM Alessandro Baggi <alessandro.baggi at gmail.com> wrote:> Il 01/07/20 17:13, Leroy Tennison ha scritto: > > I realize this shouldn't happen, the file is a tgz and isn't being > modified while being transmitted. This has happened maybe three times this > year and unfortunately I've just had to deal with it rather than invest the > time to do the research. > > > > > > Harriscomputer > > > > Leroy Tennison > > Network Information/Cyber Security Sp > > Hi Leroy, > > I think that in my case I could not use a tgz archive. I'm speaking > about full backups that reach 600/700GiB, compressing them and then > rsync them could take so much time that it will be useless. > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos >
Hi John, thank you for your answer, I already take in consideration DRBD but I need some test before start. Reading you seems that this solution is not anymore available. What do you use for this? Thank you in advance. Il 02/07/20 10:43, John Pierce ha scritto:> I setup drbd to replicate a ~50TB backuppc hive to the DR copy, an > identical box in a different DC on the same campus, with approximately gigE > speeds, and ran this for a year or two. It worked well enough but > required babysitting from time to time. Both nodes were mdraid lvm > logical volumes formatted as a single huge xfs on centos. I never > automated the failover as it never failed, and as a dev/test backup, 8 hour > response seemed adequate > > On Thu, Jul 2, 2020, 1:22 AM Alessandro Baggi <alessandro.baggi at gmail.com> > wrote: > >> Il 01/07/20 17:13, Leroy Tennison ha scritto: >>> I realize this shouldn't happen, the file is a tgz and isn't being >> modified while being transmitted. This has happened maybe three times this >> year and unfortunately I've just had to deal with it rather than invest the >> time to do the research. >>> >>> Harriscomputer >>> >>> Leroy Tennison >>> Network Information/Cyber Security Sp >> Hi Leroy, >> >> I think that in my case I could not use a tgz archive. I'm speaking >> about full backups that reach 600/700GiB, compressing them and then >> rsync them could take so much time that it will be useless. >> >> _______________________________________________ >> CentOS mailing list >> CentOS at centos.org >> https://lists.centos.org/mailman/listinfo/centos >> > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos
On 7/2/20 3:22 AM, Alessandro Baggi wrote:> Il 01/07/20 17:13, Leroy Tennison ha scritto: >> I realize this shouldn't happen, the file is a tgz and isn't being >> modified while being transmitted.? This has happened maybe three times >> this year and unfortunately I've just had to deal with it rather than >> invest the time to do the research. >> >> >> Harriscomputer >> >> Leroy Tennison >> Network Information/Cyber Security Sp > > Hi Leroy, > > I think that in my case I could not use a tgz archive. I'm speaking > about full backups that reach 600/700GiB, compressing them and then > rsync them could take so much time that it will be useless. >unless you use tape (of that high capacity), it is advantageous to restrict volume size to, say, 50GB. Then when you restore, search for specific files will be faster. And it will help your backup volumes transfers as well. Valeri> _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
Il 02/07/20 15:02, Valeri Galtsev ha scritto:> > > On 7/2/20 3:22 AM, Alessandro Baggi wrote: >> Il 01/07/20 17:13, Leroy Tennison ha scritto: >>> I realize this shouldn't happen, the file is a tgz and isn't being >>> modified while being transmitted.? This has happened maybe three >>> times this year and unfortunately I've just had to deal with it >>> rather than invest the time to do the research. >>> >>> >>> Harriscomputer >>> >>> Leroy Tennison >>> Network Information/Cyber Security Sp >> >> Hi Leroy, >> >> I think that in my case I could not use a tgz archive. I'm speaking >> about full backups that reach 600/700GiB, compressing them and then >> rsync them could take so much time that it will be useless. >> > > unless you use tape (of that high capacity), it is advantageous to > restrict volume size to, say, 50GB. Then when you restore, search for > specific files will be faster. And it will help your backup volumes > transfers as well. > > ValeriHi Valeri, thank you for your suggestion. Is bacula the right backup system when I need to replicate data offsite? There are other backup solution that simplify this process? Thank you in advance
Depending on the definition of offsite, you have a fundamental problem: either invest the time/effort compressing or take extra bandwidth, which is less costly? Hopefully a delta transfer makes sense in your situation and should save far more than compression would once the original copy is offsite. ________________________________ From: CentOS <centos-bounces at centos.org> on behalf of Valeri Galtsev <galtsev at kicp.uchicago.edu> Sent: Thursday, July 2, 2020 8:02 AM To: centos at centos.org <centos at centos.org> Subject: [EXTERNAL] Re: [CentOS] [OT] Bacula offsite replication CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Harriscomputer Leroy Tennison Network Information/Cyber Security Specialist E: leroy at datavoiceint.com [cid:Data-Voice-International-LOGO_aa3d1c6e-5cfb-451f-ba2c-af8059e69609.PNG] 2220 Bush Dr McKinney, Texas 75070 www.datavoiceint.com<http://www..com> This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc. If you prefer not to be contacted by Harris Operating Group please notify us<http://subscribe.harriscomputer.com/>. This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message. On 7/2/20 3:22 AM, Alessandro Baggi wrote:> Il 01/07/20 17:13, Leroy Tennison ha scritto: >> I realize this shouldn't happen, the file is a tgz and isn't being >> modified while being transmitted. This has happened maybe three times >> this year and unfortunately I've just had to deal with it rather than >> invest the time to do the research. >> >> >> Harriscomputer >> >> Leroy Tennison >> Network Information/Cyber Security Sp > > Hi Leroy, > > I think that in my case I could not use a tgz archive. I'm speaking > about full backups that reach 600/700GiB, compressing them and then > rsync them could take so much time that it will be useless. >unless you use tape (of that high capacity), it is advantageous to restrict volume size to, say, 50GB. Then when you restore, search for specific files will be faster. And it will help your backup volumes transfers as well. Valeri> _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++ _______________________________________________ CentOS mailing list CentOS at centos.org https://lists.centos.org/mailman/listinfo/centos