CentOS - Jul 2005 - Backup Server with rsync methods

Hi guys:

I'm about to migrate a sever to a managed server facility.  I've
alwau=ys
used tape to backup my server, now that's not an option.  I've read abut
using rsync to do backups and have several scripts to do same.  I believe
rsync is a good solution.

I am missing some important nuiances though...  I would like to have
essentially a "mirror" server at my facility ( call it
"master") that I can
backup to, and from, completely from the other server (call it
"hosted").

Obviously I have to have at least different IP addresses on "master"
and
"hosted", but other than that, I'd like them to be mirrors of each
other.  I
would like to do all my configs, setup, etc. at my leisure on
"master",
where I have hands on control, then when ready, have "hosted" setup
with a
basic Centos server install, then configure it to rsync to my "master"
and
update the entire thing (updates and all) to my "master".

1. Which files tdo I need to exclude to avoid IP address and hostname
changes?

2. What caveats do I need to be aware of?  Are there other files such as
'live" libraries that I cannot sync to?

3. The "master" server at my facility and their server may have
different
hard drive architecture (i.e. IDS and SCSI drives, different processor,
etc.).  As long as I use a generic Kernel (i.e. stock) that supposts the
hardware on both, this shoudl not be a problem, right?

4. Am I missing something, is there a better way?

Thanks,
Scott

On Thu, 2005-07-07 at 11:56 -0500, scott wrote:
> Hi guys:
> 
> I'm about to migrate a sever to a managed server facility.  I've
alwau=ys
> used tape to backup my server, now that's not an option.  I've read
abut
> using rsync to do backups and have several scripts to do same.  I believe
> rsync is a good solution.
> 
> I am missing some important nuiances though...  I would like to have
> essentially a "mirror" server at my facility ( call it
"master") that I can
> backup to, and from, completely from the other server (call it
"hosted").
> 
> Obviously I have to have at least different IP addresses on
"master" and
> "hosted", but other than that, I'd like them to be mirrors of
each other.  I
> would like to do all my configs, setup, etc. at my leisure on
"master",
> where I have hands on control, then when ready, have "hosted"
setup with a
> basic Centos server install, then configure it to rsync to my
"master" and
> update the entire thing (updates and all) to my "master".
> 
> 1. Which files tdo I need to exclude to avoid IP address and hostname
> changes?
/etc/sysconfig/network
/etc/sysconfig/network-scripts/ifcfg-eth0

Likely also want to exclude:
/etc/sysconfig/hwconf
/etc/fstab
/etc/mtab
/etc/ssh/
/root/.ssh/
/boot/grub/grub.conf
/boot/grub/device.map

Will need a /etc/hosts that is compatible with both machines.  I'm no
doubt forgetting some others to exclude.
> 2. What caveats do I need to be aware of?  Are there other files such as
> 'live" libraries that I cannot sync to?
Doing this to a live system is VERY likely to be problematic.  How about
a ping-pong approach where you keep two bootable systems on the
"hosted"
system (on different partitions or physical disks) and update the one
not being used, fix up grub and other files (see below), and reboot to
the new image.  Gives you the untouched installation to fall back on,
but could still be a problem if boot fails and you have nobody on-site
to help.
> 3. The "master" server at my facility and their server may have
different
> hard drive architecture (i.e. IDS and SCSI drives, different processor,
> etc.).  As long as I use a generic Kernel (i.e. stock) that supposts the
> hardware on both, this shoudl not be a problem, right?
May need to run mkinitrd on the "hosted" machine if the hardware is
different, edit /boot/grub/grub.conf, and possibly /etc/fstab
and /etc/mtab, before booting to a new kernel.  In the past, I've found
the existence of /etc/mtab~ to cause problems on reboot, so watch out
for cleaning up the backup file if using emacs or another editor that
uses that convention.
> 4. Am I missing something, is there a better way?
Would work better, involve less manual fix-up, and be less error-prone
to have identical hardware.

I'd consider just keeping a local backup image of the "hosted"
machine
for disaster recovery, and perhaps use a local "slave" (as opposed to
"master") system for testing updates before installing them on the
"hosted" system.  Could still use rsync for backups without worrying
about excludes.

Phil

My advice to you is rsync'ing the whole machine is never going to work 
without problems.

You should concentrate on rsync'ing data partitions and simple configs 
that may accompany the data sets eg: httpd.conf or /etc/mail

You may wish to tar up the config files for each of the setups you are 
using, and rsync the tar ball to the remote machine.

Then write a script to untar and restart services if configs (tar ball)  
changes.

Forget the idea of rsync'ing the kernel especially if the hardware is 
different, what real operational benefit is this going to give you
for something that will generate a lot of problems?.  I can see that the 
only possible benefit is that you test on the local machine and then 
rsync the config
to the hosted machine,  for offline testing of configs, the concept of 
rsync ing in the manner you describe will create so many other unknowns
or problems it will almost certainly cloud the validity of your test bed 
and its processes anyway.

The best way to truly test in the manner you describe is to have 
identical machines and then create a config on one and replicate it by 
hand onto the other when you are happy
the config is good.  Trying to do it automagically will imho screw it up 
in some other way....

P.




scott wrote:
>Hi guys:
>
>I'm about to migrate a sever to a managed server facility.  I've
alwau=ys
>used tape to backup my server, now that's not an option.  I've read
abut
>using rsync to do backups and have several scripts to do same.  I believe
>rsync is a good solution.
>
>I am missing some important nuiances though...  I would like to have
>essentially a "mirror" server at my facility ( call it
"master") that I can
>backup to, and from, completely from the other server (call it
"hosted").
>
>Obviously I have to have at least different IP addresses on
"master" and
>"hosted", but other than that, I'd like them to be mirrors of
each other.  I
>would like to do all my configs, setup, etc. at my leisure on
"master",
>where I have hands on control, then when ready, have "hosted"
setup with a
>basic Centos server install, then configure it to rsync to my
"master" and
>update the entire thing (updates and all) to my "master".
>
>1. Which files tdo I need to exclude to avoid IP address and hostname
>changes?
>
>2. What caveats do I need to be aware of?  Are there other files such as
>'live" libraries that I cannot sync to?s
>
>3. The "master" server at my facility and their server may have
different
>hard drive architecture (i.e. IDS and SCSI drives, different processor,
>etc.).  As long as I use a generic Kernel (i.e. stock) that supposts the
>hardware on both, this shoudl not be a problem, right?
>
>4. Am I missing something, is there a better way?
>
>Thanks,
>Scott
>
>
>
>_______________________________________________
>CentOS mailing list
>CentOS at centos.org
>http://lists.centos.org/mailman/listinfo/centos
>  
>

> Hi guys:
>
> I'm about to migrate a sever to a managed server facility.  I've
alwau=ys
> used tape to backup my server, now that's not an option.  I've read
abut
> using rsync to do backups and have several scripts to do same.  I believe
> rsync is a good solution.
>
> I am missing some important nuiances though...  I would like to have
> essentially a "mirror" server at my facility ( call it
"master") that I
> can
> backup to, and from, completely from the other server (call it
"hosted").
>
> Obviously I have to have at least different IP addresses on
"master" and
> "hosted", but other than that, I'd like them to be mirrors of
each other.
> I
> would like to do all my configs, setup, etc. at my leisure on
"master",
> where I have hands on control, then when ready, have "hosted"
setup with a
> basic Centos server install, then configure it to rsync to my
"master" and
> update the entire thing (updates and all) to my "master".
>
> 1. Which files tdo I need to exclude to avoid IP address and hostname
> changes?
>
> 2. What caveats do I need to be aware of?  Are there other files such as
> 'live" libraries that I cannot sync to?
>
> 3. The "master" server at my facility and their server may have
different
> hard drive architecture (i.e. IDS and SCSI drives, different processor,
> etc.).  As long as I use a generic Kernel (i.e. stock) that supposts the
> hardware on both, this shoudl not be a problem, right?
>
> 4. Am I missing something, is there a better way?
>
> Thanks,
> Scott
>
You might be able to use SystemImager (http://www.systemimager.org/) to 
handle this. It's designed to create an rsync based replica of a 'golden
server' that can then be pushed down to one or more client machines.  It has
mechanisms for changing IP addresses and other networking information on the 
client machines. You'll want to familiarize yourself with how it excludes 
files from the image so you can avoid overwriting important data.  Your live 
will be much easier if you have identical hardware, but if you're truly 
determined, it might be doable with reasonably similar hardware.

From: "scott.list" <scott.list at
mlec.net>
> My real, main objective is to be able to get a config working on a
> local machine for FTP, Postfix, Apache, etc. then quickly getting a
> remote server up and running with those configs in a quick fashion.  I
> would like to have my configuration "portable" so to speak so I
may
> change providers on a whim if desired, and to do minimal configuration
> on the remote server to quickly get it to the stage and setup of my
> main, home box.
Then you should maintain all configuration and other system-specific
files under CVS.  That way you know what may or may not need to
be changed, possibly with a couple of sed commands after checkout.

This is the greater detail of "configuration management" in general,
and not really specific to any backup scheme.



--
Bryan J. Smith   mailto:b.j.smith at ieee.org