CentOS - Jan 2011 - automated remote cloning

Good morning/day and Happy New Year.

We have a geographically distributed environment (marketing speak: cloud) where
we regularly need to migrate individual systems to new hardware (for bigger
disks or for better geographical placement, for example).  We currently use
Cobbler to do our base installs automatically and I am now looking at
integrating Clonezilla with Cobbler.  The goal is take an *individual* system
that has been customized and migrate it in an automated fashion.

We currently do this using Cobbler and then running rsync and mysqldump in a
script along with other system -> userland module configurations (such as PHP
modules) but the process is tedious and generally a PITA.  We also must have the
ability to run this across different datacenters on different continents.

Any pointers to good automated solutions?  


References:  

https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.centos.org/pipermail/centos/attachments/20110101/2eada568/attachment-0003.html>

On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz <geoff at galitz.org>
wrote:
>
> Good morning/day and Happy New Year.
>
> We have a geographically?distributed environment (marketing speak: cloud)
> where we regularly need to migrate individual systems to new?hardware (for
> bigger disks or for better?geographical placement, for example).? We
> currently use?Cobbler to do our base installs automatically?and I am now
> looking at integrating?Clonezilla with?Cobbler.? The goal is take an
> *individual* system that has been customized and migrate it in an automated
> fashion.
The "RHEL" way is to use RHN. I don't recommend it, it's
expensive and
requires an Oracle server even if you want to install your own based
on the "Spacewalk" toolkit.

More efficiently, you can use 'mock' or other tools to build a
stripped chroot cage, tarball it up, then un-tarball it and apply
patches as needed. Then push tools to partition the disk, mount the
partitions, apply the tarball on it, and configure to your particular
needs. I've probably done..... 20,000 systems this way, with the
approach dating all the way back to RedHat 5.2.

Properly done, you can even write a remote installer that grabs your
existing swap space, stuffs the OS image *there*, then rebuilds the OS
around the existing allocated swap space, then releases and
re-apportions. That's less trivial.
> We currently do this using Cobbler and then running rsync and mysqldump?in
a
> script?along with other system -> userland module configurations (such
as
> PHP modules) but the process is tedious and generally a PITA.? We also must
> have the ability to run this across different datacenters on different
> continents.
Push or pull the OS image tarball first. You  can strip out the
sensitive data. such as unlocked passwords, and push commands from
your central repository to reset those passwords with their encrypted
versions if necessary.
> Any pointers to good automated solutions?
>
>
> References:
>
> https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>
>

On 1/1/11 5:50 AM, Geoff Galitz wrote:
> Good morning/day and Happy New Year.
> We have a geographically distributed environment (marketing speak: cloud)
where
> we regularly need to migrate individual systems to new hardware (for bigger
> disks or for better geographical placement, for example). We currently use
> Cobbler to do our base installs automatically and I am now looking at
> integrating Clonezilla with Cobbler. The goal is take an *individual*
system
> that has been customized and migrate it in an automated fashion.
> We currently do this using Cobbler and then running rsync and mysqldump in
a
> script along with other system -> userland module configurations (such
as PHP
> modules) but the process is tedious and generally a PITA. We also must have
the
> ability to run this across different datacenters on different continents.
> Any pointers to good automated solutions?
> References:
> https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration
Clonezilla is probably the only thing that doesn't lock you into a single 
OS/distribution.  It works as long as the hardware is similar and the target 
disk is as large or larger than the source, but unless you use DHCP for IP 
addressing on all NICs you can expect problems with losing your static configs 
when cloning.   I don't think it would be efficient to clone a large number
of
individualized copies remotely from a central location though. You'd
probably
want a generic image you could cache locally, followed by a scripted update of 
the differences.   Clonezilla also does not have a handy way to clone an 
existing machine to one with larger disks letting you specify new 
arbitrarily-sized partitions (although it can resize them all proportionally).

-- 
   Les Mikesell
    lesmikesell at gmail.com

On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz <geoff at galitz.org>
wrote:
>
> Good morning/day and Happy New Year.
>
> We have a geographically?distributed environment (marketing speak: cloud)
> where we regularly need to migrate individual systems to new?hardware (for
> bigger disks or for better?geographical placement, for example).? We
> currently use?Cobbler to do our base installs automatically?and I am now
> looking at integrating?Clonezilla with?Cobbler.? The goal is take an
> *individual* system that has been customized and migrate it in an automated
> fashion.
>
> We currently do this using Cobbler and then running rsync and mysqldump?in
a
> script?along with other system -> userland module configurations (such
as
> PHP modules) but the process is tedious and generally a PITA.? We also must
> have the ability to run this across different datacenters on different
> continents.
>
> Any pointers to good automated solutions?
Do you use cfengine or puppet in your environment?  If you install the
base packages via post installation scripts, you can use cfengine to
layer on specific host identities (hostname, IP, configuration,
packages, etc.).  With either of these tools you can import
configurations from a running host and then layer them back to a new
host.  There is some considerable setup to get it to that point, but
lots of scripts available online so much of the work is done.

On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz <geoff at galitz.org>
wrote:
> We have a geographically?distributed environment (marketing speak: cloud)
> where we regularly need to migrate individual systems to new?hardware (for
> bigger disks or for better?geographical placement, for example).? We
> currently use?Cobbler to do our base installs automatically?and I am now
> looking at integrating?Clonezilla with?Cobbler.? The goal is take an
> *individual* system that has been customized and migrate it in an automated
> fashion.
[snip]
> Any pointers to good automated solutions?
>
Check out this document:
http://www.ibm.com/developerworks/cloud/library/cl-automatevm/index.html?ca=drs-

Not exactly your case, but it seems it could address some of the
concerns you had.