Xen users - May 2010 - migrate existing non-xen system to a VM?

I know that I''ve seen this discussed before, but can anyone point to a 
concise description of how to take an existing, non-xen machine (in this 
case a system running Debian Lenny), and,

1. shut it down
2. package up the state
3. move it into a VM (assume there''s a 2nd computer, with Dom0 up and 
running, on the same LAN)
4. start the VM and end up with a virtualized version of the original 
environment chugging away

A little context:  I''m trying to add hot-spare failover to a production
system (email, list services, web server, database) - by moving from a 
standard Debian Lenny install to Debian Lenny running as a VM, backed by 
DRBD, set for failover.  The first step is to get the production 
environment into a VM, and then I can set up all the failover stuff.

So... the above is just a little more complicated, in that there are 4 
partitions that have to be moved:

/boot - raided volume --> xen setup on new machine
/swap --> drbd device on new machine, mounted in VM
/ --> drbd device on new machine, mounted in VM
/backup (another volume) -> drbd device mounted in VM

I''m currently proceeding down the path of replicating the environment
in
a running VM, preparing to manually migrate config. and user data.  But 
if there''s a simpler way, any suggestions would be very much
appreciated.

Thanks,

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

See notes below...
>>> On 2010/05/17 at 09:29, Miles Fidelman
<mfidelman@meetinghouse.net> wrote:
> I know that I''ve seen this discussed before, but can anyone point
to a
> concise description of how to take an existing, non-xen machine (in this 
> case a system running Debian Lenny), and,
> 
0 -> before you shut it down, update kernel/initrd such that you have a
kernel and initrd combo that will boot successfully on Xen.  In many
distributions, all this requires is installing the kernel-xen package, or
something similar, and you''ll be good to go.
> 1. shut it down
> 2. package up the state
> 3. move it into a VM (assume there''s a 2nd computer, with Dom0 up
and
> running, on the same LAN)
> 4. start the VM and end up with a virtualized version of the original 
> environment chugging away
> 
> A little context:  I''m trying to add hot-spare failover to a
production
> system (email, list services, web server, database) - by moving from a 
> standard Debian Lenny install to Debian Lenny running as a VM, backed by 
> DRBD, set for failover.  The first step is to get the production 
> environment into a VM, and then I can set up all the failover stuff.
> 
> So... the above is just a little more complicated, in that there are 4 
> partitions that have to be moved:
> 
> /boot - raided volume --> xen setup on new machine
> /swap --> drbd device on new machine, mounted in VM
> / --> drbd device on new machine, mounted in VM
> /backup (another volume) -> drbd device mounted in VM
Are these partitions in the VM (domU), or on the Xen hosts themselves?  If these
are the domU partitions, just using an imaging tool (Ghost, FOG, PartImage,
etc.) to copy the entire disk over to a virtual hard disk (LVM, file-based,
etc.).

- Nick



--------
This e-mail may contain confidential and privileged material for the sole use of
the intended recipient.  If this email is not intended for you, or you are not
responsible for the delivery of this message to the intended recipient, please
note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information.  In such a case, you are strictly prohibited
from downloading, photocopying, distributing or otherwise using this message,
its contents or attachments in any way.  If you have received this message in
error, please notify us immediately by replying to this e-mail and delete the
message from your mailbox.  Information contained in this message that does not
relate to the business of SEAKR is neither endorsed by nor attributable to
SEAKR.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks, and...

Nick Couchman wrote:
> 0 ->  before you shut it down, update kernel/initrd such that you have a
kernel and initrd combo that will boot successfully on Xen.  In many
distributions, all this requires is installing the kernel-xen package, or
something similar, and you''ll be good to go.
>
>    
great point!
>> /boot - raided volume -->  xen setup on new machine
>> /swap -->  drbd device on new machine, mounted in VM
>> / -->  drbd device on new machine, mounted in VM
>> /backup (another volume) ->  drbd device mounted in VM
>>      
> Are these partitions in the VM (domU), or on the Xen hosts themselves?  If
these are the domU partitions, just using an imaging tool (Ghost, FOG,
PartImage, etc.) to copy the entire disk over to a virtual hard disk (LVM,
file-based, etc.).
>
>    
partitions on the left are on the current (physical) system --> intended 
partitions on the new VM (except for boot)

good idea....

-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

xen.independent_wallclock=1 Hi Folks,

Well, last night, I transitioned my production system from physical 
(running directly on Debian Lenny on bare metal) to running inside a VM 
on Xen.  And it went incredibly smoothly.

First off - thanks to those of you who sent advice.  You saved me a lot 
of headaches.

Second, here''s a summary of what I did - in case it might be useful to 
others:

1. Set up my target machine:
- starting condition: Dom0 = Debian Lenny, 
xen-linux-system-2.6.26-2-xen-686, plus a few tweaks
- create three logical volumes:  root, swap, backup ("backup" is
another
volume I use for backups and things)
- set these up as DRBD devices (not really relevant to this discussion)
- make filesystems on root and backup
- mkswap on swap

2. Bring down production server, reboot with a rescue CD (SysRescueCD)
- assign an unused IP
- set up to allow root login via ssh

3. Transfer Data
- mount the original root and backup volumes
- move data (took a LONG time) - thanks to whomever sent this recipe! - 
typed these on the target (new) machine

rsync -avzH --numeric-ids -e ssh root@<IP>:/mnt/mydir/backup/* /mnt/backup

rsync -avzH --numeric-ids -e ssh root@<IP>:/mnt/mydir/root/* /mnt/root


4. Set up <server>.cfg file:
# notes: this is based on an earlier file generated by xen-tools

kernel = ''/boot/vmlinuz-2.6.26-2-xen-686''

ramdisk = ''/boot/initrd.img-2.6.26-2-xen-686''

memory = ''1024''

disk = [

''drbd:root,xvda1,w'',

''drbd:swap,xvda2,w'',

''drbd:backup,xvda3,w''

]

root = ''/dev/xvda1 ro''

vif = [ ''ip=xx.xx.xx.xx,mac=00:16:3E:BE:68:F5'' ]

#where xx.xx.xx.xx = primary address of the original system

name        = ''server1''

on_poweroff = ''destroy''
on_reboot   = ''restart''
on_crash    = ''restart''

#this is to fix a bug in clock sync, per advice at
#http://wiki.debian.org/Xen#A.27clocksource.2BAC8-0.3ATimewentbackwards.27
extra="clocksource=jiffies"

4. while still mounted, edit /mnt/root/etc/fstab to read:

proc /proc proc defaults 0 0

/dev/xvda1 / ext3 errors=remount-ro 0 1

/dev/xvda2 none swap sw 0 0

/dev/xvda3 /backup ext3 defaults 0 2


5. umount the two newly initialized volumes

6. start it up

xm create -c newserver1.cfg


and it ran - the first time!!!!

note: for some reason, the console stops responding after boot, but I 
can log in via ssh, and all the processes run, mail flows, web sites are 
up, etc. - the console reports messages on shutdown


7. log into the VM and clean a few things up


- turn off some services that are not relevant inside a VM (e.g, acpi, 
hwclock)
- add " xen.independent_wallclock=1" to /etc/sysctl.conf; sysctl -p
#see above note re. time bug, note that NTP is running inside the VM


8. a few things I need to track down and fix
- getting this entry in the bootlog (doesn''t seem to be effecting 
anything, though)

Mon May 17 22:25:22 2010: modprobe: FATAL: Could not load 
/lib/modules/2.6.26-2-xen-686/modules.dep: No such file or directory
- get the console to respond


Again, thanks to all who provided advice.


Miles Fidelman



-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Le lundi 17 mai 2010 à 13:04 -0400, Miles Fidelman a écrit
:
> Thanks, and...
> 
> Nick Couchman wrote:
> > 0 ->  before you shut it down, update kernel/initrd such that you
have a kernel and initrd combo that will boot successfully on Xen.  In many
distributions, all this requires is installing the kernel-xen package, or
something similar, and you''ll be good to go.
> >
> >    
> great point!
> >> /boot - raided volume -->  xen setup on new machine
> >> /swap -->  drbd device on new machine, mounted in VM
> >> / -->  drbd device on new machine, mounted in VM
> >> /backup (another volume) ->  drbd device mounted in VM
> >>      
Hello,

Another method could be to :
- install a new system in a VM
- import data into the new system
- test and go

Regards

JPP



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

jpp@jppozzi.dyndns.org wrote:
> Another method could be to :
> - install a new system in a VM
> - import data into the new system
> - test and go
>
>    
easier said than done with a complex production system - installing, 
configuring, and wiring together things like mail, anti-spam, 
anti-virus, list management, etc. is a real pain

-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hello,


regarding the number 6.:
>6. start it up
>
>xm create -c newserver1.cfg
>
>and it ran - the first time!!!!
>
>note: for some reason, the console stops responding after boot, but I 
>can log in via ssh, and all the processes run, mail flows, web sites are 
>up, etc. - the console reports messages on shutdown

this one might help:

DomU /etc/inittab:

Replace  1:2345:respawn:/sbin/getty 38400 tty1
with 1:2345:respawn:/sbin/getty 38400 hvc0


Regards

Matej


-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Miles Fidelman
Sent: Tuesday, May 18, 2010 4:09 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] Re: migrate existing non-xen system to a VM? - SUCCESS!
(and report)

xen.independent_wallclock=1 Hi Folks,

Well, last night, I transitioned my production system from physical 
(running directly on Debian Lenny on bare metal) to running inside a VM 
on Xen.  And it went incredibly smoothly.

First off - thanks to those of you who sent advice.  You saved me a lot 
of headaches.

Second, here''s a summary of what I did - in case it might be useful to 
others:

1. Set up my target machine:
- starting condition: Dom0 = Debian Lenny, 
xen-linux-system-2.6.26-2-xen-686, plus a few tweaks
- create three logical volumes:  root, swap, backup ("backup" is
another
volume I use for backups and things)
- set these up as DRBD devices (not really relevant to this discussion)
- make filesystems on root and backup
- mkswap on swap

2. Bring down production server, reboot with a rescue CD (SysRescueCD)
- assign an unused IP
- set up to allow root login via ssh

3. Transfer Data
- mount the original root and backup volumes
- move data (took a LONG time) - thanks to whomever sent this recipe! - 
typed these on the target (new) machine

rsync -avzH --numeric-ids -e ssh root@<IP>:/mnt/mydir/backup/* /mnt/backup

rsync -avzH --numeric-ids -e ssh root@<IP>:/mnt/mydir/root/* /mnt/root


4. Set up <server>.cfg file:
# notes: this is based on an earlier file generated by xen-tools

kernel = ''/boot/vmlinuz-2.6.26-2-xen-686''

ramdisk = ''/boot/initrd.img-2.6.26-2-xen-686''

memory = ''1024''

disk = [

''drbd:root,xvda1,w'',

''drbd:swap,xvda2,w'',

''drbd:backup,xvda3,w''

]

root = ''/dev/xvda1 ro''

vif = [ ''ip=xx.xx.xx.xx,mac=00:16:3E:BE:68:F5'' ]

#where xx.xx.xx.xx = primary address of the original system

name        = ''server1''

on_poweroff = ''destroy''
on_reboot   = ''restart''
on_crash    = ''restart''

#this is to fix a bug in clock sync, per advice at
#http://wiki.debian.org/Xen#A.27clocksource.2BAC8-0.3ATimewentbackwards.27
extra="clocksource=jiffies"

4. while still mounted, edit /mnt/root/etc/fstab to read:

proc /proc proc defaults 0 0

/dev/xvda1 / ext3 errors=remount-ro 0 1

/dev/xvda2 none swap sw 0 0

/dev/xvda3 /backup ext3 defaults 0 2


5. umount the two newly initialized volumes

6. start it up

xm create -c newserver1.cfg


and it ran - the first time!!!!

note: for some reason, the console stops responding after boot, but I 
can log in via ssh, and all the processes run, mail flows, web sites are 
up, etc. - the console reports messages on shutdown


7. log into the VM and clean a few things up


- turn off some services that are not relevant inside a VM (e.g, acpi, 
hwclock)
- add " xen.independent_wallclock=1" to /etc/sysctl.conf; sysctl -p
#see above note re. time bug, note that NTP is running inside the VM


8. a few things I need to track down and fix
- getting this entry in the bootlog (doesn''t seem to be effecting 
anything, though)

Mon May 17 22:25:22 2010: modprobe: FATAL: Could not load 
/lib/modules/2.6.26-2-xen-686/modules.dep: No such file or directory
- get the console to respond


Again, thanks to all who provided advice.


Miles Fidelman



-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Matej Zary wrote:
>> >6. start it up
>> >
>> >xm create -c newserver1.cfg
>> >
>> >and it ran - the first time!!!!
>> >
>> >note: for some reason, the console stops responding after boot, but
I
>> >can log in via ssh, and all the processes run, mail flows, web
sites are
>> >up, etc. - the console reports messages on shutdown
>>      
> this one might help:
>
> DomU /etc/inittab:
>
> Replace  1:2345:respawn:/sbin/getty 38400 tty1
> with 1:2345:respawn:/sbin/getty 38400 hvc0
>
>    
Thanks - that did it!

Miles


-- 
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, May 18, 2010 at 10:24 PM, Miles Fidelman
<mfidelman@meetinghouse.net> wrote:
> jpp@jppozzi.dyndns.org wrote:
>>
>> Another method could be to :
>> - install a new system in a VM
>> - import data into the new system
>> - test and go
>>
>>
>
> easier said than done with a complex production system - installing,
> configuring, and wiring together things like mail, anti-spam, anti-virus,
> list management, etc. is a real pain
If you have an existing backup or DR strategy that can handle physical
machine/disk failure (which might involve OS-level backup or disk
image), you can also use that.

Last time I check with Ubuntu, it''s initramfs pretty much includes
everything so that you _should_ be able to move/copy the entire disk
to another physical machine (including HVM domUs) and have it running
with minimal effort. Latest -server kernels also supports running as
PV guest. Perhaps some of this is also true for Debian.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users