Xen users - May 2013 - Live Migration of a Virtual Machine without a Shared Storage

Hello everyone.

I am currently working on a Project for my thesis, which involves migrating
a VM from one host to another.

A FUSE filesystem, which I developed, is responsible for moving the Disk
contents (Virtual Disk Image) from the source to the destination host. The
VM continues to operate on the source machine until I reach the
"Hand-Over"
phase of my approach. During that phase I need to start operating the VM to
the destination host. In order to achieve this I have to move the CPU and
RAM state on the destination host.

I am aware of the fact that the live migration feature of Xen needs to have
shared storage for the disk images. In my case, the disk image is moved to
the destination machine, so that Xen can access it from its local
Filesystem. How can I command Xen to live migrate the VM and look for the
disk image in the local repository of the Destination Machine? Is there a
tutorial (or guide) that indicates how this can be done?

Thank you in advance.

Nick

P.S.: I realize that my post may be confusing, so do not hesitate to ask
questions.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Am 29.05.2013 um 16:26 schrieb Nick Katsipoulakis:
> I am aware of the fact that the live migration feature of Xen needs to have
shared storage for the disk images. In my case, the disk image is moved to the
destination machine, so that Xen can access it from its local Filesystem. How
can I command Xen to live migrate the VM and look for the disk image in the
local repository of the Destination Machine? Is there a tutorial (or guide) that
indicates how this can be done?
not shure, but if your filesystem does the correct sync it might work like with
i.e. DRBD setups:
http://backdrift.org/live-migration-and-synchronous-replicated-storage-with-xen-drbd-and-lvm

except your filesystem "copy" is not there att of migration. Depending
from your filesystem implementation it could be necessary to hack within the
migration code of xen - afaik there is no full scripting possibility or similiar
to do such things like "do that before taking over a migration" or
similiar.

You filesystem implementation sounds interesting - do you have any details about
it? If i understand right it is NOT intended for HA replication (where both
hosts requires full copies of full current filesystem) but i.e. for load
management in a cluster.


cheerioh,

Niels.

---
Niels Dettenbach
Syndicat IT & Internet
http://syndicat.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Niels thank you for your immediate answer and concern.

Actually my FUSE Filesystem works as an abstraction layer between Xen and
the underlying Filesystem (i.e. ext3). Everytime an I/O is performed by the
VM, whose disk image is stored under my filesystem''s mount point and is
about to be migrated, the I/O is intercepted and actions are taken in order
to ensure the consistency of the migrated disk image. It also manages to
hand-over the VM and start its operation on the destination machine, before
the whole disk image is migrated.



The DRBD setup is complex and I do not think I need it.


Cheers,

Nick


2013/5/29 Niels Dettenbach <nd@syndicat.com>
>
> Am 29.05.2013 um 16:26 schrieb Nick Katsipoulakis:
>
> I am aware of the fact that the live migration feature of Xen needs to
> have shared storage for the disk images. In my case, the disk image is
> moved to the destination machine, so that Xen can access it from its
> local Filesystem. How can I command Xen to live migrate the VM and look for
> the disk image in the local repository of the Destination Machine? Is there
> a tutorial (or guide) that indicates how this can be done?
>
>
> not shure, but if your filesystem does the correct sync it might work like
> with i.e. DRBD setups:
>
>
http://backdrift.org/live-migration-and-synchronous-replicated-storage-with-xen-drbd-and-lvm
>
> except your filesystem "copy" is not there att of migration.
Depending
> from your filesystem implementation it could be necessary to hack within
> the migration code of xen - afaik there is no full scripting possibility or
> similiar to do such things like "do that before taking over a
migration" or
> similiar.
>
> You filesystem implementation sounds interesting - do you have any details
> about it? If i understand right it is NOT intended for HA replication
> (where both hosts requires full copies of full current filesystem) but i.e.
> for load management in a cluster.
>
>
> cheerioh,
>
> Niels.
>
> ---
> Niels Dettenbach
> Syndicat IT & Internet
> http://syndicat.com
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Am 29.05.2013 um 17:03 schrieb Nick Katsipoulakis <popanik@gmail.com>:
> The DRBD setup is complex and I do not think I need it.
This was just an example of how xen typically handles (or not handles) migration
with underlying filesystems.

Your concept sounds really nice - hope to see some usable results when ready. ;)


good luck!
best regards,


Niels.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Ohh I see.

Well thank you very much. I am currently on the evaluation process of my
system, which features many customization options for live migrating a VM
(you do not even have to use Xen on top of it). When I have actual results
of its performance I will be happy to post them on the Xen mailing lists.

Thank you again.

Cheers,
Nick


2013/5/29 Niels Dettenbach <nd@syndicat.com>
>
> Am 29.05.2013 um 17:03 schrieb Nick Katsipoulakis
<popanik@gmail.com>:
>
> The DRBD setup is complex and I do not think I need it.
>
> This was just an example of how xen typically handles (or not handles)
> migration with underlying filesystems.
>
> Your concept sounds really nice - hope to see some usable results when
> ready. ;)
>
>
> good luck!
> best regards,
>
>
> Niels.
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On Wed, 29 May 2013 18:03:07 +0300, Nick Katsipoulakis 
 <popanik@gmail.com> wrote:
> Niels thank you for your immediate answer and concern.
>
> Actually my FUSE Filesystem works as an abstraction layer between Xen
> and the underlying Filesystem (i.e. ext3). Everytime an I/O is
> performed by the VM, whose disk image is stored under my
filesystem''s
> mount point and is about to be migrated, the I/O is intercepted and
> actions are taken in order to ensure the consistency of the migrated
> disk image. It also manages to hand-over the VM and start its
> operation on the destination machine, before the whole disk image is
> migrated.
 Most interesting. Is this FS publicly available? It sounds similar
 to GlusterFS and SeznamFS. Can you explain the differences?

 Gordan

On Wed, 2013-05-29 at 17:26 +0300, Nick Katsipoulakis
wrote:
> 
> I am aware of the fact that the live migration feature of Xen needs to
> have shared storage for the disk images. 
This is not a fundamental requirement, it is mostly a per-toolstack
decision how to deal with this. Which toolstack are you (considering)
using?

Ian.

The default XM toolstack. Why is this considered a per-toolstack decision?
I want to configure my Xen instances (on the source and destination
machines) to look for the migrated disk.img in its local store, rather on a
mounted network storage (I am targeting my system for the Share-Nothing
IaaS architecture).

Cheers,
Nick


2013/5/30 Ian Campbell <Ian.Campbell@citrix.com>
> On Wed, 2013-05-29 at 17:26 +0300, Nick Katsipoulakis wrote:
> >
> > I am aware of the fact that the live migration feature of Xen needs to
> > have shared storage for the disk images.
>
> This is not a fundamental requirement, it is mostly a per-toolstack
> decision how to deal with this. Which toolstack are you (considering)
> using?
>
> Ian.
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On Thu, 2013-05-30 at 12:50 +0300, Nick Katsipoulakis
wrote:
> The default XM toolstack.
Note that the default became xl in 4.2 and xm is deprecated in 4.3.
>  Why is this considered a per-toolstack decision?
Because different toolstacks have different models for storage
management.

xm and xl don''t really care, so long as the block device exists on the
far side they don''t really have any interest in how it got there or
whether it is indeed the same disk as was present on the source (this is
the admin''s problem)

XenAPI/xapi have a storage management framework which knows about
different types of storage repository, and can deal with concepts like
shared storage or disks which can be migrated in an offline manner.

I don''t know how libvirt deals with this.
>  I want to configure my Xen instances (on the source and destination
> machines) to look for the migrated disk.img in its local store, rather
> on a mounted network storage (I am targeting my system for the
> Share-Nothing IaaS architecture).
For xl/xm this is just a path, if you can arrange for that path to refer
to the same thing they don''t care whether that is achieved through
network mounted storage or through black magic.

Ian.

I see. Then, I will try to tweak the path that refers to the disk image.

Thank you for your answer.

Cheers,
Nick


2013/5/30 Ian Campbell <Ian.Campbell@citrix.com>
> On Thu, 2013-05-30 at 12:50 +0300, Nick Katsipoulakis wrote:
> > The default XM toolstack.
>
> Note that the default became xl in 4.2 and xm is deprecated in 4.3.
>
> >  Why is this considered a per-toolstack decision?
>
> Because different toolstacks have different models for storage
> management.
>
> xm and xl don''t really care, so long as the block device exists on
the
> far side they don''t really have any interest in how it got there
or
> whether it is indeed the same disk as was present on the source (this is
> the admin''s problem)
>
> XenAPI/xapi have a storage management framework which knows about
> different types of storage repository, and can deal with concepts like
> shared storage or disks which can be migrated in an offline manner.
>
> I don''t know how libvirt deals with this.
>
> >  I want to configure my Xen instances (on the source and destination
> > machines) to look for the migrated disk.img in its local store, rather
> > on a mounted network storage (I am targeting my system for the
> > Share-Nothing IaaS architecture).
>
> For xl/xm this is just a path, if you can arrange for that path to refer
> to the same thing they don''t care whether that is achieved through
> network mounted storage or through black magic.
>
> Ian.
>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users