Xen devel - Mar 2006 - [PATCH] External device migration support

This patch enables external devices, such as for example a mounted hard
drive image or a TPM, to be migrated to a remote machine. The patch
hooks into the checkpointing (XendCheckpoint.py) code and performs
migration in 4 different steps:

In a 1st step (step = 0 in the code) migration of all devices of a
domain is ''tested'', that means their driver implementations
(blkif.py,
netif.py, tpmif.py, usbif.py, pciif.py) are queried whether migration is
possible at all. Currently all device representations respond with a
''yes'' (=0), although probably a VM mounting a hard drive
partition
should respond with a ''no'' (-1) already. This first step is a
quick
check to see whether devices can be migrated.

The 2nd step is to do whatever can be done before the domain is
suspended. At this point migration of the device could be initiated, if
at all possible.

The 3rd step is to migrate a device after the domain has been suspended,
meaning that it is not scheduled anymore and the VM is
''settled''. All
devices are called again and a good implementation would initiate the
migration in a background process to achieve as much concurrency as
possible.

The 4th step is to synchronize with the 3rd step. At this point the
implementor has to make sure that anything that was initiated in step 3
has completed. Once all steps 4 have been processed, the VM will resume
on the remove machine.

I have implemented hooks for migration of a TPM. These hooks for now
call the os.system() call with a configurable tool but a fixed parameter
set and refuse to migrate a VM attached to a virtual TPM if no tool has
been provided for migration.
All other devices do not currently overload the ''migrate''
method defined
in the DevController.py and therefor will just let migration happen.

Please let me know what you think about this idea.

Signed-off-by: Stefan Berger <stefanb@us.ibm.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Berger wrote:
> This patch enables external devices, such as for example a mounted hard
> drive image or a TPM, to be migrated to a remote machine. The patch
> hooks into the checkpointing (XendCheckpoint.py) code and performs
> migration in 4 different steps:
...
> Please let me know what you think about this idea.
I really like the idea because it seems to work for any external device that 
can be migrated but does not allow concurrent access from both migration 
source and destination.

I have not looked at the patch in detail, but I saw the specific inclusion 
of
+# The tool used for initiating virtual TPM migration
+#(vtpm-migration-tool '''')

Could this migration tool specification be more generalized so that any 
device migration can be configured, not just vtpm devices?

Best Regards,
Michael Paesold



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

"Michael Paesold" <mpaesold@gmx.at> wrote on 03/27/2006 08:31:08
AM:
> Stefan Berger wrote:
> 
> > This patch enables external devices, such as for example a mounted 
hard
> > drive image or a TPM, to be migrated to a remote machine. The patch
> > hooks into the checkpointing (XendCheckpoint.py) code and performs
> > migration in 4 different steps:
> ...
> > Please let me know what you think about this idea.
> 
> I really like the idea because it seems to work for any external device 
that 
> can be migrated but does not allow concurrent access from both migration 
> source and destination.
> 
You mean that for example concurrent access to a disk image file is not 
possible by both systems?
> I have not looked at the patch in detail, but I saw the specific 
inclusion 
> of
> +# The tool used for initiating virtual TPM migration
> +#(vtpm-migration-tool '''')
> 
> Could this migration tool specification be more generalized so that any 
> device migration can be configured, not just vtpm devices?
I am going to change this and make this one single
''migration-tool''. I
will have that tool know by command line parameter (s.th. like ''-type 
vtpm'') what device the call for migration is about. If the external
tool
is a script you can switch over the ''-type'' parameter and call
another
external program that handles that particular device type, otherwise that 
external tool needs to know how to handle migration of all kinds of 
devices. I will change this and repost.

  Stefan
> 
> Best Regards,
> Michael Paesold
> 
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Berger wrote:
> 
> "Michael Paesold" <mpaesold@gmx.at> wrote on 03/27/2006
08:31:08 AM:
> 
>  > Stefan Berger wrote:
>  >
>  > > This patch enables external devices, such as for example a
mounted hard
>  > > drive image or a TPM, to be migrated to a remote machine. The
patch
>  > > hooks into the checkpointing (XendCheckpoint.py) code and
performs
>  > > migration in 4 different steps:
>  > ...
>  > > Please let me know what you think about this idea.
>  >
>  > I really like the idea because it seems to work for any external 
> device that
>  > can be migrated but does not allow concurrent access from both
migration
>  > source and destination.
>  >
> You mean that for example concurrent access to a disk image file is not 
> possible by both systems?
I was thinking of a DRBD-backed (see http://www.drbd.org) partition that is 
replicated between two hosts (RAID-over-network). In the current version, 
this is a failover-only soluation, so only one node of a two-node cluster 
can activate the partition in r/w mode.

So if you want to migrate from host A to host B, you first have to disable 
access on host A, and only then activate access on host B. As I understood, 
this should be possible with your proposed solution. (I hope I am right)

Another use case would be a root-partition with a cow device, where you 
would have to copy the cow to the other host. If the cow is small, this 
seem reasonable, but again, it must happen after the domain is paused at 
host A and before the domain is unpaused on host B.
>  > I have not looked at the patch in detail, but I saw the specific 
> inclusion
>  > of
>  > +# The tool used for initiating virtual TPM migration
>  > +#(vtpm-migration-tool '''')
>  >
>  > Could this migration tool specification be more generalized so that
any
>  > device migration can be configured, not just vtpm devices?
> 
> I am going to change this and make this one single
''migration-tool''. I
> will have that tool know by command line parameter (s.th. like
''-type
> vtpm'') what device the call for migration is about. If the
external tool
> is a script you can switch over the ''-type'' parameter and
call another
> external program that handles that particular device type, otherwise 
> that external tool needs to know how to handle migration of all kinds of 
> devices. I will change this and repost.
Sounds good to me.

Best Regards,
Michael Paesold

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel