Hi, I''ve been looking at various techniques to allow the the migration of the contents of the block device that a virtual machine is using from one host to another. Our aim is to allow the complete migration of a running virtual machine to its target including all the local persistent storage it is using (i.e. operating system image, user data, etc). One approach to achieve this I have heard mentioned is to proxy the local interactions with the backing block device through a degraded RAID1 array (using software RAID). The destination server then exports its local destination block device (e.g. loopback file or LVM) via NBD (network block device). This block device is hot added to the degraded array and the data is synced from the original block device to the destination. When the domain is migrated (using xm migrate) it can then use the local block device from that point on. In order to allow future migration from this point on the virtual machine must still access the block device here via a proxying degraded RAID1 array. Has anyone successfully done this? My initial attempts seem to show that it should work in practise but I experienced some problems with corrupted metadata in the filesystem. A second approach that we have devised involves adding a small hook to the vbd backend driver. This callback is fired in dispatch_rw_block_io of the blkback driver. I then have a small driver that hooks a function pointer into this callback and then records the struct phys_req in a linked list. This is then access from userspace via a simple character device. A userspace daemon then uses this information to read the blocks that have changed from the physical device attached to the vbd. These blocks are then sent to the destination server and then written to the block device there. Since we are using file backed vbds we can make the initial transfer using some existing mechanism (scp, netcat). My main worry is that when I read from the loopback device for the file that it might not reflect the update that has caused me to read the block in the first place. Will a read from the /dev/loop? always return the most recent version that the vbd backend has written? Any further comments on this idea would be much appreciated. Cheers, Rob -- Rob Bradford <rob.bradford@xenoservers.net> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2006-08-03 at 15:08 +0200, Rob Bradford wrote:> One approach to achieve this I have heard mentioned is to proxy the > local interactions with the backing block device through a degraded > RAID1 array (using software RAID). The destination server then exports > its local destination block device (e.g. loopback file or LVM) via NBD > (network block device). This block device is hot added to the degraded > array and the data is synced from the original block device to the > destination. When the domain is migrated (using xm migrate) it can then > use the local block device from that point on. In order to allow future > migration from this point on the virtual machine must still access the > block device here via a proxying degraded RAID1 array. > > Has anyone successfully done this? My initial attempts seem to show that > it should work in practise but I experienced some problems with > corrupted metadata in the filesystem.Yes I have, but that was a couple of years ago, using self-migration. It worked well back then, except, as you describe, that I had no way of ''hotswapping'' from iSCSI to the local Xen-backed version of the disk device. Also, my minimized boot loader did not have disk access, so I had to rely on an external domain providing the iSCSI service. Jacob _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel