Hi all, I am wondering, whether it''s possible to migrate a DomU from one XEN Host to another *without* using shared storage. All HowTos that I found so far, are using some kind of shared filesystem or block device. The DomU in question is using a physical device (partition /dev/sda1) that is exported as VBD from Dom0. If migration is possible with that setup, where will the DomU end up on the target system? In case, migrating this DomU won''t work, I would have to copy the VM manually. Is it possible to access the contents of /dev/sda1 from Dom0? Simply shutting down the DomU and mounting /dev/sda1 in Dom0 does not work. Regards, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Mar-31 20:38 UTC
RE: [Xen-users] migrating with physical device as VBD
Jan Schulze wrote:> > Hi all, > > I am wondering, whether it''s possible to migrate a DomU from one XEN Host to > another *without* using shared storage. All HowTos that I found so far, are > using some kind of shared filesystem or block device.Shared storage is mandatory, but it doesn''t have to be traditional multi-host scsi or fiber channel, with technology like iscsi and software like drbd shared storage can be done on the cheap.> The DomU in question is using a physical device (partition /dev/sda1) that is > exported as VBD from Dom0. If migration is possible with that setup, where will > the DomU end up on the target system?It needs to be accessible there too at the same time, thus why shared storage is mandatory. Check out drbd, the latest version does primary-primary which is what you need for live migrations.> In case, migrating this DomU won''t work, I would have to copy the VM manually. > Is it possible to access the contents of /dev/sda1 from Dom0? Simply shutting > down the DomU and mounting /dev/sda1 in Dom0 does not work.There will be a partition table within that partition, so you can run kpartx -a /dev/sda1, and access the first partition via /dev/mapper/sda11, or mount it via a loopback mount to the offset of that partition. # mount -t <fstype> -oloop,offset=<part offset> /dev/sda1 /mnt -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker wrote:> Jan Schulze wrote: >> Hi all, >> >> I am wondering, whether it''s possible to migrate a DomU from one XEN Host to >> another *without* using shared storage. All HowTos that I found so far, are >> using some kind of shared filesystem or block device. > > Shared storage is mandatory, but it doesn''t have to be traditional multi-host > scsi or fiber channel, with technology like iscsi and software like drbd > shared storage can be done on the cheap. > >> The DomU in question is using a physical device (partition /dev/sda1) that is >> exported as VBD from Dom0. If migration is possible with that setup, where will >> the DomU end up on the target system? > > It needs to be accessible there too at the same time, thus why shared storage > is mandatory. Check out drbd, the latest version does primary-primary which > is what you need for live migrations.DRBD seems really interesting. However, when using primary-primary, I would also need a cluster file system. As my existing DomU uses ext3, I won''t be able to use DRBD in a primary-primary setup, right?>> In case, migrating this DomU won''t work, I would have to copy the VM manually. >> Is it possible to access the contents of /dev/sda1 from Dom0? Simply shutting >> down the DomU and mounting /dev/sda1 in Dom0 does not work. > > There will be a partition table within that partition, so you can run > kpartx -a /dev/sda1, and access the first partition via /dev/mapper/sda11, > or mount it via a loopback mount to the offset of that partition. > > # mount -t <fstype> -oloop,offset=<part offset> /dev/sda1 /mntThanks, did not know that. Very useful. Regards, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-01 14:58 UTC
RE: [Xen-users] migrating with physical device as VBD
Jan Schulze wrote:> Ross S. W. Walker wrote: > > Jan Schulze wrote: > >> Hi all, > >> > >> I am wondering, whether it''s possible to migrate a DomU from one XEN Host to > >> another *without* using shared storage. All HowTos that I found so far, are > >> using some kind of shared filesystem or block device. > > > > Shared storage is mandatory, but it doesn''t have to be traditional multi-host > > scsi or fiber channel, with technology like iscsi and software like drbd > > shared storage can be done on the cheap. > > > >> The DomU in question is using a physical device (partition /dev/sda1) that is > >> exported as VBD from Dom0. If migration is possible with that setup, where will > >> the DomU end up on the target system? > > > > It needs to be accessible there too at the same time, thus why shared storage > > is mandatory. Check out drbd, the latest version does primary-primary which > > is what you need for live migrations. > > DRBD seems really interesting. However, when using primary-primary, I > would also need a cluster file system. As my existing DomU uses ext3, I > won''t be able to use DRBD in a primary-primary setup, right?No not in this case because the domUs file systems don''t come into play here. The Xen servers will have access to the domUs disks at the same time, but they will not write at the same time. Only 1 xen host has write access and during live migration they coordinate the hand-off of this access along with the network and running memory. Question is whether the drbd can work quick enough in this fashion to have the disks in sync by the time the hand-over is complete. You may need to up the pause state from say 50ms to 100ms just to give drbd enough time to complete the replication.> >> In case, migrating this DomU won''t work, I would have to copy the VM manually. > >> Is it possible to access the contents of /dev/sda1 from Dom0? Simply shutting > >> down the DomU and mounting /dev/sda1 in Dom0 does not work. > > > > There will be a partition table within that partition, so you can run > > kpartx -a /dev/sda1, and access the first partition via /dev/mapper/sda11, > > or mount it via a loopback mount to the offset of that partition. > > > > # mount -t <fstype> -oloop,offset=<part offset> /dev/sda1 /mnt > > Thanks, did not know that. Very useful.Always happy to pass the knowledge along as it was passed to me. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Antonio Mauriello
2008-Apr-03 10:48 UTC
Re: [Xen-users] migrating with physical device as VBD
Hi at all, as Jd knows, in my thesis project I''m developing a migration feature between not shared disks. So, when I start migration between two physical nodes, it''s copied also the virtual disk file (with the config file as ConVirt migration works). At moment the feature works but it''s needs many corrections (it needs a snapsot of state and restore in destination node yet). Basically the feature copy the virtual disk file before ConVirt migration stars, so when migrations_check starts it finds virtual disk file and doens''t go in error. Regards, Antonio Mauriello student at Federico II University, Napoli, Italy. On Tue, Apr 1, 2008 at 4:58 PM, Ross S. W. Walker <rwalker@medallion.com> wrote:> Jan Schulze wrote: > > Ross S. W. Walker wrote: > > > Jan Schulze wrote: > > >> Hi all, > > >> > > >> I am wondering, whether it''s possible to migrate a DomU from one XEN > Host to > > >> another *without* using shared storage. All HowTos that I found so > far, are > > >> using some kind of shared filesystem or block device. > > > > > > Shared storage is mandatory, but it doesn''t have to be traditional > multi-host > > > scsi or fiber channel, with technology like iscsi and software like > drbd > > > shared storage can be done on the cheap. > > > > > >> The DomU in question is using a physical device (partition /dev/sda1) > that is > > >> exported as VBD from Dom0. If migration is possible with that setup, > where will > > >> the DomU end up on the target system? > > > > > > It needs to be accessible there too at the same time, thus why shared > storage > > > is mandatory. Check out drbd, the latest version does primary-primary > which > > > is what you need for live migrations. > > > > DRBD seems really interesting. However, when using primary-primary, I > > would also need a cluster file system. As my existing DomU uses ext3, I > > won''t be able to use DRBD in a primary-primary setup, right? > > No not in this case because the domUs file systems don''t come into play > here. The Xen servers will have access to the domUs disks at the same > time, but they will not write at the same time. Only 1 xen host has > write access and during live migration they coordinate the hand-off > of this access along with the network and running memory. > > Question is whether the drbd can work quick enough in this fashion > to have the disks in sync by the time the hand-over is complete. You > may need to up the pause state from say 50ms to 100ms just to give > drbd enough time to complete the replication. > > > >> In case, migrating this DomU won''t work, I would have to copy the VM > manually. > > >> Is it possible to access the contents of /dev/sda1 from Dom0? Simply > shutting > > >> down the DomU and mounting /dev/sda1 in Dom0 does not work. > > > > > > There will be a partition table within that partition, so you can run > > > kpartx -a /dev/sda1, and access the first partition via > /dev/mapper/sda11, > > > or mount it via a loopback mount to the offset of that partition. > > > > > > # mount -t <fstype> -oloop,offset=<part offset> /dev/sda1 /mnt > > > > Thanks, did not know that. Very useful. > > Always happy to pass the knowledge along as it was passed to me. > > -Ross > > ______________________________________________________________________ > This e-mail, and any attachments thereto, is intended only for use by > the addressee(s) named herein and may contain legally privileged > and/or confidential information. If you are not the intended recipient > of this e-mail, you are hereby notified that any dissemination, > distribution or copying of this e-mail, and any attachments thereto, > is strictly prohibited. If you have received this e-mail in error, > please immediately notify the sender and permanently delete the > original and any copy or printout thereof. > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mark Williamson
2008-Apr-06 23:41 UTC
Re: [Xen-users] migrating with physical device as VBD
> I am wondering, whether it''s possible to migrate a DomU from one XEN Host > to another *without* using shared storage. All HowTos that I found so far, > are using some kind of shared filesystem or block device.Well, as others have said, basically you need the block device the guest is using to be available at the destination host by the time it gets there... As others have also noted, that doesn''t necessarily mean any horribly crazy shared cluster thingy. I guess there''s 3 things you might want and it''s worth clarifying which: 1) The /entire contents/ of the VBD moves to be stored on the new machine, the old storage can be discarded. 2) When the domU is on its "home" host it can access the block device using the normal front<->back connection with high performance. Some simple remote access setup is used when it''s migrated to a foreign host; but not using one of the very complex shared storage solutions out there. 3) You just don''t want to use a complex shared storage solution like a SAN, iSCSI or something performance-sapping like files on NFS (which is often bad practice anyhow, under Xen) For 1 there''s not currently a particularly easy solution. The closest is probably DRBD, since that avoids copying huge amounts of disk space across during the migration! DRBD has been used with Xen before, some people have had good experiences with it. You could always ask the DRBD mailing lists what the status of the multiple-primary support is and whether people have had success using it with Xen, if you want to check this option out. Number 2 could be done by hacking up a new, slightly more intelligent, block device setup script that would either use a local blkback if available or go to a network server if not. Number 3 could be done quite simply by running an enbd server to serve disk images, then putting something like (if memory serves correctly): disk = [''enbd:server:port, sda1, w''] in the config file. The control tools should automatically connect to the ENBD server correctly, including setting up a connection during a live migration. This should be quite simple to set up; you might want to test and see if the performance is OK for you.> In case, migrating this DomU won''t work, I would have to copy the VM > manually. Is it possible to access the contents of /dev/sda1 from Dom0? > Simply shutting down the DomU and mounting /dev/sda1 in Dom0 does not work.As noted by another responder, simply mounting this won''t work because it''s a whole disk image and not a filesystem image. I''ll just reiterate at this point that one should never ever mount the FS of a running (or currently paused / saved!) domU - preferably not even read-only. The safest way to mount a domU''s filesystem is to make sure the domU is shut down cleanly first. I expect you''re aware of it, but it''s worth emphasising ;-) Cheers, Mark -- Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users