I have been beating my head against a wall attempting to get virsh to migrate a simple VM from one host to another. I have two systems running Ubuntu 14.04 with libvirt 1.2.2. Yes, they are bridged. Non-shared storage. Modest guest VM that only requires 100G of disk space; there is far more available. In trying to accomplish even the simplest migration (per every page I can find from both the Fedore Project and Red Hat) I consistently get the following results: root@huey:/tmp# virsh list --all Id Name State ---------------------------------------------------- 9 e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f running - 81c50b5a-24ef-47d5-88ae-16ee7f4f9537 shut off - acd06610-c919-450b-97c8-8e49b9a0d45c shut off root@huey:/tmp# virsh migrate --live --p2p e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f qemu+ssh://10.10.11.204/system error: Failed to open file '/mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2': No such file or directory root@huey:/tmp# ls -al /mnt/store01/virt total 9140768 drwxr-xr-x 2 libvirt-qemu kvm 4096 Sep 3 10:17 . drwxr-xr-x 13 root root 4096 Aug 28 10:39 .. -rw-r--r-- 1 libvirt-qemu kvm 3271688192 Aug 29 14:39 81c50b5a-24ef-47d5-88ae-16ee7f4f9537.qcow2 -rw-r--r-- 1 libvirt-qemu kvm 2523136000 Aug 29 14:47 acd06610-c919-450b-97c8-8e49b9a0d45c.qcow2 -rw-r--r-- 1 libvirt-qemu kvm 3565486080 Sep 3 13:52 e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 root@dewey:~ >ls -al /mnt/store01/virt total 8 drwxrwxrwx 2 libvirt-qemu kvm 4096 Sep 3 12:30 . drwxr-xr-x 13 root root 4096 Aug 29 16:36 .. root@dewey:~ > (huey is 10.10.11.203, dewey is 10.10.11.204; DNS works, but I thought to eliminate that as a source of this problem.) As can be seen, the .qcow2 files exists on the source system, and does not exist on the target system. The directory shown is where images are to be stored. There is no problem starting and stopping VMs on the systems. It is not at all clear where the failure is occurring; there is no documentation that discusses this failure, of course. I believe that I am invoking a basic migration per every page I can find in a web search. Please correct any errors. I am working as root on both systems, and use ssh to get from one system to another, no passwords, no authentication required. All works well. Any observations, or suggestions as to what configuration parameters I should check, would be greatly appreciated. Thank you. Gary
Try creating a blank file on the target system at /mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 on 'dewey'. Migrations really don't go well when the target disk doesn't exist. I'm not certain why this is, I think the migration feature was mainly built with shared storage in mind. On 9/3/2014 3:05 PM, Gary Hook wrote:> I have been beating my head against a wall attempting to get virsh to > migrate a simple VM from one host to another. > > I have two systems running Ubuntu 14.04 with libvirt 1.2.2. Yes, they > are bridged. Non-shared storage. Modest guest VM that only requires > 100G of disk space; there is far more available. > > In trying to accomplish even the simplest migration (per every page I > can find from both the Fedore Project and Red Hat) I consistently get > the following results: > > root@huey:/tmp# virsh list --all > Id Name State > ---------------------------------------------------- > 9 e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f running > - 81c50b5a-24ef-47d5-88ae-16ee7f4f9537 shut off > - acd06610-c919-450b-97c8-8e49b9a0d45c shut off > > root@huey:/tmp# virsh migrate --live --p2p > e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f qemu+ssh://10.10.11.204/system > <http://10.10.11.204/system> > error: Failed to open file > '/mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2': No > such file or directory > > root@huey:/tmp# ls -al /mnt/store01/virt > total 9140768 > drwxr-xr-x 2 libvirt-qemu kvm 4096 Sep 3 10:17 . > drwxr-xr-x 13 root root 4096 Aug 28 10:39 .. > -rw-r--r-- 1 libvirt-qemu kvm 3271688192 Aug 29 14:39 > 81c50b5a-24ef-47d5-88ae-16ee7f4f9537.qcow2 > -rw-r--r-- 1 libvirt-qemu kvm 2523136000 Aug 29 14:47 > acd06610-c919-450b-97c8-8e49b9a0d45c.qcow2 > -rw-r--r-- 1 libvirt-qemu kvm 3565486080 Sep 3 13:52 > e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 > > root@dewey:~ >ls -al /mnt/store01/virt > total 8 > drwxrwxrwx 2 libvirt-qemu kvm 4096 Sep 3 12:30 . > drwxr-xr-x 13 root root 4096 Aug 29 16:36 .. > root@dewey:~ > > > (huey is 10.10.11.203, dewey is 10.10.11.204; DNS works, but I thought > to eliminate that as a source of this problem.) As can be seen, the > .qcow2 files exists on the source system, and does not exist on the > target system. The directory shown is where images are to be stored. > There is no problem starting and stopping VMs on the systems. It is > not at all clear where the failure is occurring; there is no > documentation that discusses this failure, of course. > > I believe that I am invoking a basic migration per every page I can > find in a web search. Please correct any errors. > > I am working as root on both systems, and use ssh to get from one > system to another, no passwords, no authentication required. All works > well. > > Any observations, or suggestions as to what configuration parameters I > should check, would be greatly appreciated. > > Thank you. > > Gary > > > _______________________________________________ > libvirt-users mailing list > libvirt-users@redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-users
On 09/03/2014 01:05 PM, Gary Hook wrote:> > root@huey:/tmp# virsh migrate --live --p2pThis command does not default to migrating storage,...> (huey is 10.10.11.203, dewey is 10.10.11.204; DNS works, but I thought to > eliminate that as a source of this problem.) As can be seen, the .qcow2 > files exists on the source system, and does not exist on the target system....therefore, this error message is expected if the storage is not already present on the destination. Migration requires either that you have shared storage (both source and destination can see THE SAME FILE, and if one side sees the file via NFS, then both sides must see the file via NFS [you can't mix local and NFS]). OR, you have to tell migration that you want to migrate storage as well as machine state, by adding the --copy-storage-all flag to your command line.> The directory shown is where images are to be stored. There is no problem > starting and stopping VMs on the systems. It is not at all clear where the > failure is occurring; there is no documentation that discusses this > failure, of course.http://libvirt.org/migration.html should document useful information such as this; would you like to submit a patch to help improve that web page? The sources are in libvirt.git under docs/migration.html.in. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 09/03/2014 01:12 PM, Brian Rak wrote: [please don't top-post on technical lists]> Try creating a blank file on the target system at > /mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 on 'dewey'.A pre-created blank file may be necessary if you are using --copy-storage-all (I'm not sure if libvirt has been patched to create the destination file automatically if it did not already exist), but is NOT going to work if you don't explicitly migrate storage. Since the default requires shared storage, and a blank file is not the same contents as the source, it would kill the guest. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wed, Sep 3, 2014 at 2:12 PM, Eric Blake <eblake@redhat.com> wrote:> On 09/03/2014 01:05 PM, Gary Hook wrote: > > > > > root@huey:/tmp# virsh migrate --live --p2p > > This command does not default to migrating storage,... >Oh, you mean like this? root@huey:/tmp# virsh migrate --live --p2p --copy-storage-all e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f qemu+ssh://10.10.11.204/system --verbose error: Failed to open file '/mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2': No such file or directory Apologies; I should have used this one in the first place. Too many attempts in the command history.... I'm guessing I"m still missing something...> > > > (huey is 10.10.11.203, dewey is 10.10.11.204; DNS works, but I thought to > > eliminate that as a source of this problem.) As can be seen, the .qcow2 > > files exists on the source system, and does not exist on the target > system. > > ...therefore, this error message is expected if the storage is not > already present on the destination. > > Migration requires either that you have shared storage (both source and > destination can see THE SAME FILE, and if one side sees the file via > NFS, then both sides must see the file via NFS [you can't mix local and > NFS]). OR, you have to tell migration that you want to migrate storage > as well as machine state, by adding the --copy-storage-all flag to your > command line. >Understood; perfectly sensible. See above.> http://libvirt.org/migration.html should document useful information > such as this; would you like to submit a patch to help improve that web > page? The sources are in libvirt.git under docs/migration.html.in. >Yep, read that page (several times). At such point I have anything of substance to add, absolutely, yes, I very much want to contribute. I am just starting out building my skills on KVM and libvirt (background is with PHYP on Power) but fully intend to become conversant with the packages.
On Wed, Sep 3, 2014 at 2:12 PM, Brian Rak <brak@gameservers.com> wrote:> Try creating a blank file on the target system at > /mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 on 'dewey'. >Yes, tried that before posting. At least, with a simple "touch" command.> Migrations really don't go well when the target disk doesn't exist. I'm > not certain why this is, I think the migration feature was mainly built > with shared storage in mind. >I appreciate that, but unless it's a documented and designed restriction, it seems it oughta work. And I know for a fact that it does (details which I will not include here). Thank you for your time; much appreciated.> On 9/3/2014 3:05 PM, Gary Hook wrote: > > I have been beating my head against a wall attempting to get virsh to > migrate a simple VM from one host to another. > > I have two systems running Ubuntu 14.04 with libvirt 1.2.2. Yes, they > are bridged. Non-shared storage. Modest guest VM that only requires 100G of > disk space; there is far more available. > > In trying to accomplish even the simplest migration (per every page I > can find from both the Fedore Project and Red Hat) I consistently get the > following results: > > root@huey:/tmp# virsh list --all > Id Name State > ---------------------------------------------------- > 9 e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f running > - 81c50b5a-24ef-47d5-88ae-16ee7f4f9537 shut off > - acd06610-c919-450b-97c8-8e49b9a0d45c shut off > > root@huey:/tmp# virsh migrate --live --p2p > e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f qemu+ssh://10.10.11.204/system > error: Failed to open file > '/mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2': No such > file or directory > > root@huey:/tmp# ls -al /mnt/store01/virt > total 9140768 > drwxr-xr-x 2 libvirt-qemu kvm 4096 Sep 3 10:17 . > drwxr-xr-x 13 root root 4096 Aug 28 10:39 .. > -rw-r--r-- 1 libvirt-qemu kvm 3271688192 Aug 29 14:39 > 81c50b5a-24ef-47d5-88ae-16ee7f4f9537.qcow2 > -rw-r--r-- 1 libvirt-qemu kvm 2523136000 Aug 29 14:47 > acd06610-c919-450b-97c8-8e49b9a0d45c.qcow2 > -rw-r--r-- 1 libvirt-qemu kvm 3565486080 Sep 3 13:52 > e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 > > root@dewey:~ >ls -al /mnt/store01/virt > total 8 > drwxrwxrwx 2 libvirt-qemu kvm 4096 Sep 3 12:30 . > drwxr-xr-x 13 root root 4096 Aug 29 16:36 .. > root@dewey:~ > > > (huey is 10.10.11.203, dewey is 10.10.11.204; DNS works, but I thought > to eliminate that as a source of this problem.) As can be seen, the .qcow2 > files exists on the source system, and does not exist on the target system. > The directory shown is where images are to be stored. There is no problem > starting and stopping VMs on the systems. It is not at all clear where the > failure is occurring; there is no documentation that discusses this > failure, of course. > > I believe that I am invoking a basic migration per every page I can find > in a web search. Please correct any errors. > > I am working as root on both systems, and use ssh to get from one system > to another, no passwords, no authentication required. All works well. > > Any observations, or suggestions as to what configuration parameters I > should check, would be greatly appreciated. > > Thank you. > > Gary > >
On 09/03/2014 04:10 PM, Gary Hook wrote:> On Wed, Sep 3, 2014 at 2:12 PM, Brian Rak <brak@gameservers.com> wrote: > >> Try creating a blank file on the target system at >> /mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2 on 'dewey'. >> > > Yes, tried that before posting. At least, with a simple "touch" command. > > >> Migrations really don't go well when the target disk doesn't exist. I'm >> not certain why this is, I think the migration feature was mainly built >> with shared storage in mind. >> > > I appreciate that, but unless it's a documented and designed restriction, > it seems it oughta work. And I know for a fact that it does (details which > I will not include here). > > Thank you for your time; much appreciated.Shared storage migration was implemented first. Non-shared migration was added after the fact, so you have to explicitly request it; and the fact that libvirt still can't create the destination file itself is an annoying limitation that we'd like to lift someday. It would also be nice if libvirt could detect at least obvious cases of the destination being incorrect (such as different file sizes) - except that in SOME cases, you CAN migrate to different sizes (if the source is a single raw file, but the destination is a brand-new qcow2 file with the original as its backing, then the guest contents will be identical from either source or destination perspective up until the point the destination starts writing into the qcow2 wrapper file). So I'm not sure if any heuristics can be added to help diagnose obvious problems. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 09/03/2014 04:09 PM, Gary Hook wrote:> On Wed, Sep 3, 2014 at 2:12 PM, Eric Blake <eblake@redhat.com> wrote: > >> On 09/03/2014 01:05 PM, Gary Hook wrote: >> >>> >>> root@huey:/tmp# virsh migrate --live --p2p >> >> This command does not default to migrating storage,... >> > > Oh, you mean like this? > > root@huey:/tmp# virsh migrate --live --p2p --copy-storage-all > e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f qemu+ssh://10.10.11.204/system > --verboseYes.> > > error: Failed to open file > '/mnt/store01/virt/e7f75b9b-9ed4-4f7e-aa86-e481ab911d6f.qcow2': No such > file or directoryAlas, _this_ is the case mentioned where you have to pre-create the destination file, because libvirt hasn't yet been taught to auto-create it. But with --copy-storage-all in use, you should now be fine just touch'ing the file before attempting the migration.>> http://libvirt.org/migration.html should document useful information >> such as this; would you like to submit a patch to help improve that web >> page? The sources are in libvirt.git under docs/migration.html.in. >> > > Yep, read that page (several times). At such point I have anything of > substance to add, absolutely, yes, I very much want to contribute. I am > just starting out building my skills on KVM and libvirt (background is with > PHYP on Power) but fully intend to become conversant with the packages.Welcome to the community, and hope to hear more from you! -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org