libvirt users - Jul 2011 - Associating a statefile with an image

Hi ,

I am trying to perform offline migration (i.e) Create an incremental image using
the qcow format, transfer the vm memory state to a state fie.Use the image and
statefile together as a template. Now create a new vm using the template. I can
successfully do it using the following commands :

Save phase :
stop
migrate "exec:gzip -c > STATEFILE.gz"
qemu-img 
qemu-img create -b BASE_img -f qcow2 INCRE_img

Restore phase :
qemu-kvm -m 1024 -hda INCRE_img -incoming "exec: gzip -c -d
STATEFILE.gz"

And it works fine. But I am not able to find the vm with virt-manager or other
libvirt based tools.

If I use : 
virsh save <dom_id> STATEFILE

I can restore using "virsh restore STATEFILE" but I want to associate
this with the incremental image that I created, but I dont know how. I think it
saves the existing vm's xml file along with the STATEFILE, thus I am not
able to change the disk image to the incremental image.

Thus ,
1. Either I should find a way to make vms created using qemu-kvm appear in
libvirt-based tools.

OR 

2. Find a way to associate the virsh save STATEFILE with a incremental image. 

Any help or hint with respect to these will be very helpful. I am try to do
cloning with minimal cost. This takes only 25 seconds ( to create STATEFILE) and
creating incremental image is instantatenous . But I want to use the vm
monitoring code based on libvirt. And it makes life difficult.

Thanks for your help. 

-
Regards,
Sethuraman Subbiah

Graduate Student - NC state University
M.S in Computer Science


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20110702/140f8400/attachment.htm>

On 07/02/2011 09:59 AM, sethuraman subbiah wrote:
> Hi ,
> 
> I am trying to perform offline migration (i.e) Create an incremental image
using the qcow format, transfer the vm memory state to a state fie.Use the image
and statefile together as a template. Now create a new vm using the template. I
can successfully do it using the following commands :
> 
> Save phase :
> stop
> migrate "exec:gzip -c > STATEFILE.gz"
> qemu-img 
> qemu-img create -b BASE_img -f qcow2 INCRE_img
> 
> Restore phase :
> qemu-kvm -m 1024 -hda INCRE_img -incoming "exec: gzip -c -d
STATEFILE.gz"
> 
> And it works fine. But I am not able to find the vm with virt-manager or
other libvirt based tools.
> 
> If I use : 
> virsh save <dom_id> STATEFILE
> 
> I can restore using "virsh restore STATEFILE" but I want to
associate this with the incremental image that I created, but I dont know how. I
think it saves the existing vm's xml file along with the STATEFILE, thus I
am not able to change the disk image to the incremental image.
> 
> Thus ,
> 1. Either I should find a way to make vms created using qemu-kvm appear in
libvirt-based tools.
There is a patch series pending to teach libvirt how to attach to a vm
created by raw invocation of qemu-kvm:
https://www.redhat.com/archives/libvir-list/2011-July/msg00111.html

But ultimately, our preference is to do everything through libvirt, to
isolate you from incompatible changes in qemu-kvm or to ease the
transition to any other type of hypervisor that supports similar
functionality.
> 
> OR 
> 
> 2. Find a way to associate the virsh save STATEFILE with a incremental
image.
Oh, I see what you mean.  Right now, the 'virsh save' statefile, _as
part of its saved state_, includes a verbatim copy of the XML in use at
the time the snapshot was created.  So even if you do 'virsh save', then
manually use 'qemu-img' to create incremental snapshots on top of those
images (or even my proposed RFC for virStorageVolSnapshot APIs for
managing incremental images from virsh rather than having to use
qemu-img:
https://www.redhat.com/archives/libvir-list/2011-June/msg00761.html),
you are still constrained in how to get those new disk images injected
back into the state file.  You can't just do 'virsh edit dom' to
change
the name of the files that are backing each disk device, because 'virsh
restore' will _undo_ those changes as part of reverting back to the XML
that was stored verbatim in the statefile.

Your only recourse at the moment is to actually edit the XML embedded
within the statefiles, which is not appealing (text-editing a binary
file is dangerous).

This sounds like something worth working into the existing
virDomainSnapshot APIs; right now, when invoked on an inactive domain,
those APIs create _internal_ qcow2 snapshots (using qemu-img under the
hood for all disks tied to the domain); but as you are pointing out, it
is the _external_ incremental snapshot layout (a new file with the old
file serving as the backing image) that may better suit your needs.
Furthermore, that fits in with the fact that I'm trying to extend
virDomainSnapshot APIs to support an upcoming qemu monitor command for
online disk snapshots, and that monitor command will only support
external snapshots, so I already have to figure out how to make that
command manage the aspect of changing the file name associated with a
disk to swap over to the new incremental image.  Therefore, it seems
like whether online (new qemu monitor command) or offline, it makes
sense to enhance the virDomainSnapshot behavior to support external
images, at which point you will have the behavior of properly
associating a statefile with new file names for incremental images.

Meanwhile, it may mean that it might be worth adding some APIs to make
it easier to edit the XML embedded within a statefile image.  We
recently introduced the virDomainMigrate2 API, which takes an optional
XML rendition to use on the destination, and which is validated against
the original XML as being guest-compatible (that is, items related to
the host, such as a file name used as the backing image for a disk, can
be altered, while items related to the guest, such as the fact that the
guest expects a virtual disk in the first place, must not be altered).
So a new API that lets you provide an alternate XML for use with an
existing state file could use the same XML compatibility checking, to
ensure that the XML being injected into the state file is safe; this
would allow you to change the name of a host file to use for the guest
disk on the next 'virsh restore'.  Certainly some food for thought that
I'll keep in mind as I work on snapshot code.
> 
> Any help or hint with respect to these will be very helpful.
It's a work under progress, so keep your thoughts coming to make sure we
are not precluding useful modes of operation as we introduce new APIs or
new flags to existing APIs.
> I am try to do cloning with minimal cost. This takes only 25 seconds ( to
create STATEFILE) and creating incremental image is instantatenous . But I want
to use the vm monitoring code based on libvirt. And it makes life difficult.
> 
-- 
Eric Blake   eblake at redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 619 bytes
Desc: OpenPGP digital signature
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20110705/2567fb66/attachment.sig>

[re-adding the list]

On 08/02/2011 03:18 PM, sethuraman subbiah wrote:
>> Libvirt shoots for monthly releases; the current schedule is feature
freeze this weekend, then a release around the first of August.
>>
>> The new API will allow you do to either:
>>
>> virsh dumpxml --inactive --dom>  file.xml
>> edit file.xml
>> virsh save dom dom.save --xml file.xml
>> virsh restore file.save
>>
>> or:
>>
>> virsh dumpxml --inactive --dom>  file.xml
Using --inactive here only works if you haven't made any changes to the 
inactive configuration that differs from the currently running domain. 
I'd amend this step to 'virsh dumpxml dom > file.xml' for both of
these
examples.
>> virsh save dom dom.save
>> edit file.xml
>> virsh restore file.save --xml file.xml
These two examples will be supported in 0.9.4.  Additionally, I added:

virsh save-file-dumpxml dom.save > file.xml
virsh save-file-define dom.save file.xml
virsh save-file-edit dom.save

to do things inline, and to make it so you don't have to remember to 
'virsh dumpxml dom' prior to 'virsh save'.
>
> I found that the latest libvirt (libvirt 0.9.4) is out.
That's amazing, considering it hasn't been released yet :)  Maybe you 
are referring to 0.9.4-rc2 instead?  But at any rate, the official 0.9.4 
will probably be released tomorrow with even more bug fixes above 0.9.4-rc2.

 > I tried installing in my Centos 5.6 machine. After a successful 
installation, when I start virsh , I get :
>
> virsh: /lib/libvirt.so.0: version `LIBVIRT_PRIVATE_0.8.2' not found
(required by virsh)
>
> Is it possible to get the latest version of libvirt 0.9.4 work on Centos
5.6 ?  Thanks a lot for your help.
Yes, it should be possible, although I haven't tried it myself on centos 
(but I have tried on RHEL 5.7).  Are you sure you installed your 
self-built libvirt.so correctly?  We take great pains to ensure that 
libvirt.so is backwards compatible (no symbols are ever removed).

-- 
Eric Blake   eblake at redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org