libvirt users - Apr 2016 - Re: stream finish throws exception via python API

On 04/26/2016 09:35 AM, Shahar Havivi wrote:
> On 26.04.16 15:30, Shahar Havivi wrote:
>> On 26.04.16 14:14, Shahar Havivi wrote:
>>> On 25.04.16 09:11, Cole Robinson wrote:
>>>> On 04/25/2016 08:10 AM, Shahar Havivi wrote:
>>>>> On 17.04.16 15:41, Shahar Havivi wrote:
>>>>>> Hi,
>>>>>> The following snippet works fine e.g. receiving the
data but when calling
>>>>>> stream.finish() we get the following error:
>>>>>>
>>>>>> stream = con.newStream()
>>>>>> vol.download(stream, 0, 0, 0)
>>>>>> buf = stream.recv(1024)
>>>>>> stream.finish()
>>>>>>
>>>>>> libvirt: I/O Stream Utils error : internal error: I/O
helper exited abnormally
>>>>>> Traceback (most recent call last):
>>>>>>   File "./helpers/kvm2ovirt", line 149, in
<module>
>>>>>>     download_volume(vol, item[1], diskno, disksitems,
pksize)
>>>>>>   File "./helpers/kvm2ovirt", line 102, in
download_volume
>>>>>>     stream.finish()
>>>>>>   File
"/usr/lib64/python2.7/site-packages/libvirt.py", line 5501, in finish
>>>>>>     if ret == -1: raise libvirtError
('virStreamFinish() failed')
>>>>>> libvirt.libvirtError: internal error: I/O helper exited
abnormally
>>>>>>  
>>>>
>>>> The error message sucks, I'll send patches to improve it a
little at least.
>>>>
>>>> What's happening here is that the you haven't read all
the data you requested
>>>> (it's vol.download(path, offset, length, flags), length ==
0 means read the
>>>> whole file which I suspect you haven't done). In this case
the iohelper
>>>> program that libvirt uses won't complete feeding us all the
data, and it exits
>>>> with SIGPIPE when we close the read end of the pipe.
>>>>
>>>> Now whether that should actually be an error condition is open
to debate.
>>>> virStreamFinish docs make it sound like it's legitimate to
throw an error if
>>>> it appears that all requested data wasn't read.
>>> Thanks checking...
>> Thanks Cole,
>> I modify our script and for checking if stream.recv() returns zero the
finish
>> works fine.
>> Our API needs to know the size of bytes that you are going to stream
back
>> which is not provided by the allocation and capacity.
>> (ie the same as du -s /path/to/disk)
>>
>>   Shahar.
> We need the size of the image "Image end offset" you get from
"qemu-img check",
> Does libvirt have an API for that?
> 
Libvirt doesn't invoke qemu-img check anywhere AFAIK, so if that's the
only
way to get that info, then it isn't available

- Cole

On Tue, Apr 26, 2016 at 4:37 PM, Cole Robinson <crobinso@redhat.com>
wrote:
> On 04/26/2016 09:35 AM, Shahar Havivi wrote:
>> On 26.04.16 15:30, Shahar Havivi wrote:
>>> On 26.04.16 14:14, Shahar Havivi wrote:
>>>> On 25.04.16 09:11, Cole Robinson wrote:
>>>>> On 04/25/2016 08:10 AM, Shahar Havivi wrote:
>>>>>> On 17.04.16 15:41, Shahar Havivi wrote:
>>>>>>> Hi,
>>>>>>> The following snippet works fine e.g. receiving the
data but when calling
>>>>>>> stream.finish() we get the following error:
>>>>>>>
>>>>>>> stream = con.newStream()
>>>>>>> vol.download(stream, 0, 0, 0)
>>>>>>> buf = stream.recv(1024)
>>>>>>> stream.finish()
>>>>>>>
>>>>>>> libvirt: I/O Stream Utils error : internal error:
I/O helper exited abnormally
>>>>>>> Traceback (most recent call last):
>>>>>>>   File "./helpers/kvm2ovirt", line 149,
in <module>
>>>>>>>     download_volume(vol, item[1], diskno,
disksitems, pksize)
>>>>>>>   File "./helpers/kvm2ovirt", line 102,
in download_volume
>>>>>>>     stream.finish()
>>>>>>>   File
"/usr/lib64/python2.7/site-packages/libvirt.py", line 5501, in finish
>>>>>>>     if ret == -1: raise libvirtError
('virStreamFinish() failed')
>>>>>>> libvirt.libvirtError: internal error: I/O helper
exited abnormally
>>>>>>>
>>>>>
>>>>> The error message sucks, I'll send patches to improve
it a little at least.
>>>>>
>>>>> What's happening here is that the you haven't read
all the data you requested
>>>>> (it's vol.download(path, offset, length, flags), length
== 0 means read the
>>>>> whole file which I suspect you haven't done). In this
case the iohelper
>>>>> program that libvirt uses won't complete feeding us all
the data, and it exits
>>>>> with SIGPIPE when we close the read end of the pipe.
>>>>>
>>>>> Now whether that should actually be an error condition is
open to debate.
>>>>> virStreamFinish docs make it sound like it's legitimate
to throw an error if
>>>>> it appears that all requested data wasn't read.
>>>> Thanks checking...
>>> Thanks Cole,
>>> I modify our script and for checking if stream.recv() returns zero
the finish
>>> works fine.
>>> Our API needs to know the size of bytes that you are going to
stream back
>>> which is not provided by the allocation and capacity.
>>> (ie the same as du -s /path/to/disk)
>>>
>>>   Shahar.
>> We need the size of the image "Image end offset" you get from
"qemu-img check",
>> Does libvirt have an API for that?
>>
>
> Libvirt doesn't invoke qemu-img check anywhere AFAIK, so if that's
the only
> way to get that info, then it isn't available
We would like to report progress for downloads, so we need to know in advance
what is the size of the download.

I guess we can use an estimate (e.g. capacity * 1.1), or maybe someone have
a better idea how to estimate the download size?

A second issue, libvirt does not seem to understand sparseness, so we
are copying
around gigabytes of zeros.

For example, I created 102M empty image (using virt manager):

$ qemu-img info /home/nsoffer/var/libvirt/images/tiny.qcow2
image: /home/nsoffer/var/libvirt/images/tiny.qcow2
file format: qcow2
virtual size: 102M (107374592 bytes)
disk size: 304K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

Downloading it using the attach test script:

$ python download.py /home/nsoffer/var/libvirt/images/tiny.qcow2
written 107741184 bytes

$ qemu-img info download.qcow2
image: download.qcow2
file format: qcow2
virtual size: 102M (107374592 bytes)
disk size: 103M
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

Same operation using qemu-img convert:

$ qemu-img convert -O qcow2
/home/nsoffer/var/libvirt/images/tiny.qcow2 convert.qcow2

$ qemu-img info convert.qcow2
image: convert.qcow2
file format: qcow2
virtual size: 102M (107374592 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

So with libvirt we downloaded 102M, and qemu copied only 196K.

Nir

On 04/26/2016 02:56 PM, Nir Soffer wrote:
> On Tue, Apr 26, 2016 at 4:37 PM, Cole Robinson <crobinso@redhat.com>
wrote:
>> On 04/26/2016 09:35 AM, Shahar Havivi wrote:
>>> On 26.04.16 15:30, Shahar Havivi wrote:
>>>> On 26.04.16 14:14, Shahar Havivi wrote:
>>>>> On 25.04.16 09:11, Cole Robinson wrote:
>>>>>> On 04/25/2016 08:10 AM, Shahar Havivi wrote:
>>>>>>> On 17.04.16 15:41, Shahar Havivi wrote:
>>>>>>>> Hi,
>>>>>>>> The following snippet works fine e.g. receiving
the data but when calling
>>>>>>>> stream.finish() we get the following error:
>>>>>>>>
>>>>>>>> stream = con.newStream()
>>>>>>>> vol.download(stream, 0, 0, 0)
>>>>>>>> buf = stream.recv(1024)
>>>>>>>> stream.finish()
>>>>>>>>
>>>>>>>> libvirt: I/O Stream Utils error : internal
error: I/O helper exited abnormally
>>>>>>>> Traceback (most recent call last):
>>>>>>>>   File "./helpers/kvm2ovirt", line
149, in <module>
>>>>>>>>     download_volume(vol, item[1], diskno,
disksitems, pksize)
>>>>>>>>   File "./helpers/kvm2ovirt", line
102, in download_volume
>>>>>>>>     stream.finish()
>>>>>>>>   File
"/usr/lib64/python2.7/site-packages/libvirt.py", line 5501, in finish
>>>>>>>>     if ret == -1: raise libvirtError
('virStreamFinish() failed')
>>>>>>>> libvirt.libvirtError: internal error: I/O
helper exited abnormally
>>>>>>>>
>>>>>>
>>>>>> The error message sucks, I'll send patches to
improve it a little at least.
>>>>>>
>>>>>> What's happening here is that the you haven't
read all the data you requested
>>>>>> (it's vol.download(path, offset, length, flags),
length == 0 means read the
>>>>>> whole file which I suspect you haven't done). In
this case the iohelper
>>>>>> program that libvirt uses won't complete feeding us
all the data, and it exits
>>>>>> with SIGPIPE when we close the read end of the pipe.
>>>>>>
>>>>>> Now whether that should actually be an error condition
is open to debate.
>>>>>> virStreamFinish docs make it sound like it's
legitimate to throw an error if
>>>>>> it appears that all requested data wasn't read.
>>>>> Thanks checking...
>>>> Thanks Cole,
>>>> I modify our script and for checking if stream.recv() returns
zero the finish
>>>> works fine.
>>>> Our API needs to know the size of bytes that you are going to
stream back
>>>> which is not provided by the allocation and capacity.
>>>> (ie the same as du -s /path/to/disk)
>>>>
>>>>   Shahar.
>>> We need the size of the image "Image end offset" you get
from "qemu-img check",
>>> Does libvirt have an API for that?
>>>
>>
>> Libvirt doesn't invoke qemu-img check anywhere AFAIK, so if
that's the only
>> way to get that info, then it isn't available
> 
> We would like to report progress for downloads, so we need to know in
advance
> what is the size of the download.
> 
> I guess we can use an estimate (e.g. capacity * 1.1), or maybe someone have
> a better idea how to estimate the download size?
> 
Hmm, I didn't realize <capacity> for a qcow2 volume isn't the full
size on
disk, but instead the size of the virtual disk image. We should probably add
an extra volume field to report the actual on disk size too. Please file a
RHEL bug
> A second issue, libvirt does not seem to understand sparseness, so we
> are copying
> around gigabytes of zeros.
> 
The sparseness issue is being worked on presently:

https://bugzilla.redhat.com/show_bug.cgi?id=1282859

- Cole