libvirt users - Mar 2016 - Why are virsh vol-upload/vol-download so slow?

I was interested in using 'virsh vol-upload' and 'virsh
vol-download',
rather than mucking about directly with the filesystem.  But while
simply copying the image is relatively quick:

  # ls -l /var/lib/libvirt/images/centos-7-cloud.qcow2
  -rw-r--r--. 1 qemu qemu 1004994560 Apr 15  2015 centos-7-cloud.qcow2
  # time cp /var/lib/libvirt/images/centos-7-cloud.qcow2 /tmp/myimage.qcow2
  real	0m10.829s
  user	0m0.024s
  sys	0m1.668s

Trying to do the same thing with `virsh vol-download` is glacial:

  # time virsh vol-download --pool default centos-7-cloud.qcow2
/tmp/myimage.qcow2

I killed it after 10 minutes:

  real	10m20.775s
  user	10m19.451s
  sys	0m1.887s

After which is was about halfway through:

  # ls -l /tmp/myimage.qcow2
  -rw-r--r--. 1 root root 564920248 Mar  4 11:11 /tmp/myimage.qcow2

What is up with that?

-- 
Lars Kellogg-Stedman <lars@redhat.com> | larsks @
{freenode,twitter,github}
Cloud Engineering / OpenStack          | http://blog.oddbit.com/

On Fri, Mar 04, 2016 at 11:14:44AM -0500, Lars Kellogg-Stedman
wrote:
> I was interested in using 'virsh vol-upload' and 'virsh
vol-download',
> rather than mucking about directly with the filesystem.  But while
> simply copying the image is relatively quick:
> 
>   # ls -l /var/lib/libvirt/images/centos-7-cloud.qcow2
>   -rw-r--r--. 1 qemu qemu 1004994560 Apr 15  2015 centos-7-cloud.qcow2
>   # time cp /var/lib/libvirt/images/centos-7-cloud.qcow2 /tmp/myimage.qcow2
>   real	0m10.829s
>   user	0m0.024s
>   sys	0m1.668s
> 
> Trying to do the same thing with `virsh vol-download` is glacial:
> 
>   # time virsh vol-download --pool default centos-7-cloud.qcow2
/tmp/myimage.qcow2
> 
> I killed it after 10 minutes:
> 
>   real	10m20.775s
>   user	10m19.451s
>   sys	0m1.887s
> 
> After which is was about halfway through:
> 
>   # ls -l /tmp/myimage.qcow2
>   -rw-r--r--. 1 root root 564920248 Mar  4 11:11 /tmp/myimage.qcow2
> 
> What is up with that?
What version of libvirt ?  We had an unfortunate implementation mistake
which caused us to waste tonnes of time in memmove(). versions since
1.2.19 have a fix that addresses it

commit d9c9e138f22c48626f719f880920e04c639e0177
Author: Ossi Herrala <oherrala@gmail.com>
Date:   Mon Jul 20 12:44:32 2015 +0000

    rpc: Fix slow volume download (virsh vol-download)
    
    Use I/O vector (iovec) instead of one huge memory buffer as suggested
    in https://bugzilla.redhat.com/show_bug.cgi?id=1026137#c7. This avoids
    doing memmove() to big buffers and performance doesn't degrade if
    source (virNetClientStreamQueuePacket()) is faster than sink
    (virNetClientStreamRecvPacket()).
    
    Resolves: http://bugzilla.redhat.com/1026137
    
    Signed-off-by: Martin Kletzander <mkletzan@redhat.com>

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

On Fri, Mar 04, 2016 at 04:57:30PM +0000, Daniel P. Berrange
wrote:
> What version of libvirt ?  We had an unfortunate implementation mistake
> which caused us to waste tonnes of time in memmove(). versions since
> 1.2.19 have a fix that addresses it
Ah, this is on F23, which has:

  libvirt-1.2.18.2-2.fc23.x86_64

There's nothing more recent right now in either updates or
updates-testing (and I was hoping to target (RHEL/CentOS)7, which have
libvirt-1.2.17-13.el7_2.2.x86_64).

Thanks,

-- 
Lars Kellogg-Stedman <lars@redhat.com> | larsks @
{freenode,twitter,github}
Cloud Engineering / OpenStack          | http://blog.oddbit.com/