Libguestfs - Apr 2018 - Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

On Thu, Apr 12, 2018 at 2:07 AM Nir Soffer <nirsof@gmail.com> wrote:
> On Tue, Apr 10, 2018 at 6:53 PM Richard W.M. Jones
<rjones@redhat.com>
> wrote:
>
...
> Dan Berrange pointed out earlier on that it might be easier if imageio
>>
> just exposed NBD, or if we found a way to tunnel NBD requests over web
>> sockets (in the format case nbdkit would not be needed, in the latter
>> case nbdkit could act as a bridge).
>
>
I think we can expose NBD using ndb-server and dynamic exports.
It can work like this:

0. Install nbd and enable nbd-server on a host, running
   as vdsm:kvm, not exporting anything.

1. User starts transfer session via oVirt API with protocol="nbd"
    (if not specified, use "https" for backward compatibility)

2. oVirt selects a host for the transfer and configures an export
   in that host - for example:

[bbff5ab7-2e8e-45b8-a497-423c2d2b3ff1]
exportname = /path/to/image
trim = yes
timeout = 300

nbd-server supports reloading configuration via SIGHUP.
Looks like it should work for adding export dynamically:
https://github.com/NetworkBlockDevice/nbd/blob/cb6e816c4d2c536cfff5ba9859f210aabe39539b/nbd-server.c#L3007

I could find code handling removing exports.

3. User gets back
- transfer_url: "nbd:hostname:port"
- export_name: bbff5ab7-2e8e-45b8-a497-423c2d2b3ff1

Assuming that if NBD_OPT_EXPORT_NAME is bbff5ab7-2e8e-45b8-a497-423c2d2b3ff1
ndb-server will export the /path/to/image mentioned in the config
exportname.

This seems to be the case based on:
- https://github.com/NetworkBlockDevice/nbd/blob/master/nbd-server.c#L998
-
https://github.com/NetworkBlockDevice/nbd/blob/cb6e816c4d2c536cfff5ba9859f210aabe39539b/nbd-server.c#L2074

4. User transmit data using nbd-client or qemu-img or maybe
   a wrapper script to make it easier

5. During the transfer, oVirt monitors progress and activity.

I don't see how we can do this with current nbd-server.

6. User ends the transfer session via oVirt API

7. oVirt removes the export and finalize the transfer.

I guess this can work in a similar way for KubeVirt. The way
the image is provisioned and attached to a host will be
different, but the way a user will transfer image data can be
the same.

Seems that nbd-server is missing:
- way to remove exports dynamically
- way to monitor export activity and progress

Nir

On Thu, Apr 12, 2018 at 09:22:16AM +0000, Nir Soffer
wrote:
> I think we can expose NBD using ndb-server and dynamic exports.
> It can work like this:
> 
> 0. Install nbd and enable nbd-server on a host, running
>    as vdsm:kvm, not exporting anything.
> 
> 1. User starts transfer session via oVirt API with protocol="nbd"
>     (if not specified, use "https" for backward compatibility)
> 
> 2. oVirt selects a host for the transfer and configures an export
>    in that host - for example:
> 
> [bbff5ab7-2e8e-45b8-a497-423c2d2b3ff1]
> exportname = /path/to/image
> trim = yes
> timeout = 300
> 
> nbd-server supports reloading configuration via SIGHUP.
> Looks like it should work for adding export dynamically:
>
https://github.com/NetworkBlockDevice/nbd/blob/cb6e816c4d2c536cfff5ba9859f210aabe39539b/nbd-server.c#L3007
I don't think we have nbd-server in RHEL, and in any case wouldn't it
be better to use qemu-nbd?

You just start a new qemu-nbd process instead of faffing around with
configuration files, kill the qemu-nbd process when you're done, and
qemu-nbd supports qcow2 already.

It doesn't support progress, but it's not very clear what
"progress"
means for an NBD connection, since it's quite reasonable to go back
and rewrite blocks, or (as with qemu-img convert) issue zeroes, or
trims, etc.  It might be better simply to derive progress from ‘du -s
target_file’, ‘qemu-img info target.qcow2’ or similar.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On 04/12/2018 05:24 AM, Richard W.M. Jones wrote:
> I don't think we have nbd-server in RHEL, and in any case wouldn't
it
> be better to use qemu-nbd?
> 
> You just start a new qemu-nbd process instead of faffing around with
> configuration files, kill the qemu-nbd process when you're done, and
> qemu-nbd supports qcow2 already.
That, and qemu-nbd supports extensions such as NBD_CMD_BLOCK_STATUS and
NBD_OPT_STRUCTURED_REPLY that nbd-server has not implemented yet; a qemu
NBD client talking to a qemu-nbd server is thus going to be able to take
advantage of those extensions for better performance that would not be
possible with a qemu NBD client talking to an nbd-server instance (at
least, not without someone implementing the new features there).  And
this is no different from the situation where nbdkit as the server lacks
several features; the current rhv-upload patches use a python plugin to
nbdkit, which is implemented as serializing all requests; while using
qemu-nbd as the server would allow parallel requests to be in flight
simultaneously.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org