similar to: v2v: -o rhv-upload - oVirt imageio random I/O APIs

On Wed, Mar 21, 2018 at 11:20:13AM +0000, Nir Soffer wrote: > Hi, > > I updated the random I/O documentation patch: > https://gerrit.ovirt.org/#/c/89022/ > > I would to get your comments on this before we complete the implementation. I want to check a couple of things: (1) The OPTIONS request uses path ‘/images/%2A’ (ascii encoding of ‘*’). That is literally what is sent

On Wed, Mar 21, 2018 at 11:20:13AM +0000, Nir Soffer wrote: > Hi, > > I updated the random I/O documentation patch: > https://gerrit.ovirt.org/#/c/89022/ > > I would to get your comments on this before we complete the implementation. Looks great to me, we can work with this. There's no mention of alignment, so can I assume that we can make arbitrarily aligned requests?

On Fri, Mar 9, 2018 at 4:25 PM Richard W.M. Jones <rjones@redhat.com> wrote: > It has to be said it would be really convenient to have a 'zero' > and/or 'trim' method of some sort. > 'trim' means discard? Currently we cannot support discard on block storage since ovirt may need to wipe lvs when deleting a disk, and discarding may leave unwiped user data.

Hi Richard, We've added zero and flush functionality to imageio-daemon. You can download and test the latest build (for el7/fc) from: - http://jenkins.ovirt.org/job/ovirt-imageio_master_build-artifacts-el7-x86_64/200/artifact/exported-artifacts/ - http://jenkins.ovirt.org/job/ovirt-imageio_master_build-artifacts-fc27-x86_64/53/artifact/exported-artifacts/ -

On Mon, Mar 12, 2018 at 2:57 PM Eric Blake <eblake@redhat.com> wrote: > On 03/12/2018 07:13 AM, Nir Soffer wrote: > > On Mon, Mar 12, 2018 at 12:32 PM Richard W.M. Jones <rjones@redhat.com> > > wrote: > > > >> On Mon, Mar 12, 2018 at 07:13:52AM +0000, Nir Soffer wrote: > >>> On Fri, Mar 9, 2018 at 4:25 PM Richard W.M. Jones

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00015.html v2 fixes all the issues raised in Nir's review. I tested it against ovirt-engine-4.2.4.1-1.el7.noarch and it works, although I wasn't easily able to trigger the error path so that code is not really tested (except for syntax checking). Rich.

On Mon, Mar 12, 2018 at 12:32 PM Richard W.M. Jones <rjones@redhat.com> wrote: > On Mon, Mar 12, 2018 at 07:13:52AM +0000, Nir Soffer wrote: > > On Fri, Mar 9, 2018 at 4:25 PM Richard W.M. Jones <rjones@redhat.com> > wrote: > > > > > It has to be said it would be really convenient to have a 'zero' > > > and/or 'trim' method of some

v2 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00109.html v3: - Added/fixed all suggestions from Nir in previous review. Q: I wasn't sure if we still need the "UnsupportedError" class so I left it in. Q: Does the Unix socket always have the same name? What happens if there's more than one transfer happening? I tested this both ways, and it worked both

When sending request with small or no payload, it is simpler and possibly more efficient to use the high level HTTPSConnection.request(), instead of the lower level APIs. The only reason to use the lower level APIs is to avoid copying the payload, or on python 2, to use a bigger buffer size when streaming a file-like object. --- v2v/rhv-upload-plugin.py | 35 ++++++++++++++++-------------------

On Wed, Mar 14, 2018 at 06:56:19PM +0000, Nir Soffer wrote: > I posted documentation for the new API optimized for random I/O: > https://gerrit.ovirt.org/#/c/89022/ Wish I'd had this documentation when I started the patch :-) Yes, it's much clearer. > I changed POST to PATCH to match the existing /tickets API, and > this also seems to be more standard way to do such

On Tue, Jun 26, 2018 at 4:25 PM Richard W.M. Jones <rjones@redhat.com> wrote: > In the case where virt-v2v runs on the same server as the imageio > daemon that we are talking to, it may be possible to optimize access > using a Unix domain socket. > > This is only an optimization. If it fails or if we're not running on > the same server it will fall back to the usual

I posted this patch for vdsm, adding NBD APIs: https://gerrit.ovirt.org/c/96079/ The main purpose of these APIs are enabling incremental restore, but they also enable a more efficient rhv-upload via NBD, and importing to thin disks, which is not possible with current solution. The same idea can work for KubeVirt or other targets, minimizing specific target code. Here is how rhv-upload can work

These two patches add support for using a Unix domain socket to directly access imageio in the case where imageio is running on the conversion host (usually that means virt-v2v is running on the RHV node and something else -- eg. CFME scripts -- arranges that the RHV node is the same one running imageio). Conversions in the normal case are not affected - they happen over TCP as usual. This was

On Thu, Mar 08, 2018 at 12:13:01PM +0000, Nir Soffer wrote: > On Wed, Mar 7, 2018 at 12:18 AM Richard W.M. Jones <rjones@redhat.com> > wrote: > > > Previous versions: > > v3: https://www.redhat.com/archives/libguestfs/2018-March/msg00000.html > > v2: > > https://www.redhat.com/archives/libguestfs/2018-February/msg00177.html > > v1: > >

On Tue, Apr 10, 2018 at 03:25:47PM +0000, Nir Soffer wrote: > On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones <rjones@redhat.com> > wrote: > > > On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote: > > > This makes sense if the device is backed by a block device on oVirt side, > > > and the NBD support efficient zeroing. But in this case the device

On Thu, Jul 9, 2020 at 4:12 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > On Thu, Jul 09, 2020 at 03:30:22PM +0300, Nir Soffer wrote: > > Testing with a real server is easy. I have incomplete patch using > > imageio server with some manual setup. > [...] > > For a certain definition of "easy" :-) > > Our QE team tests -o rhv-upload from time

On Mon, Jun 18, 2018 at 08:55:13PM +0300, Nir Soffer wrote: > On Mon, Jun 18, 2018 at 1:37 PM Richard W.M. Jones <rjones@redhat.com> > wrote: > > > On Thu, Jun 14, 2018 at 09:24:48PM +0300, Nir Soffer wrote: > > > On Thu, Jun 14, 2018 at 9:16 PM Nir Soffer <nirsof@gmail.com> wrote: > > > > + headers = {"Content-Type":

On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote: > This makes sense if the device is backed by a block device on oVirt side, > and the NBD support efficient zeroing. But in this case the device is backed > by an empty sparse file on NFS, and oVirt does not support yet efficient > zeroing, we just write zeros manually. > > I think should be handled on virt-v2v plugin

We now have true zeroing support in oVirt imageio, thanks for that. However a problem is that ‘qemu-img convert’ issues zero requests for the whole disk before starting the transfer. It does this using 32 MB requests which take approx. 1 second each to execute on the oVirt side. Two problems therefore: (1) Zeroing the disk can take a long time (eg. 40 GB is approx. 20 minutes).

On Thu, Jun 14, 2018 at 09:24:48PM +0300, Nir Soffer wrote: > On Thu, Jun 14, 2018 at 9:16 PM Nir Soffer <nirsof@gmail.com> wrote: > > + headers = {"Content-Type": "application/json", > > + "Content-Length", str(len(buf))} There were a few Python syntax errors such as this one. They can be found by running: make -C v2v check