similar to: [PATCH] v2v: rhv-upload-plugin: Use BrokenPipeError

On Tue, Aug 28, 2018 at 11:50:56PM +0300, Nir Soffer wrote: > With python 3, we have a nicer way to handle socket.error with errno set > to EPIPE (or ESHUTDOWN). > > This is also more correct since in some cases (that I could not > reproduce yet with v2v), using e[0] with BrokenPipeError will fail with: > > >>> OSError(errno.EPIPE, "Broken pipe")[0] >

Nir, can you confirm this is correct for RHEL 7 / Python 2? Rich.

These patches improve error handling when PUT request fail, including the error response from oVirt server. This will make it easier to debug issue when oVirt server logs have been rotated. Nir Soffer (2): v2v: rhv-upload-plugin: Handle send send failures v2v: rhv-upload-plugin: Fix error formatting v2v/rhv-upload-plugin.py | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7

Replace python 2 only "buffer" with "memoryview". Falling back to emulated zero would fail with: NameError: name 'buffer' is not defined I did not test the changed code but it was not tested before so it is unlikely to be worse. Detected by pylint. --- v2v/rhv-upload-plugin.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

From: "Richard W.M. Jones" <rjones@redhat.com> Updates commit 8f250c00c8b028a614815f63b2713748504d7ef9 for RHEL 7.6 LP. --- v2v/rhv-upload-plugin.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 42c7065..3e561e6 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -383,8

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00015.html v2 fixes all the issues raised in Nir's review. I tested it against ovirt-engine-4.2.4.1-1.el7.noarch and it works, although I wasn't easily able to trigger the error path so that code is not really tested (except for syntax checking). Rich.

Thanks: Nir Soffer --- v2v/rhv-upload-plugin.py | 66 ++++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 27 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 791c9e7d2..c3de7d555 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -228,6 +228,29 @@ def can_flush(h): def get_size(h): return

Old imageio proxy was using Authorization header for GET and PUT requests. Remove unneeded authorization when sending OPTIONS request. Remove unneeded duplicated comments about authorization for old imageio, and replace them with a comment when we set needs_auth. --- v2v/rhv-upload-plugin.py | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/v2v/rhv-upload-plugin.py

We can use now ImageioClient to communicate with ovirt-imageio server on oVirt host. Using the client greatly simplifies the plugin, and enables new features like transparent proxy support. The client will use transfer_url if possible, or fall back to proxy_url. Since the client implements the buffer protocol, move to version 2 of the API for more efficient pread(). Another advantage the client

Currently we handle only HTTP errors like getting unexpected response code from imageio server. However if we could not send the request, or some other error is raised, we failed to mark the handle as failed, and finalized the transfer in close(). This may fool virt-v2v to create a VM with an empty or partially uploaded disk. Decorate all the NBD hander functions with a @failing decorator,

On Fri, Dec 07, 2018 at 07:55:28PM +0200, Nir Soffer wrote: > Replace python 2 only "buffer" with "memoryview". > > Falling back to emulated zero would fail with: > > NameError: name 'buffer' is not defined > > I did not test the changed code but it was not tested before so it is > unlikely to be worse. > > Detected by pylint.

When sending request with small or no payload, it is simpler and possibly more efficient to use the high level HTTPSConnection.request(), instead of the lower level APIs. The only reason to use the lower level APIs is to avoid copying the payload, or on python 2, to use a bigger buffer size when streaming a file-like object. --- v2v/rhv-upload-plugin.py | 35 ++++++++++++++++-------------------

v7 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00143.html Since then: - Earlier patches are now upstream. - The to-do list is moved from the commit message to the TODO file. - This version forces -of raw + -oa sparse and gives an error in any other mode. We intend to lift these restrictions later. - Tested against latest imageio which supports longer timeouts,

v8 was here: https://www.redhat.com/archives/libguestfs/2018-April/msg00022.html v8 -> v9: - Addresses the only feedback from Tomáš. Rich.

The oVirt server may fail a PUT request before reading all request body. However before closing the connection, it writes a detailed error response, that will make debugging issues like expired tickets much easier to handle. If we don't get the response and log the error, the error may be lost when the daemon log is rotated. Change pwrite() and emulate_zero() to get the response after EPIPE,

Only compile tested. I'm keeping this as a separate patch for the moment until the relevant change on the oVirt side goes upstream. Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00099.html v2: - Just fixes the two problems noted in the review of the previous version. Rich.

On Mon, Mar 26, 2018 at 11:41 AM Richard W.M. Jones <rjones@redhat.com> wrote: > On Sun, Mar 25, 2018 at 08:05:14PM +0000, Nir Soffer wrote: > > When using the sdk, you can select both the image format and sparse, so > you > > can create invalid combinations. oVirt selects the allocation for you. > > > > storage type image format sparse | allocation

v5 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00032.html There is only a single patch in this version because the other patches went upstream. This patch adds the virt-v2v -o rhv-upload mode (https://bugzilla.redhat.com/show_bug.cgi?id=1557273). Compared to v5, this adds the ability to make zero, trim and flush requests to the oVirt imageio server

On Tue, Nov 19, 2019 at 04:14:45PM +0200, Nir Soffer wrote: >On Tue, Nov 19, 2019 at 3:36 PM Martin Kletzander <mkletzan@redhat.com> >wrote: > >> On Tue, Nov 19, 2019 at 01:48:31PM +0100, Martin Kletzander wrote: >> >On Tue, Nov 19, 2019 at 02:24:11PM +0200, Nir Soffer wrote: >> >>On Tue, Nov 19, 2019 at 2:16 PM Martin Kletzander