search for: proxy_url

We can use now ImageioClient to communicate with ovirt-imageio server on oVirt host. Using the client greatly simplifies the plugin, and enables new features like transparent proxy support. The client will use transfer_url if possible, or fall back to proxy_url. Since the client implements the buffer protocol, move to version 2 of the API for more efficient pread(). Another advantage the client is maintained by oVirt, and fixes are delivered quickly in oVirt, without depending on RHEL release schedule. Not ready yet, we have several issues: - The clie...

...only works " "when virt-v2v is run within the oVirt/RHV " "environment, eg. on an oVirt node.") destination_url = urlparse(transfer.transfer_url) else: destination_url = urlparse(transfer.proxy_url) if destination_url.scheme == "https": context = \ ssl.create_default_context(purpose = ssl.Purpose.SERVER_AUTH, - cafile = cafile) + cafile = params['rhv_cafile']) if...

...) > > diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py > index 5cd6d5cab..6e35b5057 100644 > --- a/v2v/rhv-upload-plugin.py > +++ b/v2v/rhv-upload-plugin.py > @@ -207,8 +207,11 @@ def open(readonly): > else: > destination_url = urlparse(transfer.proxy_url) > > - context = ssl.create_default_context() > - context.load_verify_locations(cafile = params['rhv_cafile']) > This line was never needed. In imageio client we use: context = ssl.create_default_context( purpose=ssl.Purpose.SERVER_AUTH, cafile=cafile) i...

...changed, 5 insertions(+), 2 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 5cd6d5cab..6e35b5057 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -207,8 +207,11 @@ def open(readonly): else: destination_url = urlparse(transfer.proxy_url) - context = ssl.create_default_context() - context.load_verify_locations(cafile = params['rhv_cafile']) + if destination_url.scheme == "https": + context = ssl.create_default_context() + context.load_verify_locations(cafile = params['rhv_cafile'])...

...anged, 19 insertions(+), 8 deletions(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 5cd6d5cab..1a217b6dc 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -207,14 +207,25 @@ def open(readonly): else: destination_url = urlparse(transfer.proxy_url) - context = ssl.create_default_context() - context.load_verify_locations(cafile = params['rhv_cafile']) - - http = HTTPSConnection( - destination_url.hostname, - destination_url.port, - context = context - ) + if destination_url.scheme == "https&q...

..."when virt-v2v is run within the oVirt/RHV " > "environment, eg. on an oVirt node.") > destination_url = urlparse(transfer.transfer_url) > else: > destination_url = urlparse(transfer.proxy_url) > > if destination_url.scheme == "https": > context = \ > ssl.create_default_context(purpose = ssl.Purpose.SERVER_AUTH, > - cafile = cafile) > + cafile = params['...

...ir Soffer wrote: > We can use now ImageioClient to communicate with ovirt-imageio server > on oVirt host. > > Using the client greatly simplifies the plugin, and enables new features > like transparent proxy support. The client will use transfer_url if > possible, or fall back to proxy_url. > > Since the client implements the buffer protocol, move to version 2 of > the API for more efficient pread(). This will require nbdkit >= 1.17.3 which implies RHEL AV 8.3.0, but that's fine since we only ship the new virt-v2v (>= 1.42) on AV 8.3.0. > Another advantage th...

This adds a test of -o rhv-upload. Obviously for an upstream test we cannot require a working oVirt server. This test works by faking the ovirtsdk4 Python module, setting PYTHONPATH so that the fake module is picked up instead of the real module (if installed). However it's more complex than that because the nbdkit plugin also expects to talk to a working imageio HTTPS server. Therefore

..."when virt-v2v is run within the > oVirt/RHV " > > "environment, eg. on an oVirt node.") > > destination_url = urlparse(transfer.transfer_url) > > else: > > destination_url = urlparse(transfer.proxy_url) > > > > if destination_url.scheme == "https": > > context = \ > > ssl.create_default_context(purpose = > ssl.Purpose.SERVER_AUTH, > > - cafile = cafile) > > +...

...why you cannot communicate with > >imageio daemon? > > That is something I would love to find out, but I don't know what to try next. Can you share entire virt-v2v log, and /var/log/ovirt-imageio-daemon/daemon.log from the host you are uploading to? if you use the proxy (transfer.proxy_url), which you should not, also the proxy log from the engine host (/var/log/ovirt-imageio-proxy/image-proxy.log).

On Tue, Nov 19, 2019 at 04:14:45PM +0200, Nir Soffer wrote: >On Tue, Nov 19, 2019 at 3:36 PM Martin Kletzander <mkletzan@redhat.com> >wrote: > >> On Tue, Nov 19, 2019 at 01:48:31PM +0100, Martin Kletzander wrote: >> >On Tue, Nov 19, 2019 at 02:24:11PM +0200, Nir Soffer wrote: >> >>On Tue, Nov 19, 2019 at 2:16 PM Martin Kletzander

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00121.html v2: - Rewrote patch 2 from scratch so it incorporates Nir's suggestions. - Add fake module to EXTRA_DIST. - Retested. Unfortunately I am no longer able to test the ordinary conversion path because ovirtsdk4 is incompatible with Fedora 29 / Python 3.7:

This series extract oVirt SDK and imageio code to make it eaiser to follow the code and improve error handing in open() and close(). Tested with virt-v2v master. Changes since v1: - Rebase on merged patches from v1 - Fix regression introduced by "rhv-upload: Fix cleanup after errors" - Remove "rhv-upload: Try to remove disk on timeout" since it cannot succeed - Add more

...er = transfer_service.get() + +if %s: + if transfer.transfer_url is None: + print(\"Direct upload to host not supported, requires ovirt-engine >= 4.2\") + sys.exit(1) + destination_url = urlparse(transfer.transfer_url) +else: + destination_url = urlparse(transfer.proxy_url) + +context = ssl.create_default_context() +context.load_verify_locations(cafile = %s) + +proxy_connection = HTTPSConnection( + destination_url.hostname, + destination_url.port, + context = context +) + +image_path = %s +image_size = %Ld + +proxy_connection.putrequest(\"PUT\", de...

...can use now ImageioClient to communicate with ovirt-imageio server > > on oVirt host. > > > > Using the client greatly simplifies the plugin, and enables new features > > like transparent proxy support. The client will use transfer_url if > > possible, or fall back to proxy_url. > > > > Since the client implements the buffer protocol, move to version 2 of > > the API for more efficient pread(). > > This will require nbdkit >= 1.17.3 which implies RHEL AV 8.3.0, but > that's fine since we only ship the new virt-v2v (>= 1.42) on AV 8.3.0...

This series extract oVirt SDK and imageio code to make it eaiser to follow the code and improve error handing in open() and close(). The first small patches can be consider as fixes for downstream. Tested based on libguestfs v1.41.5, since I had trouble building virt-v2v and libguestfs from master. Nir Soffer (18): rhv-upload: Remove unused exception class rhv-upload: Check status more

...raise RuntimeError(\"Direct upload to host not supported, requires ovirt-engine >= 4.2 and only works when virt-v2v is run within the oVirt/RHV environment, eg. on an ovirt node.\") + destination_url = urlparse(transfer.transfer_url) +else: + destination_url = urlparse(transfer.proxy_url) + +context = ssl.create_default_context() +context.load_verify_locations(cafile = %s) + +proxy_connection = HTTPSConnection( + destination_url.hostname, + destination_url.port, + context = context +) + +image_path = %s +image_size = %Ld + +proxy_connection.putrequest(\"PUT\", de...

This patch set is still for discussion only. See 3/3 for the current list of problems. However this will upload an image to an oVirt or RHV server, although you must have absolutely the latest snapshot version of 4.2 for it to work. Rich.

...+if %s: > + if transfer.transfer_url is None: > + print(\"Direct upload to host not supported, requires ovirt-engine >= 4.2\") > + sys.exit(1) > + destination_url = urlparse(transfer.transfer_url) > +else: > + destination_url = urlparse(transfer.proxy_url) > + > +context = ssl.create_default_context() > +context.load_verify_locations(cafile = %s) > + > +proxy_connection = HTTPSConnection( > + destination_url.hostname, > + destination_url.port, > + context = context > +) > + > +image_path = %s > +image_siz...

The first four patches are straightforward. The final patch adds the -o rhv-upload output mode. It is still spooling into a temporary file because I've had some trouble getting streaming conversions working. There are other problems as outlined in the commit message, so this patch is not ready for upstream but is good for discussion. Also I hit this, which I'm assuming for now will be