search for: transfer_url

...serving the image for imageio: qemu-nbd --socket=/tmp/nbd.sock --persistent --shared=8 --format=qcow2 \ --aio=native --cache=none --discard=unmap /var/tmp/disk.qcow2 Start nbdkit with this plugin: ./nbdkit -U nbd.sock -t4 -f python ./plugins/python/examples/imageio.py \ transfer_url=https://localhost:54322/images/nbd connections=4 secure=no Finally, upload the image using qemu-img: time qemu-img convert -n -f raw -O raw -W /var/tmp/fedora-32.raw \ nbd+unix:///?socket=./nbd.sock I tested with 1 and 4 threads/connections, creating new empty qcow2 image before each...

...> qemu-nbd --socket=/tmp/nbd.sock --persistent --shared=8 --format=qcow2 \ > --aio=native --cache=none --discard=unmap /var/tmp/disk.qcow2 > > Start nbdkit with this plugin: > > ./nbdkit -U nbd.sock -t4 -f python ./plugins/python/examples/imageio.py \ > transfer_url=https://localhost:54322/images/nbd connections=4 secure=no > > Finally, upload the image using qemu-img: > > time qemu-img convert -n -f raw -O raw -W /var/tmp/fedora-32.raw \ > nbd+unix:///?socket=./nbd.sock > > I tested with 1 and 4 threads/connections, creating n...

...out waiting for transfer status " + + raise RuntimeError("timed out waiting for transfer status " "!= INITIALIZING") # Now we have permission to start the transfer. if params['rhv_direct']: if transfer.transfer_url is None: - raise RuntimeError("direct upload to host not supported, " + - "requires ovirt-engine >= 4.2 and only works " + - "when virt-v2v is run within the oVirt/RHV " + + raise Runt...

...break if time.time() > endt: raise RuntimeError("timed out waiting for transfer status " "!= INITIALIZING") # Now we have permission to start the transfer. if params['rhv_direct']: if transfer.transfer_url is None: raise RuntimeError("direct upload to host not supported, " "requires ovirt-engine >= 4.2 and only works " "when virt-v2v is run within the oVirt/RHV "...

...t;transport>nbd</transport> </image_transfer> Engine does not implement <transport> yet, but this should be an easy change. This will use the new NBD APIs to export the disk using NBD over unix socket. We can support later also NBD over TLS/PSK. Engine will return NBD url in transfer_url: <transfer_url>nbd:unix:/run/vdsm/nbd/<transfer_uuid>.sock</tansfer_url> 3. v2v use the trasfer_url to start qem-img with the NBD server: nbdkit (vddk) -> qemu-img convert -> qemu-nbd Note that nbdkit is removed from the rhv side of the pipeline. This is expected...

We can use now ImageioClient to communicate with ovirt-imageio server on oVirt host. Using the client greatly simplifies the plugin, and enables new features like transparent proxy support. The client will use transfer_url if possible, or fall back to proxy_url. Since the client implements the buffer protocol, move to version 2 of the API for more efficient pread(). Another advantage the client is maintained by oVirt, and fixes are delivered quickly in oVirt, without depending on RHEL release schedule. Not ready y...

...us " + > + raise RuntimeError("timed out waiting for transfer status " > "!= INITIALIZING") > > # Now we have permission to start the transfer. > if params['rhv_direct']: > if transfer.transfer_url is None: > - raise RuntimeError("direct upload to host not supported, " + > - "requires ovirt-engine >= 4.2 and only works " + > - "when virt-v2v is run within the oVirt/RHV " + > +...

...> endt: > raise RuntimeError("timed out waiting for transfer status " > "!= INITIALIZING") > > # Now we have permission to start the transfer. > if params['rhv_direct']: > if transfer.transfer_url is None: > raise RuntimeError("direct upload to host not supported, " > "requires ovirt-engine >= 4.2 and only works " > "when virt-v2v is run within the oVirt/RHV " >...

...ge_transfer> > > Engine does not implement <transport> yet, but this should be an easy > change. > > This will use the new NBD APIs to export the disk using NBD over unix > socket. > We can support later also NBD over TLS/PSK. > > Engine will return NBD url in transfer_url: > > <transfer_url>nbd:unix:/run/vdsm/nbd/<transfer_uuid>.sock</tansfer_url> > > 3. v2v use the trasfer_url to start qem-img with the NBD server: > > nbdkit (vddk) -> qemu-img convert -> qemu-nbd > > Note that nbdkit is removed from the rh...

...(object): ): pass - id = 123 + id = "123" status = DiskStatus.OK class ImageTransferPhase(Enum): @@ -104,7 +104,7 @@ class ImageTransfer(object): ): pass - id = 456 + id = "456" phase = ImageTransferPhase.TRANSFERRING transfer_url = "http://localhost:" + str(imageio_port) + "/" -- 2.21.0

...t: + transfer_service.cancel() raise RuntimeError("timed out waiting for transfer status " "!= INITIALIZING") # Now we have permission to start the transfer. if params['rhv_direct']: if transfer.transfer_url is None: + transfer_service.cancel() raise RuntimeError("direct upload to host not supported, " "requires ovirt-engine >= 4.2 and only works " "when virt-v2v is run within the oVirt/R...

We were not considering failures while initializing the transfer. In this case the transfer phase can change to PAUSED_SYSTEM or FINISHED_FAILURE, and transfer_url will be None, which failed the upload with a misleading error: RuntimeError: direct upload to host not supported, requires ovirt-engine >= 4.2 and only works when virt-v2v is run within the oVirt/RHV environment, eg. on an oVirt node Change the wait loop to consider all cases: - Tr...

...raise RuntimeError("timed out waiting for transfer status " > > "!= INITIALIZING") > > > > # Now we have permission to start the transfer. > > if params['rhv_direct']: > > if transfer.transfer_url is None: > > raise RuntimeError("direct upload to host not supported, " > > "requires ovirt-engine >= 4.2 and only > works " > > "when virt-v2v is run within the > oVirt/...

...hard W.M. Jones <rjones@redhat.com> wrote: > > On Thu, Nov 28, 2019 at 09:34:18PM +0200, Nir Soffer wrote: > > We were not considering failures while initializing the transfer. In > > this case the transfer phase can change to PAUSED_SYSTEM or > > FINISHED_FAILURE, and transfer_url will be None, which failed the > > upload with a misleading error: > > > > RuntimeError: direct upload to host not supported, requires > > ovirt-engine >= 4.2 and only works when virt-v2v is run within the > > oVirt/RHV environment, eg. on an oVirt node...

...raise RuntimeError("timed out waiting for transfer status " > > "!= INITIALIZING") > > > > # Now we have permission to start the transfer. > > if params['rhv_direct']: > > if transfer.transfer_url is None: > > - raise RuntimeError("direct upload to host not supported, " + > > - "requires ovirt-engine >= 4.2 and only > works " + > > - "when virt-v2v is run within the > oV...

...t the trasnfer URL for this image transfer. s/trasnfer/transfer/ > +# > +# Run this example from the build directory: > +# > +# ./nbdkit -t4 -f -v -U /tmp/nbd.sock -t4 python \ -t4 does not need to be listed twice. > +# ./plugins/python/examples/imageio.py \ > +# transfer_url=https://server:54322/images/ticket-id \ > +# connections=4 \ > +# secure=no > +# > +# Note that number of nbdkit threads and imageio connections should match. > +# > +# To upload an image run: > +# > +# qemu-img convert -f qcow2 -O raw disk.img nbd:///?socket=t...

On Thu, Jul 09, 2020 at 01:51:44AM +0300, Nir Soffer wrote: > We can use now ImageioClient to communicate with ovirt-imageio server > on oVirt host. > > Using the client greatly simplifies the plugin, and enables new features > like transparent proxy support. The client will use transfer_url if > possible, or fall back to proxy_url. > > Since the client implements the buffer protocol, move to version 2 of > the API for more efficient pread(). This will require nbdkit >= 1.17.3 which implies RHEL AV 8.3.0, but that's fine since we only ship the new virt-v2v (>= 1...

This series extract oVirt SDK and imageio code to make it eaiser to follow the code and improve error handing in open() and close(). Tested with virt-v2v master. Changes since v1: - Rebase on merged patches from v1 - Fix regression introduced by "rhv-upload: Fix cleanup after errors" - Remove "rhv-upload: Try to remove disk on timeout" since it cannot succeed - Add more

On Thu, Nov 28, 2019 at 09:34:18PM +0200, Nir Soffer wrote: > We were not considering failures while initializing the transfer. In > this case the transfer phase can change to PAUSED_SYSTEM or > FINISHED_FAILURE, and transfer_url will be None, which failed the > upload with a misleading error: > > RuntimeError: direct upload to host not supported, requires > ovirt-engine >= 4.2 and only works when virt-v2v is run within the > oVirt/RHV environment, eg. on an oVirt node > > Change the wa...

...ice(transfer.id) + +# After adding a new transfer for the disk, the transfer's status will +# be INITIALIZING. Wait until the init phase is over. +while transfer.phase == types.ImageTransferPhase.INITIALIZING: + time.sleep(1) + transfer = transfer_service.get() + +if %s: + if transfer.transfer_url is None: + print(\"Direct upload to host not supported, requires ovirt-engine >= 4.2\") + sys.exit(1) + destination_url = urlparse(transfer.transfer_url) +else: + destination_url = urlparse(transfer.proxy_url) + +context = ssl.create_default_context() +context.load_...