search for: emulate_zero

http was missing in emulate_zero, so the code would fail with NameError. This can happen only when communicating with old imageio versions not supporting the "zero" feature. Testing with qemu-img 2.12 show that we never send emulated zero request because of the highestwrite mechanism, but it can break with older qemu-im...

...ly. > > I think should be handled on virt-v2v plugin side. When zeroing a file raw > image, > you can ignore zero requests after the highest write offset, since the > plugin > created a new image, and we know that the image is empty. Didn't Rich already try to do that? +def emulate_zero(h, count, offset): + # qemu-img convert starts by trying to zero/trim the whole device. + # Since we've just created a new disk it's safe to ignore these + # requests as long as they are smaller than the highest write seen. + # After that we must emulate them with writes. + i...

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00015.html v2 fixes all the issues raised in Nir's review. I tested it against ovirt-engine-4.2.4.1-1.el7.noarch and it works, although I wasn't easily able to trigger the error path so that code is not really tested (except for syntax checking). Rich.

...ntimeError("could not zero sector (%d, %d): %d: %s" % - (offset, count, r.status, r.reason)) + unexpected_response(h, r, + "could not zero sector offset %d size %d" % + (offset, count)) def emulate_zero(h, count, offset): # qemu-img convert starts by trying to zero/trim the whole device. @@ -334,15 +352,13 @@ def emulate_zero(h, count, offset): r = http.getresponse() if r.status != 200: - transfer_service.pause() - h['failed'] = True -...

..., h['path'] + "?flush=n") - # Authorization is only needed for old imageio. if h['needs_auth']: http.putheader("Authorization", transfer.signed_ticket) # The oVirt server only uses the first part of the range, and the @@ -378,7 +377,6 @@ def emulate_zero(h, count, offset): # After that we must emulate them with writes. if offset+count < h['highestwrite']: http.putrequest("PUT", h['path']) - # Authorization is only needed for old imageio. if h['needs_auth']: http.pu...

...ort socket @@ -361,9 +360,8 @@ def pwrite(h, buf, offset): try: http.send(buf) - except socket.error as e: - if e[0] != errno.EPIPE: - raise + except BrokenPipeError: + pass r = http.getresponse() if r.status != 200: @@ -425,9 +423,8 @@ def emulate_zero(h, count, offset): http.send(buf) count -= len(buf) http.send(buffer(buf, 0, count)) - except socket.error as e: - if e[0] != errno.EPIPE: - raise + except BrokenPipeError: + pass r = http.g...

"http" and "transfer" variables were missing in emulate_zero, so the code would fail with NameError. This can happen only when communicating with old imageio versions not supporting the "zero" feature. Testing with qemu-img 2.12 show that we never send emulated zero request because of the highestwrite mechanism, but it can break with older qemu-i...

On Mon, Jun 25, 2018 at 8:30 PM Nir Soffer <nirsof@gmail.com> wrote: > http was missing in emulate_zero, so the code would fail with NameError. > This can happen only when communicating with old imageio versions not > supporting the "zero" feature. > > Testing with qemu-img 2.12 show that we never send emulated zero request > because of the highestwrite mechanism, but it can...

...ng all request body. However before closing the connection, it writes a detailed error response, that will make debugging issues like expired tickets much easier to handle. If we don't get the response and log the error, the error may be lost when the daemon log is rotated. Change pwrite() and emulate_zero() to get the response after EPIPE, failing with the error response from the oVirt server. Here is an example error log when a ticket expires during import: nbdkit: python[1]: error: /home/nsoffer/src/libguestfs/tmp/v2v.pRoyXm/rhv-upload-plugin.py: pwrite: error: ('%s: %d %s: %r', 'cou...

Nir, can you confirm this is correct for RHEL 7 / Python 2? Rich.

...buf, offset): def zero(h, count, offset, may_trim): http = h['http'] - transfer = h['transfer'] # Unlike the trim and flush calls, there is no 'can_zero' method # so nbdkit could call this even if the server doesn't support @@ -371,7 +369,6 @@ def emulate_zero(h, count, offset): def trim(h, count, offset): http = h['http'] - transfer = h['transfer'] # Construct the JSON request for trimming. buf = json.dumps({'op': "trim", @@ -394,7 +391,6 @@ def trim(h, count, offset): def flush(h): http =...

...ro(h, count, offset, may_trim): http = h['http'] transfer = h['transfer'] @@ -330,6 +335,8 @@ def zero(h, count, offset, may_trim): "could not zero sector offset %d size %d" % (offset, count)) + r.read() + def emulate_zero(h, count, offset): # qemu-img convert starts by trying to zero/trim the whole device. # Since we've just created a new disk it's safe to ignore these @@ -357,6 +364,8 @@ def emulate_zero(h, count, offset): "could not write zeroes offset %d size %d&...

...raise RuntimeError("could not zero sector (%d, %d): %d: %s" % - (offset, count, r.status, r.reason)) + request_failed(h, r, + "could not zero sector offset %d size %d" % + (offset, count)) def emulate_zero(h, count, offset): # qemu-img convert starts by trying to zero/trim the whole device. @@ -334,15 +355,13 @@ def emulate_zero(h, count, offset): r = http.getresponse() if r.status != 200: - transfer_service.pause() - h['failed'] = True -...

...worse. Detected by pylint. --- v2v/rhv-upload-plugin.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2v/rhv-upload-plugin.py b/v2v/rhv-upload-plugin.py index 3272c3ce3..12d4e68f7 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -416,41 +416,41 @@ def emulate_zero(h, count, offset): transfer = h['transfer'] # qemu-img convert starts by trying to zero/trim the whole device. # Since we've just created a new disk it's safe to ignore these # requests as long as they are smaller than the highest write seen. # After that...

These patches improve error handling when PUT request fail, including the error response from oVirt server. This will make it easier to debug issue when oVirt server logs have been rotated. Nir Soffer (2): v2v: rhv-upload-plugin: Handle send send failures v2v: rhv-upload-plugin: Fix error formatting v2v/rhv-upload-plugin.py | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7

...ero sector (%d, %d): %d: %s" % > - (offset, count, r.status, r.reason)) > + unexpected_response(h, r, > + "could not zero sector offset %d size %d" % > + (offset, count)) > > def emulate_zero(h, count, offset): > # qemu-img convert starts by trying to zero/trim the whole device. > @@ -334,15 +352,13 @@ def emulate_zero(h, count, offset): > > r = http.getresponse() > if r.status != 200: > - transfer_service.pause() > - h[&...

...+ > > + # Unlike the trim and flush calls, there is no 'can_zero' method > > + # so nbdkit could call this even if the server doesn't support > > + # zeroing. If this is the case we must emulate. > > + if not h['can_zero']: > > + emulate_zero(h, count, offset) > > + return > > + > > + # Construct the JSON request for zeroing. > > + buf = json.dumps({'op': "zero", > > + 'offset': offset, > > + 'size': count, > &...

...ot;?flush=n") > - # Authorization is only needed for old imageio. > if h['needs_auth']: > http.putheader("Authorization", transfer.signed_ticket) > # The oVirt server only uses the first part of the range, and the > @@ -378,7 +377,6 @@ def emulate_zero(h, count, offset): > # After that we must emulate them with writes. > if offset+count < h['highestwrite']: > http.putrequest("PUT", h['path']) > - # Authorization is only needed for old imageio. > if h['needs_auth...

...gt; > try: > http.send(buf) > - except socket.error as e: > - if e[0] != errno.EPIPE: > - raise > + except BrokenPipeError: > + pass > > r = http.getresponse() > if r.status != 200: > @@ -425,9 +423,8 @@ def emulate_zero(h, count, offset): > http.send(buf) > count -= len(buf) > http.send(buffer(buf, 0, count)) > - except socket.error as e: > - if e[0] != errno.EPIPE: > - raise > + except BrokenPipeError: >...

...@@ def pwrite(h, buf, offset): try: http.send(buf) - except BrokenPipeError: - pass + except socket.error as e: + if e.args[0] not in (errno.EPIPE, errno.ESHUTDOWN): + raise r = http.getresponse() if r.status != 200: @@ -446,8 +447,9 @@ def emulate_zero(h, count, offset): http.send(buf) count -= len(buf) http.send(buffer(buf, 0, count)) - except BrokenPipeError: - pass + except socket.error as e: + if e.args[0] not in (errno.EPIPE, errno.ESHUTDOWN): +...