search for: copyonread

From: "Richard W.M. Jones" <rjones@redhat.com> We were dropping the add_drive copyonread flag when using the libvirt backend. This resulted in significant performance degradation (2x-3x slower) when running virt-v2v against VMware servers. Thanks: Kun Wei. --- lib/launch-libvirt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/launch-libvirt.c b/lib/launch...

...re us to answer the previous one. > - Do we want to add the same optional parameters to 'add_drive_scratch' > API method? I think it would be nice but it is up to you. It should also be added to add_domain and add_libvirt_dom (note all the APIs which have 'discard' and 'copyonread' already). It could be added to add_drive_scratch, I guess. However it doesn't seem very useful for scratch drives (why create a scratch drive with 4K sectors which will be thrown away in the end?) > - What about 'add_dom', 'add_libvirt_dom' API methods? Should they a...

...OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username"; OString "secret"; OString "cachemode"; OString "discard"; OBool "copyonread"]; + style = RErr, [String (PlainString, "filename")], [OBool "readonly"; OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username&quot...

...OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username"; OString "secret"; OString "cachemode"; OString "discard"; OBool "copyonread"]; + style = RErr, [String (PlainString, "filename")], [OBool "readonly"; OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username&quot...

...OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username"; OString "secret"; OString "cachemode"; OString "discard"; OBool "copyonread"]; + style = RErr, [String (PlainString, "filename")], [OBool "readonly"; OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username&quot...

...() then g#set_trace true; - if verbose () then g#set_verbose true; - g#set_network true; - List.iter ( - fun ({ov_overlay_file = overlay_file}) -> - g#add_drive_opts overlay_file - ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" - ~copyonread:true - ) overlays; + let g = open_guestfs () in + populate_overlays g overlays; g#launch (); @@ -261,6 +251,24 @@ and init_targets overlays source output output_format = output#prepare_targets source targets +and open_guestfs () = + (* Open the guestfs handle. *) + let g = new G....

This is only lightly tested at the moment. For context see: https://bugzilla.redhat.com/show_bug.cgi?id=1138203#c40 https://bugzilla.redhat.com/show_bug.cgi?id=1075143 https://bugzilla.redhat.com/show_bug.cgi?id=1075164 Note this is not a complete fix. At least one more libguestfs patch is required (to implement virDomainPtr in the python bindings). Plus a virt-manager patch. Rich.

On 6/19/20 7:47 AM, Richard W.M. Jones wrote: > This appears to be the cause of timeouts during the conversion step > where VMware VCenter server's Tomcat HTTPS server stops responding to > requests (or rather, responds only with 503 errors). The server later > recovers and in fact because of the retry filter the conversion > usually succeeds, but I found that we can avoid the

This completes the implementation on the libguestfs side, allowing python-libvirt dom pointers to be passed to guestfs_add_libvirt_dom. For context see: https://bugzilla.redhat.com/show_bug.cgi?id=1138203#c40 https://bugzilla.redhat.com/show_bug.cgi?id=1075143 https://bugzilla.redhat.com/show_bug.cgi?id=1075164 Rich.

A hopefully cleaner implementation this time. It doesn't require any special insights into how libvirt-python is implemented. Instead, it requires a change to libvirt-python to add a .c_pointer() method: https://www.redhat.com/archives/libvir-list/2014-December/msg00615.html Rich.

Since v3: - Fix labelling over overlays (see 6/6) - Tested it with a test program which simulates what virt-manager will do. See the attachment here: https://bugzilla.redhat.com/show_bug.cgi?id=1075164#c7 Rich.

...() then g#set_trace true; - if verbose () then g#set_verbose true; - g#set_network true; - List.iter ( - fun ({ov_overlay_file = overlay_file}) -> - g#add_drive_opts overlay_file - ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" - ~copyonread:true - ) overlays; - - g#launch (); - +let init_targets overlays source output output_format = (* Work out where we will write the final output. Do this early * just so we can display errors to the user before doing too much * work. @@ -213,7 +180,43 @@ let rec main () = ta...

On 12/28/18 12:45 PM, Richard W.M. Jones wrote: > The same as qemu's copyonread flag, this caches read requests. > --- > filters/cache/nbdkit-cache-filter.pod | 11 +++++ > filters/cache/cache.c | 37 +++++++++++++-- > tests/Makefile.am | 4 +- > tests/test-cache-on-read.sh | 66 +++++++++++++++++++++++++++ > 4...

Hi All, Here's the latest patch. I think this should address the problem. The query string is now only appended to the end of a URI in the HTTP and HTTPS cases. The add-uri test now passes, and 'make check' still passes. -- Gabriel

...OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username"; OString "secret"; OString "cachemode"; OString "discard"; OBool "copyonread"]; + style = RErr, [String "filename"], [OBool "readonly"; OString "format"; OString "iface"; OString "name"; OString "label"; OString "protocol"; OStringList "server"; OString "username"; OString &quo...

...erbose () then g#set_verbose true; g#set_network true; List.iter ( - fun (overlay_file, _) -> + fun ({ov_overlay_file = overlay_file}) -> g#add_drive_opts overlay_file ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" ~copyonread:true @@ -108,20 +75,6 @@ let rec main () = g#launch (); - (* Create the list of overlays structs. Query each disk for its - * virtual size, and fill in a few other fields. - *) - let overlays = - mapi ( - fun i (overlay_file, source) -> - let sd = "sd" ^ dr...

...) in if trace () then g#set_trace true; if verbose () then g#set_verbose true; g#set_network true; - List.iter ( - fun (overlay_file, _) -> - g#add_drive_opts overlay_file - ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" - ~copyonread:true - ) overlays; + if in_place then ( + List.iter ( + fun ({s_qemu_uri = qemu_uri; s_format = format}) -> + match format with + | None -> + g#add_drive_opts qemu_uri ~cachemode:"unsafe" ~discard:"besteffort" + | Some fmt -> +...

...e true; > if verbose () then g#set_verbose true; > g#set_network true; > - List.iter ( > - fun (overlay_file, _) -> > - g#add_drive_opts overlay_file > - ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" > - ~copyonread:true > - ) overlays; > + if in_place then ( > + List.iter ( > + fun ({s_qemu_uri = qemu_uri; s_format = format}) -> > + match format with > + | None -> > + g#add_drive_opts qemu_uri ~cachemode:"unsafe" ~discard:"besteffort...

...erbose () then g#set_verbose true; g#set_network true; List.iter ( - fun (overlay_file, _) -> + fun ({ov_overlay_file = overlay_file}) -> g#add_drive_opts overlay_file ~format:"qcow2" ~cachemode:"unsafe" ~discard:"besteffort" ~copyonread:true @@ -172,20 +178,6 @@ let rec main () = g#launch (); - (* Create the list of overlays structs. Query each disk for its - * virtual size, and fill in a few other fields. - *) - let overlays = - mapi ( - fun i (overlay_file, source) -> - let sd = "sd" ^ d...

This series is a second attempt to add a mode of virt-v2v operation where it leaves the config and disk image conversion, rollback on errors, registering with the destination hypervisor, etc. to a third-party toolset, and performs only tuning of the guest OS to run in the KVM-based hypervisor. The first 14 patches are just refactoring and rearrangement of the code, factoring the implementation