search for: find_file_in_tar

...--- > v2v/utils.ml | 23 +++++++++++++++++++++++ > v2v/utils.mli | 11 +++++++++++ > 2 files changed, 34 insertions(+) > > diff --git a/v2v/utils.ml b/v2v/utils.ml > index 1ceba94cd..bf4106620 100644 > --- a/v2v/utils.ml > +++ b/v2v/utils.ml > @@ -196,3 +196,26 @@ let find_file_in_tar tar filename = > ) > in > loop lines > + > +let run_python ?(python = "python") code = > + (* In debug mode output the python code into the log. *) > + debug "running python code using ‘%s’:\n%s" python code; > + > + (* Write the Python c...

...json:%s'%s" + (quote tmp1) + (quote (JSON.string_of_doc ~fmt:JSON.Compact json)) + (if verbose () then "" else " >/dev/null 2>&1") in + + let r = Sys.command cmd in + + Unix.unlink tmp1; + Unix.unlink tmp2; + + r = 0 let find_file_in_tar tar filename = let lines = external_command (sprintf "tar tRvf %s" (Filename.quote tar)) in diff --git a/v2v/utils.mli b/v2v/utils.mli index eb47d03..b75baa7 100644 --- a/v2v/utils.mli +++ b/v2v/utils.mli @@ -51,10 +51,9 @@ val du : string -> int64 This can raise either [Failur...

v1 -> v2: - Use 'qemu-img info' which avoids one of the temporary files (thanks Tomáš). - Add a unit test. Unfortunately when I run 'make -C v2v check' I don't think this code is being run at all, so there may be something else going on here which I don't understand. Rich.

Not tested yet, am running the tests now. Rich.

--- v2v/utils.ml | 23 +++++++++++++++++++++++ v2v/utils.mli | 11 +++++++++++ 2 files changed, 34 insertions(+) diff --git a/v2v/utils.ml b/v2v/utils.ml index 1ceba94cd..bf4106620 100644 --- a/v2v/utils.ml +++ b/v2v/utils.ml @@ -196,3 +196,26 @@ let find_file_in_tar tar filename = ) in loop lines + +let run_python ?(python = "python") code = + (* In debug mode output the python code into the log. *) + debug "running python code using ‘%s’:\n%s" python code; + + (* Write the Python code to a temporary file so we don't have...

v2: - 1/3: same as in previous version - 2/3: add check for libvirt version - 3/3: restore the disabled test The libvirt patch [2] that should fix our problem with 'raw' driver that is mentioned in [1] was merged. To realy fix things, there is at least one thing we have to change on our side. We have to be explicit about the driver in the JSON we generate. [1]

...this function. I think one improvement would be to move it to v2v/utils.ml. It clutters up the code being buried here, when really it's a utility function. > + (* Find file in [tar] archive and return at which byte it starts and how > + * long it is. > + *) > + let find_file_in_tar tar filename = This function is saying "utility" to me as well. An advantage of putting these functions into utils.ml is that they will have a properly defined and typed interface (in utils.mli) so that people won't need to understand exactly how they work in order to work out the i...

...ich we currently understand for '-i ova' are: tar (uncompressed, compress with gzip or xz), zip") ova ) @@ -135,6 +146,49 @@ object loop [dir] in + (* Find file in [tar] archive and return at which byte it starts and how + * long it is. + *) + let find_file_in_tar tar filename = + let cmd1 = [ "tar"; "tRvf"; tar ] in + let cmd2 = [ "awk"; sprintf + "$8 == \"%s\" {print substr($2, 1, index($2, \":\")-1), $5}" + filename ] + in + let lines = + external_command...

...ich we currently understand for '-i ova' are: tar (uncompressed, compress with gzip or xz), zip") ova ) @@ -135,6 +183,59 @@ object loop [dir] in + (* Find file in [tar] archive and return at which byte it starts and how + * long it is. + *) + let find_file_in_tar tar filename = + let lines = external_command (stringify_args [ "tar"; "tRvf"; tar ]) in + let rec loop lines = + match lines with + | [] -> raise Not_found + | line :: lines -> ( + (* Lines have the form: + * block <off...

v7: - rebased because patch 1/3 has been pushed - changes to nsplit have been dropped (2/3) - addressed Richard's comments, notably the subfolder function was moved to mllib and renamed to subdirectory v6: - just rebase v5: - rebase, patches 1,3,5 were merged - 1/3: we still need to discuss whether to detect compressed discs - 2/3: - renamed argument noempty to keep_empty - tests were

...AUTH_SOCK") + with Not_found -> + error (f_"ssh-agent authentication has not been set up ($SSH_AUTH_SOCK is not set). This is required by qemu to do passwordless ssh access. See the virt-v2v(1) man page for more information.") + let ws = PCRE.compile "\\s+" let find_file_in_tar tar filename = diff --git a/v2v/utils.mli b/v2v/utils.mli index 8d902a53a..422fde298 100644 --- a/v2v/utils.mli +++ b/v2v/utils.mli @@ -53,6 +53,8 @@ val qemu_img_supports_offset_and_size : unit -> bool val backend_is_libvirt : unit -> bool (** Return true iff the current backend is libvirt...

...* we must uncompress it into the tmpdir. @@ -302,9 +353,39 @@ object ) else filename in + let qemu_uri = + if not partial then ( + filename + ) + else ( + let offset, size = + try find_file_in_tar ova filename + with + | Not_found -> + error (f_"file '%s' not found in the ova") filename + | Failure msg -> error (f_"%s") msg in + (* QEMU requires size aligned to 512 bytes. This is safe...

Originally an attempt to fix: https://bugzilla.redhat.com/show_bug.cgi?id=1570407 However this isn't a complete fix. The OVA supplied doesn't even conform to VMware's own "specification" (I use the word loosely). The OVF inside the OVA references the disk.vmdk file, but the OVA doesn't contain that disk.vmdk file, only a snapshot called disk.vmdk.000000000. Therefore

...30 minutes for extracting the > tarball, and about 2 hours to merge 250-300GB image) I guess the nbdkit approach would be better given the multiple and missing files within the tar file. You'll have to use ‘tar tRvf file.xva’ to extract the offset and size of each file. (See the function ‘find_file_in_tar’ in virt-v2v source for exactly how). This will give you one offset/size/filename tuple for each file. The plugin will then simply need to calculate which file to access to resolve each virtual file range (or substitute zeroes for missing files). Note that nbdkit lets you write plugins in high-l...

v6: - just rebase v5: - rebase, patches 1,3,5 were merged - 1/3: we still need to discuss whether to detect compressed discs - 2/3: - renamed argument noempty to keep_empty - tests were not run - 3/3: - using JSON module to generate JSON (as suggested by Pino) - all the other comments raised by Pino v4: - rebase to more recent master - 1/6: using just "quote" instead of

This patch set is still for discussion only. See 3/3 for the current list of problems. However this will upload an image to an oVirt or RHV server, although you must have absolutely the latest snapshot version of 4.2 for it to work. Rich.

...patch. As pointed out before I didn't find a way how to detect that by using the tar tool only and would probably require use of some external library. Tomáš Golembiovský (4): mllib: add subdirectory function to common_utils v2v: add function qemu_img_version to utils v2v: add function find_file_in_tar to utils v2v: ova: don't extract files from OVA if it's not needed mllib/common_utils.ml | 9 +++ mllib/common_utils.mli | 10 +++ mllib/common_utils_tests.ml | 7 ++ test-data/test-utils.sh | 19 +++++ v2v/Makefile.am...

...er error messages") + filename; + new_filename + ) + else filename in - let qemu_uri = - if not partial then ( - filename - ) - else ( - let offset, size = - try find_file_in_tar ova filename - with - | Not_found -> - error (f_"file '%s' not found in the ova") filename - | Failure msg -> error (f_"%s") msg in - (* QEMU requires size aligned to 512 bytes. This is safe...

https://bugzilla.redhat.com/show_bug.cgi?id=1570407 This turned into quite an in-depth refactoring of how we handle OVAs. It also fixes a potential security issue. Rich.

The first patch was previously posted here: https://www.redhat.com/archives/libguestfs/2017-December/msg00018.html That patch hasn't changed except that I made the ‘input_transport’ variable type-safe. The second patch adds a significant new mode for liberating data from VMware: the ability to copy VMs over SSH directly from ESXi hypervisors. Although this requires enabling SSH access (a