thr3ads.net - Libguestfs - [Libguestfs] [PATCH v3 3/6] v2v: ova: move the untar function [Dec 2016]

If this information is useful, please help other people find it:
Share via:

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

[Libguestfs] [PATCH v3 0/6] Import directly from OVA tar archive if possible

v3: Addressed Pino's comments, namely:
- input_ova.ml
  - untar takes list of paths
  - renamed untar_partial to untar_metadata
  - replaced uggly regex with nsplit
- tests
  - test changes are part of the main commit
  - renamed test-data/guestfs-hashsums.sh to test-data/test-utils.sh
  - renamed qemu_version to qemu_is_version and moved it to
    test-data/test-utils.sh
  - normalize paths in expect files


v2:
- rewritten the tar invocations, the output processing is now done in
  OcaML rather than with a shell code; it turned out to be easier and
  more readable than I have feared.
- added QEMU version check
- addressed Pino's comments
- changed tests; the expected result is now based on the QEMU used
  during testing

This series is related to the problem of inefficient import of OVA
files. The needed enhancements of QEMU were merged into the codebase and
should be available in QEMU 2.8. From there we can use 'size' and
'offset' options in raw driver to tell QEMU to use only subset of a file
as an image.

The patch set is more or less complete. The only outstanding issue is
the missing detection of sparse files inside tar. But this can be done
in a separate patch. As pointed out before I didn't find a way how to
detect that by using the tar tool only and would probably require use of
some external library.

The first three patches are just preparation. The main work is in patch
four. Last patch fixes the tests.


Tomáš Golembiovský (6):
  mllib: compute checksum of file inside tar
  v2v: ova: don't detect compressed disks, read the OVF instead
  v2v: ova: move the untar function
  mllib: modify nsplit to take optional noempty and count arguments
  tests: rename guestfs-hashsums.sh to test-utils.sh
  v2v: ova: don't extract files from OVA if it's not needed

 mllib/checksums.ml                               |  11 +-
 mllib/checksums.mli                              |   7 +-
 mllib/common_utils.ml                            |  12 +-
 mllib/common_utils.mli                           |  12 +-
 test-data/Makefile.am                            |   2 +-
 test-data/{guestfs-hashsums.sh => test-utils.sh} |  20 +++
 tests/qemu/qemu-liveness.sh                      |   2 +-
 tests/qemu/qemu-snapshot-isolation.sh            |   2 +-
 v2v/Makefile.am                                  |   1 +
 v2v/input_ova.ml                                 | 211 ++++++++++++++++++++---
 v2v/test-v2v-i-ova-formats.sh                    |   7 +-
 v2v/test-v2v-i-ova-gz.ovf                        |   2 +-
 v2v/test-v2v-i-ova-gz.sh                         |   2 +-
 v2v/test-v2v-i-ova-subfolders.expected2          |  18 ++
 v2v/test-v2v-i-ova-subfolders.sh                 |  15 +-
 v2v/test-v2v-i-ova-tar.expected                  |  18 ++
 v2v/test-v2v-i-ova-tar.expected2                 |  18 ++
 v2v/test-v2v-i-ova-tar.ovf                       | 138 +++++++++++++++
 v2v/test-v2v-i-ova-tar.sh                        |  71 ++++++++
 v2v/test-v2v-i-ova-two-disks.expected2           |  19 ++
 v2v/test-v2v-i-ova-two-disks.sh                  |  15 +-
 v2v/test-v2v-i-ova.sh                            |   2 +-
 v2v/test-v2v-in-place.sh                         |   2 +-
 23 files changed, 553 insertions(+), 54 deletions(-)
 rename test-data/{guestfs-hashsums.sh => test-utils.sh} (73%)
 create mode 100644 v2v/test-v2v-i-ova-subfolders.expected2
 create mode 100644 v2v/test-v2v-i-ova-tar.expected
 create mode 100644 v2v/test-v2v-i-ova-tar.expected2
 create mode 100644 v2v/test-v2v-i-ova-tar.ovf
 create mode 100755 v2v/test-v2v-i-ova-tar.sh
 create mode 100644 v2v/test-v2v-i-ova-two-disks.expected2

-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 mllib/checksums.ml  | 11 +++++++++--
 mllib/checksums.mli |  7 +++++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/mllib/checksums.ml b/mllib/checksums.ml
index dfa8c3a..a6c995b 100644
--- a/mllib/checksums.ml
+++ b/mllib/checksums.ml
@@ -45,7 +45,7 @@ let of_string csum_type csum_value    | "sha512"
-> SHA512 csum_value
   | _ -> invalid_arg csum_type
 
-let verify_checksum csum filename +let verify_checksum csum ?tar filename   
let prog, csum_ref      match csum with
     | SHA1 c -> "sha1sum", c
@@ -53,7 +53,14 @@ let verify_checksum csum filename      | SHA512 c ->
"sha512sum", c
   in
 
-  let cmd = sprintf "%s %s" prog (Filename.quote filename) in
+  let cmd +    match tar with
+    | None ->
+      sprintf "%s %s" prog (Filename.quote filename)
+    | Some tar ->
+      sprintf "tar xOf %s %s | %s"
+        (Filename.quote tar) (Filename.quote filename) prog
+  in
   let lines = external_command cmd in
   match lines with
   | [] ->
diff --git a/mllib/checksums.mli b/mllib/checksums.mli
index 0074837..9f7041b 100644
--- a/mllib/checksums.mli
+++ b/mllib/checksums.mli
@@ -29,8 +29,11 @@ val of_string : string -> string -> csum_t
 
     Raise [Invalid_argument] if the checksum type is not known. *)
 
-val verify_checksum : csum_t -> string -> unit
-(** Verify the checksum of the file. *)
+val verify_checksum : csum_t -> ?tar:string -> string -> unit
+(** [verify_checksum type filename] Verify the checksum of the file.
+
+    When optional [tar] is used it is path to uncompressed tar archive
+    and the [filename] is a path in the tar archive. *)
 
 val verify_checksums : csum_t list -> string -> unit
 (** Verify all the checksums of the file. *)
-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

The information whether the disk is gzip compressed or not is stored
in the OVF. There is no reason to do the detection.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 v2v/input_ova.ml          | 28 +++++++++++++++++-----------
 v2v/test-v2v-i-ova-gz.ovf |  2 +-
 2 files changed, 18 insertions(+), 12 deletions(-)

diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
index b283629..61930f0 100644
--- a/v2v/input_ova.ml
+++ b/v2v/input_ova.ml
@@ -275,6 +275,13 @@ object
             | None -> error (f_"no href in ovf:File (id=%s)")
file_ref
             | Some s -> s in
 
+          let expr = sprintf
"/ovf:Envelope/ovf:References/ovf:File[@ovf:id='%s']/@ovf:compression"
file_ref in
+          let compressed +            match xpath_string expr with
+            | None | Some "identity" -> false
+            | Some "gzip" -> true
+            | Some s -> error (f_"unsupported compression in OVF:
%s") s in
+
           (* Does the file exist and is it readable? *)
           let filename = ovf_folder // filename in
           Unix.access filename [Unix.R_OK];
@@ -282,17 +289,16 @@ object
           (* The spec allows the file to be gzip-compressed, in which case
            * we must uncompress it into the tmpdir.
            *)
-          let filename -            if detect_file_type filename = `GZip then (
-              let new_filename = tmpdir // String.random8 () ^
".vmdk" in
-              let cmd -                sprintf "zcat %s > %s"
(quote filename) (quote new_filename) in
-              if shell_command cmd <> 0 then
-                error (f_"error uncompressing %s, see earlier error
messages")
-                  filename;
-              new_filename
-            )
-            else filename in
+          let filename = if compressed then (
+            let new_filename = tmpdir // String.random8 () ^ ".vmdk"
in
+            let cmd +              sprintf "zcat %s > %s" (quote
filename) (quote new_filename) in
+            if shell_command cmd <> 0 then
+              error (f_"error uncompressing %s, see earlier error
messages")
+                filename;
+            new_filename
+          )
+          else filename in
 
           let disk = {
             s_disk_id = i;
diff --git a/v2v/test-v2v-i-ova-gz.ovf b/v2v/test-v2v-i-ova-gz.ovf
index e10ad2b..4a03e85 100644
--- a/v2v/test-v2v-i-ova-gz.ovf
+++ b/v2v/test-v2v-i-ova-gz.ovf
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <Envelope vmw:buildId="build-1750787"
xmlns="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common"
xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData"
xmlns:vmw="http://www.vmware.com/schema/ovf"
xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <References>
-    <File ovf:href="disk1.vmdk.gz" ovf:id="file1"
ovf:size="7804077568"/>
+    <File ovf:href="disk1.vmdk.gz" ovf:id="file1"
ovf:size="7804077568" ovf:compression="gzip"/>
   </References>
   <DiskSection>
     <Info>Virtual disk information</Info>
-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 3/6] v2v: ova: move the untar function

Move the untar function so it can be used later in the code.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 v2v/input_ova.ml | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
index 61930f0..85954a3 100644
--- a/v2v/input_ova.ml
+++ b/v2v/input_ova.ml
@@ -38,6 +38,12 @@ object
   method as_options = "-i ova " ^ ova
 
   method source () +
+    let untar ?(format = "") file outdir +      let cmd = [
"tar"; sprintf "-x%sf" format; file; "-C"; outdir
] in
+      if run_command cmd <> 0 then
+        error (f_"error unpacking %s, see earlier error messages")
ova in
+
     (* Extract ova file. *)
     let exploded        (* The spec allows a directory to be specified as an
ova.  This
@@ -61,11 +67,6 @@ object
 
           tmpfile in
 
-        let untar ?(format = "") file outdir -          let cmd = [
"tar"; sprintf "-x%sf" format; file; "-C"; outdir
] in
-          if run_command cmd <> 0 then
-            error (f_"error unpacking %s, see earlier error
messages") ova in
-
         match detect_file_type ova with
         | `Tar ->
           (* Normal ovas are tar file (not compressed). *)
-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 4/6] mllib: modify nsplit to take optional noempty and count arguments

Added two new optional arguments to nsplit:

* noempty: if set to false empty elements are not stored in the returned
  list. The default is to keep the empty elements

* count: specifies how many splits to perform; negative count
  (the default) means do as many splits as possible

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 mllib/common_utils.ml  | 12 +++++++++---
 mllib/common_utils.mli | 12 ++++++++++--
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/mllib/common_utils.ml b/mllib/common_utils.ml
index 78618f5..2d373f9 100644
--- a/mllib/common_utils.ml
+++ b/mllib/common_utils.ml
@@ -92,15 +92,21 @@ module String = struct
         s' ^ s2 ^ replace s'' s1 s2
       )
 
-    let rec nsplit sep str +    let rec nsplit ?(noempty = true) ?(count = -1)
sep str        let len = length str in
       let seplen = length sep in
       let i = find str sep in
-      if i = -1 then [str]
+      if i = -1 || count = 0 then [str]
       else (
         let s' = sub str 0 i in
         let s'' = sub str (i+seplen) (len-i-seplen) in
-        s' :: nsplit sep s''
+        let elem, count +          if s' = "" && noempty
then
+            [], count
+          else
+            [s'], if count > 0 then count-1 else count
+        in
+        elem @ nsplit ~noempty:noempty ~count:count sep s''
       )
 
     let split sep str diff --git a/mllib/common_utils.mli
b/mllib/common_utils.mli
index ad43345..bcd459d 100644
--- a/mllib/common_utils.mli
+++ b/mllib/common_utils.mli
@@ -64,9 +64,17 @@ module String : sig
     val replace : string -> string -> string -> string
     (** [replace str s1 s2] replaces all instances of [s1] appearing in
         [str] with [s2]. *)
-    val nsplit : string -> string -> string list
+    val nsplit : ?noempty:bool -> ?count:int -> string -> string ->
string list
     (** [nsplit sep str] splits [str] into multiple strings at each
-        separator [sep]. *)
+        separator [sep].
+
+        If [noempty] is set to true empty elements are not included in the
+        list. By default empty elements are kept.
+
+        If [count] is specified it says how many splits to perform. I.e. the
+        returned array will have at most [count]+1 elements. Negative [count]
+        (the default) means do as many splits as possible.
+        *)
     val split : string -> string -> string * string
     (** [split sep str] splits [str] at the first occurrence of the
         separator [sep], returning the part before and the part after.
-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 5/6] tests: rename guestfs-hashsums.sh to test-utils.sh

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 test-data/Makefile.am                            | 2 +-
 test-data/{guestfs-hashsums.sh => test-utils.sh} | 0
 tests/qemu/qemu-liveness.sh                      | 2 +-
 tests/qemu/qemu-snapshot-isolation.sh            | 2 +-
 v2v/test-v2v-i-ova-formats.sh                    | 2 +-
 v2v/test-v2v-i-ova-gz.sh                         | 2 +-
 v2v/test-v2v-i-ova-subfolders.sh                 | 2 +-
 v2v/test-v2v-i-ova-two-disks.sh                  | 2 +-
 v2v/test-v2v-i-ova.sh                            | 2 +-
 v2v/test-v2v-in-place.sh                         | 2 +-
 10 files changed, 9 insertions(+), 9 deletions(-)
 rename test-data/{guestfs-hashsums.sh => test-utils.sh} (100%)

diff --git a/test-data/Makefile.am b/test-data/Makefile.am
index 86cd5a2..4ac4c91 100644
--- a/test-data/Makefile.am
+++ b/test-data/Makefile.am
@@ -27,7 +27,7 @@ SUBDIRS += files
 SUBDIRS += .
 
 EXTRA_DIST = \
-        guestfs-hashsums.sh
+        test-utils.sh
 
 # Build an ISO containing various files from the subdirectories, which
 # is used by tests/c-api and a few guestfish tests and regression
diff --git a/test-data/guestfs-hashsums.sh b/test-data/test-utils.sh
similarity index 100%
rename from test-data/guestfs-hashsums.sh
rename to test-data/test-utils.sh
diff --git a/tests/qemu/qemu-liveness.sh b/tests/qemu/qemu-liveness.sh
index fda3cc2..ee22b66 100755
--- a/tests/qemu/qemu-liveness.sh
+++ b/tests/qemu/qemu-liveness.sh
@@ -22,7 +22,7 @@
 
 set -e
 
-. $srcdir/../../test-data/guestfs-hashsums.sh
+. $srcdir/../../test-data/test-utils.sh
 
 rm -f liveness1.img
 
diff --git a/tests/qemu/qemu-snapshot-isolation.sh
b/tests/qemu/qemu-snapshot-isolation.sh
index 4d60e59..78cc69c 100755
--- a/tests/qemu/qemu-snapshot-isolation.sh
+++ b/tests/qemu/qemu-snapshot-isolation.sh
@@ -22,7 +22,7 @@
 
 set -e
 
-. $srcdir/../../test-data/guestfs-hashsums.sh
+. $srcdir/../../test-data/test-utils.sh
 
 # UML backend doesn't support qcow2 format.
 supports_qcow2=yes
diff --git a/v2v/test-v2v-i-ova-formats.sh b/v2v/test-v2v-i-ova-formats.sh
index d113994..3105c62 100755
--- a/v2v/test-v2v-i-ova-formats.sh
+++ b/v2v/test-v2v-i-ova-formats.sh
@@ -46,7 +46,7 @@ fi
 
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=test-v2v-i-ova-formats.d
 rm -rf $d
diff --git a/v2v/test-v2v-i-ova-gz.sh b/v2v/test-v2v-i-ova-gz.sh
index a38e1b4..e394ed7 100755
--- a/v2v/test-v2v-i-ova-gz.sh
+++ b/v2v/test-v2v-i-ova-gz.sh
@@ -34,7 +34,7 @@ fi
 
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=test-v2v-i-ova-gz.d
 rm -rf $d
diff --git a/v2v/test-v2v-i-ova-subfolders.sh b/v2v/test-v2v-i-ova-subfolders.sh
index c49f7cb..4fd1ace 100755
--- a/v2v/test-v2v-i-ova-subfolders.sh
+++ b/v2v/test-v2v-i-ova-subfolders.sh
@@ -34,7 +34,7 @@ fi
 
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=test-v2v-i-ova-subfolders.d
 rm -rf $d
diff --git a/v2v/test-v2v-i-ova-two-disks.sh b/v2v/test-v2v-i-ova-two-disks.sh
index aefd90e..a6ca671 100755
--- a/v2v/test-v2v-i-ova-two-disks.sh
+++ b/v2v/test-v2v-i-ova-two-disks.sh
@@ -35,7 +35,7 @@ fi
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 export VIRTIO_WIN="$srcdir/../test-data/fake-virtio-win"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=test-v2v-i-ova-two-disks.d
 rm -rf $d
diff --git a/v2v/test-v2v-i-ova.sh b/v2v/test-v2v-i-ova.sh
index 716cd33..3c93c63 100755
--- a/v2v/test-v2v-i-ova.sh
+++ b/v2v/test-v2v-i-ova.sh
@@ -41,7 +41,7 @@ fi
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 export VIRTIO_WIN="$srcdir/../test-data/fake-virtio-win"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=test-v2v-i-ova.d
 rm -rf $d
diff --git a/v2v/test-v2v-in-place.sh b/v2v/test-v2v-in-place.sh
index 444790c..9424b7f 100755
--- a/v2v/test-v2v-in-place.sh
+++ b/v2v/test-v2v-in-place.sh
@@ -44,7 +44,7 @@ fi
 export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
 export VIRTIO_WIN="$srcdir/../test-data/fake-virtio-win"
 
-. $srcdir/../test-data/guestfs-hashsums.sh
+. $srcdir/../test-data/test-utils.sh
 
 d=$PWD/test-v2v-in-place.d
 rm -rf $d
-- 
2.10.2

Tomáš Golembiovský

2016-Dec-07 16:13 UTC

head link

[Libguestfs] [PATCH v3 6/6] v2v: ova: don't extract files from OVA if it's not needed

We don't have to always extract all files from the OVA archive. The OVA,
as defined in the standard, is plain tar. We can work directly over the
tar archive if we use correct 'offset' and 'size' options when
defining
the backing file for QEMU. This puts much lower requirement on available
disk space.

Since the virt-v2v behaviour for OVA input now depends on QEMU version
available this affects some of the tests. The affected tests will have
two *.expect files and the expected result also has to depend on the
QEMU used.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
---
 test-data/test-utils.sh                 |  20 ++++
 v2v/Makefile.am                         |   1 +
 v2v/input_ova.ml                        | 180 +++++++++++++++++++++++++++++---
 v2v/test-v2v-i-ova-formats.sh           |   5 +-
 v2v/test-v2v-i-ova-subfolders.expected2 |  18 ++++
 v2v/test-v2v-i-ova-subfolders.sh        |  13 ++-
 v2v/test-v2v-i-ova-tar.expected         |  18 ++++
 v2v/test-v2v-i-ova-tar.expected2        |  18 ++++
 v2v/test-v2v-i-ova-tar.ovf              | 138 ++++++++++++++++++++++++
 v2v/test-v2v-i-ova-tar.sh               |  71 +++++++++++++
 v2v/test-v2v-i-ova-two-disks.expected2  |  19 ++++
 v2v/test-v2v-i-ova-two-disks.sh         |  13 ++-
 12 files changed, 491 insertions(+), 23 deletions(-)
 create mode 100644 v2v/test-v2v-i-ova-subfolders.expected2
 create mode 100644 v2v/test-v2v-i-ova-tar.expected
 create mode 100644 v2v/test-v2v-i-ova-tar.expected2
 create mode 100644 v2v/test-v2v-i-ova-tar.ovf
 create mode 100755 v2v/test-v2v-i-ova-tar.sh
 create mode 100644 v2v/test-v2v-i-ova-two-disks.expected2

diff --git a/test-data/test-utils.sh b/test-data/test-utils.sh
index 86a5aaf..04e8333 100755
--- a/test-data/test-utils.sh
+++ b/test-data/test-utils.sh
@@ -54,3 +54,23 @@ do_sha256 ()
       ;;
   esac
 }
+
+# Returns 0 if QEMU version is greater or equal to the arguments
+qemu_is_version() {
+    if [ $# -ne 2 ] ; then
+        echo "Usage: $0 <major_version> <minor_version>"
>&2
+        return 3
+    fi
+
+    QV=$(expr match "$(qemu-img --version)" 'qemu-img version
\([0-9]\+\.[0-9]\+\)')
+    [ -z "$QV" ] && return 2
+
+    QMAJ=$(echo "$QV" | cut -d. -f1)
+    QMIN=$(echo "$QV" | cut -d. -f2)
+
+    if [ \( $QMAJ -gt $1 \) -o \( $QMAJ -eq $1 -a $QMIN -ge $2 \) ] ; then
+        return 0
+    fi
+
+    return 1
+}
diff --git a/v2v/Makefile.am b/v2v/Makefile.am
index 5e3c3eb..621ba10 100644
--- a/v2v/Makefile.am
+++ b/v2v/Makefile.am
@@ -258,6 +258,7 @@ TESTS_ENVIRONMENT = $(top_builddir)/run --test
 
 TESTS = \
 	test-v2v-docs.sh \
+	test-v2v-i-ova-tar.sh \
 	test-v2v-i-ova-formats.sh \
 	test-v2v-i-ova-gz.sh \
 	test-v2v-i-ova-subfolders.sh \
diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
index 85954a3..b7828b4 100644
--- a/v2v/input_ova.ml
+++ b/v2v/input_ova.ml
@@ -39,17 +39,23 @@ object
 
   method source ()  
-    let untar ?(format = "") file outdir -      let cmd = [
"tar"; sprintf "-x%sf" format; file; "-C"; outdir
] in
+    (* Untar part or all files from tar archive. If [paths] is specified it is
+     * a list of paths in the tar archive.
+     *)
+    let untar ?(format = "") ?paths file outdir +      let cmd +     
[ "tar"; sprintf "-x%sf" format; file; "-C";
outdir ]
+        @ (match paths with None -> [] | Some p -> p)
+      in
       if run_command cmd <> 0 then
         error (f_"error unpacking %s, see earlier error messages")
ova in
 
     (* Extract ova file. *)
-    let exploded +    let exploded, partial        (* The spec allows a
directory to be specified as an ova.  This
        * is also pretty convenient.
        *)
-      if is_directory ova then ova
+      if is_directory ova then ova, false
       else (
         let uncompress_head zcat file            let cmd = sprintf "%s
%s" zcat (quote file) in
@@ -67,11 +73,56 @@ object
 
           tmpfile in
 
+        (* Untar only ovf and manifest from the archive *)
+        let untar_metadata ova outdir +          let files +           
external_command (sprintf "tar -tf %s" (Filename.quote ova)) in
+          let files +            filter_map (fun f ->
+              if Filename.check_suffix f ".ovf" ||
+                  Filename.check_suffix f ".mf" then
+                Some f
+              else None
+            ) files in
+          untar ~paths:files ova outdir in
+
+        let qemu_img_version () +          let cmd = "qemu-img
--version" in
+          let lines = external_command cmd in
+          match lines with
+          | [] -> error ("'qemu-img --version' returned no
output")
+          | line :: _ ->
+              let rex = Str.regexp "qemu-img version
\\([0-9]+\\)\\.\\([0-9]+\\)" in
+              if Str.string_match rex line 0 then (
+                try
+                  int_of_string (Str.matched_group 1 line),
+                  int_of_string (Str.matched_group 2 line)
+                with Failure _ ->
+                  warning (f_"failed to parse qemu-img version(%S),
assuming 0.9")
+                  line;
+                  0, 9
+              ) else (
+                warning (f_"failed to read qemu-img version(%S), assuming
0.9")
+                  line;
+                0, 9
+              )
+        in
+
         match detect_file_type ova with
         | `Tar ->
           (* Normal ovas are tar file (not compressed). *)
-          untar ova tmpdir;
-          tmpdir
+          let qmajor, qminor = qemu_img_version () in
+          if qmajor > 2 || (qmajor == 2 && qminor >= 8) then (
+            (* If QEMU is recent enough we don't have to extract
everything. We
+             * can access disks inside the tar archive.
+             *)
+            untar_metadata ova tmpdir;
+            tmpdir, true
+          ) else (
+            untar ova tmpdir;
+            tmpdir, false
+          )
+
         | `Zip ->
           (* However, although not permitted by the spec, people ship
            * zip files as ova too.
@@ -81,7 +132,7 @@ object
             [ "-j"; "-d"; tmpdir; ova ] in
           if run_command cmd <> 0 then
             error (f_"error unpacking %s, see earlier error
messages") ova;
-          tmpdir
+          tmpdir, false
         | (`GZip|`XZ) as format ->
           let zcat, tar_fmt              match format with
@@ -94,7 +145,7 @@ object
           (match tmpfiletype with
           | `Tar ->
             untar ~format:tar_fmt ova tmpdir;
-            tmpdir
+            tmpdir, false
           | `Zip | `GZip | `XZ | `Unknown ->
             error (f_"%s: unsupported file format\n\nFormats which we
currently understand for '-i ova' are: tar (uncompressed, compress with
gzip or xz), zip") ova
           )
@@ -135,6 +186,68 @@ object
       loop [dir]
     in
 
+    (* Find file in [tar] archive and return at which byte it starts and how
+     * long it is.
+     *)
+    let find_file_in_tar tar filename +      let lines = external_command
(sprintf "tar tRvf %s" (Filename.quote tar)) in
+      let rec loop lines +        match lines with
+        | [] -> raise Not_found
+        | line :: lines -> (
+          (* Lines have the form:
+           * block <offset>: <perms> <owner>/<group>
<size> <mdate> <mtime> <file>
+           *)
+          let elems = String.nsplit ~noempty:true ~count:7 " " line
in
+          if (List.length elems) = 8 && (List.hd elems) =
"block" then (
+            let elems = Array.of_list elems in
+            let offset = elems.(1) in
+            let size = elems.(4) in
+            let fname = elems.(7) in
+
+            if fname <> filename then
+              loop lines
+            else (
+              let offset +                try
+                  (* There should be a colon at the end *)
+                  let i = String.rindex offset ':' in
+                  if i == (String.length offset)-1 then
+                    Int64.of_string (String.sub offset 0 i)
+                  else
+                    raise (Failure "colon at wrong position")
+                with Failure _ | Not_found ->
+                  error (f_"invalid offset returned by tar: %S")
offset in
+
+              let size +                try Int64.of_string size
+                with Failure _ ->
+                  error (f_"invalid size returned by tar: %S") size
in
+
+              (* Note: Offset is actualy block number and there is a single
+               * block with tar header at the beginning of the file. So skip
+               * the header and convert the block number to bytes before
+               * returning.
+               *)
+              (offset +^ 1L) *^ 512L, size
+            )
+          ) else
+            error (f_"failed to parse line returned by tar: %S") line
+        )
+      in
+      loop lines
+    in
+
+    let subfolder folder parent +      if folder = parent then
+        ""
+      else if String.is_prefix folder (parent // "") then
+        let len = String.length parent in
+        String.sub folder (len+1) (String.length folder-len-1)
+      else
+        assert false
+    in
+
     (* Search for the ovf file. *)
     let ovf = find_files exploded ".ovf" in
     let ovf @@ -152,6 +265,7 @@ object
       fun mf ->
         debug "processing manifest %s" mf;
         let mf_folder = Filename.dirname mf in
+        let mf_subfolder = subfolder mf_folder exploded in
         let chan = open_in mf in
         let rec loop ()            let line = input_line chan in
@@ -160,7 +274,11 @@ object
             let disk = Str.matched_group 2 line in
             let expected = Str.matched_group 3 line in
             let csum = Checksums.of_string mode expected in
-            try Checksums.verify_checksum csum (mf_folder // disk)
+            try
+              if partial then
+                Checksums.verify_checksum csum ~tar:ova (mf_subfolder // disk)
+              else
+                Checksums.verify_checksum csum (mf_folder // disk)
             with Checksums.Mismatched_checksum (_, actual) ->
               error (f_"checksum of disk %s does not match manifest %s
(actual %s(%s) = %s, expected %s(%s) = %s)")
                 disk mf mode disk actual mode disk expected;
@@ -283,9 +401,25 @@ object
             | Some "gzip" -> true
             | Some s -> error (f_"unsupported compression in OVF:
%s") s in
 
-          (* Does the file exist and is it readable? *)
-          let filename = ovf_folder // filename in
-          Unix.access filename [Unix.R_OK];
+          let partial +            if compressed && partial then (
+              (* We cannot access compressed disk inside the tar; we have to
+               * extract it *)
+              untar ~paths:[(subfolder ovf_folder exploded) // filename]
+                ova tmpdir;
+              false
+            )
+            else
+              partial in
+
+          let filename +            if partial then
+              (subfolder ovf_folder exploded) // filename
+            else (
+              (* Does the file exist and is it readable? *)
+              Unix.access (ovf_folder // filename) [Unix.R_OK];
+              ovf_folder // filename
+            ) in
 
           (* The spec allows the file to be gzip-compressed, in which case
            * we must uncompress it into the tmpdir.
@@ -301,9 +435,29 @@ object
           )
           else filename in
 
+          let qemu_uri +            if not partial then (
+              filename
+            )
+            else (
+              let offset, size +                try find_file_in_tar ova
filename
+                with Not_found ->
+                  error (f_"file '%s' not found in the ova")
filename
+              in
+              (* QEMU requires size aligned to 512 bytes. This is safe because
+               * tar also works with 512 byte blocks.
+               *)
+              let size = roundup64 size 512L in
+              sprintf
+                "json:{ \"file\": {
\"driver\":\"raw\", \"offset\":\"%Ld\",
\"size\":\"%Ld\", \"file\": {
\"filename\":\"%s\" } } }"
+                offset size ova
+            )
+          in
+
           let disk = {
             s_disk_id = i;
-            s_qemu_uri = filename;
+            s_qemu_uri = qemu_uri;
             s_format = Some "vmdk";
             s_controller = controller;
           } in
diff --git a/v2v/test-v2v-i-ova-formats.sh b/v2v/test-v2v-i-ova-formats.sh
index 3105c62..46406cf 100755
--- a/v2v/test-v2v-i-ova-formats.sh
+++ b/v2v/test-v2v-i-ova-formats.sh
@@ -22,7 +22,7 @@ unset CDPATH
 export LANG=C
 set -e
 
-formats="tar zip tar-gz tar-xz"
+formats="zip tar-gz tar-xz"
 
 if [ -n "$SKIP_TEST_V2V_I_OVA_FORMATS_SH" ]; then
     echo "$0: test skipped because environment variable is set"
@@ -62,9 +62,6 @@ echo -e "SHA1(disk1.vmdk)= $sha\r" > disk1.mf
 
 for format in $formats; do
     case "$format" in
-        tar)
-            tar -cf test-$format.ova ../test-v2v-i-ova-formats.ovf disk1.vmdk
disk1.mf
-            ;;
         zip)
             zip -r test ../test-v2v-i-ova-formats.ovf disk1.vmdk disk1.mf
             mv test.zip test-$format.ova
diff --git a/v2v/test-v2v-i-ova-subfolders.expected2
b/v2v/test-v2v-i-ova-subfolders.expected2
new file mode 100644
index 0000000..b4b3a57
--- /dev/null
+++ b/v2v/test-v2v-i-ova-subfolders.expected2
@@ -0,0 +1,18 @@
+Source guest information (--print-source option):
+
+    source name: 2K8R2EESP1_2_Medium
+hypervisor type: vmware
+         memory: 1073741824 (bytes)
+       nr vCPUs: 1
+   CPU features: 
+       firmware: uefi
+        display: 
+          video: 
+          sound: 
+disks:
+	json:{ "file": { "driver":"raw",
"offset":"2048", "size":"10240",
"file": { "filename":"test.ova" } } } (vmdk)
[scsi]
+removable media:
+	CD-ROM [ide] in slot 0
+NICs:
+	Network "Network adapter 1"
+
diff --git a/v2v/test-v2v-i-ova-subfolders.sh b/v2v/test-v2v-i-ova-subfolders.sh
index 4fd1ace..4229590 100755
--- a/v2v/test-v2v-i-ova-subfolders.sh
+++ b/v2v/test-v2v-i-ova-subfolders.sh
@@ -56,10 +56,17 @@ popd
 # normalize the output.
 $VG virt-v2v --debug-gc --quiet \
     -i ova $d/test.ova \
-    --print-source |
-sed 's,[^ \t]*\(subfolder/disk.*\.vmdk\),\1,' > $d/source
+    --print-source > $d/source
 
 # Check the parsed source is what we expect.
-diff -u test-v2v-i-ova-subfolders.expected $d/source
+if qemu_is_version 2 8 ; then
+    # normalize the output
+    sed -i -e "s,\"$d/,\"," $d/source
+    diff -u test-v2v-i-ova-subfolders.expected2 $d/source
+else
+    # normalize the output
+    sed -i -e 's,[^ \t]*\(subfolder/disk.*\.vmdk\),\1,' $d/source
+    diff -u test-v2v-i-ova-subfolders.expected $d/source
+fi
 
 rm -rf $d
diff --git a/v2v/test-v2v-i-ova-tar.expected b/v2v/test-v2v-i-ova-tar.expected
new file mode 100644
index 0000000..7049aee
--- /dev/null
+++ b/v2v/test-v2v-i-ova-tar.expected
@@ -0,0 +1,18 @@
+Source guest information (--print-source option):
+
+    source name: 2K8R2EESP1_2_Medium
+hypervisor type: vmware
+         memory: 1073741824 (bytes)
+       nr vCPUs: 1
+   CPU features: 
+       firmware: uefi
+        display: 
+          video: 
+          sound: 
+disks:
+	disk1.vmdk (vmdk) [scsi]
+removable media:
+	CD-ROM [ide] in slot 0
+NICs:
+	Network "Network adapter 1"
+
diff --git a/v2v/test-v2v-i-ova-tar.expected2 b/v2v/test-v2v-i-ova-tar.expected2
new file mode 100644
index 0000000..adfcd9f
--- /dev/null
+++ b/v2v/test-v2v-i-ova-tar.expected2
@@ -0,0 +1,18 @@
+Source guest information (--print-source option):
+
+    source name: 2K8R2EESP1_2_Medium
+hypervisor type: vmware
+         memory: 1073741824 (bytes)
+       nr vCPUs: 1
+   CPU features: 
+       firmware: uefi
+        display: 
+          video: 
+          sound: 
+disks:
+	json:{ "file": { "driver":"raw",
"offset":"9216", "size":"10240",
"file": { "filename":"test-tar.ova" } } } (vmdk)
[scsi]
+removable media:
+	CD-ROM [ide] in slot 0
+NICs:
+	Network "Network adapter 1"
+
diff --git a/v2v/test-v2v-i-ova-tar.ovf b/v2v/test-v2v-i-ova-tar.ovf
new file mode 100644
index 0000000..4827c7e
--- /dev/null
+++ b/v2v/test-v2v-i-ova-tar.ovf
@@ -0,0 +1,138 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Envelope vmw:buildId="build-1750787"
xmlns="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common"
xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"
xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData"
xmlns:vmw="http://www.vmware.com/schema/ovf"
xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+  <References>
+    <File ovf:href="disk1.vmdk" ovf:id="file1"
ovf:size="7804077568"/>
+  </References>
+  <DiskSection>
+    <Info>Virtual disk information</Info>
+    <Disk ovf:capacity="50" ovf:capacityAllocationUnits="byte
* 2^30" ovf:diskId="vmdisk1" ovf:fileRef="file1"
ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized"
ovf:populatedSize="18975752192"/>
+  </DiskSection>
+  <NetworkSection>
+    <Info>The list of logical networks</Info>
+    <Network ovf:name="PG-VLAN60">
+      <Description>The PG-VLAN60 network</Description>
+    </Network>
+  </NetworkSection>
+  <VirtualSystem ovf:id="2K8R2EESP1_2_Medium">
+    <Info>A virtual machine</Info>
+    <Name>2K8R2EESP1_2_Medium</Name>
+    <OperatingSystemSection ovf:id="103"
vmw:osType="windows7Server64Guest">
+      <Info>The kind of installed guest operating system</Info>
+      <Description>Microsoft Windows Server 2008 R2
(64-bit)</Description>
+    </OperatingSystemSection>
+    <VirtualHardwareSection>
+      <Info>Virtual hardware requirements</Info>
+      <System>
+        <vssd:ElementName>Virtual Hardware
Family</vssd:ElementName>
+        <vssd:InstanceID>0</vssd:InstanceID>
+       
<vssd:VirtualSystemIdentifier>2K8R2EESP1_2_Medium</vssd:VirtualSystemIdentifier>
+        <vssd:VirtualSystemType>vmx-10</vssd:VirtualSystemType>
+      </System>
+      <Item>
+        <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
+        <rasd:Description>Number of Virtual CPUs</rasd:Description>
+        <rasd:ElementName>1 virtual CPU(s)</rasd:ElementName>
+        <rasd:InstanceID>1</rasd:InstanceID>
+        <rasd:ResourceType>3</rasd:ResourceType>
+        <rasd:VirtualQuantity>1</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:AllocationUnits>byte * 2^20</rasd:AllocationUnits>
+        <rasd:Description>Memory Size</rasd:Description>
+        <rasd:ElementName>1024MB of memory</rasd:ElementName>
+        <rasd:InstanceID>2</rasd:InstanceID>
+        <rasd:ResourceType>4</rasd:ResourceType>
+        <rasd:VirtualQuantity>1024</rasd:VirtualQuantity>
+      </Item>
+      <Item>
+        <rasd:Address>0</rasd:Address>
+        <rasd:Description>SCSI Controller</rasd:Description>
+        <rasd:ElementName>SCSI controller 0</rasd:ElementName>
+        <rasd:InstanceID>3</rasd:InstanceID>
+        <rasd:ResourceSubType>lsilogicsas</rasd:ResourceSubType>
+        <rasd:ResourceType>6</rasd:ResourceType>
+        <vmw:Config ovf:required="false"
vmw:key="slotInfo.pciSlotNumber" vmw:value="160"/>
+      </Item>
+      <Item>
+        <rasd:Address>1</rasd:Address>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>IDE 1</rasd:ElementName>
+        <rasd:InstanceID>4</rasd:InstanceID>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:Address>0</rasd:Address>
+        <rasd:Description>IDE Controller</rasd:Description>
+        <rasd:ElementName>IDE 0</rasd:ElementName>
+        <rasd:InstanceID>5</rasd:InstanceID>
+        <rasd:ResourceType>5</rasd:ResourceType>
+      </Item>
+      <Item ovf:required="false">
+        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
+        <rasd:ElementName>Video card</rasd:ElementName>
+        <rasd:InstanceID>6</rasd:InstanceID>
+        <rasd:ResourceType>24</rasd:ResourceType>
+        <vmw:Config ovf:required="false"
vmw:key="enable3DSupport" vmw:value="false"/>
+        <vmw:Config ovf:required="false"
vmw:key="use3dRenderer" vmw:value="automatic"/>
+        <vmw:Config ovf:required="false"
vmw:key="useAutoDetect" vmw:value="true"/>
+        <vmw:Config ovf:required="false"
vmw:key="videoRamSizeInKB" vmw:value="4096"/>
+      </Item>
+      <Item ovf:required="false">
+        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
+        <rasd:ElementName>VMCI device</rasd:ElementName>
+        <rasd:InstanceID>7</rasd:InstanceID>
+        <rasd:ResourceSubType>vmware.vmci</rasd:ResourceSubType>
+        <rasd:ResourceType>1</rasd:ResourceType>
+        <vmw:Config ovf:required="false"
vmw:key="allowUnrestrictedCommunication"
vmw:value="false"/>
+        <vmw:Config ovf:required="false"
vmw:key="slotInfo.pciSlotNumber" vmw:value="32"/>
+      </Item>
+      <Item ovf:required="false">
+        <rasd:AddressOnParent>0</rasd:AddressOnParent>
+        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
+        <rasd:ElementName>CD/DVD drive 1</rasd:ElementName>
+        <rasd:InstanceID>8</rasd:InstanceID>
+        <rasd:Parent>4</rasd:Parent>
+       
<rasd:ResourceSubType>vmware.cdrom.atapi</rasd:ResourceSubType>
+        <rasd:ResourceType>15</rasd:ResourceType>
+      </Item>
+      <Item>
+        <rasd:AddressOnParent>0</rasd:AddressOnParent>
+        <rasd:ElementName>Hard disk 1</rasd:ElementName>
+        <rasd:HostResource>ovf:/disk/vmdisk1</rasd:HostResource>
+        <rasd:InstanceID>9</rasd:InstanceID>
+        <rasd:Parent>3</rasd:Parent>
+        <rasd:ResourceType>17</rasd:ResourceType>
+        <vmw:Config ovf:required="false"
vmw:key="backing.writeThrough" vmw:value="false"/>
+      </Item>
+      <Item>
+        <rasd:AddressOnParent>7</rasd:AddressOnParent>
+        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
+        <rasd:Connection>PG-VLAN60</rasd:Connection>
+        <rasd:Description>E1000 ethernet adapter on
&quot;PG-VLAN60&quot;</rasd:Description>
+        <rasd:ElementName>Network adapter 1</rasd:ElementName>
+        <rasd:InstanceID>11</rasd:InstanceID>
+        <rasd:ResourceSubType>E1000</rasd:ResourceSubType>
+        <rasd:ResourceType>10</rasd:ResourceType>
+        <vmw:Config ovf:required="false"
vmw:key="slotInfo.pciSlotNumber" vmw:value="33"/>
+        <vmw:Config ovf:required="false"
vmw:key="wakeOnLanEnabled" vmw:value="true"/>
+      </Item>
+      <vmw:Config ovf:required="false"
vmw:key="cpuHotAddEnabled" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="cpuHotRemoveEnabled" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="firmware" vmw:value="efi"/>
+      <vmw:Config ovf:required="false"
vmw:key="virtualICH7MPresent" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="virtualSMCPresent" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="memoryHotAddEnabled" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="nestedHVEnabled" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="powerOpInfo.powerOffType" vmw:value="soft"/>
+      <vmw:Config ovf:required="false"
vmw:key="powerOpInfo.resetType" vmw:value="soft"/>
+      <vmw:Config ovf:required="false"
vmw:key="powerOpInfo.standbyAction"
vmw:value="checkpoint"/>
+      <vmw:Config ovf:required="false"
vmw:key="powerOpInfo.suspendType" vmw:value="hard"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.afterPowerOn" vmw:value="true"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.afterResume" vmw:value="true"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
+      <vmw:Config ovf:required="false"
vmw:key="tools.toolsUpgradePolicy"
vmw:value="upgradeAtPowerCycle"/>
+    </VirtualHardwareSection>
+  </VirtualSystem>
+</Envelope>                                 
diff --git a/v2v/test-v2v-i-ova-tar.sh b/v2v/test-v2v-i-ova-tar.sh
new file mode 100755
index 0000000..a2d56a7
--- /dev/null
+++ b/v2v/test-v2v-i-ova-tar.sh
@@ -0,0 +1,71 @@
+#!/bin/bash -
+# libguestfs virt-v2v test script
+# Copyright (C) 2014-2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Test -i ova option with ova file compressed in different ways
+
+unset CDPATH
+export LANG=C
+set -e
+
+if [ -n "$SKIP_TEST_V2V_I_OVA_FORMATS_SH" ]; then
+    echo "$0: test skipped because environment variable is set"
+    exit 77
+fi
+
+if [ "$(guestfish get-backend)" = "uml" ]; then
+    echo "$0: test skipped because UML backend does not support
network"
+    exit 77
+fi
+
+export VIRT_TOOLS_DATA_DIR="$srcdir/../test-data/fake-virt-tools"
+
+. $srcdir/../test-data/test-utils.sh
+
+d=test-v2v-i-ova-tar.d
+rm -rf $d
+mkdir $d
+
+pushd $d
+
+# Create a phony OVA.  This is only a test of source parsing, not
+# conversion, so the contents of the disks doesn't matter.
+truncate -s 10k disk1.vmdk
+sha=`do_sha1 disk1.vmdk`
+echo -e "SHA1(disk1.vmdk)= $sha\r" > disk1.mf
+tar -cf test-tar.ova ../test-v2v-i-ova-tar.ovf disk1.vmdk disk1.mf
+
+popd
+
+# Run virt-v2v but only as far as the --print-source stage
+$VG virt-v2v --debug-gc --quiet \
+    -i ova $d/test-tar.ova \
+    --print-source > $d/source
+
+# Check the parsed source is what we expect.
+if qemu_is_version 2 8 ; then
+    # normalize the output
+    sed -i -e "s,\"$d/,\"," $d/source
+    diff -u test-v2v-i-ova-tar.expected2 $d/source
+else
+    # normalize the output
+    sed -i -e 's,[^ \t]*\(disk.*.vmdk\),\1,' $d/source
+    diff -u test-v2v-i-ova-tar.expected $d/source
+fi
+
+
+rm -rf $d
diff --git a/v2v/test-v2v-i-ova-two-disks.expected2
b/v2v/test-v2v-i-ova-two-disks.expected2
new file mode 100644
index 0000000..0c29d9b
--- /dev/null
+++ b/v2v/test-v2v-i-ova-two-disks.expected2
@@ -0,0 +1,19 @@
+Source guest information (--print-source option):
+
+    source name: 2K8R2EESP1_2_Medium
+hypervisor type: vmware
+         memory: 1073741824 (bytes)
+       nr vCPUs: 1
+   CPU features: 
+       firmware: bios
+        display: 
+          video: 
+          sound: 
+disks:
+	json:{ "file": { "driver":"raw",
"offset":"9728", "size":"10240",
"file": { "filename":"test.ova" } } } (vmdk)
[scsi]
+	json:{ "file": { "driver":"raw",
"offset":"21504", "size":"102400",
"file": { "filename":"test.ova" } } } (vmdk)
[scsi]
+removable media:
+	CD-ROM [ide] in slot 0
+NICs:
+	Network "Network adapter 1"
+
diff --git a/v2v/test-v2v-i-ova-two-disks.sh b/v2v/test-v2v-i-ova-two-disks.sh
index a6ca671..f34df2d 100755
--- a/v2v/test-v2v-i-ova-two-disks.sh
+++ b/v2v/test-v2v-i-ova-two-disks.sh
@@ -59,10 +59,17 @@ popd
 # normalize the output.
 $VG virt-v2v --debug-gc --quiet \
     -i ova $d/test.ova \
-    --print-source |
-sed 's,[^ \t]*\(disk.*.vmdk\),\1,' > $d/source
+    --print-source  > $d/source
 
 # Check the parsed source is what we expect.
-diff -u test-v2v-i-ova-two-disks.expected $d/source
+if qemu_is_version 2 8 ; then
+    # normalize the output
+    sed -i -e "s,\"$d/,\"," $d/source
+    diff -u test-v2v-i-ova-two-disks.expected2 $d/source
+else
+    # normalize the output
+    sed -i -e 's,[^ \t]*\(disk.*.vmdk\),\1,' $d/source
+    diff -u test-v2v-i-ova-two-disks.expected $d/source
+fi
 
 rm -rf $d
-- 
2.10.2

Pino Toscano

2016-Dec-09 09:52 UTC

head link

Re: [Libguestfs] [PATCH v3 4/6] mllib: modify nsplit to take optional noempty and count arguments

On Wednesday, 7 December 2016 17:13:08 CET Tomáš Golembiovský
wrote:> Added two new optional arguments to nsplit:
> 
> * noempty: if set to false empty elements are not stored in the returned
>   list. The default is to keep the empty elements
> 
> * count: specifies how many splits to perform; negative count
>   (the default) means do as many splits as possible
> 
> Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
> ---
LGTM -- while you are here, what about adding tests for it in
mllib/common_utils_tests.ml?

Thanks,
-- 
Pino Toscano

Richard W.M. Jones

2016-Dec-09 12:46 UTC

head link

Re: [Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

On Wed, Dec 07, 2016 at 05:13:06PM +0100, Tomáš Golembiovský
wrote:> +          let filename = if compressed then (
This is nit-picking but it would be more consistent if written as:

        let filename           if compressed then (

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

Richard W.M. Jones

2016-Dec-09 12:47 UTC

head link

Re: [Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

On Wed, Dec 07, 2016 at 05:13:05PM +0100, Tomáš Golembiovský
wrote:> Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
> ---
>  mllib/checksums.ml  | 11 +++++++++--
>  mllib/checksums.mli |  7 +++++--
>  2 files changed, 14 insertions(+), 4 deletions(-)
> 
> diff --git a/mllib/checksums.ml b/mllib/checksums.ml
> index dfa8c3a..a6c995b 100644
> --- a/mllib/checksums.ml
> +++ b/mllib/checksums.ml
> @@ -45,7 +45,7 @@ let of_string csum_type csum_value >    |
"sha512" -> SHA512 csum_value
>    | _ -> invalid_arg csum_type
>  
> -let verify_checksum csum filename > +let verify_checksum csum ?tar
filename >    let prog, csum_ref >      match csum with
>      | SHA1 c -> "sha1sum", c
> @@ -53,7 +53,14 @@ let verify_checksum csum filename >      | SHA512 c
-> "sha512sum", c
>    in
>  
> -  let cmd = sprintf "%s %s" prog (Filename.quote filename) in
> +  let cmd > +    match tar with
> +    | None ->
> +      sprintf "%s %s" prog (Filename.quote filename)
> +    | Some tar ->
> +      sprintf "tar xOf %s %s | %s"
> +        (Filename.quote tar) (Filename.quote filename) prog
> +  in
>    let lines = external_command cmd in
>    match lines with
>    | [] ->
> diff --git a/mllib/checksums.mli b/mllib/checksums.mli
> index 0074837..9f7041b 100644
> --- a/mllib/checksums.mli
> +++ b/mllib/checksums.mli
> @@ -29,8 +29,11 @@ val of_string : string -> string -> csum_t
>  
>      Raise [Invalid_argument] if the checksum type is not known. *)
>  
> -val verify_checksum : csum_t -> string -> unit
> -(** Verify the checksum of the file. *)
> +val verify_checksum : csum_t -> ?tar:string -> string -> unit
> +(** [verify_checksum type filename] Verify the checksum of the file.
> +
> +    When optional [tar] is used it is path to uncompressed tar archive
> +    and the [filename] is a path in the tar archive. *)
>  
>  val verify_checksums : csum_t list -> string -> unit
>  (** Verify all the checksums of the file. *)
This one looks fine to me.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Richard W.M. Jones

2016-Dec-09 12:47 UTC

head link

Re: [Libguestfs] [PATCH v3 3/6] v2v: ova: move the untar function

On Wed, Dec 07, 2016 at 05:13:07PM +0100, Tomáš Golembiovský
wrote:> Move the untar function so it can be used later in the code.
> 
> Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
> ---
>  v2v/input_ova.ml | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
> index 61930f0..85954a3 100644
> --- a/v2v/input_ova.ml
> +++ b/v2v/input_ova.ml
> @@ -38,6 +38,12 @@ object
>    method as_options = "-i ova " ^ ova
>  
>    method source () > +
> +    let untar ?(format = "") file outdir > +      let cmd = [
"tar"; sprintf "-x%sf" format; file; "-C"; outdir
] in
> +      if run_command cmd <> 0 then
> +        error (f_"error unpacking %s, see earlier error
messages") ova in
> +
>      (* Extract ova file. *)
>      let exploded >        (* The spec allows a directory to be
specified as an ova.  This
> @@ -61,11 +67,6 @@ object
>  
>            tmpfile in
>  
> -        let untar ?(format = "") file outdir > -          let
cmd = [ "tar"; sprintf "-x%sf" format; file; "-C";
outdir ] in
> -          if run_command cmd <> 0 then
> -            error (f_"error unpacking %s, see earlier error
messages") ova in
> -
>          match detect_file_type ova with
>          | `Tar ->
>            (* Normal ovas are tar file (not compressed). *)
Simple code motion, so ACK.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

Pino Toscano

2016-Dec-09 13:01 UTC

head link

Re: [Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

On Wednesday, 7 December 2016 17:13:06 CET Tomáš Golembiovský
wrote:> The information whether the disk is gzip compressed or not is stored
> in the OVF. There is no reason to do the detection.
> 
> Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
> ---
>  v2v/input_ova.ml          | 28 +++++++++++++++++-----------
>  v2v/test-v2v-i-ova-gz.ovf |  2 +-
>  2 files changed, 18 insertions(+), 12 deletions(-)
> 
> diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
> index b283629..61930f0 100644
> --- a/v2v/input_ova.ml
> +++ b/v2v/input_ova.ml
> @@ -275,6 +275,13 @@ object
>              | None -> error (f_"no href in ovf:File (id=%s)")
file_ref
>              | Some s -> s in
>  
> +          let expr = sprintf
"/ovf:Envelope/ovf:References/ovf:File[@ovf:id='%s']/@ovf:compression"
file_ref in
> +          let compressed > +            match xpath_string expr with
> +            | None | Some "identity" -> false
> +            | Some "gzip" -> true
> +            | Some s -> error (f_"unsupported compression in OVF:
%s") s in
I'd still do the detection when no ovf:compression attribute is found
(that is, the None case above).
After all, right now we do the detection for any ova already, so a
"broken ova" (with no compression attribute in the ovf but with a
compressed image) is already handled by the current code.

It could look something like this:

  let compressed     match xpath_string expr with
    | None -> None
    | Some "identity" -> Some false
    | Some "gzip" -> Some true
    | Some s -> error (f_"unsupported compression in OVF: %s") s in

  let compressed     match compressed with
    | None -> (* do detection *)
    | Some c -> c in

Thanks,
-- 
Pino Toscano

Richard W.M. Jones

2016-Dec-10 13:18 UTC

head link

Re: [Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

On Wed, Dec 07, 2016 at 05:13:05PM +0100, Tomáš Golembiovský
wrote:> -  let cmd = sprintf "%s %s" prog (Filename.quote filename) in
> +  let cmd > +    match tar with
> +    | None ->
> +      sprintf "%s %s" prog (Filename.quote filename)
> +    | Some tar ->
> +      sprintf "tar xOf %s %s | %s"
> +        (Filename.quote tar) (Filename.quote filename) prog
You can actually use just "quote" instead of
"Filename.quote"
everywhere now, since 07fb30b1618310e5a944a947499c4d121d8bd3d8 went
upstream yesterday.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Apparently Analagous Threads

Search for more reasonably related threads

Libguestfs - Dec 2016 - [PATCH v3 3/6] v2v: ova: move the untar function

[Libguestfs] [PATCH v3 0/6] Import directly from OVA tar archive if possible

[Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

[Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

[Libguestfs] [PATCH v3 3/6] v2v: ova: move the untar function

[Libguestfs] [PATCH v3 4/6] mllib: modify nsplit to take optional noempty and count arguments

[Libguestfs] [PATCH v3 5/6] tests: rename guestfs-hashsums.sh to test-utils.sh

[Libguestfs] [PATCH v3 6/6] v2v: ova: don't extract files from OVA if it's not needed

Re: [Libguestfs] [PATCH v3 4/6] mllib: modify nsplit to take optional noempty and count arguments

Re: [Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

Re: [Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

Re: [Libguestfs] [PATCH v3 3/6] v2v: ova: move the untar function

Re: [Libguestfs] [PATCH v3 2/6] v2v: ova: don't detect compressed disks, read the OVF instead

Re: [Libguestfs] [PATCH v3 1/6] mllib: compute checksum of file inside tar

Apparently Analagous Threads