Richard W.M. Jones
2015-Nov-19 19:37 UTC
[Libguestfs] [PATCH 0/4] v2v: Add a new tool virt-v2v-copy-to-local to handle Xen and ESXi
It turns out that RHEL 5 Xen conversions don't work if the source disk is located on a block device. See patch 1/4 for the gory details. This patch series proposes a new tool called virt-v2v-copy-to-local which essentially is a way to make new virt-v2v work like the old virt-v2v, ie. copy first, convert after. Of course this is very slow and would only be used as a last resort, but I identified two situations in which this would currently be useful (see explanation in patch 4/4). This is also available in my git repo: https://github.com/rwmjones/libguestfs/commits/master Rich.
Richard W.M. Jones
2015-Nov-19 19:37 UTC
[Libguestfs] [PATCH 1/4] v2v: Detect conversions where the source disk has zero size (RHBZ#1283588).
If you try to convert a guest that has a zero-sized disk, it
will currently fail in a rather strange way. Usually you will
see errors in the debug output:
[ 0.562714] sd 2:0:0:0: [sda] Read Capacity(16) failed: Result:
hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 0.563884] sd 2:0:0:0: [sda] Sense Key : Not Ready [current]
[ 0.564587] sd 2:0:0:0: [sda] Add. Sense: Logical unit not ready, manual
intervention required
followed by virt-v2v failing with:
libguestfs: trace: v2v: inspect_os = []
virt-v2v: error: inspection could not detect the source guest (or physical
machine).
Additionally, because of a problem with the ssh driver in qemu (or
perhaps, with sftp-server on the remote side) it is not possible to
use the ssh driver to open a block device path on the remote server.
The drive will appear as zero-sized, triggering the above error.
Therefore detect this situation, and emit an error (see below). Also
add a section to the manual describing the workaround required for
converting RHEL 5 Xen guests which are located on block devices.
virt-v2v: error: guest disk sda appears to be zero bytes in size.
There could be several reasons for this:
Check that the guest doesn't really have a zero-sized disk. virt-v2v
cannot convert such a guest.
If you are converting a guest from an ssh source and the guest has a disk
on a block device (eg. on a host partition or host LVM LV), then
conversions of this type are not supported. See "XEN OR SSH CONVERSIONS
FROM BLOCK DEVICES" in the virt-v2v(1) manual for a workaround.
---
v2v/v2v.ml | 7 ++++++
v2v/virt-v2v.pod | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 83 insertions(+)
diff --git a/v2v/v2v.ml b/v2v/v2v.ml
index 4b04834..d633601 100644
--- a/v2v/v2v.ml
+++ b/v2v/v2v.ml
@@ -228,6 +228,13 @@ and create_overlays src_disks
let vsize = (open_guestfs ())#disk_virtual_size overlay_file in
+ (* If the virtual size is 0, then something went badly wrong.
+ * It could be RHBZ#1283588 or some other problem with qemu.
+ *)
+ if vsize = 0L then
+ error (f_"guest disk %s appears to be zero bytes in size.\n\nThere
could be several reasons for this:\n\nCheck that the guest doesn't really
have a zero-sized disk. virt-v2v cannot convert such a guest.\n\nIf you are
converting a guest from an ssh source and the guest has a disk on a block device
(eg. on a host partition or host LVM LV), then conversions of this type are not
supported. See \"XEN OR SSH CONVERSIONS FROM BLOCK DEVICES\" in the
virt-v2v(1) manual for a workaround.")
+ sd;
+
{ ov_overlay_file = overlay_file; ov_sd = sd;
ov_virtual_size = vsize; ov_source = source }
) src_disks
diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod
index 2e83168..0b7be7d 100644
--- a/v2v/virt-v2v.pod
+++ b/v2v/virt-v2v.pod
@@ -1226,6 +1226,10 @@ B<If the above commands do not work, then virt-v2v is
not going to
work either>. Fix your libvirt configuration or the remote server
before continuing.
+B<If the guest disks are located on a host block device>, then the
+conversion will fail. See L</XEN OR SSH CONVERSIONS FROM BLOCK DEVICES>
+below for a workaround.
+
=head2 XEN: IMPORTING A GUEST
To import a particular guest from a Xen server, do:
@@ -1241,6 +1245,78 @@ In this case the output flags are set to write the
converted guest to
a temporary directory as this is just an example, but you can also
write to libvirt or any other supported target.
+=head2 XEN OR SSH CONVERSIONS FROM BLOCK DEVICES
+
+It is currently not possible to convert a Xen guest (or any guest
+located remotely over ssh) if that guest's disks are located on host
+block devices.
+
+To tell if a Xen guest uses host block devices, look at the guest XML.
+You will see:
+
+ <disk type='block' device='disk'>
+ ...
+ <source dev='/dev/VG/guest'/>
+
+where C<type='block'>, C<source dev=> and C</dev/...>
are all
+indications that the disk is located on a host block device.
+
+This happens because the qemu ssh block driver that we use to access
+remote disks uses the ssh sftp protocol, and this protocol cannot
+correctly detect the size of host block devices.
+
+The workaround is to copy the guest over to the conversion server.
+You will need sufficient space on the conversion server to store a
+full copy of the guest.
+
+=over 4
+
+=item 1.
+
+From the conversion host, dump the guest XML to a local file:
+
+ virsh -c xen+ssh://root@xen.example.com dumpxml guest > guest.xml
+
+=item 2.
+
+From the conversion host, copy the guest disk(s) over. You may need
+to read the C<E<lt>diskE<gt>> sections from the guest XML to
find the
+names of the guest disks.
+
+ ssh root@xen.example.com 'dd if=/dev/VG/guest' | dd conv=sparse
of=guest-disk1
+
+Notice C<conv=sparse> which adds sparseness, so the copied disk will
+use as little space as possible.
+
+=item 3.
+
+Edit the guest XML file, replacing C<E<lt>diskE<gt>>
section(s) that
+refer to the remote disks with references to the local copies.
+
+Three changes have to be made. Firstly in:
+
+ <disk type='block' device='disk'>
+
+the C<type> must be changed to C<file>:
+
+ <disk type='file' device='disk'>
+
+The last two changes are in the C<E<lt>sourceE<gt>> line
where:
+
+ <source dev='/dev/VG/guest'/>
+
+C<source dev=> becomes C<source file=> pointing at the local file:
+
+ <source file='guest-disk1'/>
+
+=item 4.
+
+Convert the guest using C<virt-v2v -i libvirtxml> mode like this:
+
+ virt-v2v -i libvirtxml guest.xml [-o options as usual ...]
+
+=back
+
=head1 OUTPUT TO LIBVIRT
The I<-o libvirt> option lets you upload the converted guest to
--
2.5.0
Richard W.M. Jones
2015-Nov-19 19:37 UTC
[Libguestfs] [PATCH 2/4] v2v: vcenter: Split out map_source_to_https from map_source_to_uri.
Split the map_source_to_uri function up so that we can get at just the
https://... URI.
This is almost refactoring. It doesn't quite handle the case where we
are passed a source path from the libvirt VMware driver that doesn't
match the expected "[datastore] path" format, but probably if we hit
that case it's a bug anyway.
---
v2v/vCenter.ml | 79 ++++++++++++++++++++++++++++++---------------------------
v2v/vCenter.mli | 6 +++++
2 files changed, 48 insertions(+), 37 deletions(-)
diff --git a/v2v/vCenter.ml b/v2v/vCenter.ml
index 4c5ca1c..c411f6e 100644
--- a/v2v/vCenter.ml
+++ b/v2v/vCenter.ml
@@ -174,9 +174,9 @@ let guess_dcPath uri = function
let source_re = Str.regexp "^\\[\\(.*\\)\\] \\(.*\\)\\.vmdk$"
-let map_source_to_uri readahead dcPath password uri scheme server path +let
map_source_to_https dcPath uri server path if not (Str.string_match source_re
path 0) then
- path
+ (path, true)
else (
let datastore = Str.matched_group 1 path
and path = Str.matched_group 2 path in
@@ -208,39 +208,44 @@ let map_source_to_uri readahead dcPath password uri scheme
server path (* XXX only works if the query string is not URI-quoted *)
String.find query "no_verify=1" = -1 in
- (* Now we have to query the server to get the session cookie. *)
- let session_cookie = get_session_cookie password scheme uri sslverify url
in
-
- (* Construct the JSON parameters. *)
- let json_params = [
- "file.driver", JSON.String "https";
- "file.url", JSON.String url;
- (* https://bugzilla.redhat.com/show_bug.cgi?id=1146007#c10 *)
- "file.timeout", JSON.Int 2000;
- ] in
-
- let json_params - match readahead with
- | None -> json_params
- | Some readahead ->
- ("file.readahead", JSON.Int readahead) :: json_params in
-
- let json_params - if sslverify then json_params
- else ("file.sslverify", JSON.String "off") ::
json_params in
-
- let json_params - match session_cookie with
- | None -> json_params
- | Some cookie -> ("file.cookie", JSON.String cookie) ::
json_params in
-
- if verbose () then
- printf "vcenter: json parameters: %s\n" (JSON.string_of_doc
json_params);
-
- (* Turn the JSON parameters into a 'json:' protocol string.
- * Note this requires qemu-img >= 2.1.0.
- *)
- let qemu_uri = "json: " ^ JSON.string_of_doc json_params in
-
- qemu_uri
+ (url, sslverify)
)
+
+let map_source_to_uri readahead dcPath password uri scheme server path + let
url, sslverify = map_source_to_https dcPath uri server path in
+
+ (* Now we have to query the server to get the session cookie. *)
+ let session_cookie = get_session_cookie password scheme uri sslverify url in
+
+ (* Construct the JSON parameters. *)
+ let json_params = [
+ "file.driver", JSON.String "https";
+ "file.url", JSON.String url;
+ (* https://bugzilla.redhat.com/show_bug.cgi?id=1146007#c10 *)
+ "file.timeout", JSON.Int 2000;
+ ] in
+
+ let json_params + match readahead with
+ | None -> json_params
+ | Some readahead ->
+ ("file.readahead", JSON.Int readahead) :: json_params in
+
+ let json_params + if sslverify then json_params
+ else ("file.sslverify", JSON.String "off") ::
json_params in
+
+ let json_params + match session_cookie with
+ | None -> json_params
+ | Some cookie -> ("file.cookie", JSON.String cookie) ::
json_params in
+
+ if verbose () then
+ printf "vcenter: json parameters: %s\n" (JSON.string_of_doc
json_params);
+
+ (* Turn the JSON parameters into a 'json:' protocol string.
+ * Note this requires qemu-img >= 2.1.0.
+ *)
+ let qemu_uri = "json: " ^ JSON.string_of_doc json_params in
+
+ qemu_uri
diff --git a/v2v/vCenter.mli b/v2v/vCenter.mli
index 15b5143..fedddbd 100644
--- a/v2v/vCenter.mli
+++ b/v2v/vCenter.mli
@@ -52,3 +52,9 @@ val map_source_to_uri : int option -> string -> string
option -> Xml.uri -> stri
["[datastore1] Fedora 20/Fedora 20.vmdk"]
including those literal spaces in the string. *)
+
+val map_source_to_https : string -> Xml.uri -> string -> string ->
string * bool
+(** [map_source_to_https dcPath uri server path] is the same as
+ {!map_source_to_uri} but it produces a regular [https://...] URL.
+ The returned boolean is whether TLS certificate verification
+ should be done. *)
--
2.5.0
Richard W.M. Jones
2015-Nov-19 19:37 UTC
[Libguestfs] [PATCH 3/4] v2v: xml: Add a binding for XmlUnsetProp.
---
v2v/xml-c.c | 12 ++++++++++++
v2v/xml.ml | 3 +++
v2v/xml.mli | 3 +++
3 files changed, 18 insertions(+)
diff --git a/v2v/xml-c.c b/v2v/xml-c.c
index 97b8a20..a6d9231 100644
--- a/v2v/xml-c.c
+++ b/v2v/xml-c.c
@@ -353,6 +353,18 @@ v2v_xml_node_ptr_set_prop (value nodev, value namev, value
valv)
}
value
+v2v_xml_node_ptr_unset_prop (value nodev, value namev)
+{
+ CAMLparam2 (nodev, namev);
+ xmlNodePtr node = (xmlNodePtr) nodev;
+ int r;
+
+ r = xmlUnsetProp (node, BAD_CAST String_val (namev));
+
+ CAMLreturn (r == 0 ? Val_true : Val_false);
+}
+
+value
v2v_xml_node_ptr_unlink_node (value nodev)
{
CAMLparam1 (nodev);
diff --git a/v2v/xml.ml b/v2v/xml.ml
index dbb2b41..27c8566 100644
--- a/v2v/xml.ml
+++ b/v2v/xml.ml
@@ -95,6 +95,9 @@ let new_text_child (doc_ptr, node_ptr) name content external
node_ptr_set_prop : node_ptr -> string -> string -> unit =
"v2v_xml_node_ptr_set_prop"
let set_prop (doc_ptr, node_ptr) = node_ptr_set_prop node_ptr
+external node_ptr_unset_prop : node_ptr -> string -> bool =
"v2v_xml_node_ptr_unset_prop"
+let unset_prop (doc_ptr, node_ptr) = node_ptr_unset_prop node_ptr
+
external node_ptr_unlink_node : node_ptr -> unit =
"v2v_xml_node_ptr_unlink_node"
let unlink_node (doc_ptr, node_ptr) = node_ptr_unlink_node node_ptr
diff --git a/v2v/xml.mli b/v2v/xml.mli
index b039d09..9664c73 100644
--- a/v2v/xml.mli
+++ b/v2v/xml.mli
@@ -73,6 +73,9 @@ val new_text_child : node -> string -> string -> node
val set_prop : node -> string -> string -> unit
(** xmlSetProp *)
+val unset_prop : node -> string -> bool
+(** xmlUnsetProp (returns [true] if the property was removed) *)
+
val unlink_node : node -> unit
(** xmlUnlinkNode
{b NB:} This frees the [node], do not use it afterwards. *)
--
2.5.0
Richard W.M. Jones
2015-Nov-19 19:37 UTC
[Libguestfs] [PATCH 4/4] v2v: Add a new tool virt-v2v-copy-to-local.
This allows certain guests which virt-v2v cannot access to be copied
off the remote hypervisor and converted. Essentially this just
automates the process of copying the guest's disks and adjusting the
libvirt XML.
---
.gitignore | 3 +
po/POTFILES-ml | 1 +
v2v/Makefile.am | 65 ++++++++-
v2v/copy_to_local.ml | 311 +++++++++++++++++++++++++++++++++++++++++
v2v/virt-v2v-copy-to-local.pod | 206 +++++++++++++++++++++++++++
v2v/virt-v2v.pod | 72 ++++------
6 files changed, 606 insertions(+), 52 deletions(-)
create mode 100644 v2v/copy_to_local.ml
create mode 100644 v2v/virt-v2v-copy-to-local.pod
diff --git a/.gitignore b/.gitignore
index a43c243..11557b6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -562,6 +562,7 @@ Makefile.in
/v2v/rhel-6.5.img
/v2v/rhel-7.0.img
/v2v/stamp-virt-v2v.pod
+/v2v/stamp-virt-v2v-copy-to-local.pod
/v2v/test-harness/.depend
/v2v/test-harness/META
/v2v/test-harness/dllv2v_test_harness.so
@@ -570,6 +571,8 @@ Makefile.in
/v2v/v2v_unit_tests
/v2v/virt-v2v
/v2v/virt-v2v.1
+/v2v/virt-v2v-copy-to-local
+/v2v/virt-v2v-copy-to-local.1
/website/*.html
/website/README.txt
/website/TODO.txt
diff --git a/po/POTFILES-ml b/po/POTFILES-ml
index c02ffc0..00a9d63 100644
--- a/po/POTFILES-ml
+++ b/po/POTFILES-ml
@@ -101,6 +101,7 @@ v2v/changeuid.ml
v2v/cmdline.ml
v2v/convert_linux.ml
v2v/convert_windows.ml
+v2v/copy_to_local.ml
v2v/curl.ml
v2v/domainxml.ml
v2v/input_disk.ml
diff --git a/v2v/Makefile.am b/v2v/Makefile.am
index 5dfef6e..4b6cbcc 100644
--- a/v2v/Makefile.am
+++ b/v2v/Makefile.am
@@ -35,7 +35,8 @@ EXTRA_DIST = \
test-v2v-networks-and-bridges-expected.xml \
test-v2v-networks-and-bridges.xml \
test-v2v-sound.xml \
- virt-v2v.pod
+ virt-v2v.pod \
+ virt-v2v-copy-to-local.pod
CLEANFILES = *~ *.annot *.cmi *.cmo *.cmx *.cmxa *.o virt-v2v
@@ -113,7 +114,7 @@ SOURCES_C = \
if HAVE_OCAML
-bin_PROGRAMS = virt-v2v
+bin_PROGRAMS = virt-v2v virt-v2v-copy-to-local
virt_v2v_SOURCES = $(SOURCES_C)
virt_v2v_CPPFLAGS = \
@@ -177,6 +178,46 @@ virt_v2v_LINK = \
$(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \
$(OBJECTS) -o $@
+virt_v2v_copy_to_local_SOURCES = \
+ domainxml-c.c \
+ utils-c.c \
+ xml-c.c
+virt_v2v_copy_to_local_CPPFLAGS = \
+ -I. \
+ -I$(top_builddir) \
+ -I$(shell $(OCAMLC) -where) \
+ -I$(top_srcdir)/src
+virt_v2v_copy_to_local_CFLAGS = \
+ $(WARN_CFLAGS) $(WERROR_CFLAGS) \
+ $(LIBXML2_CFLAGS) \
+ $(LIBVIRT_CFLAGS)
+
+COPY_TO_LOCAL_BOBJECTS = \
+ $(top_builddir)/mllib/guestfs_config.cmo \
+ $(top_builddir)/mllib/common_gettext.cmo \
+ $(top_builddir)/mllib/common_utils.cmo \
+ $(top_builddir)/mllib/JSON.cmo \
+ xml.cmo \
+ utils.cmo \
+ curl.cmo \
+ vCenter.cmo \
+ domainxml.cmo \
+ copy_to_local.cmo
+COPY_TO_LOCAL_XOBJECTS = $(COPY_TO_LOCAL_BOBJECTS:.cmo=.cmx)
+
+if !HAVE_OCAMLOPT
+COPY_TO_LOCAL_OBJECTS = $(COPY_TO_LOCAL_BOBJECTS)
+else
+COPY_TO_LOCAL_OBJECTS = $(COPY_TO_LOCAL_XOBJECTS)
+endif
+
+virt_v2v_copy_to_local_DEPENDENCIES = \
+ $(COPY_TO_LOCAL_OBJECTS) $(top_srcdir)/ocaml-link.sh
+virt_v2v_copy_to_local_LINK = \
+ $(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \
+ $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \
+ $(COPY_TO_LOCAL_OBJECTS) -o $@
+
.mli.cmi:
$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
.ml.cmo:
@@ -192,9 +233,11 @@ virttoolsdatadir = $(datadir)/virt-tools
# Manual pages and HTML files for the website.
-man_MANS = virt-v2v.1
+man_MANS = virt-v2v.1 virt-v2v-copy-to-local.1
-noinst_DATA = $(top_builddir)/website/virt-v2v.1.html
+noinst_DATA = \
+ $(top_builddir)/website/virt-v2v.1.html \
+ $(top_builddir)/website/virt-v2v-copy-to-local.1.html
virt-v2v.1 $(top_builddir)/website/virt-v2v.1.html: stamp-virt-v2v.pod
@@ -206,9 +249,21 @@ stamp-virt-v2v.pod: virt-v2v.pod
$<
touch $@
+virt-v2v-copy-to-local.1 $(top_builddir)/website/virt-v2v-copy-to-local.1.html:
stamp-virt-v2v-copy-to-local.pod
+
+stamp-virt-v2v-copy-to-local.pod: virt-v2v-copy-to-local.pod
+ $(PODWRAPPER) \
+ --man virt-v2v-copy-to-local.1 \
+ --html $(top_builddir)/website/virt-v2v-copy-to-local.1.html \
+ --license GPLv2+ \
+ $<
+ touch $@
+
CLEANFILES += \
stamp-virt-v2v.pod \
- virt-v2v.1
+ stamp-virt-v2v-copy-to-local.pod \
+ virt-v2v.1 \
+ virt-v2v-copy-to-local.1
# Tests.
diff --git a/v2v/copy_to_local.ml b/v2v/copy_to_local.ml
new file mode 100644
index 0000000..ed967be
--- /dev/null
+++ b/v2v/copy_to_local.ml
@@ -0,0 +1,311 @@
+(* virt-v2v
+ * Copyright (C) 2009-2015 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+
+(* The separate virt-v2v-copy-to-local tool. *)
+
+open Printf
+
+open Common_gettext.Gettext
+open Common_utils
+
+open Utils
+
+type source_t = Xen_ssh | ESXi
+
+let rec main () + let input_conn = ref None in
+
+ let set_string_option_once optname optref arg + match !optref with
+ | Some _ ->
+ error (f_"%s option used more than once on the command line")
optname
+ | None ->
+ optref := Some arg
+ in
+
+ (* Handle the command line. *)
+ let argspec = [
+ "-ic", Arg.String (set_string_option_once "-ic"
input_conn),
+ "uri " ^ s_"Libvirt
URI";
+ ] in
+ let argspec = set_standard_options argspec in
+ let args = ref [] in
+ let anon_fun s = args := s :: !args in
+ let usage_msg + sprintf (f_"\
+%s: copy a remote guest to the local machine
+
+Copy the remote guest:
+
+ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest
+
+ virt-v2v-copy-to-local -ic esx://esxi.example.com guest
+
+Then perform the conversion step:
+
+ virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp
+
+To clean up:
+
+ rm guest.xml guest-disk*
+
+A short summary of the options is given below. For detailed help please
+read the man page virt-v2v-copy-to-local(1).
+")
+ prog in
+ Arg.parse argspec anon_fun usage_msg;
+
+ let args = !args in
+ let input_conn = !input_conn in
+
+ let input_conn + match input_conn with
+ | None ->
+ error (f_"the -ic parameter is required") (* at the moment *)
+ | Some ic -> ic in
+
+ (* Check this is a libvirt URI we can understand. *)
+ let parsed_uri + try Xml.parse_uri input_conn
+ with Invalid_argument msg ->
+ error (f_"could not parse '-ic %s'. Original error message
was: %s")
+ input_conn msg in
+ let source, server + match parsed_uri.Xml.uri_server,
parsed_uri.Xml.uri_scheme with
+ | Some server, Some "xen+ssh" -> (* Xen over SSH *)
+ Xen_ssh, server
+ | Some server, Some "esx" -> (* esxi over https *)
+ ESXi, server
+
+ (* We can probably extend this list in future. *)
+ | _ ->
+ error (f_"only copies from VMware ESXi or Xen over SSH are
supported. See the virt-v2v-copy-to-local(1) manual page.") in
+
+ (* We expect a single extra argument, which is the guest name. *)
+ let guest_name + match args with
+ | [] ->
+ error (f_"missing guest name. See the virt-v2v-copy-to-local(1)
manual page.")
+ | [arg] -> arg
+ | _ ->
+ error (f_"too many command line parameters. See the
virt-v2v-copy-to-local(1) manual page.") in
+
+ (* Print the version, easier than asking users to tell us. *)
+ if verbose () then
+ printf "%s: %s %s (%s)\n%!"
+ prog Guestfs_config.package_name Guestfs_config.package_version
Guestfs_config.host_cpu;
+
+ (* Get the remote libvirt XML. *)
+ message (f_"Fetching the remote libvirt XML metadata ...");
+ let xml = Domainxml.dumpxml ~conn:input_conn guest_name in
+
+ if verbose () then
+ printf "libvirt XML from remote server:\n%s\n" xml;
+
+ (* Get the disk remote paths from the XML. *)
+ message (f_"Parsing the remote libvirt XML metadata ...");
+ let disks, dcpath, xml = parse_libvirt_xml guest_name xml in
+
+ if verbose () then
+ printf "libvirt XML after modifying for local disks:\n%s\n" xml;
+
+ (* For VMware ESXi source, we have to massage the disk path. *)
+ let disks + match source with
+ | ESXi ->
+ List.map (
+ fun (remote_disk, local_disk) ->
+ let url, sslverify + VCenter.map_source_to_https dcpath
parsed_uri
+ server remote_disk in
+ if verbose () then
+ printf "esxi: source disk %s (sslverify=%b)\n" url
sslverify;
+ let cookie + VCenter.get_session_cookie None
"esx" parsed_uri sslverify url in
+ (url, local_disk, sslverify, cookie)
+ ) disks
+ | Xen_ssh ->
+ List.map (fun (remote_disk, local_disk) ->
+ (remote_disk, local_disk, false, None)) disks in
+
+ (* Delete the disks on exit, unless we finish everything OK. *)
+ let delete_on_exit = ref true in
+ at_exit (
+ fun () ->
+ if !delete_on_exit then (
+ List.iter (
+ fun (_, local_disk, _, _) ->
+ try Unix.unlink local_disk with _ -> ()
+ ) disks
+ )
+ );
+
+ (* Copy the disks. *)
+ let n = List.length disks in
+ iteri (
+ fun i (remote_disk, local_disk, sslverify, cookie) ->
+ message (f_"Copying remote disk %d/%d to %s")
+ (i+1) n local_disk;
+
+ (* How we copy it depends on the source. *)
+ match source with
+ | Xen_ssh ->
+ let { Xml.uri_server = server; uri_user = user; uri_port = port } +
parsed_uri in
+
+ let cmd + sprintf "set -o pipefail; ssh%s %s%s dd bs=1M
if=%s | dd%s conv=sparse bs=1M of=%s"
+ (match port with
+ | n when n >= 1 -> sprintf " -p %d" n
+ | _ -> "")
+ (match user with
+ | None -> ""
+ | Some u -> sprintf "%s@" (quote u))
+ (match server with
+ | None -> assert false
+ | Some server -> server)
+ (quote remote_disk)
+ (if quiet () then ""
+ else " status=progress")
+ (quote local_disk) in
+ if verbose () then
+ printf "%s\n%!" cmd;
+ if Sys.command cmd <> 0 then
+ error (f_"ssh copy command failed, see earlier errors");
+
+ | ESXi ->
+ let curl_args = [
+ "url", Some remote_disk;
+ "output", Some local_disk;
+ ] in
+ let curl_args + if sslverify then curl_args
+ else ("insecure", None) :: curl_args in
+ let curl_args + match cookie with
+ | None -> curl_args
+ | Some cookie -> ("cookie", Some cookie) :: curl_args in
+ let curl_args + if quiet () then ("silent", None) ::
curl_args
+ else curl_args in
+
+ if verbose () then
+ Curl.print_curl_command stdout curl_args;
+ ignore (Curl.run curl_args)
+ ) disks;
+
+ let guest_xml = guest_name ^ ".xml" in
+ message (f_"Writing libvirt XML metadata to %s.xml ...") guest_xml;
+ let chan = open_out guest_xml in
+ output_string chan xml;
+ close_out chan;
+
+ (* Finished, so don't delete the disks on exit. *)
+ message (f_"Finishing off");
+ delete_on_exit := false
+
+(* This is a greatly simplified version of the parsing function
+ * in virt-v2v input_libvirtxml.ml:parse_libvirt_xml
+ * It also modifies the XML <disk> elements to point to local disks.
+ *)
+and parse_libvirt_xml guest_name xml + (* Parse the XML. *)
+ let doc = Xml.parse_memory xml in
+ let xpathctx = Xml.xpath_new_context doc in
+ Xml.xpath_register_ns xpathctx
+ "vmware"
"http://libvirt.org/schemas/domain/vmware/1.0";
+ let xpath_string = xpath_string xpathctx
+ and xpath_string_default = xpath_string_default xpathctx in
+
+ (* Get the dcpath, only present for libvirt >= 1.2.20 so use a
+ * sensible default for older versions.
+ *)
+ let dcpath + xpath_string_default
"/domain/vmware:datacenterpath" "ha-datacenter" in
+
+ (* Parse the disks. *)
+ let get_disks, add_disk + let disks = ref [] and i = ref 0 in
+ let get_disks () = List.rev !disks in
+ let add_disk remote_disk + (* Generate a unique name for each output
disk. *)
+ incr i;
+ let local_disk = sprintf "%s-disk%d" guest_name !i in
+
+ disks := (remote_disk, local_disk) :: !disks;
+ local_disk
+ in
+ get_disks, add_disk
+ in
+
+ (* node is a <disk> node, containing a <source> element. Update
the
+ * node to point to a local file.
+ *)
+ let update_disk_node node local_disk + Xml.set_prop node "type"
"file";
+ let obj = Xml.xpath_eval_expression xpathctx "source" in
+ let nr_nodes = Xml.xpathobj_nr_nodes obj in
+ assert (nr_nodes >= 1);
+ for i = 0 to nr_nodes-1 do
+ let source_node = Xml.xpathobj_node obj i in
+ ignore (Xml.unset_prop source_node "dev");
+ Xml.set_prop source_node "file" local_disk
+ done
+ in
+
+ let obj + Xml.xpath_eval_expression xpathctx
+
"/domain/devices/disk[@device='disk']" in
+ let nr_nodes = Xml.xpathobj_nr_nodes obj in
+ if nr_nodes < 1 then
+ error (f_"this guest has no non-removable disks");
+
+ for i = 0 to nr_nodes-1 do
+ let node = Xml.xpathobj_node obj i in
+ Xml.xpathctx_set_current_context xpathctx node;
+
+ (* The <disk type='...'> attribute may be 'block' or
'file'.
+ * We ignore any other types.
+ *)
+ match xpath_string "@type" with
+ | None ->
+ warning (f_"<disk> element with no type attribute
ignored")
+
+ | Some "block" ->
+ (match xpath_string "source/@dev" with
+ | Some path ->
+ let local_disk = add_disk path in
+ update_disk_node node local_disk
+ | None -> ()
+ );
+ | Some "file" ->
+ (match xpath_string "source/@file" with
+ | Some path ->
+ let local_disk = add_disk path in
+ update_disk_node node local_disk
+ | None -> ()
+ );
+
+ | Some disk_type ->
+ warning (f_"<disk type='%s'> was ignored")
disk_type
+ done;
+
+ let xml = Xml.to_string doc ~format:true in
+ get_disks (), dcpath, xml
+
+let () = run_main_and_handle_errors main
diff --git a/v2v/virt-v2v-copy-to-local.pod b/v2v/virt-v2v-copy-to-local.pod
new file mode 100644
index 0000000..85e6d4e
--- /dev/null
+++ b/v2v/virt-v2v-copy-to-local.pod
@@ -0,0 +1,206 @@
+=head1 NAME
+
+virt-v2v-copy-to-local - Copy a remote guest to the local machine
+
+=head1 SYNOPSIS
+
+ virt-v2v-copy-to-local -ic LIBVIRT_URI GUEST
+
+ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com xen_guest
+
+ virt-v2v-copy-to-local -ic esx://root@esxi.example.com vmware_guest
+
+=head1 DESCRIPTION
+
+C<virt-v2v-copy-to-local> copies a guest from a remote hypervisor to
+the local machine, in preparation for conversion by L<virt-v2v(1)>.
+Note this tool alone B<does not> do the virt-v2v conversion.
+
+=head2 When to use this tool
+
+This tool is not usually necessary, but there are a few special cases
+(see list below) where you might need it.
+
+If your case does not fit one of these special cases, then ignore this
+tool and read L<virt-v2v(1)> instead. The virt-v2v-copy-to-local
+process is slower than using virt-v2v directly, because it has to copy
+unused parts of the guest disk.
+
+=over 4
+
+=item *
+
+You have a Xen guest using host block devices. Virt-v2v cannot
+convert such guests directly.
+
+See L<virt-v2v(1)/XEN OR SSH CONVERSIONS FROM BLOCK DEVICES>.
+
+=item *
+
+You have VMware ESXi hypervisors, and are not using VMware vCenter to
+manage them. Virt-v2v cannot directly access ESXi hypervisor, so you
+either have to export the guest as an OVA (eg. using VMware's
+C<ovftool>); or you can use this tool to copy the guest to a local
+file on the conversion server, from where virt-v2v will be able to
+access it.
+
+=back
+
+=head2 How this tool works
+
+This tool uses libvirt to get the libvirt XML (metadata) of the remote
+guest, essentially equivalent to running C<virsh dumpxml guest>.
+
+It then uses the XML to locate the remote guest disks, which are
+copied over using a hypervisor-specific method. It uses ssh for
+remote Xen hypervisors, and HTTPS (curl) for remote ESXi hypervisors.
+
+It then modifies the libvirt XML so that it points at the local copies
+of the guest disks.
+
+The libvirt XML is output to a file called F<guest.xml> (where
+I<guest> is the name of the guest). The disk(s) are output to file(s)
+called F<guest-disk1>, F<guest-disk2> and so on.
+
+After copying the guest locally, you can convert it using:
+
+ virt-v2v -i libvirtxml guest.xml [-o options ...]
+
+Virt-v2v finds the local copies of the disks by looking in the XML.
+
+=head1 EXAMPLES
+
+=head2 Copy and convert from Xen hypervisor that uses host block devices
+
+ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com xen_guest
+ virt-v2v -i libvirtxml xen_guest.xml -o local -os /var/tmp
+ rm xen_guest.xml xen_guest-disk*
+
+=head2 Copy and convert from ESXi hypervisor
+
+ virt-v2v-copy-to-local -ic esx://root@esxi.example.com?no_verify=1
vmware_guest
+ virt-v2v -i libvirtxml vmware_guest.xml -o local -os /var/tmp
+ rm vmware_guest.xml vmware_guest-disk*
+
+=head1 COPYING FROM XEN HYPERVISOR
+
+=head2 XEN: LIBVIRT URI
+
+The libvirt URI for remote Xen hosts will look something like this:
+
+ xen+ssh://root@xen.example.com
+
+The remote Xen server must allow root logins over ssh.
+
+To test it and list the remote guests available, use L<virsh(1)>:
+
+ $ virsh -c xen+ssh://root@xen.example.com list --all
+ Id Name State
+ ----------------------------------------------------
+ 0 Domain-0 running
+ - guest shut off
+
+=head2 XEN: COPY THE GUEST
+
+Using the libvirt URI as the I<-ic> option, copy one of the guests to
+the local machine:
+
+ $ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest
+
+This creates F<guest.xml>, F<guest-disk1>, ...
+
+=head2 XEN: DO THE CONVERSION
+
+Perform the conversion of the guest using virt-v2v:
+
+ $ virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp
+
+=head2 XEN: CLEAN UP
+
+Remove the F<guest.xml> and F<guest-disk*> files.
+
+=head1 COPYING FROM VMware ESXi HYPERVISOR
+
+=head2 ESXi: LIBVIRT URI
+
+The libvirt URI for VMware ESXi hypervisors will look something like this:
+
+ esx://root@esxi.example.com?no_verify=1
+
+The C<?no_verify=1> parameter disables TLS certificate checking.
+
+To test it and list the remote guests available, use L<virsh(1)>:
+
+ $ virsh -c esx://root@esxi.example.com?no_verify=1 list --all
+ Enter root's password for esxi.example.com: ***
+ Id Name State
+ ----------------------------------------------------
+ - guest shut off
+
+=head2 ESXi: COPY THE GUEST
+
+Using the libvirt URI as the I<-ic> option, copy one of the guests to
+the local machine:
+
+ $ virt-v2v-copy-to-local -ic esx://root@esxi.example.com?no_verify=1 guest
+
+This creates F<guest.xml>, F<guest-disk1>, ...
+
+=head2 ESXi: DO THE CONVERSION
+
+Perform the conversion of the guest using virt-v2v:
+
+ $ virt-v2v -i libvirtxml guest.xml -o local -os /var/tmp
+
+=head2 ESXi: CLEAN UP
+
+Remove the F<guest.xml> and F<guest-disk*> files.
+
+=head1 OPTIONS
+
+=over 4
+
+=item B<--help>
+
+Display help.
+
+=item B<-ic> libvirtURI
+
+Specify a libvirt connection URI
+
+=item B<-q>
+
+=item B<--quiet>
+
+This disables progress bars and other unnecessary output.
+
+=item B<-v>
+
+=item B<--verbose>
+
+Enable verbose messages for debugging.
+
+=item B<-V>
+
+=item B<--version>
+
+Display version number and exit.
+
+=back
+
+=head1 SEE ALSO
+
+L<virt-v2v(1)>,
+L<virsh(1)>,
+L<http://libguestfs.org/>,
+L<https://libvirt.org/uri.html>,
+L<https://libvirt.org/remote.html>,
+L<https://libvirt.org/drvesx.html>.
+
+=head1 AUTHORS
+
+Richard W.M. Jones L<http://people.redhat.com/~rjones/>
+
+=head1 COPYRIGHT
+
+Copyright (C) 2009-2015 Red Hat Inc.
diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod
index 0b7be7d..0aa0c56 100644
--- a/v2v/virt-v2v.pod
+++ b/v2v/virt-v2v.pod
@@ -1177,6 +1177,19 @@ directory containing the files:
$ virt-v2v -i ova /path/to/files -o local -os /var/tmp
+=head1 INPUT FROM VMWARE ESXi HYPERVISOR
+
+Virt-v2v cannot access an ESXi hypervisor directly.
+
+However you can use the L<virt-v2v-copy-to-local(1)> tool to copy the
+guest off the hypervisor into a local file, and then convert it:
+
+ virt-v2v-copy-to-local -ic esx://esxi.example.com vmware_guest
+ virt-v2v -i libvirtxml vmware_guest.xml [-o options ...]
+ rm vmware_guest.xml vmware_guest-disk*
+
+See L<virt-v2v-copy-to-local(1)> for further information.
+
=head1 INPUT FROM RHEL 5 XEN
Virt-v2v is able to import Xen guests from RHEL 5 Xen hosts.
@@ -1269,53 +1282,9 @@ The workaround is to copy the guest over to the
conversion server.
You will need sufficient space on the conversion server to store a
full copy of the guest.
-=over 4
-
-=item 1.
-
-From the conversion host, dump the guest XML to a local file:
-
- virsh -c xen+ssh://root@xen.example.com dumpxml guest > guest.xml
-
-=item 2.
-
-From the conversion host, copy the guest disk(s) over. You may need
-to read the C<E<lt>diskE<gt>> sections from the guest XML to
find the
-names of the guest disks.
-
- ssh root@xen.example.com 'dd if=/dev/VG/guest' | dd conv=sparse
of=guest-disk1
-
-Notice C<conv=sparse> which adds sparseness, so the copied disk will
-use as little space as possible.
-
-=item 3.
-
-Edit the guest XML file, replacing C<E<lt>diskE<gt>>
section(s) that
-refer to the remote disks with references to the local copies.
-
-Three changes have to be made. Firstly in:
-
- <disk type='block' device='disk'>
-
-the C<type> must be changed to C<file>:
-
- <disk type='file' device='disk'>
-
-The last two changes are in the C<E<lt>sourceE<gt>> line
where:
-
- <source dev='/dev/VG/guest'/>
-
-C<source dev=> becomes C<source file=> pointing at the local file:
-
- <source file='guest-disk1'/>
-
-=item 4.
-
-Convert the guest using C<virt-v2v -i libvirtxml> mode like this:
-
- virt-v2v -i libvirtxml guest.xml [-o options as usual ...]
-
-=back
+ virt-v2v-copy-to-local -ic xen+ssh://root@xen.example.com guest
+ virt-v2v -i libvirtxml guest.xml [-o options ...]
+ rm guest.xml guest-disk*
=head1 OUTPUT TO LIBVIRT
@@ -1786,6 +1755,14 @@ For other environment variables, see
L<guestfs(3)/ENVIRONMENT VARIABLES>.
=over 4
+=item L<virt-v2v-copy-to-local(1)>
+
+There are some special cases where virt-v2v cannot directly access the
+remote hypervisor. In that case you have to use
+L<virt-v2v-copy-to-local(1)> to make a local copy of the guest first,
+followed by running C<virt-v2v -i libvirtxml> to perform the
+conversion.
+
=item L<engine-image-uploader(8)>
Variously called C<engine-image-uploader>, C<ovirt-image-uploader>
or
@@ -1816,6 +1793,7 @@ L<guestfs(3)>,
L<guestfish(1)>,
L<qemu-img(1)>,
L<fstrim(8)>,
+L<virt-v2v-copy-to-local(1)>,
L<virt-v2v-test-harness(1)>,
L<engine-image-uploader(8)>,
L<import-to-ovirt.pl|http://git.annexia.org/?p=import-to-ovirt.git>,
--
2.5.0
Possibly Parallel Threads
- [PATCH 0/5] v2v: Handle disks with snapshots (RHBZ#1172425).
- [PATCH v2v 0/4] v2v: vcenter: Implement cookie scripts.
- [PATCH 0/4] v2v: Use libvirt-supplied <vmware:datacenterpath> if available.
- ocaml tools: Use a common debug function.
- [PATCH v3 0/8] v2v: Move Curl wrapper to mllib and more.