Richard W.M. Jones
2019-Apr-05 09:33 UTC
[Libguestfs] [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK
From: Martin Kletzander <mkletzan@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
---
plugins/vddk/nbdkit-vddk-plugin.pod | 7 ++++++-
plugins/vddk/vddk-structs.h | 1 +
plugins/vddk/vddk.c | 10 ++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod
b/plugins/vddk/nbdkit-vddk-plugin.pod
index 586aab4..25a6511 100644
--- a/plugins/vddk/nbdkit-vddk-plugin.pod
+++ b/plugins/vddk/nbdkit-vddk-plugin.pod
@@ -8,7 +8,7 @@ nbdkit-vddk-plugin - nbdkit VMware VDDK plugin
[vm=moref=ID] [server=HOSTNAME] [user=USERNAME]
[password=PASSWORD | password=- | password=+FILENAME]
[cookie=COOKIE] [thumbprint=THUMBPRINT]
- [port=PORT] [nfchostport=PORT]
+ [port=PORT] [nfchostport=PORT] [single-link=true]
[snapshot=MOREF] [transports=MODE:MODE:...]
nbdkit vddk --dump-plugin
@@ -128,6 +128,11 @@ Optional. The port on the VCenter/ESXi host. Defaults to
443.
Optional (required for remote connections). The hostname or IP
address of VCenter or ESXi host.
+=item B<single-link=true>
+
+Optional. Open the current link, not the entire chain. This
+corresponds to the C<VIXDISKLIB_FLAG_OPEN_SINGLE_LINK> flag.
+
=item B<snapshot=>MOREF
Optional. The Managed Object Reference of the snapshot.
diff --git a/plugins/vddk/vddk-structs.h b/plugins/vddk/vddk-structs.h
index 24b3259..bc68ac6 100644
--- a/plugins/vddk/vddk-structs.h
+++ b/plugins/vddk/vddk-structs.h
@@ -43,6 +43,7 @@
typedef uint64_t VixError;
#define VIX_OK 0
+#define VIXDISKLIB_FLAG_OPEN_SINGLE_LINK 2
#define VIXDISKLIB_FLAG_OPEN_READ_ONLY 4
#define VIXDISKLIB_SECTOR_SIZE 512
diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c
index 8ea923a..78a4c88 100644
--- a/plugins/vddk/vddk.c
+++ b/plugins/vddk/vddk.c
@@ -91,6 +91,7 @@ static int nfc_host_port = 0; /* nfchostport */
static char *password = NULL; /* password */
static int port = 0; /* port */
static const char *server_name = NULL; /* server */
+static bool single_link = false; /* single-link */
static const char *snapshot_moref = NULL; /* snapshot */
static const char *thumb_print = NULL; /* thumbprint */
static const char *transport_modes = NULL; /* transports */
@@ -272,6 +273,13 @@ vddk_config (const char *key, const char *value)
else if (strcmp (key, "vm") == 0) {
vmx_spec = value;
}
+ else if (strcmp (key, "single-link") == 0) {
+ int r = nbdkit_parse_bool (value);
+
+ if (r == -1)
+ return -1;
+ single_link = r;
+ }
else {
nbdkit_error ("unknown parameter '%s'", key);
return -1;
@@ -464,6 +472,8 @@ vddk_open (int readonly)
flags = 0;
if (readonly)
flags |= VIXDISKLIB_FLAG_OPEN_READ_ONLY;
+ if (single_link)
+ flags |= VIXDISKLIB_FLAG_OPEN_SINGLE_LINK;
DEBUG_CALL ("VixDiskLib_Open",
"connection, %s, %d, &handle", filename, flags);
--
2.20.1
Martin Kletzander
2019-Apr-05 12:19 UTC
Re: [Libguestfs] [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK
On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote:>From: Martin Kletzander <mkletzan@redhat.com> > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com>Just to mention, this is not complete yet. The disk needs to be opened as SINGLE_LINK only for the purpose of extents(), but without that flag for reading. I hope VDDK let's us have two concurrent handles to the same disk with different open flags. If not, there are other ways, of course.>--- > plugins/vddk/nbdkit-vddk-plugin.pod | 7 ++++++- > plugins/vddk/vddk-structs.h | 1 + > plugins/vddk/vddk.c | 10 ++++++++++ > 3 files changed, 17 insertions(+), 1 deletion(-) > >diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod b/plugins/vddk/nbdkit-vddk-plugin.pod >index 586aab4..25a6511 100644 >--- a/plugins/vddk/nbdkit-vddk-plugin.pod >+++ b/plugins/vddk/nbdkit-vddk-plugin.pod >@@ -8,7 +8,7 @@ nbdkit-vddk-plugin - nbdkit VMware VDDK plugin > [vm=moref=ID] [server=HOSTNAME] [user=USERNAME] > [password=PASSWORD | password=- | password=+FILENAME] > [cookie=COOKIE] [thumbprint=THUMBPRINT] >- [port=PORT] [nfchostport=PORT] >+ [port=PORT] [nfchostport=PORT] [single-link=true] > [snapshot=MOREF] [transports=MODE:MODE:...] > nbdkit vddk --dump-plugin > >@@ -128,6 +128,11 @@ Optional. The port on the VCenter/ESXi host. Defaults to 443. > Optional (required for remote connections). The hostname or IP > address of VCenter or ESXi host. > >+=item B<single-link=true> >+ >+Optional. Open the current link, not the entire chain. This >+corresponds to the C<VIXDISKLIB_FLAG_OPEN_SINGLE_LINK> flag. >+ > =item B<snapshot=>MOREF > > Optional. The Managed Object Reference of the snapshot. >diff --git a/plugins/vddk/vddk-structs.h b/plugins/vddk/vddk-structs.h >index 24b3259..bc68ac6 100644 >--- a/plugins/vddk/vddk-structs.h >+++ b/plugins/vddk/vddk-structs.h >@@ -43,6 +43,7 @@ > typedef uint64_t VixError; > #define VIX_OK 0 > >+#define VIXDISKLIB_FLAG_OPEN_SINGLE_LINK 2 > #define VIXDISKLIB_FLAG_OPEN_READ_ONLY 4 > #define VIXDISKLIB_SECTOR_SIZE 512 > >diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c >index 8ea923a..78a4c88 100644 >--- a/plugins/vddk/vddk.c >+++ b/plugins/vddk/vddk.c >@@ -91,6 +91,7 @@ static int nfc_host_port = 0; /* nfchostport */ > static char *password = NULL; /* password */ > static int port = 0; /* port */ > static const char *server_name = NULL; /* server */ >+static bool single_link = false; /* single-link */ > static const char *snapshot_moref = NULL; /* snapshot */ > static const char *thumb_print = NULL; /* thumbprint */ > static const char *transport_modes = NULL; /* transports */ >@@ -272,6 +273,13 @@ vddk_config (const char *key, const char *value) > else if (strcmp (key, "vm") == 0) { > vmx_spec = value; > } >+ else if (strcmp (key, "single-link") == 0) { >+ int r = nbdkit_parse_bool (value); >+ >+ if (r == -1) >+ return -1; >+ single_link = r; >+ } > else { > nbdkit_error ("unknown parameter '%s'", key); > return -1; >@@ -464,6 +472,8 @@ vddk_open (int readonly) > flags = 0; > if (readonly) > flags |= VIXDISKLIB_FLAG_OPEN_READ_ONLY; >+ if (single_link) >+ flags |= VIXDISKLIB_FLAG_OPEN_SINGLE_LINK; > > DEBUG_CALL ("VixDiskLib_Open", > "connection, %s, %d, &handle", filename, flags); >-- >2.20.1 >
Richard W.M. Jones
2019-Apr-05 13:35 UTC
Re: [Libguestfs] [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK
On Fri, Apr 05, 2019 at 02:19:25PM +0200, Martin Kletzander wrote:> On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote: > >From: Martin Kletzander <mkletzan@redhat.com> > > > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com> > > Just to mention, this is not complete yet. The disk needs to be opened as > SINGLE_LINK only for the purpose of extents(), but without that flag for > reading. I hope VDDK let's us have two concurrent handles to the same disk with > different open flags. If not, there are other ways, of course.I think there are a couple of implicit questions here: (1) Can VDDK (actually, VMware server) handle two connections to the same disk image, both open for reading and with different flags? The answer to that is yes. Running these two sets of commands in parallel works fine for me: $ LD_LIBRARY_PATH=vddk/vmware-vix-disklib-distrib/lib64 \ ./nbdkit vddk file="[datastore1] Fedora 28/Fedora 28.vmdk" \ libdir=vddk/vmware-vix-disklib-distrib \ server=vmware user=root password=- \ thumbprint=xx vm=moref=3 \ -fv -r -p 10809 single-link=true & $ qemu-img convert nbd:localhost:10809 /var/tmp/out $ LD_LIBRARY_PATH=vddk/vmware-vix-disklib-distrib/lib64 \ ./nbdkit vddk file="[datastore1] Fedora 28/Fedora 28.vmdk" \ libdir=vddk/vmware-vix-disklib-distrib \ server=vmware user=root password=- \ thumbprint=xx vm=moref=3 \ -fv -r -p 10810 & $ qemu-img convert nbd:localhost:10810 /var/tmp/out2 (2) Can we do this all in one nbdkit process? Maybe but it's a large architectural for the VDDK plugin from what we have right now. We could fork the VDDK plugin or hack things in the current plugin, but instead is it possible to do this from two nbdkit processes? ie: We'd have one which does the map and a second which does the read from the mapped areas? This pushes the complexity up to qemu-img, but if we think that qemu-img cannot actually do what we want for other reasons and we're going to need to write a small "copier" program, then having it open two NBD connections instead of one isn't a big deal IMO. nbdkit itself is explicitly designed for this situation however. Soon virt-v2v could be running half a dozen nbdkit subprocesses for common conversion scenarios. nbdkit is designed to be used as a captive subprocess (https://github.com/libguestfs/nbdkit/blob/master/docs/nbdkit-captive.pod) If we do this then the patch as it stands would be fine. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Apparently Analagous Threads
- [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK
- Re: nbdkit, VDDK, extents, readahead, etc
- Re: More parallelism in VDDK driver (was: Re: CFME-5.11.7.3 Perf. Tests)
- Re: More parallelism in VDDK driver (was: Re: CFME-5.11.7.3 Perf. Tests)
- Re: nbdkit, VDDK, extents, readahead, etc