similar to: [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_UNBUFFERED.

We had a query yesterday about the VDDK plugin and making it actually obey the weird "Multithreading Considerations" rules in the VDDK documentation (https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-4d67-bb6c-d18a1d101540/ef536a47-27cd-481a-90ef-76b38e75353c/vsphere-vddk-671-programming-guide.pdf) This patch is my attempt to implement this. The idea is that the

From: Martin Kletzander <mkletzan@redhat.com> Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- plugins/vddk/nbdkit-vddk-plugin.pod | 7 ++++++- plugins/vddk/vddk-structs.h | 1 + plugins/vddk/vddk.c | 10 ++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod

This is my second attempt at this. The first version (also not working) was here: https://www.redhat.com/archives/libguestfs/2019-October/msg00062.html In part 1/2 I introduce a new .ready_to_serve plugin method which is called after forking and just before accepting any client connection. The idea would be that plugins could start background threads here. However this doesn't work well in

Couple of fixes to return correct data and one nice-to-have clean-up which is not needed. I just find it nicer to read. Martin Kletzander (3): vddk: Use a separate handle for single-link=true vddk: Do not report hole extents to be zero with single-link=true vddk: Eliminate one needless goto plugins/vddk/vddk.c | 48 +++++++++++++++++++++++++++++++++------------ 1 file changed, 36

I believe this roughly implements Nir's proposal here: https://www.redhat.com/archives/libguestfs/2020-August/msg00028.html Unfortunately it doesn't work for me. It actually slows things down quite a lot, for reasons I don't understand. Note the adjustment of the pool-max parameter and how it affects the total time. The results are quite reproducible. $ ./nbdkit -r -U - vddk

In v7: everything should work now! The re-exec code is slightly simplified, with Rich's suggestion to pass the original LD_LIBRARY_PATH rather than just the prefix being added, and I've now finished wiring up the initial dlopen() check into code that correctly computes the right prefix dir to add to LD_LIBRARY_PATH. Eric Blake (1): vddk: Add re-exec with altered environment Richard

On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote: >From: Martin Kletzander <mkletzan@redhat.com> > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Just to mention, this is not complete yet. The disk needs to be opened as SINGLE_LINK only for the purpose of extents(), but without that flag for reading. I hope VDDK let's us have two concurrent

When using VIXDISKLIB_FLAG_OPEN_SINGLE_LINK, parent images are not taken into account. However, the data we get from VixDiskLib_QueryAllocatedBlocks() has such granularity that it might return incorrect information. For example when the top image has only one block allocated, we get information about the whole chunk being allocated. This results in the consequent read to return invalid data -

Probably needs a bit of cleanup, but seems like it is generally the right direction. One thing I've noticed is that the expect test randomly (but rarely) hangs :-( I guess something is racey but I don't know what at the moment. Rich.

The pool is only used for readonly connections, since writable connections usually take a lock on the server side and therefore you cannot open more than one. --- plugins/vddk/nbdkit-vddk-plugin.pod | 7 + plugins/vddk/vddk.c | 201 ++++++++++++++++++++++------ 2 files changed, 164 insertions(+), 44 deletions(-) diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod

On Thu, Aug 6, 2020, 16:16 Richard W.M. Jones <rjones@redhat.com> wrote: > The pool is only used for readonly connections, since writable > connections usually take a lock on the server side and therefore you > cannot open more than one. > --- > plugins/vddk/nbdkit-vddk-plugin.pod | 7 + > plugins/vddk/vddk.c | 201 ++++++++++++++++++++++------ > 2

On Fri, Apr 05, 2019 at 02:19:25PM +0200, Martin Kletzander wrote: > On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote: > >From: Martin Kletzander <mkletzan@redhat.com> > > > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com> > > Just to mention, this is not complete yet. The disk needs to be opened as > SINGLE_LINK only for the

This patch isn't quite complete (see ‘assert false’). Currently we have a bunch of ad hoc options like --vddk* and --vdsm* (and proposed to add --rhv*) to handle extra parameters for input and output modes/transports. This complicates the command line parsing and also the clarity of the command line (becauseit's not very obvious which options apply to which side of the conversion).

Proposed small change to the command line of virt-v2v when specifying that you want VDDK mode. Rich.

v6 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00126.html This makes a number of significant changes: - Input and output options now use a uniform set of -io and -oo parameters. - For -o rhv-upload, we use ‘-oo rhv-cafile=/tmp/ca.pem’ etc. The ‘--rhv*’ options have been dropped. - Rearranges the documentation. - As before includes (untested) support for zero,

This series (except the last one) changes virt-v2v to use nbdkit for several input modes: -i vmx -it vddk: No change in functionality, as this already uses nbdkit-vddk-plugin, but the code is refactored for the other modes to use. -i libvirtxml: Use nbdkit-curl-plugin instead of qemu curl. vCenter: Use nbdkit-curl-plugin instead of qemu curl. xen: Use nbdkit-ssh-plugin instead of qemu

The first patch was previously posted here: https://www.redhat.com/archives/libguestfs/2017-December/msg00018.html That patch hasn't changed except that I made the ‘input_transport’ variable type-safe. The second patch adds a significant new mode for liberating data from VMware: the ability to copy VMs over SSH directly from ESXi hypervisors. Although this requires enabling SSH access (a

v2 was posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00115.html This also has links to earlier versions. v3: - The 01/11 patch in v2 included a bunch of unnecessary plus one necessary change to how input_password is passed around. I moved the necessary change into the final patch (implementing SSH password authentication) and dropped the rest. - The 01/11

Originally posted here: https://www.redhat.com/archives/libguestfs/2019-April/thread.html#00054 https://www.redhat.com/archives/libguestfs/2019-April/msg00076.html https://www.redhat.com/archives/libguestfs/2019-April/msg00126.html This is a rebase on top of current master branch with no other changes. The first patch in the old series was pushed a while back, and the last "TEMPORARY"

v3 posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00200.html v4: - The first patch in the v3 series was just a trivial doc whitespace fix so I pushed it. - There's a new patch using the nbdkit-retry-filter. This is not actually upstream in nbdkit but we know enough about how it will work. - Rebased against master and reran the tests. Rich.