similar to: [PATCH nbdkit] vddk: Add support for VIXDISKLIB_FLAG_OPEN_SINGLE_LINK

On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote: >From: Martin Kletzander <mkletzan@redhat.com> > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com> Just to mention, this is not complete yet. The disk needs to be opened as SINGLE_LINK only for the purpose of extents(), but without that flag for reading. I hope VDDK let's us have two concurrent

I suppose we may as well implement the only other flag too ... It's not clear what this does, something like O_DIRECT I imagine. Rich.

We had a query yesterday about the VDDK plugin and making it actually obey the weird "Multithreading Considerations" rules in the VDDK documentation (https://vdc-download.vmware.com/vmwb-repository/dcr-public/8f96698a-0e7b-4d67-bb6c-d18a1d101540/ef536a47-27cd-481a-90ef-76b38e75353c/vsphere-vddk-671-programming-guide.pdf) This patch is my attempt to implement this. The idea is that the

Proposed small change to the command line of virt-v2v when specifying that you want VDDK mode. Rich.

I believe this roughly implements Nir's proposal here: https://www.redhat.com/archives/libguestfs/2020-August/msg00028.html Unfortunately it doesn't work for me. It actually slows things down quite a lot, for reasons I don't understand. Note the adjustment of the pool-max parameter and how it affects the total time. The results are quite reproducible. $ ./nbdkit -r -U - vddk

On Fri, Apr 05, 2019 at 02:19:25PM +0200, Martin Kletzander wrote: > On Fri, Apr 05, 2019 at 10:33:05AM +0100, Richard W.M. Jones wrote: > >From: Martin Kletzander <mkletzan@redhat.com> > > > >Signed-off-by: Martin Kletzander <mkletzan@redhat.com> > > Just to mention, this is not complete yet. The disk needs to be opened as > SINGLE_LINK only for the

This patch isn't quite complete (see ‘assert false’). Currently we have a bunch of ad hoc options like --vddk* and --vdsm* (and proposed to add --rhv*) to handle extra parameters for input and output modes/transports. This complicates the command line parsing and also the clarity of the command line (becauseit's not very obvious which options apply to which side of the conversion).

Couple of fixes to return correct data and one nice-to-have clean-up which is not needed. I just find it nicer to read. Martin Kletzander (3): vddk: Use a separate handle for single-link=true vddk: Do not report hole extents to be zero with single-link=true vddk: Eliminate one needless goto plugins/vddk/vddk.c | 48 +++++++++++++++++++++++++++++++++------------ 1 file changed, 36

In v7: everything should work now! The re-exec code is slightly simplified, with Rich's suggestion to pass the original LD_LIBRARY_PATH rather than just the prefix being added, and I've now finished wiring up the initial dlopen() check into code that correctly computes the right prefix dir to add to LD_LIBRARY_PATH. Eric Blake (1): vddk: Add re-exec with altered environment Richard

When using VIXDISKLIB_FLAG_OPEN_SINGLE_LINK, parent images are not taken into account. However, the data we get from VixDiskLib_QueryAllocatedBlocks() has such granularity that it might return incorrect information. For example when the top image has only one block allocated, we get information about the whole chunk being allocated. This results in the consequent read to return invalid data -

On Fri, Apr 12, 2019 at 03:52:58PM +0200, Martin Kletzander wrote: > On Thu, Apr 11, 2019 at 06:55:55PM +0100, Richard W.M. Jones wrote: > >This seems to be a bug in VDDK, possibly connected with the fact that > >we fork after initializing VDDK but before doing the > >VixDiskLib_ConnectEx. I suspect it's something to do with the PID > >changing. > > >

On Wed, Aug 5, 2020 at 3:47 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > > Here are some results anyway. The command I'm using is: > > $ ./nbdkit -r -U - vddk \ > libdir=/path/to/vmware-vix-disklib-distrib \ > user=root password='***' \ > server='***' thumbprint=aa:bb:cc:... \ > vm=moref=3 \ >

This is my second attempt at this. The first version (also not working) was here: https://www.redhat.com/archives/libguestfs/2019-October/msg00062.html In part 1/2 I introduce a new .ready_to_serve plugin method which is called after forking and just before accepting any client connection. The idea would be that plugins could start background threads here. However this doesn't work well in

The first patch was previously posted here: https://www.redhat.com/archives/libguestfs/2017-December/msg00018.html That patch hasn't changed except that I made the ‘input_transport’ variable type-safe. The second patch adds a significant new mode for liberating data from VMware: the ability to copy VMs over SSH directly from ESXi hypervisors. Although this requires enabling SSH access (a

Nir, BTW what are you using for performance testing? As far as I can tell it's not possible to make qemu-img convert use multi-conn when connecting to the source (which is going to be a problem if we want to use this stuff in virt-v2v). Instead I've hacked up a copy of this program from libnbd: https://github.com/libguestfs/libnbd/blob/master/examples/threaded-reads-and-writes.c so

v6 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00126.html This makes a number of significant changes: - Input and output options now use a uniform set of -io and -oo parameters. - For -o rhv-upload, we use ‘-oo rhv-cafile=/tmp/ca.pem’ etc. The ‘--rhv*’ options have been dropped. - Rearranges the documentation. - As before includes (untested) support for zero,

The pool is only used for readonly connections, since writable connections usually take a lock on the server side and therefore you cannot open more than one. --- plugins/vddk/nbdkit-vddk-plugin.pod | 7 + plugins/vddk/vddk.c | 201 ++++++++++++++++++++++------ 2 files changed, 164 insertions(+), 44 deletions(-) diff --git a/plugins/vddk/nbdkit-vddk-plugin.pod

On Mon, Apr 15, 2019 at 04:30:47PM +0200, Martin Kletzander wrote: > On Fri, Apr 12, 2019 at 04:30:02PM +0100, Richard W.M. Jones wrote: > > +-- nbdkit monitoring process > > | > > +-- first child = nbdkit > > | > > +-- second child = ‘--run’ command > > > >so when the second child exits, the monitoring process (which is doing > >nothing

This series (except the last one) changes virt-v2v to use nbdkit for several input modes: -i vmx -it vddk: No change in functionality, as this already uses nbdkit-vddk-plugin, but the code is refactored for the other modes to use. -i libvirtxml: Use nbdkit-curl-plugin instead of qemu curl. vCenter: Use nbdkit-curl-plugin instead of qemu curl. xen: Use nbdkit-ssh-plugin instead of qemu

As I've spent really too long today investigating this, I want to document this in a public email, even though there's nothing really that interesting here. One thing you find from search for VDD 6.7 / VixDiskLib_QueryAllocatedBlocks issues with Google is that we must be one of the very few users out there. And the other thing is that it's quite broken. All testing was done using