similar to: [nbdkit PATCH v4 0/4] vddk: Drive library loading from libdir parameter.

Differences from v4: Patch 1 is simplified: I realized that since we already use -rdynamic for nbdkit (after all, we WANT our dlopen()d plugins to be able to call our nbdkit_* exports), it is only a matter of adding dlopen to the set of symbols that we export. With that done, there is no separate shared library needed; our dlopen shim is now part of nbdkit proper, and we don't have to tweak

I couldn't get this to work in the end. This is the latest non-working version. This email documents what doesn't work for the permanent record. The central problem is that VDDK InitEx() appears to dlopen() various of its own plugins. Although I wasn't able to capture exactly what dlopen() command it is running, the plugins cannot be loaded because they rely on the recompiled

We were previously dlopen-ing it in the load() method. This is very early and in particular means that the only possible way to configure where we find the library is through environment variables and not through config parameters. Also it's not necessary as we don't call any functions from the library (such as VixDiskLib_InitEx) until config_complete. This change is neutral refactoring

In v7: everything should work now! The re-exec code is slightly simplified, with Rich's suggestion to pass the original LD_LIBRARY_PATH rather than just the prefix being added, and I've now finished wiring up the initial dlopen() check into code that correctly computes the right prefix dir to add to LD_LIBRARY_PATH. Eric Blake (1): vddk: Add re-exec with altered environment Richard

On Tue, Feb 18, 2020 at 11:05:11AM -0600, Eric Blake wrote: > static const char *sonames[] = { > - /* Prefer the newest library in case multiple exist. */ > + /* Prefer the newest library in case multiple exist. Check two > + * possible directories: the usual VDDK installation puts .so > + * files in an arch-specific subdirectory of $libdir (although > + *

Unfortunately this series doesn't work with the real VDDK library :-( I see: nbdkit: debug: vddk: config_complete nbdkit: debug: dlopen shim prefix set to /home/rjones/tmp/vddk-6.7.3/vmware-vix-disklib-distrib/lib64 Then a little bit later during VDDK initialization it fails with: nbdkit: debug: VixDiskLib: Could not load default plugins from

That version depends on libexpat.so but does not ship it, and it appears that VMware no longer supports it. Since VDDK 5.5.5 (the next oldest version) dropped support for 32-bit platforms, we can slightly simplify our code by documenting our minimum supported version. Signed-off-by: Eric Blake <eblake@redhat.com> --- v2: scrub for more places to clean up, based on Rich's review of v1

From: "Richard W.M. Jones" <rjones@redhat.com> Do not use LD_LIBRARY_PATH to locate the VDDK library. Setting this always causes problems because VDDK comes bundled with broken replacements for system libraries, such as libcrypto.so and libstdc++.so. Two problems this causes which we have seen in the real world: (1) User does ‘export LD_LIBRARY_PATH=vmware-vix-disklib-distrib’

On Fri, Feb 21, 2020 at 01:19:34PM +0100, Florian Weimer wrote: > I think what confuses me is that keep talking about a single binary, but > clearly there is this separate vddk DSO, and there is talk of plugins. > So it seems to me that multiple files are involved already? nbdkit is a standalone binary that happens to be able to load plugins from a well-known path, eg

On 2/13/20 8:01 AM, Richard W.M. Jones wrote: > Do not use LD_LIBRARY_PATH to locate the VDDK library. Setting this > always causes problems because VDDK comes bundled with broken > replacements for system libraries, such as libcrypto.so and > libstdc++.so. Two problems this causes which we have seen in the real > world: > > (1) User does ‘export

On Fri, Feb 21, 2020 at 04:00:30PM +0100, Florian Weimer wrote: > * Richard W. M. Jones: > > > On Fri, Feb 21, 2020 at 01:19:34PM +0100, Florian Weimer wrote: > >> I think what confuses me is that keep talking about a single binary, but > >> clearly there is this separate vddk DSO, and there is talk of plugins. > >> So it seems to me that multiple files are

In the next patch, we want to get rid of the requirement for the user to supply LD_LIBRARY_PATH pointing to vddk libs, if we can derive it ourselves from libdir. However, VDDK itself requires LD_LIBRARY_PATH to be set (because it tries to load libraries that in turn depend on a bare library name, which no manner of dlopen() hacking can work around, and implementing la_objsearch() is no better for

On Fri, Feb 21, 2020 at 05:02:12PM +0100, Florian Weimer wrote: > * Richard W. M. Jones: > > > On Fri, Feb 21, 2020 at 04:00:30PM +0100, Florian Weimer wrote: > >> * Richard W. M. Jones: > >> > >> > On Fri, Feb 21, 2020 at 01:19:34PM +0100, Florian Weimer wrote: > >> >> I think what confuses me is that keep talking about a single binary,

On 2/17/20 9:12 AM, Florian Weimer wrote: > * Eric Blake: > >> I'm just now noticing that 'man ld' reports that you may pass '--audit >> LIB' during linking to add a DT_DEPAUDIT dependency on a library >> implementing the audit interface, which sounds like it might be an >> alternative to LD_AUDIT for getting a library with la_objsearch() to

I've got a situation where I need to hook a dlopen() made by VDDK, a proprietary library, where it passes a relative name expecting to resolve to a copy of several libraries, including libstdc++.so, that it installs alongside itself, and fails to load if that resolves to the system libstdc++.so. The simplest solution of providing LD_LIBRARY_PATH is enough to load VDDK, but then poisons

--- configure.ac | 5 +++++ plugins/vddk/vddk.c | 4 ++++ 2 files changed, 9 insertions(+) diff --git a/configure.ac b/configure.ac index d71f06e4..57626a76 100644 --- a/configure.ac +++ b/configure.ac @@ -321,6 +321,11 @@ AC_SEARCH_LIBS([dlsym], [dl dld], [ ]) LIBS="$old_LIBS" +old_LIBS="$LIBS" +LIBS="$LIBS -ldl" +AC_CHECK_FUNCS([dlmopen])

From: "Richard W.M. Jones" <rjones@redhat.com> Do not use LD_LIBRARY_PATH to locate the VDDK library. Setting this always causes problems because VDDK comes bundled with broken replacements for system libraries, such as libcrypto.so and libstdc++.so. Two problems this causes which we have seen in the real world: (1) User does ‘export LD_LIBRARY_PATH=vmware-vix-disklib-distrib’

Proposed small change to the command line of virt-v2v when specifying that you want VDDK mode. Rich.

Since it is required, it might as well be magic ;) Signed-off-by: Eric Blake <eblake@redhat.com> --- As written, applies on top of my v3 patch for dlopen; but it would be easy enough to rebase and take this one now even if we aren't sure about the dlopen stuff plugins/vddk/nbdkit-vddk-plugin.pod | 5 ++++- plugins/vddk/vddk.c | 3 ++- tests/test-vddk.sh

Probably needs a bit of cleanup, but seems like it is generally the right direction. One thing I've noticed is that the expect test randomly (but rarely) hangs :-( I guess something is racey but I don't know what at the moment. Rich.