Libguestfs - Feb 2020 - [PATCH nbdkit] NOT WORKING vddk: Use dlmopen to isolate VDDK.

---
 configure.ac        | 5 +++++
 plugins/vddk/vddk.c | 4 ++++
 2 files changed, 9 insertions(+)

diff --git a/configure.ac b/configure.ac
index d71f06e4..57626a76 100644
--- a/configure.ac
+++ b/configure.ac
@@ -321,6 +321,11 @@ AC_SEARCH_LIBS([dlsym], [dl dld], [
 ])
 LIBS="$old_LIBS"
 
+old_LIBS="$LIBS"
+LIBS="$LIBS -ldl"
+AC_CHECK_FUNCS([dlmopen])
+LIBS="$old_LIBS"
+
 dnl Test if <iconv.h> header can build working binaries.
 dnl
 dnl On FreeBSD: iconv and libiconv both exist, both can be installed
diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c
index c49eebcd..b988946b 100644
--- a/plugins/vddk/vddk.c
+++ b/plugins/vddk/vddk.c
@@ -267,7 +267,11 @@ load_library (void)
       exit (EXIT_FAILURE);
     }
 
+#ifdef HAVE_DLMOPEN
+    dl = dlmopen (LM_ID_NEWLM, path, RTLD_NOW);
+#else
     dl = dlopen (path, RTLD_NOW);
+#endif
     if (dl != NULL)
       break;
     if (i == 0) {
-- 
2.25.0

On Thu, Feb 13, 2020 at 02:06:47PM +0000, Richard W.M. Jones
wrote:
> ---
>  configure.ac        | 5 +++++
>  plugins/vddk/vddk.c | 4 ++++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/configure.ac b/configure.ac
> index d71f06e4..57626a76 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -321,6 +321,11 @@ AC_SEARCH_LIBS([dlsym], [dl dld], [
>  ])
>  LIBS="$old_LIBS"
>  
> +old_LIBS="$LIBS"
> +LIBS="$LIBS -ldl"
> +AC_CHECK_FUNCS([dlmopen])
> +LIBS="$old_LIBS"
> +
>  dnl Test if <iconv.h> header can build working binaries.
>  dnl
>  dnl On FreeBSD: iconv and libiconv both exist, both can be installed
> diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c
> index c49eebcd..b988946b 100644
> --- a/plugins/vddk/vddk.c
> +++ b/plugins/vddk/vddk.c
> @@ -267,7 +267,11 @@ load_library (void)
>        exit (EXIT_FAILURE);
>      }
>  
> +#ifdef HAVE_DLMOPEN
> +    dl = dlmopen (LM_ID_NEWLM, path, RTLD_NOW);
> +#else
>      dl = dlopen (path, RTLD_NOW);
> +#endif
>      if (dl != NULL)
>        break;
>      if (i == 0) {
The error when this patch is applied is:

nbdkit: debug: VDDK call: VixDiskLib_InitEx (5, 1, &debug_fn, &error_fn,
&error_fn, /home/rjones/tmp/vddk-5.1.1/vmware-vix-disklib-distrib, NULL)
nbdkit: debug: VixDiskLib: linuxVerifySSLCertificates is 0
nbdkit: debug: VixDiskLib: config options: libdir
'/home/rjones/tmp/vddk-5.1.1/vmware-vix-disklib-distrib', tmpDir
'/tmp/vmware-rjones'.
nbdkit: debug: OBJLIB-LIB : Objlib initialized.
nbdkit: debug: VixDiskLib: Attempting to locate advanced transport module in
"/home/rjones/tmp/vddk-5.1.1/vmware-vix-disklib-distrib".
/home/rjones/d/nbdkit/server/nbdkit: error while loading shared libraries:
libcrypto.so.0.9.8: cannot open shared object file: No such file or directory

What appears to happen here is that libvixDiskLib.so.6 is loaded and
runs (see the VixDiskLib debug messages which come from VDDK itself).

However VDDK tries to open "Advanced Transport" (which is a VDDK
extension) and I think it's trying itself to use dlopen() and that is
failing.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/