[Libguestfs] [PATCH nbdkit] build: Use dlsym as sentinel function for -ldl.

Richard W.M. Jones
2019-04-24T20:42:02Z

When testing which “dl library” we must use for dl* symbols, autoconf
runs a test similar to:

  $ cat conftest.c
  char dlopen ();
  int main () { return dlopen (); }
  $ gcc -o conftest $CFLAGS conftest.c [try various -ldl options here]

When using ‘CFLAGS="-fsanitize=address"’ this succeeds even if no dl
libraries are used at all, since it appears that using this option
causes dlopen to be included in the final binary implicitly.  This
causes configure to set DL_LIBS='', but unfortunately dlsym is not
included implicitly and so linking fails.

Although I believe this is a bug in GCC 9, as a workaround use dlsym
as the sentinel function instead.

The new output from ‘./configure CFLAGS="-fsanitize=address"’ is:

  checking for library containing dlsym... -ldl
  [...]
  checking for dladdr... yes

See also: https://bugzilla.redhat.com/show_bug.cgi?id=1702761
---
 configure.ac | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/configure.ac b/configure.ac
index 4624bf8..1d3aa7b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -224,11 +224,11 @@ AS_IF([test "x$nbdkit_cv_func_printf_percent_m"
= xyes],
     [Define to 1 if vfprintf supports %m.])])

 old_LIBS="$LIBS"
-AC_SEARCH_LIBS([dlopen], [dl dld], [
-        AS_IF([test "x$ac_cv_search_dlopen" != "xnone
required"],
-	    [DL_LIBS="$ac_cv_search_dlopen"], [DL_LIBS=])
+AC_SEARCH_LIBS([dlsym], [dl dld], [
+        AS_IF([test "x$ac_cv_search_dlsym" != "xnone
required"],
+	    [DL_LIBS="$ac_cv_search_dlsym"], [DL_LIBS=])
         AC_SUBST([DL_LIBS])
-    ], [AC_MSG_ERROR([unable to find the dlopen() function])
+    ], [AC_MSG_ERROR([unable to find the dlsym() function])
 ])
 LIBS="$old_LIBS"

--
2.20.1