Libguestfs - Aug 2023 - [nbdkit PATCH] cc: Allow configuration without absolute paths

In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports
that in a cross-compilation environment, nbdkit embeds the absolute
name of the cross-compiler into the resulting cc plugin, even though
running the plugin should be favoring the bare name 'cc'.  This in
turn leads to non-reproducible builds.  As the goal of cross-compiling
nbdkit is to produce a binary that behaves identically regardless of
the build environment used, this means we need to give the user
control over the defaults for CC and CFLAGS embedded into the cc
plugin.

However, instead of trying to munge the build environment variable as
suggested in that merge request, I found it cleaner to just add
additional precious variables to be set at configure time, as in:

./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc' ...

Reported-by: Khem Raj
Signed-off-by: Eric Blake <eblake at redhat.com>
---

gitlab doesn't let me see the right email address to cc; if I can
figure that out, I'll tweak the Reported-by line as appropriate before
committing...
---
 plugins/cc/nbdkit-cc-plugin.pod |  9 ++++++---
 configure.ac                    | 11 +++++++++++
 plugins/cc/Makefile.am          |  4 ++--
 3 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/plugins/cc/nbdkit-cc-plugin.pod b/plugins/cc/nbdkit-cc-plugin.pod
index 2974890c..2bc3cfb8 100644
--- a/plugins/cc/nbdkit-cc-plugin.pod
+++ b/plugins/cc/nbdkit-cc-plugin.pod
@@ -45,9 +45,12 @@ To replace the compiler flags:

 The plugin parameters C<CC>, C<CFLAGS> and C<EXTRA_CFLAGS>
(written in
 uppercase) can be used to control which C compiler and C compiler
-flags are used.  If not set, the default compiler and flags from when
-nbdkit was itself compiled from source are used.  To see what those
-were you can do:
+flags are used.  If not set, you can hardcode the defaults for C<CC>
+and C<CFLAGS> at the time nbdkit is compiled from source by
+configuring with C<CC_PLUGIN_CC=...> and C<CC_PLUGIN_CFLAGS=...>,
+otherwise, the configuration for compiling nbdkit itself is used
+(C<EXTRA_CFLAGS> can only be set from the command line when starting
+the cc plugin).  To see what those were you can do:

  $ nbdkit cc --dump-plugin
  ...
diff --git a/configure.ac b/configure.ac
index afc5ddab..e5e261c8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -820,6 +820,15 @@ AC_ARG_ENABLE([plugins],
                     [disable all bundled plugins and filters])])
 AM_CONDITIONAL([HAVE_PLUGINS], [test "x$enable_plugins" !=
"xno"])

+dnl For the cc plugin, let the user hard-code their preferred compiler setup
+dnl Default to the settings used for nbdkit itself
+AC_ARG_VAR([CC_PLUGIN_CC],
+  [Value to use for CC when building the cc plugin, default $CC])
+: "${CC_PLUGIN_CC:=$CC}"
+AC_ARG_VAR([CC_PLUGIN_CFLAGS],
+  [Value to use for CFLAGS when building the cc plugin, default $CFLAGS])
+: "${CC_PLUGIN_CFLAGS:=$CFLAGS}"
+
 dnl Check for Perl, for embedding in the perl plugin.
 dnl Note that the perl binary is checked above.
 AC_ARG_ENABLE([perl],
@@ -1716,6 +1725,8 @@ feature "tests using libguestfs" \
                               test "x$HAVE_LIBGUESTFS_TRUE" =
"x" && \
                               test "x$USE_LIBGUESTFS_FOR_TESTS_TRUE"
= "x"
 feature "zlib-ng"             test "x$ZLIB_NG_LIBS" !=
"x"
+print cc-plugin-CC            "$CC_PLUGIN_CC"
+print cc-plugin-CFLAGS        "$CC_PLUGIN_CFLAGS"

 echo
 echo "If any optional component is configured ???no??? when you expected
???yes???"
diff --git a/plugins/cc/Makefile.am b/plugins/cc/Makefile.am
index 935d125f..088b5ff3 100644
--- a/plugins/cc/Makefile.am
+++ b/plugins/cc/Makefile.am
@@ -45,8 +45,8 @@ nbdkit_cc_plugin_la_SOURCES = \
 	$(NULL)

 nbdkit_cc_plugin_la_CPPFLAGS = \
-	-DCC="\"$(CC)\"" \
-	-DCFLAGS="\"$(CFLAGS)\"" \
+	-DCC="\"$(CC_PLUGIN_CC)\"" \
+	-DCFLAGS="\"$(CC_PLUGIN_CFLAGS)\"" \
 	-I$(top_srcdir)/include \
 	-I$(top_builddir)/include \
 	-I$(top_srcdir)/common/include \
-- 
2.41.0

On Fri, Aug 18, 2023 at 08:27:45AM -0500, Eric Blake
wrote:
> In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports
> that in a cross-compilation environment, nbdkit embeds the absolute
> name of the cross-compiler into the resulting cc plugin, even though
> running the plugin should be favoring the bare name 'cc'.  This in
> turn leads to non-reproducible builds.  As the goal of cross-compiling
> nbdkit is to produce a binary that behaves identically regardless of
> the build environment used, this means we need to give the user
> control over the defaults for CC and CFLAGS embedded into the cc
> plugin.
> 
> However, instead of trying to munge the build environment variable as
> suggested in that merge request, I found it cleaner to just add
> additional precious variables to be set at configure time, as in:
> 
> ./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc'
...
> 
> Reported-by: Khem Raj
> Signed-off-by: Eric Blake <eblake at redhat.com>
> ---
> 
> gitlab doesn't let me see the right email address to cc; if I can
> figure that out, I'll tweak the Reported-by line as appropriate before
> committing...
> ---
>  plugins/cc/nbdkit-cc-plugin.pod |  9 ++++++---
>  configure.ac                    | 11 +++++++++++
>  plugins/cc/Makefile.am          |  4 ++--
>  3 files changed, 19 insertions(+), 5 deletions(-)
> 
> diff --git a/plugins/cc/nbdkit-cc-plugin.pod
b/plugins/cc/nbdkit-cc-plugin.pod
> index 2974890c..2bc3cfb8 100644
> --- a/plugins/cc/nbdkit-cc-plugin.pod
> +++ b/plugins/cc/nbdkit-cc-plugin.pod
> @@ -45,9 +45,12 @@ To replace the compiler flags:
> 
>  The plugin parameters C<CC>, C<CFLAGS> and
C<EXTRA_CFLAGS> (written in
>  uppercase) can be used to control which C compiler and C compiler
> -flags are used.  If not set, the default compiler and flags from when
> -nbdkit was itself compiled from source are used.  To see what those
> -were you can do:
> +flags are used.  If not set, you can hardcode the defaults for C<CC>
> +and C<CFLAGS> at the time nbdkit is compiled from source by
> +configuring with C<CC_PLUGIN_CC=...> and
C<CC_PLUGIN_CFLAGS=...>,
> +otherwise, the configuration for compiling nbdkit itself is used
> +(C<EXTRA_CFLAGS> can only be set from the command line when starting
> +the cc plugin).  To see what those were you can do:
> 
>   $ nbdkit cc --dump-plugin
>   ...
Widening the context,

    ...
    CC=gcc
    CFLAGS=-g -O2 -fPIC -shared

  The C<CFLAGS> parameter overrides the built-in flags completely.  The
  C<EXTRA_CFLAGS> parameter adds extra flags to the built-in flags.

Since we already mention EXTRA_CFLAGS below the example, I'm not sure
if my addition of a parenthetical about EXTRA_CFLAGS above is
worthwhile, or just adds noise.
> diff --git a/configure.ac b/configure.ac
> index afc5ddab..e5e261c8 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -820,6 +820,15 @@ AC_ARG_ENABLE([plugins],
>                      [disable all bundled plugins and filters])])
>  AM_CONDITIONAL([HAVE_PLUGINS], [test "x$enable_plugins" !=
"xno"])
> 
> +dnl For the cc plugin, let the user hard-code their preferred compiler
setup
> +dnl Default to the settings used for nbdkit itself
> +AC_ARG_VAR([CC_PLUGIN_CC],
> +  [Value to use for CC when building the cc plugin, default $CC])
I'm wondering if there is a better way to word this (it shows up in
'./configure --help' output).  Maybe:

[Value to hard-code into the cc plugin's default for CC, instead of $CC]
> +: "${CC_PLUGIN_CC:=$CC}"
> +AC_ARG_VAR([CC_PLUGIN_CFLAGS],
> +  [Value to use for CFLAGS when building the cc plugin, default $CFLAGS])
> +: "${CC_PLUGIN_CFLAGS:=$CFLAGS}"
and similar

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org