Libguestfs - Apr 2020 - Re: [PATCH nbdkit] golang: Compile against the local nbdkit build, not installed.

When compiling when an older nbdkit is installed, the build would fail
because certain symbols such as .get_ready were not defined:

  ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready undefined
(type _Ctype_struct_nbdkit_plugin has no field or method get_ready)

This happens because we were using the installed <nbdkit-plugin.h>
rather than the local copy.

We don't want to modify the *.go files themselves as they might be
copied into other projects.  Instead we can set PKG_CONFIG to point to
a fake pkg-config binary which will return the correct CFLAGS.

Fixes: commit 1ff44288ae1cf95428283e252edd9474c3fe3b55
Thanks: Dan Berrangé, Eric Blake
---
 .gitignore                          |  1 +
 configure.ac                        |  2 ++
 plugins/golang/Makefile.am          |  4 +++
 plugins/golang/golang-pkgconf.sh.in | 40 +++++++++++++++++++++++++++++
 4 files changed, 47 insertions(+)

diff --git a/.gitignore b/.gitignore
index 4f6fcf1b..bda3b05a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -74,6 +74,7 @@ plugins/*/*.3
 /plugins/eval/methods.c
 /plugins/golang/examples/*/nbdkit-*-plugin.h
 /plugins/golang/examples/*/nbdkit-*-plugin.so
+/plugins/golang/golang-pkgconf.sh
 /plugins/golang/test/nbdkit-gotest-plugin.h
 /plugins/golang/test/nbdkit-gotest-plugin.so
 /plugins/ocaml/nbdkit-ocamlexample-plugin.so
diff --git a/configure.ac b/configure.ac
index b71d9d8c..405890f5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1007,6 +1007,8 @@ AC_CONFIG_FILES([podwrapper.pl],
                 [chmod +x,-w podwrapper.pl])
 AC_CONFIG_FILES([common/protocol/generate-protostrings.sh],
                 [chmod +x,-w common/protocol/generate-protostrings.sh])
+AC_CONFIG_FILES([plugins/golang/golang-pkgconf.sh],
+                [chmod +x,-w plugins/golang/golang-pkgconf.sh])
 AC_CONFIG_FILES([Makefile
                  bash/Makefile
                  common/bitmap/Makefile
diff --git a/plugins/golang/Makefile.am b/plugins/golang/Makefile.am
index f189184c..60539229 100644
--- a/plugins/golang/Makefile.am
+++ b/plugins/golang/Makefile.am
@@ -66,18 +66,21 @@ noinst_DATA = \
 examples/dump-plugin/nbdkit-godump-plugin.so: \
 	    $(plugin_sources) examples/dump-plugin/dumpplugin.go
 	cd examples/dump-plugin && \
+	PKG_CONFIG=$(abs_builddir)/golang-pkgconf.sh \
 	GOPATH="$(abs_builddir)" \
 	$(GOLANG) build -o nbdkit-godump-plugin.so -buildmode=c-shared
 
 examples/minimal/nbdkit-gominimal-plugin.so: \
 	    $(plugin_sources) examples/minimal/minimal.go
 	cd examples/minimal && \
+	PKG_CONFIG=$(abs_builddir)/golang-pkgconf.sh \
 	GOPATH="$(abs_builddir)" \
 	$(GOLANG) build -o nbdkit-gominimal-plugin.so -buildmode=c-shared
 
 examples/ramdisk/nbdkit-goramdisk-plugin.so: \
 	    $(plugin_sources) examples/ramdisk/ramdisk.go
 	cd examples/ramdisk && \
+	PKG_CONFIG=$(abs_builddir)/golang-pkgconf.sh \
 	GOPATH="$(abs_builddir)" \
 	$(GOLANG) build -o nbdkit-goramdisk-plugin.so -buildmode=c-shared
 
@@ -87,6 +90,7 @@ check_DATA = test/nbdkit-gotest-plugin.so
 
 test/nbdkit-gotest-plugin.so: $(plugin_sources) test/test.go
 	cd test && \
+	PKG_CONFIG=$(abs_builddir)/golang-pkgconf.sh \
 	GOPATH="$(abs_builddir)" \
 	$(GOLANG) build -o nbdkit-gotest-plugin.so -buildmode=c-shared
 
diff --git a/plugins/golang/golang-pkgconf.sh.in
b/plugins/golang/golang-pkgconf.sh.in
new file mode 100644
index 00000000..11af3718
--- /dev/null
+++ b/plugins/golang/golang-pkgconf.sh.in
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+# nbdkit
+# Copyright (C) 2020 Red Hat Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+# This fake pkg-config program is used to trick cgo so that
+# "#cgo pkg-config nbdkit" lines are processed relative to the local
+# directory and not the installed nbdkit.
+
+case "$1" in
+    --cflags*) echo "-I@abs_top_builddir@/include" ;;
+    *) ;;
+esac
-- 
2.18.2

On Thu, Apr 23, 2020 at 05:55:14PM +0100, Richard W.M. Jones
wrote:
> When compiling when an older nbdkit is installed, the build would fail
> because certain symbols such as .get_ready were not defined:
> 
>   ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready
undefined (type _Ctype_struct_nbdkit_plugin has no field or method get_ready)
> 
> This happens because we were using the installed <nbdkit-plugin.h>
> rather than the local copy.
> 
> We don't want to modify the *.go files themselves as they might be
> copied into other projects.  Instead we can set PKG_CONFIG to point to
> a fake pkg-config binary which will return the correct CFLAGS.
> 
> Fixes: commit 1ff44288ae1cf95428283e252edd9474c3fe3b55
> Thanks: Dan Berrangé, Eric Blake
> +# This fake pkg-config program is used to trick cgo so that
> +# "#cgo pkg-config nbdkit" lines are processed relative to the
local
> +# directory and not the installed nbdkit.
> +
> +case "$1" in
> +    --cflags*) echo "-I@abs_top_builddir@/include" ;;
> +    *) ;;
> +esac
Don't you need something like this too:

   --libs)  echo "-L@abs_top_builddir@/lib -lnbdkit"

The installed nbdkit.pc would do this I presume. Or are you happy relying
on the allow undefined symbols LDFLAGS

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Thu, Apr 23, 2020 at 05:58:25PM +0100, Daniel P. Berrangé
wrote:
> On Thu, Apr 23, 2020 at 05:55:14PM +0100, Richard W.M. Jones wrote:
> > When compiling when an older nbdkit is installed, the build would fail
> > because certain symbols such as .get_ready were not defined:
> > 
> >   ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready
undefined (type _Ctype_struct_nbdkit_plugin has no field or method get_ready)
> > 
> > This happens because we were using the installed
<nbdkit-plugin.h>
> > rather than the local copy.
> > 
> > We don't want to modify the *.go files themselves as they might be
> > copied into other projects.  Instead we can set PKG_CONFIG to point to
> > a fake pkg-config binary which will return the correct CFLAGS.
> > 
> > Fixes: commit 1ff44288ae1cf95428283e252edd9474c3fe3b55
> > Thanks: Dan Berrangé, Eric Blake
> 
> > +# This fake pkg-config program is used to trick cgo so that
> > +# "#cgo pkg-config nbdkit" lines are processed relative to
the local
> > +# directory and not the installed nbdkit.
> > +
> > +case "$1" in
> > +    --cflags*) echo "-I@abs_top_builddir@/include" ;;
> > +    *) ;;
> > +esac
> 
> Don't you need something like this too:
> 
>    --libs)  echo "-L@abs_top_builddir@/lib -lnbdkit"
We don't actually have "libnbdkit".  It was proposed a while back
but
we didn't get sufficient review feedback, and the code was very
complex and unmaintainable so it never made it upstream.

cgo currently issues the following two commands:

  $PKG_CONFIG --cflags -- nbdkit nbdkit nbdkit
  $PKG_CONFIG --libs -- nbdkit nbdkit nbdkit

and yes I also don't know why it prints the package name 3 times.
> The installed nbdkit.pc would do this I presume. Or are you happy relying
> on the allow undefined symbols LDFLAGS
A "local" nbdkit.pc might indeed be better because we'd be using
pkg-config itself to parse the command line.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On 4/23/20 11:58 AM, Daniel P. Berrangé wrote:
> On Thu, Apr 23, 2020 at 05:55:14PM +0100, Richard W.M. Jones wrote:
>> When compiling when an older nbdkit is installed, the build would fail
>> because certain symbols such as .get_ready were not defined:
>>
>>    ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready
undefined (type _Ctype_struct_nbdkit_plugin has no field or method get_ready)
>>
>> This happens because we were using the installed
<nbdkit-plugin.h>
>> rather than the local copy.
>>
>> We don't want to modify the *.go files themselves as they might be
>> copied into other projects.  Instead we can set PKG_CONFIG to point to
>> a fake pkg-config binary which will return the correct CFLAGS.
>>
>> Fixes: commit 1ff44288ae1cf95428283e252edd9474c3fe3b55
>> Thanks: Dan Berrangé, Eric Blake
> 
>> +# This fake pkg-config program is used to trick cgo so that
>> +# "#cgo pkg-config nbdkit" lines are processed relative to
the local
>> +# directory and not the installed nbdkit.
>> +
>> +case "$1" in
>> +    --cflags*) echo "-I@abs_top_builddir@/include" ;;
>> +    *) ;;
>> +esac
> 
> Don't you need something like this too:
> 
>     --libs)  echo "-L@abs_top_builddir@/lib -lnbdkit"
Also, if you're installing this in PATH prior to the real pkg-config, 
shouldn't you start by checking whether 'nbdkit' is one of the 
arguments?  If so, fall through to the 'case "$1"' code, if
not,
short-circuit into 'exec /path/to/pkg-config "$@"', so that
whatever
_other_ pkg-config queries cgo makes will be answered as usual?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org