Richard W.M. Jones
2020-Apr-23 09:03 UTC
Re: [Libguestfs] [PATCH nbdkit v2] Add the ability to write plugins in golang.
On Tue, Apr 21, 2020 at 05:07:43PM +0100, Daniel P. Berrangé wrote:> On Tue, Apr 21, 2020 at 11:44:59AM +0100, Richard W.M. Jones wrote: > > Thanks: Dan Berrangé > > > > XXX UNFINISHED: > > > > - Is using uintptr for the handle a good idea? Plugins must return > > something != 0. In other languages we would allow plugins to > > return an arbitrary object here, but this is not possible in golang > > because of lack of GC roots. > > Yeah, this is the bit that looks wierd to me when thinking about this > from a Go dev POV. You asked me on IRC whether we should have a separate > interface for a connection object, and this makes me think that we should > indeed do that.... This is what the patch looks like with a separate connection object. It appears to work. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ --qKOY9RTXA4DvnaVz Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename="0001-Add-the-ability-to-write-plugins-in-golang.patch" Content-Transfer-Encoding: 8bit>From a929cff8745f21df913ea3230741dc7f2a1c016e Mon Sep 17 00:00:00 2001From: "Richard W.M. Jones" <rjones@redhat.com> Date: Thu, 9 Apr 2020 12:45:10 +0100 Subject: [PATCH] Add the ability to write plugins in golang. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thanks: Dan Berrangé XXX UNFINISHED: - Default can_* methods are hard to implement. Ideally we would be able to test if a user plugin implements a particular callback (eg. TestPlugin.PWrite) or else defaults to the default callback, but even with reflection I don't think this is possible in golang. [Discussed on IRC: We decided the only option was to go for nbdkit-sh-plugin style can_* methods] - Write wrappers etc for all the other methods. - Write documentation. --- plugins/golang/nbdkit-golang-plugin.pod | 34 +++ configure.ac | 32 +++ plugins/golang/Makefile.am | 80 ++++++ .../src/libguestfs.org/nbdkit/wrappers.h | 41 +++ plugins/golang/config-test.go | 38 +++ .../src/libguestfs.org/nbdkit/nbdkit.go | 270 ++++++++++++++++++ .../golang/src/libguestfs.org/nbdkit/utils.go | 75 +++++ .../src/libguestfs.org/nbdkit/wrappers.go | 102 +++++++ plugins/golang/test/run-test.sh | 42 +++ plugins/golang/test/test.go | 119 ++++++++ .gitignore | 2 + README | 4 + 12 files changed, 839 insertions(+) create mode 100644 plugins/golang/nbdkit-golang-plugin.pod create mode 100644 plugins/golang/Makefile.am create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/wrappers.h create mode 100644 plugins/golang/config-test.go create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/nbdkit.go create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/utils.go create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/wrappers.go create mode 100755 plugins/golang/test/run-test.sh create mode 100644 plugins/golang/test/test.go diff --git a/plugins/golang/nbdkit-golang-plugin.pod b/plugins/golang/nbdkit-golang-plugin.pod new file mode 100644 index 00000000..b449a830 --- /dev/null +++ b/plugins/golang/nbdkit-golang-plugin.pod @@ -0,0 +1,34 @@ +=head1 NAME + +nbdkit-golang-plugin - writing nbdkit plugins in Go + +=head1 SYNOPSIS + + nbdkit /path/to/plugin.so [arguments...] + +=head1 DESCRIPTION + +This manual page describes how to write nbdkit plugins in compiled +Golang code. Go plugins are compiled to F<*.so> files (the same as +plugins written in C) and are used in the same way. + +XXX MORE DOCS + + + +=head1 VERSION + +Golang plugins first appeared in nbdkit 1.20. + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-plugin(3)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright (C) 2020 Red Hat Inc. diff --git a/configure.ac b/configure.ac index c1aec8fa..7fbf7abe 100644 --- a/configure.ac +++ b/configure.ac @@ -49,6 +49,7 @@ LT_INIT dnl List of plugins and filters. lang_plugins="\ + golang \ lua \ ocaml \ perl \ @@ -754,6 +755,34 @@ AS_IF([test "x$enable_lua" != "xno"],[ ]) AM_CONDITIONAL([HAVE_LUA],[test "x$enable_lua" = "xyes"]) +dnl Check for golang. +AC_ARG_ENABLE([golang], + AS_HELP_STRING([--disable-golang], [disable Go language plugin]), + [], + [enable_golang=yes]) +AS_IF([test "x$enable_golang" != "xno"],[ + AC_CHECK_PROG([GOLANG],[go],[go],[no]) + AS_IF([test "x$GOLANG" != "xno"],[ + AC_MSG_CHECKING([if $GOLANG is usable]) + AS_IF([$GOLANG run $srcdir/plugins/golang/config-test.go 2>&AS_MESSAGE_LOG_FD],[ + AC_MSG_RESULT([yes]) + + # Substitute some golang environment. + GOOS=`$GOLANG env GOOS` + GOARCH=`$GOLANG env GOARCH` + GOROOT=`$GOLANG env GOROOT` + AC_SUBST([GOOS]) + AC_SUBST([GOARCH]) + AC_SUBST([GOROOT]) + ],[ + AC_MSG_RESULT([no]) + AC_MSG_WARN([golang ($GOLANG) is installed but not usable]) + GOLANG=no + ]) + ]) +],[GOLANG=no]) +AM_CONDITIONAL([HAVE_GOLANG],[test "x$GOLANG" != "xno"]) + dnl Check for curl (only if you want to compile the curl plugin). AC_ARG_WITH([curl], [AS_HELP_STRING([--without-curl], @@ -1002,6 +1031,7 @@ AC_CONFIG_FILES([Makefile plugins/file/Makefile plugins/floppy/Makefile plugins/full/Makefile + plugins/golang/Makefile plugins/guestfs/Makefile plugins/gzip/Makefile plugins/info/Makefile @@ -1125,6 +1155,8 @@ feature "vddk ................................... " \ echo echo "Languages:" echo +feature "go ..................................... " \ + test "x$HAVE_GOLANG_TRUE" = "x" feature "lua .................................... " \ test "x$HAVE_LUA_TRUE" = "x" feature "ocaml .................................. " \ diff --git a/plugins/golang/Makefile.am b/plugins/golang/Makefile.am new file mode 100644 index 00000000..4f69f03f --- /dev/null +++ b/plugins/golang/Makefile.am @@ -0,0 +1,80 @@ +# nbdkit +# Copyright (C) 2020 Red Hat Inc. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +EXTRA_DIST = \ + nbdkit-golang-plugin.pod \ + src/libguestfs.org/nbdkit/nbdkit.go \ + src/libguestfs.org/nbdkit/utils.go \ + src/libguestfs.org/nbdkit/wrappers.go \ + src/libguestfs.org/nbdkit/wrappers.h \ + test/run-test.sh \ + test/test.go \ + $(NULL) + +if HAVE_GOLANG + +# There is nothing to build. Everything is statically compiled and +# linked together when we compile the test. + +TESTS = test/run-test.sh +check_DATA = test/nbdkit-gotest-plugin.so + +test/nbdkit-gotest-plugin.so: \ + src/libguestfs.org/nbdkit/nbdkit.go \ + src/libguestfs.org/nbdkit/utils.go \ + src/libguestfs.org/nbdkit/wrappers.go \ + src/libguestfs.org/nbdkit/wrappers.h \ + test/test.go + cd test && \ + GOPATH="$(abs_builddir)" \ + $(GOLANG) build \ + -o nbdkit-gotest-plugin.so -buildmode=c-shared + +CLEANFILES += \ + test/nbdkit-gotest-plugin.h \ + test/nbdkit-gotest-plugin.so \ + $(NULL) + +if HAVE_POD + +man_MANS = nbdkit-golang-plugin.3 +CLEANFILES += $(man_MANS) + +nbdkit-golang-plugin.3: nbdkit-golang-plugin.pod + $(PODWRAPPER) --section=3 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD + +endif HAVE_GOLANG diff --git a/plugins/golang/src/libguestfs.org/nbdkit/wrappers.h b/plugins/golang/src/libguestfs.org/nbdkit/wrappers.h new file mode 100644 index 00000000..efa92623 --- /dev/null +++ b/plugins/golang/src/libguestfs.org/nbdkit/wrappers.h @@ -0,0 +1,41 @@ +/* cgo wrappers. + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +extern void wrapper_load (); +extern void wrapper_unload (); +extern int wrapper_config (const char *key, const char *value); +extern int wrapper_config_complete (void); +extern void * wrapper_open (int readonly); +extern void wrapper_close (void *handle); +extern int64_t wrapper_get_size (void *handle); +extern int wrapper_pread (void *handle, void *buf, + uint32_t count, uint64_t offset, uint32_t flags); diff --git a/plugins/golang/config-test.go b/plugins/golang/config-test.go new file mode 100644 index 00000000..0f5cfe6b --- /dev/null +++ b/plugins/golang/config-test.go @@ -0,0 +1,38 @@ +/* Go configuration test + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* Used by ./configure to check golang is functional. */ + +package main + +func main() { +} diff --git a/plugins/golang/src/libguestfs.org/nbdkit/nbdkit.go b/plugins/golang/src/libguestfs.org/nbdkit/nbdkit.go new file mode 100644 index 00000000..cddf64a5 --- /dev/null +++ b/plugins/golang/src/libguestfs.org/nbdkit/nbdkit.go @@ -0,0 +1,270 @@ +/* Go helper functions. + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +package nbdkit + +/* +#cgo pkg-config: nbdkit +#cgo LDFLAGS: -Wl,--unresolved-symbols=ignore-in-object-files + +#include <stdio.h> +#include <stdlib.h> + +#define NBDKIT_API_VERSION 2 +#include <nbdkit-plugin.h> +#include "wrappers.h" +*/ +import "C" + +import ( + "fmt" + "syscall" + "unsafe" +) + +// The plugin may raise errors by returning this struct (instead of nil). +type PluginError struct { + Errmsg string // string (passed to nbdkit_error) + Errno syscall.Errno // errno (optional, use 0 if not available) +} + +func (e PluginError) String() string { + if e.Errno != 0 { + return e.Errmsg + } else { + return fmt.Sprintf("%s (errno %d)", e.Errmsg, e.Errno) + } +} + +func (e PluginError) Error() string { + return e.String() +} + +// The plugin interface. +type PluginInterface interface { + // Open is required for all plugins. + // Other methods are optional. + Load() + Unload() + Config(key string, value string) error + ConfigComplete() error + Open(readonly bool) (ConnectionInterface, error) +} + +// The client connection interface. +type ConnectionInterface interface { + // GetSize and PRead are required for all plugins. + // Other methods are optional. + Close() + GetSize() (uint64, error) + PRead(buf []byte, offset uint64, flags uint32) error +} + +// Default implementations for plugin interface methods. +type Plugin struct{} +type Connection struct{} + +func (p Plugin) Load() { +} + +func (p Plugin) Unload() { +} + +func (p Plugin) Config(key string, value string) error { + return nil +} + +func (p Plugin) ConfigComplete() error { + return nil +} + +func (p Plugin) Open(readonly bool) (ConnectionInterface, error) { + panic("plugin must implement Open()") +} + +func (c Connection) Close() { +} + +func (c Connection) GetSize() (uint64, error) { + panic("plugin must implement GetSize()") +} + +func (c Connection) PRead(buf []byte, offset uint64, flags uint32) error { + panic("plugin must implement PRead()") +} + +// The implementation of the user plugin. +var pluginImpl PluginInterface +var nextConnectionId uintptr +var connectionMap map[uintptr]ConnectionInterface + +// Callbacks from the server. These translate C to Go and back. + +func set_error(err error) { + perr, ok := err.(PluginError) + if ok { + if perr.Errno != 0 { + SetError(perr.Errno) + } + Error(perr.Errmsg) + } else { + Error(err.Error()) + } +} + +//export implLoad +func implLoad() { + pluginImpl.Load() +} + +//export implUnload +func implUnload() { + pluginImpl.Unload() +} + +//export implConfig +func implConfig(key *C.char, value *C.char) C.int { + err := pluginImpl.Config(C.GoString(key), C.GoString(value)) + if err != nil { + set_error(err) + return -1 + } + return 0 +} + +//export implConfigComplete +func implConfigComplete() C.int { + err := pluginImpl.ConfigComplete() + if err != nil { + set_error(err) + return -1 + } + return 0 +} + +//export implOpen +func implOpen(c_readonly C.int) unsafe.Pointer { + readonly := false + if c_readonly != 0 { + readonly = true + } + h, err := pluginImpl.Open(readonly) + if err != nil { + set_error(err) + return nil + } + id := nextConnectionId + nextConnectionId++ + connectionMap[id] = h + return unsafe.Pointer(id) +} + +func getConn(handle unsafe.Pointer) ConnectionInterface { + id := uintptr(handle) + h, ok := connectionMap[id] + if !ok { + panic(fmt.Sprintf("connection %d was not open", id)) + } + return h +} + +//export implClose +func implClose(handle unsafe.Pointer) { + h := getConn(handle) + h.Close() + id := uintptr(handle) + delete(connectionMap, id) +} + +//export implGetSize +func implGetSize(handle unsafe.Pointer) C.int64_t { + h := getConn(handle) + size, err := h.GetSize() + if err != nil { + set_error(err) + return -1 + } + return C.int64_t(size) +} + +//export implPRead +func implPRead(handle unsafe.Pointer, buf unsafe.Pointer, + count C.uint32_t, offset C.uint64_t, flags C.uint32_t) C.int { + h := getConn(handle) + err := h.PRead(C.GoBytes(buf, C.int(count)), + uint64(offset), uint32(flags)) + if err != nil { + set_error(err) + return -1 + } + return 0 +} + +// Called from C plugin_init function. +func PluginInitialize(name string, impl PluginInterface) unsafe.Pointer { + // Initialize the connection map. Note that connection IDs + // must start counting from 1 since we must never return what + // looks like a NULL pointer to the C code. + connectionMap = make(map[uintptr]ConnectionInterface) + nextConnectionId = 1 + + pluginImpl = impl + + plugin := C.struct_nbdkit_plugin{} + + // Set up the hidden plugin fields as for C. + struct_size := C.ulong(unsafe.Sizeof(plugin)) + plugin._struct_size = struct_size + plugin._api_version = C.NBDKIT_API_VERSION + plugin._thread_model = C.NBDKIT_THREAD_MODEL_PARALLEL + + // Set up the other fields. + plugin.name = C.CString(name) + plugin.load = (*[0]byte)(C.wrapper_load) + plugin.unload = (*[0]byte)(C.wrapper_unload) + plugin.config = (*[0]byte)(C.wrapper_config) + plugin.config_complete = (*[0]byte)(C.wrapper_config_complete) + plugin.open = (*[0]byte)(C.wrapper_open) + plugin.close = (*[0]byte)(C.wrapper_close) + plugin.get_size = (*[0]byte)(C.wrapper_get_size) + plugin.pread = (*[0]byte)(C.wrapper_pread) + + // Golang plugins don't preserve errno correctly. + plugin.errno_is_preserved = 0 + + // Return a newly malloced copy of the struct. This must be + // globally available to the C code in the server, so it is + // never freed. + p := (*C.struct_nbdkit_plugin)(C.malloc(struct_size)) + *p = plugin + return unsafe.Pointer(p) +} diff --git a/plugins/golang/src/libguestfs.org/nbdkit/utils.go b/plugins/golang/src/libguestfs.org/nbdkit/utils.go new file mode 100644 index 00000000..d9c0c188 --- /dev/null +++ b/plugins/golang/src/libguestfs.org/nbdkit/utils.go @@ -0,0 +1,75 @@ +/* cgo wrappers. + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +package nbdkit + +/* +#cgo pkg-config: nbdkit + +#define NBDKIT_API_VERSION 2 +#include <nbdkit-plugin.h> + +// cgo cannot call varargs functions. +void +_nbdkit_debug (const char *s) +{ + nbdkit_debug ("%s", s); +} + +// cgo cannot call varargs functions. +void +_nbdkit_error (const char *s) +{ + nbdkit_error ("%s", s); +} +*/ +import "C" +import "syscall" + +// Utility functions. + +func Debug(s string) { + C._nbdkit_debug(C.CString(s)) +} + +// This function is provided but plugins would rarely need to call +// this explicitly since returning an error from a plugin callback +// will call it implicitly. +func Error(s string) { + C._nbdkit_error(C.CString(s)) +} + +// Same applies as for Error(). Callers should not usually need to +// call this. +func SetError(err syscall.Errno) { + C.nbdkit_set_error(C.int(err)) +} diff --git a/plugins/golang/src/libguestfs.org/nbdkit/wrappers.go b/plugins/golang/src/libguestfs.org/nbdkit/wrappers.go new file mode 100644 index 00000000..79b6d329 --- /dev/null +++ b/plugins/golang/src/libguestfs.org/nbdkit/wrappers.go @@ -0,0 +1,102 @@ +/* cgo wrappers. + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +package nbdkit + +/* +#cgo pkg-config: nbdkit + +#include <stdio.h> +#include <stdlib.h> + +#define NBDKIT_API_VERSION 2 +#include <nbdkit-plugin.h> +#include "wrappers.h" + +extern void implLoad (); +void +wrapper_load (void) +{ + implLoad (); +} + +extern void implUnload (); +void +wrapper_unload (void) +{ + implUnload (); +} + +extern int implConfig (); +int +wrapper_config (const char *key, const char *value) +{ + return implConfig (key, value); +} + +extern int implConfigComplete (); +int +wrapper_config_complete (void) +{ + return implConfigComplete (); +} + +extern void *implOpen (); +void * +wrapper_open (int readonly) +{ + return implOpen (readonly); +} + +extern void implClose (); +void +wrapper_close (void *handle) +{ + return implClose (handle); +} + +extern int64_t implGetSize (); +int64_t +wrapper_get_size (void *handle) +{ + return implGetSize (handle); +} + +extern int implPRead (); +int +wrapper_pread (void *handle, void *buf, + uint32_t count, uint64_t offset, uint32_t flags) +{ + return implPRead (handle, buf, count, offset, flags); +} +*/ +import "C" diff --git a/plugins/golang/test/run-test.sh b/plugins/golang/test/run-test.sh new file mode 100755 index 00000000..f4da139e --- /dev/null +++ b/plugins/golang/test/run-test.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# nbdkit +# Copyright (C) 2020 Red Hat Inc. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +set -e +set -x + +if ! qemu-img --version; then + echo "qemu-img is required to run this test" + exit 1 +fi + +../../nbdkit -f -v test/nbdkit-gotest-plugin.so size=$((1024 * 1024)) \ + --run 'qemu-img info $nbd' diff --git a/plugins/golang/test/test.go b/plugins/golang/test/test.go new file mode 100644 index 00000000..7186ffa8 --- /dev/null +++ b/plugins/golang/test/test.go @@ -0,0 +1,119 @@ +/* Test plugin. + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +package main + +import ( + "C" + "libguestfs.org/nbdkit" + "strconv" + "unsafe" +) + +var pluginName = "test" + +type TestPlugin struct { + nbdkit.Plugin +} + +type TestConnection struct { + nbdkit.Connection +} + +var size uint64 +var size_set = false + +func (p TestPlugin) Load() { + nbdkit.Debug("golang code running in the .load callback") +} + +func (p TestPlugin) Unload() { + nbdkit.Debug("golang code running in the .unload callback") +} + +func (p TestPlugin) Config(key string, value string) error { + if key == "size" { + var err error + size, err = strconv.ParseUint(value, 0, 64) + if err != nil { + return err + } + size_set = true + return nil + } else { + return nbdkit.PluginError{Errmsg: "unknown parameter"} + } +} + +func (p TestPlugin) ConfigComplete() error { + if !size_set { + return nbdkit.PluginError{Errmsg: "size parameter is required"} + } + return nil +} + +func (p TestPlugin) Open(readonly bool) (nbdkit.ConnectionInterface, error) { + nbdkit.Debug("golang code running in the .open callback") + return &TestConnection{}, nil +} + +func (c TestConnection) GetSize() (uint64, error) { + nbdkit.Debug("golang code running in the .get_size callback") + return size, nil +} + +func (c TestConnection) PRead(buf []byte, offset uint64, flags uint32) error { + nbdkit.Debug("golang code running in the .pread callback") + for i := 0; i < len(buf); i++ { + buf[i] = 0 + } + return nil +} + +//---------------------------------------------------------------------- +// +// The boilerplate below this line is required by all golang plugins, +// as well as importing "C" and "unsafe" modules at the top of the +// file. + +//export plugin_init +func plugin_init() unsafe.Pointer { + // If your plugin needs to do any initialization, you can + // either put it here or implement a Load() method. + // ... + + // Then you must call the following function. + return nbdkit.PluginInitialize(pluginName, &TestPlugin{}) +} + +// This is never(?) called, but must exist. +func main() {} diff --git a/.gitignore b/.gitignore index 11cd976b..f681f526 100644 --- a/.gitignore +++ b/.gitignore @@ -72,6 +72,8 @@ plugins/*/*.3 /plugins/example4/nbdkit-example4-plugin /plugins/eval/call.c /plugins/eval/methods.c +/plugins/golang/test/nbdkit-gotest-plugin.h +/plugins/golang/test/nbdkit-gotest-plugin.so /plugins/ocaml/nbdkit-ocamlexample-plugin.so /plugins/rust/Cargo.lock /plugins/rust/Cargo.toml diff --git a/README b/README index a33c5693..62fc93e9 100644 --- a/README +++ b/README @@ -148,6 +148,10 @@ For the Rust plugin: - cargo (other dependencies will be downloaded at build time) +To be able to write plugins in golang: + + - go >= 1.5 + For bash tab completion: - bash-completion >= 1.99 -- 2.25.0 --qKOY9RTXA4DvnaVz--
Daniel P. Berrangé
2020-Apr-23 13:49 UTC
Re: [Libguestfs] [PATCH nbdkit v2] Add the ability to write plugins in golang.
On Thu, Apr 23, 2020 at 10:03:45AM +0100, Richard W.M. Jones wrote:> On Tue, Apr 21, 2020 at 05:07:43PM +0100, Daniel P. Berrangé wrote: > > On Tue, Apr 21, 2020 at 11:44:59AM +0100, Richard W.M. Jones wrote: > > > Thanks: Dan Berrangé > > > > > > XXX UNFINISHED: > > > > > > - Is using uintptr for the handle a good idea? Plugins must return > > > something != 0. In other languages we would allow plugins to > > > return an arbitrary object here, but this is not possible in golang > > > because of lack of GC roots. > > > > Yeah, this is the bit that looks wierd to me when thinking about this > > from a Go dev POV. You asked me on IRC whether we should have a separate > > interface for a connection object, and this makes me think that we should > > indeed do that. > ... > > This is what the patch looks like with a separate connection object. > It appears to work. > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > virt-df lists disk usage of guests without needing to install any > software inside the virtual machine. Supports Linux and Windows. > http://people.redhat.com/~rjones/virt-df/> From a929cff8745f21df913ea3230741dc7f2a1c016e Mon Sep 17 00:00:00 2001 > From: "Richard W.M. Jones" <rjones@redhat.com> > Date: Thu, 9 Apr 2020 12:45:10 +0100 > Subject: [PATCH] Add the ability to write plugins in golang. > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > Thanks: Dan Berrangé > > XXX UNFINISHED: > > - Default can_* methods are hard to implement. Ideally we would be > able to test if a user plugin implements a particular callback > (eg. TestPlugin.PWrite) or else defaults to the default callback, > but even with reflection I don't think this is possible in golang. > [Discussed on IRC: We decided the only option was to go for > nbdkit-sh-plugin style can_* methods] > > - Write wrappers etc for all the other methods. > > - Write documentation. > --- > plugins/golang/nbdkit-golang-plugin.pod | 34 +++ > configure.ac | 32 +++ > plugins/golang/Makefile.am | 80 ++++++ > .../src/libguestfs.org/nbdkit/wrappers.h | 41 +++ > plugins/golang/config-test.go | 38 +++ > .../src/libguestfs.org/nbdkit/nbdkit.go | 270 ++++++++++++++++++ > .../golang/src/libguestfs.org/nbdkit/utils.go | 75 +++++ > .../src/libguestfs.org/nbdkit/wrappers.go | 102 +++++++ > plugins/golang/test/run-test.sh | 42 +++ > plugins/golang/test/test.go | 119 ++++++++ > .gitignore | 2 + > README | 4 + > 12 files changed, 839 insertions(+) > create mode 100644 plugins/golang/nbdkit-golang-plugin.pod > create mode 100644 plugins/golang/Makefile.am > create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/wrappers.h > create mode 100644 plugins/golang/config-test.go > create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/nbdkit.go > create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/utils.go > create mode 100644 plugins/golang/src/libguestfs.org/nbdkit/wrappers.go > create mode 100755 plugins/golang/test/run-test.sh > create mode 100644 plugins/golang/test/test.goI think this design with two interfaces looks pretty nice from the Go plugin implementors POV now. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Richard W.M. Jones
2020-Apr-23 13:51 UTC
Re: [Libguestfs] [PATCH nbdkit v2] Add the ability to write plugins in golang.
On Thu, Apr 23, 2020 at 02:49:25PM +0100, Daniel P. Berrangé wrote:> I think this design with two interfaces looks pretty nice from the > Go plugin implementors POV now.Thanks. While implementing PWrite I discovered that C.GoBytes makes a copy (the docs were unclear on this), so it wasn't actually reading and writing from the server at all :-( All fixed now after reading this very long issue: https://github.com/golang/go/issues/13656 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Seemingly Similar Threads
- Re: [PATCH nbdkit v2] Add the ability to write plugins in golang.
- Re: [PATCH nbdkit v2] Add the ability to write plugins in golang.
- [PATCH nbdkit v2] Add the ability to write plugins in golang.
- Re: [PATCH nbdkit UNFINISHED] Add the ability to write plugins in golang.
- [PATCH nbdkit] golang: Pass Plugin and Connection by reference not value.