similar to: [PATCH nbdkit UNFINISHED] Add the ability to write plugins in golang.

On Fri, Apr 10, 2020 at 02:51:52PM +0100, Richard W.M. Jones wrote: > Similar to C, OCaml and Rust, this is not a plugin per se. Instead > it's more of a method and set of tests around writing plugins in > golang. They are standalone programs that compile into shared objects > that nbdkit can then load (so there is no "go plugin" between nbdkit > and the user plugin,

Thanks: Dan Berrangé XXX UNFINISHED: - Is using uintptr for the handle a good idea? Plugins must return something != 0. In other languages we would allow plugins to return an arbitrary object here, but this is not possible in golang because of lack of GC roots. - Default can_* methods are hard to implement. Ideally we would be able to test if a user plugin implements a

Version 1 was here: https://www.redhat.com/archives/libguestfs/2020-April/thread.html#00160 Version 2 side-steps the objections to the first patch by using a well-formed alternate nbdkit.pc file and running ordinary pkg-config against it, so any parsing of --cflags etc will be done by pkg-config. The first patch is essentially the same idea as:

On Tue, Apr 21, 2020 at 05:07:43PM +0100, Daniel P. Berrangé wrote: > On Tue, Apr 21, 2020 at 11:44:59AM +0100, Richard W.M. Jones wrote: > > Thanks: Dan Berrangé > > > > XXX UNFINISHED: > > > > - Is using uintptr for the handle a good idea? Plugins must return > > something != 0. In other languages we would allow plugins to > > return an

Version 2: https://www.redhat.com/archives/libguestfs/2020-April/thread.html#00166 Version 3 contains all changes discussed in the previous review. Rich.

On 4/23/20 2:13 PM, Richard W.M. Jones wrote: > Compiling nbdkit from source when an older nbdkit is installed would > fail because certain symbols such as .get_ready are not defined in the > (installed) <nbdkit-plugin.h>: > > ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready undefined (type _Ctype_struct_nbdkit_plugin has no field or method get_ready) >

Similar to C, OCaml and Rust, this is not a plugin per se. Instead it's more of a method and set of tests around writing plugins in golang. They are standalone programs that compile into shared objects that nbdkit can then load (so there is no "go plugin" between nbdkit and the user plugin, unlike in scripting languages like Perl). --- plugins/golang/nbdkit-golang-plugin.pod

When compiling when an older nbdkit is installed, the build would fail because certain symbols such as .get_ready were not defined: ../../src/libguestfs.org/nbdkit/nbdkit.go:541:8: plugin.get_ready undefined (type _Ctype_struct_nbdkit_plugin has no field or method get_ready) This happens because we were using the installed <nbdkit-plugin.h> rather than the local copy. We don't want

This feature is roughly finished now, although it needs a few more tests and some examples. It's pretty much up to par with all the other bindings, but it lacks a completely safe AIO buffer. It won't stop you from freeing the buffer too early) because golang's GC inexplicably lacks a way to declare a root from C. I can probably do it with a global variable and ref counting on the

Now runs a complete set of tests, notably including the AIO test. File descriptors are passed in and out as plain ints (instead of *os.File) for a couple of reasons: (1) We have to pass the plain int to syscall.Select. (2) Turning an fd into an os.File causes golang to set the blocking flag which is deeply unhelpful. Rich.

[Adding libguestfs mailing list] I reproduced it locally - the difference was installing "gcc-go". I only had golang-bin installed previously. With gcc-go installed: ../run go install libguestfs.org/libnbd write of Go pointer 0xc000016060 to non-Go memory 0x7f5fe8297390 fatal error: Go pointer stored into non-Go memory runtime stack: runtime_mstart

These bindings get as far as running very simple connections. However there are many missing parts still: * No callbacks. * No functions which handle buffers (pread/pwrite!) This is posted just for general early interest, not even for review. Rich.

I took the easy route of crippling what I couldn't get working, on the grounds that partial coverage is better than none now that we have Cirrus CI checking commits on additional platforms. This series got me to a green checkmark: https://gitlab.com/ebblake/nbdkit/-/pipelines/793156983 but depends on an as-yet uncommitted patch in libvirt-ci:

On Wed, Apr 15, 2020 at 06:04:26PM +0100, Richard W.M. Jones wrote: > On Wed, Apr 15, 2020 at 05:27:07PM +0100, Daniel P. Berrangé wrote: > > On Fri, Apr 10, 2020 at 02:51:52PM +0100, Richard W.M. Jones wrote: > > > Similar to C, OCaml and Rust, this is not a plugin per se. Instead > > > it's more of a method and set of tests around writing plugins in > > >

Go 1.17 or newer is required to use unsafe.Slice(), which in turn allows us to write a simpler conversion from a C array to a Go object during callbacks. To check if this makes sense, look at https://repology.org/project/go/versions compared to our list in ci/manifest.yml, at the time I made this commit: Alpine 3.15: 1.17.10 AlmaLinux 8: 1.19.10 CentOS Stream 8: 1.20.4 Debian 10: 1.11.6 Debian

Commit 6725fa0e12 changed copy_uint32_array() to utilize a Go hack for accessing a C array as a Go slice in order to potentially benefit from any optimizations in Go's copy() for bulk transfer of memory over naive one-at-a-time iteration. But that commit also acknowledged that no benchmark timings were performed, which would have been useful to demonstrat an actual benefit for using hack in

[To continue the conversation from https://github.com/libguestfs/nbdkit/issues/5] > Currently there's no way for the Rust plugin to report an error. The > idiomatic way to do it would be for each callback to return a Result > object, much like how the Go plugin currently does it. I'm sure it's not idiomatic for Rust, but I just wanted to say that there is a way to return an

2014-07-12 14:00 GMT+04:00 Richard W.M. Jones <rjones@redhat.com>: > It depends entirely on what the Linux kernel and tools can do. If > there are properly maintained Linux tools for resizing BSD slices, > then it should be no problem to add this. > > Compare with 'ntfsresize', where ntfs-3g exists and is well-maintained > and available in the majority of Linux

On 2/8/19 5:55 AM, Richard W.M. Jones wrote: > --- > +* Consider supporting a more idiomatic style for writing Rust plugins. > + Considering I've written nothing in Rust, I'm no good at deciding what forms idiomatic Rust plugins. > +++ b/configure.ac > @@ -545,6 +545,15 @@ AS_IF([test "x$OCAMLOPT" != "xno" && test "x$enable_ocaml" !=

On Mon, Nov 05, 2018 at 09:04:23PM +0000, Olly Betts wrote: > Dave Courtois wrote: > > Hi there just a little word to tell you that Xapian 1.4.6 work in > > Golang 1.9. There is some issue with the MSet ESet, the namespace part > > of the name is drop by swig. There is also some error with define > > macro in .i files, but with minor change all work perfectly. I had a