Libguestfs - Nov 2021 - [PATCH libnbd 0/2] Publish the Go binding via a proxy server

Consuming the Go binding now requires downloading the tarball, and
copying the source to your project. This is not the standard way that Go
modules should be consumed.

We can solve this issue in several ways:

- Publish the Go module from the libnbd repo
- Publish the Go module from a new distribution repo
- Publish the Go module via a proxy server

The first option requires that we keep the generated sources in the
repo, and change the package name to it can be imported from the gitlab
subdirectory:

    import "gitlab.com/nbdkit/libnbd/-/tree/master/golang"

Since the Go tools do not support yet a module in a sub-directory of a
git repo:
https://github.com/golang/go/issues/34055

I think this is horrible.

The second option requires that we push the Go binding source to a new
repo:

   https://gitlab.com/nbdkit/libnbd-golang

I think this may confuse users and will be harder to maintain.

The third option seems most suitable for libnbd use case - serve the Go
module from the same server serving the tarballs. This requires creating
a module zip file, and uploading it to a special directory structure on
the web server, with some metadata files.

For the second and third options, we can keep the name of the module as
"libguestfs.org/libnb" by serving html document with the required meta
tag at:

    https://libguestfs.org/libnbd

This series implement the third option by adding a script to create the
distribution tree for the web server.

Nir Soffer (2):
  golang: Create distribution for a proxy server
  golang/README.md: Remove the hacks

 golang/README.md    |  31 -----------
 golang/make-dist.sh | 123 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 123 insertions(+), 31 deletions(-)
 create mode 100755 golang/make-dist.sh

-- 
2.31.1

Add make-dist.sh script, creating a distribution tree for a proxy
server[1]. The created tree must be served by the web server serving the
libnbd module, for example:

    https://download.libguestfs.org/libnbd-golang

The Go tools will find this URL by looking at:

    https://libguestfs.org/linbd

Which serve html document with the "go-import" meta tag, pointing to
the server serving the module.

When the module is available, we need to register the package here:
https://pkg.go.dev/libguestfs.org/libnbd

[1] https://golang.org/ref/mod#serving-from-proxy

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 golang/make-dist.sh | 123 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100755 golang/make-dist.sh

diff --git a/golang/make-dist.sh b/golang/make-dist.sh
new file mode 100755
index 0000000..634e306
--- /dev/null
+++ b/golang/make-dist.sh
@@ -0,0 +1,123 @@
+#!/bin/sh -e
+# nbd client library in userspace
+# Copyright (C) 2021 Red Hat Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+# Create a distribution tree for serving the libnbd Go module.
+#
+# The web server must serve tree:
+#
+# libguestfs.org
+# ??? libnbd
+#    ??? @latest
+#    ??? @v
+#        ??? list
+#        ??? v1.11.4.info
+#        ??? v1.11.4.mod
+#        ??? v1.11.4.zip
+#
+# For example:
+#
+#   https://download.libguestfs.org/libnbd-golang/libguestfs.org/libnbd/@v/list
+#
+# To test the web server use:
+#
+#   GOPROXY=https://download.libguestfs.org go get libguestfs.org/libnbd
+#
+# This should download the module and add it to the local cache, by
+# default in:
+#
+#   ~/go/pkg/mod/libguestfs.org/libnbd at v1.11.4/
+#
+# The Go tools will find the web server by looking at:
+#
+#   https://libguestfs.org/libnbd
+#
+# Which must serve html document with this meta tag:
+#
+#   <meta name="go-import" content="libguestfs.org/libnbd mod
https://download.libguestfs.org/libnbd-golang">
+#
+# With this, users can consume the libnbd Go bindings using:
+#
+#   go get libguestfs.org/libnbd
+#
+
+version=$(git describe)
+
+# Create module zip file
+#
+# libguestfs.org
+# ??? libnbd at v1.11.4
+#    ??? go.mod
+#    ??? handle.go
+#    ...
+#
+# See https://golang.org/ref/mod#zip-files
+
+version_dir="libguestfs.org/libnbd at v$version"
+
+echo "## Creating module zip file: $version.zip"
+
+mkdir -p $version_dir
+
+tar cf - \
+    --exclude examples \
+    --exclude configure \
+    --exclude "*_test.go" \
+    go.mod README.md LICENSE *.go *.h \
+    | tar xvf - -C $version_dir
+
+zip -rv $version.zip $version_dir/*
+
+rm -rf libguestfs.org
+
+# Create web server tree.
+#
+# libguestfs.org
+# ??? libnbd
+#    ??? @latest
+#    ??? @v
+#        ??? list
+#        ??? v1.11.4.info
+#        ??? v1.11.4.mod
+#        ??? v1.11.4.zip
+#
+# See https://golang.org/ref/mod#serving-from-proxy
+
+echo "## Creating a tarball: libnbd-golang-$version.tar.gz"
+
+module_dir=libguestfs.org/libnbd
+v_dir=$module_dir/@v
+
+mkdir -p $v_dir
+
+echo "{\"Version\": \"$version\"}" >
$module_dir/@latest
+echo "{\"Version\": \"$version\"}" >
$v_dir/$version.info
+
+# This is not entirely corect. This file should have a list of all
+# version avaialable, here we create only single version. This should
+# really be done on the server by appending the new version to the list
+# file.
+echo $version > $v_dir/list
+
+cp go.mod $v_dir/$version.mod
+mv $version.zip $v_dir
+
+# Create tarball to upload and extract on the webserver. It should be
+# extracted in the directory pointed by the "go-import" meta tag.
+tar cvzf libnbd-golang-$version.tar.gz libguestfs.org
+
+rm -rf libguestfs.org
-- 
2.31.1

The module should be usable via Go tools now.

Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
 golang/README.md | 31 -------------------------------
 1 file changed, 31 deletions(-)

diff --git a/golang/README.md b/golang/README.md
index aeadcf6..2eb0006 100644
--- a/golang/README.md
+++ b/golang/README.md
@@ -9,37 +9,6 @@ https://libguestfs.org/libnbd-golang.3.html
 This is part of libnbd, please check the project at:
 https://gitlab.com/nbdkit/libnbd
 
-## How consume this module
-
-The Go binding cannot be consumed yet using the Go tools, but you
-extract it from the tarball.
-
-1. Download a tarball
-
-        wget
https://download.libguestfs.org/libnbd/1.10-stable/libnbd-1.10.1.tar.gz
-
-2. Extract the sources from the golang directory
-
-        mkdir pkg/libnbd
-
-        tar xvf libnbd-1.11.1.tar.gz \
-            --directory pkg/libnbd \
-            --strip 2 \
-            --exclude "*_test.go" \
-            --exclude "examples" \
-            --exclude "configure" \
-            libnbd-1.11.1/golang/{go.mod,README.md,LICENSE,*.go,*.h}
-
-3. Edit your go mode file to use the local copy
-
-        go mod edit -replace libguestfs.org/libnbd=./pkg/libnbd
-
-4. Install the libnbd development package
-
-        dnf install libnbd-devel
-
-   The package may be named differently on your distro.
-
 ## License
 
 The software is copyright ? Red Hat Inc. and licensed under the GNU
-- 
2.31.1

I'll grab you on IRC today and we can try option (3) to see if it works.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top