Libguestfs - Apr 2021 - [libnbd RFC PATCH] First stab at CI infrastructure

On Tue, Mar 30, 2021 at 06:12:06PM +0200, Martin Kletzander
wrote:
>This is a first try for adding a CI to libnbd.  It uses the libvirt-ci to
>get as much coverage as possible with the ease of use provided by that
>repository.  Not all the data are available there at the time of posting
>this patch, so if anyone wants to recreate the Containerfiles and
>variable files (for cirrus CI) my temporary branch of libvirt-ci called
>nbd_prep:
>
>  https://gitlab.com/nertpinx/libvirt-ci/-/tree/nbd_prep
>
>The result of this branch CI run is available here:
>
>  https://gitlab.com/nertpinx/libnbd/-/pipelines
>
>As you can see there are errors.  I went down a rabbit hole of trying to
>figure out one of them, but ended up not being sure what the preferred
>way of fixing that particular issue would be.  So instead of trying
>myself and raising various questions every single day I am posting this
>here as handling it myself would take too much time and I would be
>bothering other people throughout days and days going forward.
>
>If there are any questions related to how the CI is running, how it
>works, how to replicate CI builds locally or how to change anything,
>then I am more than happy to help.
>
>Actually recreating the builds locally (at least for Linux distributions
>and setups) is pretty straightforward. Choose a file from ci/containers
>which represents the desired setup, for our example let's pick fedora
>rawhide, and build your container and tag it, e.g. using podman (or feel
>free to substitute "podman" with "docker"):
>
>  podman build ci/containers/fedora-33.Dockerfile -t libnbd-fedora-rawhide
>
>That will get you a container tagged `libnbd-fedora-rawhide` that you
>can execute the tests on.  You can then run whatever you want inside
>that container with the current repository passed through like this:
>
>  podman run -it --rm -v .:/repo -w /repo libnbd-fedora-rawhide bash
>
>which will bind-mount the current directory onto /repo inside the
>container and also use that path as the working directory (just so you
>do not have to `cd /repo` before any commands.  I prefer running bash,
>but of course you can just run the build script used in the CI.  I have
>put all the commands into one file for simplicity, so that you can
>simply specifically `ci/build_script.sh`.  So simply executing that
>script will give you the results and you can experiment right inside
>that environment to figure out what is needed.  At the same time you can
>easily modify any files inside that repository on your host, just like
>you are used to, so that you can use your editor and other setups that
>work for you.
>
>Last few things to note:
>
>- You should make sure that build files do not interfere between the
>  host and the container, if you want to replicate a clean build you
>  need to either use VPATH or just clean everything.
>
>- No tests include running make distcheck as that seems a bit more
>  broken and could be fixed after more pressing issues are dealt with,
>  just so the output does not interfere in the meantime.
>
>Let me know what you think, and have a nice day.
Polite ping.  Any thoughts?  Anything you'd like to change?

Thanks,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210407/6d2a931c/attachment.sig>

On Wed, Apr 07, 2021 at 04:55:17PM +0200, Martin Kletzander
wrote:
> Polite ping.  Any thoughts?  Anything you'd like to change?
I didn't particularly understand any of it; but it's what we need so
looks good!

One bug: All the new files that have been added probably need to be
added to the top-level Makefile.am EXTRA_DIST, certainly if they would
be in any way useful in the tarball.  However if they are not useful
in the tarball then the target "maintainer-check-extra-dist" (also in
Makefile.am) must be modified to ignore these files when I do
"make && make dist && make
maintainer-check-extra-dist".

I also looked at the results:

  https://gitlab.com/nertpinx/libnbd/-/pipelines?scope=all&page=1

and choosing the top "failed" link arbitrarily takes me to:

  https://gitlab.com/nertpinx/libnbd/-/pipelines/278989630

where there are some failures.  We don't support mingw* platforms for
libnbd (we do for nbdkit), so for libnbd those platforms can
immediately be dropped.

But there are some other non-mingw failures, again picking one
at random:

  https://gitlab.com/nertpinx/libnbd/-/jobs/1140619393

the failure is:

  go install: version is required when current directory is not in a module

which is (or was) a real bug!  I've since fixed it upstream, but it
seems to show the thing works.

As far as I know libnbd should currently build on Rawhide, so could
you pull in the updates, drop mingw*, and see if the tests start to
pass?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW