Libguestfs - May 2021 - [libnbd PATCH 0/5] Introduce automated testing using GitLab CI

On Thu, May 06, 2021 at 01:30:05PM +0200, Martin Kletzander
wrote:
> This is reworked version of
> 
>   https://listman.redhat.com/archives/libguestfs/2021-March/msg00134.html
> 
> with more packages installed so that more code paths are tested, also some
fixes
> in the code (mostly requirements).
> 
> There are still some things that fail.
> 
> FreeBSD
> ======> 
> The build fails because there is no fallocate() there.
I'm actually not seeing a failure (FreeBSD 12.2).  Do you happen to
know what the build error was?

I'm seeing test failures though which I'll fix if they are easy.  I
think for non-Linux right now I'm most concerned about build failures.
> Many platforms (those that have nbd-server in repositories)
> ==========================================================> 
> interop/interop-nbd-server fails with:
> 
>   Error: inetd mode requires syslog
>   Exiting.
>
> which makes sense as it is run with port 0, but how to check for
> whether it is compiled with or without syslog (so that we know to
> skip the test) or whether it is something else that is missing
> (capabilities in a container) is beyond me.
I wasn't able to reproduce this (on FreeBSD).  Do you know which
operating systems have nbd-server compiled without syslog?
> interop/list-exports-nbd-server gets stuck and in the CI times out the
whole
> test suite.  Locally I need to manually exec into the container:
>
>   podman exec -itu bash
>
> and I see the nbd-server is just not doing anything.  After killing
> it the test ends and fails.  I did not debug nbd-server for this,
> maybe it is again some requisite that is missing and we are not
> checking for it (capability, nbd module access, etc.), I do not
> know.
FreeBSD seems OK here.  Do you know which OSes this affected?
> There are also some skips that I did not go into any more after
> trying to figure out every "... no" in config.log:
I wouldn't worry too much about these in terms of getting things
working.  We can always extend the CI later.
> Both interop-qemu-nbd-tls-certs and interop-qemu-nbd-tls-psk get
> skipped because they are missing the keys, but I presume those are
> supposed to be generated.
These require "certtool" (part of gnutls).
> I think that all these can be figured out after the final patch is
> merged so that the CI runs truly automatically.  I am posting it for
> review though as I am not sure the solutions in first four patches
> are the preferred ones.
I agree.

Thanks!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On Tue, May 11, 2021 at 12:16:19PM +0100, Richard W.M. Jones
wrote:
> On Thu, May 06, 2021 at 01:30:05PM +0200, Martin Kletzander wrote:
> > FreeBSD
> > ======> > 
> > The build fails because there is no fallocate() there.
> 
> I'm actually not seeing a failure (FreeBSD 12.2).  Do you happen to
> know what the build error was?
> 
> I'm seeing test failures though which I'll fix if they are easy.  I
> think for non-Linux right now I'm most concerned about build failures.
Another problem with the libnbd tests is that they use the installed
nbdkit, but -- presumably for political rather than technical reasons --
nbdkit on FreeBSD is built without GnuTLS support.  While I could add
'requires' lines everywhere to test this, it may be better (on
FreeBSD) to either not test against nbdkit at all or to test against
the latest nbdkit build.  Could CI tests allow this?  Don't worry
about implementing this, I just want to know what's possible.

The situation for testing nbdkit is similar but reversed.  FreeBSD
does not package libnbd at all so far, but I'm quite sure they'll also
build a TLS-less version if they get around to it.

Someone requested a version of libnbd that supported OpenSSL as an
alternative crypto backend recently.  I wonder if they were a *BSD
developer?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

On Tue, May 11, 2021 at 12:16:19PM +0100, Richard W.M. Jones
wrote:
>On Thu, May 06, 2021 at 01:30:05PM +0200, Martin Kletzander wrote:
>> This is reworked version of
>>
>>  
https://listman.redhat.com/archives/libguestfs/2021-March/msg00134.html
>>
>> with more packages installed so that more code paths are tested, also
some fixes
>> in the code (mostly requirements).
>>
>> There are still some things that fail.
>>
>> FreeBSD
>> ======>>
>> The build fails because there is no fallocate() there.
>
>I'm actually not seeing a failure (FreeBSD 12.2).  Do you happen to
>know what the build error was?
>
The error is visible here:

   https://gitlab.com/nertpinx/libnbd/-/jobs/1237433032#L2185

is it possible that I did not include some dependency in the system?  Or
maybe it is compiling a file that is not compiled on your setup?  I
could not find anything fallocate() related for FreeBSD, only
posix_fallocate().
>I'm seeing test failures though which I'll fix if they are easy.  I
>think for non-Linux right now I'm most concerned about build failures.
>
>> Many platforms (those that have nbd-server in repositories)
>> ==========================================================>>
>> interop/interop-nbd-server fails with:
>>
>>   Error: inetd mode requires syslog
>>   Exiting.
>>
>> which makes sense as it is run with port 0, but how to check for
>> whether it is compiled with or without syslog (so that we know to
>> skip the test) or whether it is something else that is missing
>> (capabilities in a container) is beyond me.
>
>I wasn't able to reproduce this (on FreeBSD).  Do you know which
>operating systems have nbd-server compiled without syslog?
>
This was some Linux distro and I do not know if it actually is "build
without syslog", it only looked like it based on the nbd-server code
that I went through.  But it might be some missing capability or
anything else that is actually affected by the fact that the builds and
tests are running in containers.

I might have mixed this one and the below.  You can try it on Ubuntu
18.04, but you would have to try it locally, you cannot see it in the CI
because it times out and it requires manual intervention.  Basically see
below.
>> interop/list-exports-nbd-server gets stuck and in the CI times out the
whole
>> test suite.  Locally I need to manually exec into the container:
>>
>>   podman exec -itu bash
>>
>> and I see the nbd-server is just not doing anything.  After killing
>> it the test ends and fails.  I did not debug nbd-server for this,
>> maybe it is again some requisite that is missing and we are not
>> checking for it (capability, nbd module access, etc.), I do not
>> know.
>
>FreeBSD seems OK here.  Do you know which OSes this affected?
>
I think Ubuntu 18.04, see above.
>> There are also some skips that I did not go into any more after
>> trying to figure out every "... no" in config.log:
>
>I wouldn't worry too much about these in terms of getting things
>working.  We can always extend the CI later.
>
>> Both interop-qemu-nbd-tls-certs and interop-qemu-nbd-tls-psk get
>> skipped because they are missing the keys, but I presume those are
>> supposed to be generated.
>
>These require "certtool" (part of gnutls).
>
I made sure I installed gnutls-bin (or whatever gnutls package provides
certtool), but I might've missed that certool is in another package on
some particular distro.
>> I think that all these can be figured out after the final patch is
>> merged so that the CI runs truly automatically.  I am posting it for
>> review though as I am not sure the solutions in first four patches
>> are the preferred ones.
>
>I agree.
>
>Thanks!
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
>Read my programming and virtualization blog: http://rwmj.wordpress.com
>libguestfs lets you edit virtual machines.  Supports shell scripting,
>bindings from many languages.  http://libguestfs.org
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210512/b7be980b/attachment.sig>