Libguestfs - Apr 2021 - [libnbd RFC PATCH] First stab at CI infrastructure

On Thu, Apr 08, 2021 at 05:39:17PM +0200, Martin Kletzander
wrote:
>  - clang failures:
>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504401
I don't usually enable warnings with clang, but I've pushed this which
fixes the problem above (and a further one) for me:

 
https://gitlab.com/nbdkit/libnbd/-/commit/28816a5f8829f474d07351611140f461facd0e4d
>  - golang test failures on OpenSUSE:
>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504416
This is actually because:

  checking for nbdkit... no

but we go ahead and run the golang tests anyway.  Fixed here:

 
https://gitlab.com/nbdkit/libnbd/-/commit/b15dde588f47894f9656ea62eb632156b7638b8b

but there may be more problems after this, we'll see.
>  - one weird ubuntu 18.04 failure:
>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504418
This has golang 1.10 which predates the inclusion of modules.  There's
basically no sane way to make libnbd work with both old and new
golang, so I would suggest --disable-golang on this platform.  (In
fact is it possible to disable features on different CI platforms?)
> and if you want to be able to run `make distcheck` as well, there are
> some builddir/srcdir issues in golang/Makefile.am I believe, so VPATH
> building actually fails.  I added the `make distcheck` step to one of
> the jobs here:
> 
>   https://gitlab.com/nertpinx/libnbd/-/jobs/1163504412
I think "make distcheck" is a nice-to-have but ISTR that making it
work was a lot of work for libguestfs.
> In order not to bother you with every single e-mail I would suggest that
> I push some incomplete version in the repository and you can see
> yourself (and others as well) how the build behaves.  What do you say?
> I won't look at it earlier than tomorrow anyway.
Sure, go for it.

Thanks again,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

On Thu, Apr 08, 2021 at 05:19:25PM +0100, Richard W.M. Jones
wrote:
>On Thu, Apr 08, 2021 at 05:39:17PM +0200, Martin Kletzander wrote:
>>  - clang failures:
>>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504401
>
>I don't usually enable warnings with clang, but I've pushed this
which
>fixes the problem above (and a further one) for me:
>
> 
https://gitlab.com/nbdkit/libnbd/-/commit/28816a5f8829f474d07351611140f461facd0e4d
>
We can disable anything if you want.  Bear in mind this is a preparation
for other projects, like nbdkit, so make sure you express all your
desires so that I can adjust appropriately.  We can just keep clang
running optionally or not enable warnings.
>>  - golang test failures on OpenSUSE:
>>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504416
>
>This is actually because:
>
>  checking for nbdkit... no
>
Oh, missed that.  There might be some extra dependencies missing and
these are pain in the butt to search for.  But nbdkit is missing because
we will want to do two things:

   - test it with the distro's nbdkit

   - test it with current nbdkit's master branch

I did not get to that yet, I have some preparation done, but we first
need CI for nbdkit running properly and then it'll be easier to join
them together.

I added a lot of dependencies (what took a tremendous amount of time).
Some are not added on all platforms, I'll be working on that, slowly.
Even though the mapping from libvirt-ci helps a lot, the last 10% is
still checking each missing dependency and skipped test on every
distro. I am just about done with OpenSUSE Tumbleweed.
>but we go ahead and run the golang tests anyway.  Fixed here:
>
> 
https://gitlab.com/nbdkit/libnbd/-/commit/b15dde588f47894f9656ea62eb632156b7638b8b
>
>but there may be more problems after this, we'll see.
>
>>  - one weird ubuntu 18.04 failure:
>>    https://gitlab.com/nertpinx/libnbd/-/jobs/1163504418
>
>This has golang 1.10 which predates the inclusion of modules.  There's
>basically no sane way to make libnbd work with both old and new
>golang, so I would suggest --disable-golang on this platform.  (In
>fact is it possible to disable features on different CI platforms?)
>
We can do anything!  If disabling golang is the best way to go (no pun
intended), then we can do that.  Or disable it for any golang older than
some 1.xx version.  For now I disabled golang on that old Ubuntu.
>> and if you want to be able to run `make distcheck` as well, there are
>> some builddir/srcdir issues in golang/Makefile.am I believe, so VPATH
>> building actually fails.  I added the `make distcheck` step to one of
>> the jobs here:
>>
>>   https://gitlab.com/nertpinx/libnbd/-/jobs/1163504412
>
>I think "make distcheck" is a nice-to-have but ISTR that making it
>work was a lot of work for libguestfs.
>
If distcheck is not a priority, then we can just not run it. VPATH
builds with autotools are hard and unless there is someone running
VPATH builds all the time then they will break every now and then.

I'll keep the `make distcheck` guarded under `if $DISTCHECK ==
"force"`
so that we can enable it later with one env var.
>> In order not to bother you with every single e-mail I would suggest
that
>> I push some incomplete version in the repository and you can see
>> yourself (and others as well) how the build behaves.  What do you say?
>> I won't look at it earlier than tomorrow anyway.
>
>Sure, go for it.
>
I added some more optional packages that were missing, so that most of
the code is build/tested, changed a few more things and I'll at least
make it consistent where possible, otherwise it would be a mess for
anyone trying to make sense of it.  I hope that if I do it properly
right away I will save some time for later.

There are still some issues, couple of them due to (probably just older)
libev on centos-7 and ubuntu-18.04, FreeBSD not having fallocate() and
some others that I have fixes for.  I'll let you know, thanks for the
patience.
>Thanks again,
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
>Read my programming and virtualization blog: http://rwmj.wordpress.com
>virt-builder quickly builds VMs from scratch
>http://libguestfs.org/virt-builder.1.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://listman.redhat.com/archives/libguestfs/attachments/20210415/0cff13c0/attachment.sig>