Martin Kletzander
2021-Apr-08 13:24 UTC
[Libguestfs] [libnbd RFC PATCH] First stab at CI infrastructure
On Wed, Apr 07, 2021 at 05:49:30PM +0100, Richard W.M. Jones wrote:>On Wed, Apr 07, 2021 at 04:55:17PM +0200, Martin Kletzander wrote: >> Polite ping. Any thoughts? Anything you'd like to change? > >I didn't particularly understand any of it; but it's what we need so >looks good! >Well, any question you might have I'll do my best to answer it. It took me a while as well, but it is way more versatile and stable than other CI setups.>One bug: All the new files that have been added probably need to be >added to the top-level Makefile.am EXTRA_DIST, certainly if they would >be in any way useful in the tarball. However if they are not useful >in the tarball then the target "maintainer-check-extra-dist" (also in >Makefile.am) must be modified to ignore these files when I do >"make && make dist && make maintainer-check-extra-dist". >I did not really check more than just running the tests, but if there is any workflow that you have, then we can incorporate it in there, so that it checks as much as possible. I tried running `make distcheck` and there were some harder failures than these, so I wanted to start small.>I also looked at the results: > > https://gitlab.com/nertpinx/libnbd/-/pipelines?scope=all&page=1 > >and choosing the top "failed" link arbitrarily takes me to: > > https://gitlab.com/nertpinx/libnbd/-/pipelines/278989630 > >where there are some failures. We don't support mingw* platforms for >libnbd (we do for nbdkit), so for libnbd those platforms can >immediately be dropped. >Good to know. I started with the list from libvirt (which is the most complete I know) and figured we will tune it later. I'm removing the mingw builds then, thanks.>But there are some other non-mingw failures, again picking one >at random: > > https://gitlab.com/nertpinx/libnbd/-/jobs/1140619393 > >the failure is: > > go install: version is required when current directory is not in a module > >which is (or was) a real bug! I've since fixed it upstream, but it >seems to show the thing works. > >As far as I know libnbd should currently build on Rawhide, so could >you pull in the updates, drop mingw*, and see if the tests start to >pass? >There are couple more, but they are the same failure. Basically anything that builds with clang (BSD and MacOS by default and CentOS 8 is running both GCC and clang variants) will error out due to unused string vector functions. Unfortunately those are generated. Example: https://gitlab.com/nertpinx/libnbd/-/jobs/1140619387 GCC does not find this even with --enable-gcc-warnings (already used). One last thing that I forgot to mention, but I definitely wanted to ask about is whether there is a reason behind not erroring out when some configure option is requested, but cannot be fulfilled. For example `--with-libxml2` when pkg-config cannot find it will just result in a warning that most people will miss. If new dependencies are required and we forget to add them to the list of packages for CI, then instead of the CI failing and us noticing pretty quickly we will just build without the support for some part of the code and completely missing it in the tests as well. Anyway, I updated some details and ran it again: https://gitlab.com/nertpinx/libnbd/-/pipelines/283351379 Let's see what's going to pop up next.>Rich. > >-- >Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones >Read my programming and virtualization blog: http://rwmj.wordpress.com >Fedora Windows cross-compiler. Compile Windows programs, test, and >build Windows installers. Over 100 libraries supported. >http://fedoraproject.org/wiki/MinGW-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210408/9cec66a7/attachment.sig>
Richard W.M. Jones
2021-Apr-08 13:36 UTC
[Libguestfs] [libnbd RFC PATCH] First stab at CI infrastructure
On Thu, Apr 08, 2021 at 03:24:20PM +0200, Martin Kletzander wrote:> On Wed, Apr 07, 2021 at 05:49:30PM +0100, Richard W.M. Jones wrote: > >One bug: All the new files that have been added probably need to be > >added to the top-level Makefile.am EXTRA_DIST, certainly if they would > >be in any way useful in the tarball. However if they are not useful > >in the tarball then the target "maintainer-check-extra-dist" (also in > >Makefile.am) must be modified to ignore these files when I do > >"make && make dist && make maintainer-check-extra-dist". > > > > I did not really check more than just running the tests, but if there is > any workflow that you have, then we can incorporate it in there, so that > it checks as much as possible. I tried running `make distcheck` and > there were some harder failures than these, so I wanted to start small.I meant the patch needs to be updated so it either adds these files to EXTRA_DIST (ie. the tarball) or the maintainer-check-extra-dist rule is fixed to ignore them.> > go install: version is required when current directory is not in a module...> > There are couple more, but they are the same failure. Basically > anything that builds with clang (BSD and MacOS by default and CentOS 8 > is running both GCC and clang variants) will error out due to unused > string vector functions. Unfortunately those are generated.libnbd supports only Linux, FreeBSD and OpenBSD. It probably would work on MacOS but I haven't tried it. We do support clang on the supported platforms, but because we don't use CI it's not tested very often so I don't know if it works right now.> Example: > > https://gitlab.com/nertpinx/libnbd/-/jobs/1140619387 > > GCC does not find this even with --enable-gcc-warnings (already used).I'm not sure to what extent we want to use --enable-gcc-warnings on the non-Linux platforms. What does libvirt do here? Do they aim to have GCC warnings clean even on non-Linux?> One last thing that I forgot to mention, but I definitely wanted to ask > about is whether there is a reason behind not erroring out when some > configure option is requested, but cannot be fulfilled. For example > `--with-libxml2` when pkg-config cannot find it will just result in a > warning that most people will miss.If ?./configure --with-libxml2? is used then it's a bug if libxml2 is not present. But libxml2 is an optional dependency so if they didn't specify ?--with-libxml2? and libxml2 is not available it should work. So yes it's a bug.> Anyway, I updated some details and ran it again: > > https://gitlab.com/nertpinx/libnbd/-/pipelines/283351379 > > Let's see what's going to pop up next.golang seems to be failing hard there :-( Looks it fails with libnbd is not already installed (which is a bug). I'll see if I can fix that one. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html