Libguestfs - Sep 2021 - [libguestfs-common PATCH 0/2] build guestfs-tools and virt-v2v against locally built libguestfs

Hi,

I'll post two small patch sets as followups to this "higher order"
cover
letter. The goal is to build guestfs-tools and virt-v2v against a
freshly built libguestfs (git checkout), using libguestfs's "run"
script. The modifications affect two projects, libguestfs-common and
guestfs-tools, hence the two (upcoming) patch series.

libguestfs-common is tricky because it is consumed as a submodule by
three git superprojects (libguestfs itself, then guestfs-tools and
virt-v2v). Yet more trickily, the three superprojects don't all seem to
consume libguestfs-common at the same submodule git commit, at the
moment. (libguestfs and guestfs-tools consume the submodule at older
commit 74bc5c5c5cb4, while virt-v2v consumes the submodule at current
HEAD commit 6d26b6eac9da.)

For verifying the libguestfs-common updates, I first attempted various
git-pushes into the submodule checkouts of the superprojects. That
proved super unwieldy, as it would require a whole lot of git massaging
just to (incrementally) rebuild all three superprojects after each
libguestfs-common update. Instead, in each superproject, I created a
development branch, and as first commit on that branch, I removed the
submodule altogether, and replaced it with a directory tree of symbolic
links into my stand-alone libguestfs-common worktree -- refer to the
"lndir" command. I didn't expect creating new files in
libguestfs-common, so once established, the symlink set could be
considered final. Furthermore, "lndir" was required for symlinking
individual regular files: symlinking subdirectories from
libguestfs-common into the superprojects' "common" directories
doesn't
work, as the "common" content refers back to the superproject, via
relative pathnames such as "../blah". For such pathnames to work, the
deep "common" directory structure actually needs to exist within each
superproject, only the leaves (the regular files) can be replaced with
symlinks. Thankfully, "lndir" implements just that.

This effectively bumped libguestfs and guestfs-tools to
libguestfs-common HEAD commit 6d26b6eac9da, as a basis for the needed
libguestfs-common fixes.

With the "libguestfs-common" patch set applied (on top of commit
6d26b6eac9da), libguestfs can be built and checked without regressions.
Furthermore, virt-v2v too can be built and checked against the
just-built libguestfs (using the latter's "run" script) without
regressions.

The same applies to "guestfs-tools", assuming the other patch set is
applied to it (on top of commit 9ba463545fa0).

Once these patch sets are up-stream, the three superprojects should
advance their submodule references to the new libguestfs-common head commit.

Thanks!
Laszlo

This series lets (a) virt-v2v, and (b) in combination with the sibling
series for guestfs-tools, guestfs-tools, be built against a just-built
libguestfs (e.g. with the libguest-devel package absent), using the
commands

$ git submodule update --init --force
$ autoreconf -i
$ ../libguestfs/run ./configure CFLAGS=-fPIC
$ ../libguestfs/run make -j $(getconf _NPROCESSORS_ONLN)
$ ../libguestfs/run make -j $(getconf _NPROCESSORS_ONLN) check

Thanks,
Laszlo

Laszlo Ersek (2):
  Makefile.am: supply missing $(LIBGUESTFS_CFLAGS)
  Makefile.am: use $(LIBGUESTFS_LIBS) for linking OCaml test programs

 edit/Makefile.am     |  3 ++-
 mltools/Makefile.am  | 24 ++++++++++++++++++------
 mlvisit/Makefile.am  | 10 +++++++++-
 parallel/Makefile.am |  1 +
 progress/Makefile.am |  1 +
 structs/Makefile.am  |  1 +
 visit/Makefile.am    |  1 +
 windows/Makefile.am  |  3 ++-
 8 files changed, 35 insertions(+), 9 deletions(-)


base-commit: 6d26b6eac9dab6bc8fddf9f6564a13f39c51defe
-- 
2.19.1.3.g30247aa5d201

This series (in combination with the sibling series for
libguestfs-common) allows guestfs-tools to be built against a just-built
libguestfs (e.g. with the libguest-devel package absent), using the
commands

$ git submodule update --init --force
$ autoreconf -i
$ ../libguestfs/run ./configure CFLAGS=-fPIC
$ ../libguestfs/run make -j $(getconf _NPROCESSORS_ONLN)
$ ../libguestfs/run make -j $(getconf _NPROCESSORS_ONLN) check

Thanks,
Laszlo

Laszlo Ersek (3):
  Makefile.am: fix $(LIBGUESTFS_CFLAGS) typo for virt-ls
  Makefile.am: supply missing $(LIBGUESTFS_CFLAGS)
  Makefile.am: use $(LIBGUESTFS_LIBS) for linking OCaml programs

 builder/Makefile.am    | 5 ++++-
 cat/Makefile.am        | 2 +-
 customize/Makefile.am  | 2 ++
 dib/Makefile.am        | 1 +
 get-kernel/Makefile.am | 1 +
 resize/Makefile.am     | 1 +
 sparsify/Makefile.am   | 1 +
 sysprep/Makefile.am    | 1 +
 8 files changed, 12 insertions(+), 2 deletions(-)


base-commit: 9ba463545fa017910a5077434f87ae67e146c9e0
-- 
2.19.1.3.g30247aa5d201

On Mon, Sep 06, 2021 at 12:57:56PM +0200, Laszlo Ersek
wrote:
> Hi,
> 
> I'll post two small patch sets as followups to this "higher
order" cover
> letter. The goal is to build guestfs-tools and virt-v2v against a
> freshly built libguestfs (git checkout), using libguestfs's
"run"
> script. The modifications affect two projects, libguestfs-common and
> guestfs-tools, hence the two (upcoming) patch series.
> 
> libguestfs-common is tricky because it is consumed as a submodule by
> three git superprojects (libguestfs itself, then guestfs-tools and
> virt-v2v). Yet more trickily, the three superprojects don't all seem to
> consume libguestfs-common at the same submodule git commit, at the
> moment. (libguestfs and guestfs-tools consume the submodule at older
> commit 74bc5c5c5cb4, while virt-v2v consumes the submodule at current
> HEAD commit 6d26b6eac9da.)
This is because I ad-hoc update the common submodule when I need to.

In theory all the projects should work against the latest common
submodule, and a commit against any of them which just bumps the
common submodule to the latest version should always be valid.
> For verifying the libguestfs-common updates, I first attempted various
> git-pushes into the submodule checkouts of the superprojects. That
> proved super unwieldy, as it would require a whole lot of git massaging
> just to (incrementally) rebuild all three superprojects after each
> libguestfs-common update. Instead, in each superproject, I created a
> development branch, and as first commit on that branch, I removed the
> submodule altogether, and replaced it with a directory tree of symbolic
> links into my stand-alone libguestfs-common worktree -- refer to the
> "lndir" command. I didn't expect creating new files in
> libguestfs-common, so once established, the symlink set could be
> considered final. Furthermore, "lndir" was required for
symlinking
> individual regular files: symlinking subdirectories from
> libguestfs-common into the superprojects' "common"
directories doesn't
> work, as the "common" content refers back to the superproject,
via
> relative pathnames such as "../blah". For such pathnames to work,
the
> deep "common" directory structure actually needs to exist within
each
> superproject, only the leaves (the regular files) can be replaced with
> symlinks. Thankfully, "lndir" implements just that.
> 
> This effectively bumped libguestfs and guestfs-tools to
> libguestfs-common HEAD commit 6d26b6eac9da, as a basis for the needed
> libguestfs-common fixes.
> 
> With the "libguestfs-common" patch set applied (on top of commit
> 6d26b6eac9da), libguestfs can be built and checked without regressions.
> Furthermore, virt-v2v too can be built and checked against the
> just-built libguestfs (using the latter's "run" script)
without regressions.
> 
> The same applies to "guestfs-tools", assuming the other patch set
is
> applied to it (on top of commit 9ba463545fa0).
> 
> Once these patch sets are up-stream, the three superprojects should
> advance their submodule references to the new libguestfs-common head
commit.
Yup, submodules are horrible.  But a bit better than code duplication.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v