Libguestfs - Jan 2014 - Notes on building libguestfs in a systemd-nspawn container

Last night I was tinkering with `systemd-nspawn` -- namespace based
container for testing, I thought I'll post  what I tried with libguestfs
here:


Prerequisite
------------

Because of an audit subsystem incompatibility bug - rhbz#966807[1], turn
off auditing by booting the host w/ 'audit=0' on Kernel command line.
(NOTE: There's work in progress[2] in upstream Kernel to fix this.)


Create container
----------------

On the host, specify an installroot (/srv) and install a minimal Fedora
20 distribution:

    $ yum -y --releasever=20 --nogpg \
        --installroot=/srv/testcontainer \
        --disablerepo='*' --enablerepo=fedora \
        install systemd passwd yum \
        fedora-release vim-minimal

Boot into the container, set a password:

    $ systemd-nspawn -D /srv/testcontainer
    [. . .]
    -bash-4.2# passwd

Start the container w/ systemd:

    $ systemd-nspawn -bD /srv/testcontainer
    [. . .]
    -bash-4.2#


Building Libguestfs
-------------------

Inside the minimal Fedora 20 container, install libguestfs dependencies,
clone the libguestfs git repository:

    -bash-4.2# yum-builddep libguestfs -y

    -bash-4.2# git clone git://github.com/libguestfs/libguestfs.git

Build and test libguestfs in the container:

    -bash-4.2# cd libguestfs

    -bash-4.2# ./autogen.sh && time make 2>&1 \
                | tee /tmp/libguestfs-compile.log

     -bash-4.2# time make -k check \
     LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 2>&1 \
     | tee /tmp/libguestfs-test.log


Notes
-----

- If you need to build a container without networking (once all the
  relevant dependencies are cloned/and network related operations
  are done), the container can be invoked with no network
  devices (except loopback):

    $ systemd-nspawn -bD /srv/testcontainer --private-network
    [. . .]
    -bash-4.2#

- Single `make` job timing to compile everything:

    real    31m9.792s
    user    17m18.359s
    sys     13m17.868s

- `make -k check` is still running as I write this, albeit
   a bit slow.



  [1] https://bugzilla.redhat.com/show_bug.cgi?id=966807
  [2] https://www.redhat.com/archives/linux-audit/2013-May/msg00065.html


-- 
/kashyap

On 01/30/2014 07:41 AM, Kashyap Chamarthy wrote:
> Last night I was tinkering with `systemd-nspawn` -- namespace based
> container for testing, I thought I'll post  what I tried with
libguestfs
> here:
> 
> 
> Prerequisite
> ------------
> 
> Because of an audit subsystem incompatibility bug - rhbz#966807[1], turn
> off auditing by booting the host w/ 'audit=0' on Kernel command
line.
> (NOTE: There's work in progress[2] in upstream Kernel to fix this.)
> 
> 
> Create container
> ----------------
> 
> On the host, specify an installroot (/srv) and install a minimal Fedora
> 20 distribution:
> 
>     $ yum -y --releasever=20 --nogpg \
>         --installroot=/srv/testcontainer \
>         --disablerepo='*' --enablerepo=fedora \
>         install systemd passwd yum \
>         fedora-release vim-minimal
> 
> Boot into the container, set a password:
> 
>     $ systemd-nspawn -D /srv/testcontainer
>     [. . .]
>     -bash-4.2# passwd
> 
> Start the container w/ systemd:
> 
>     $ systemd-nspawn -bD /srv/testcontainer
>     [. . .]
>     -bash-4.2#
> 
> 
> Building Libguestfs
> -------------------
> 
> Inside the minimal Fedora 20 container, install libguestfs dependencies,
> clone the libguestfs git repository:
> 
>     -bash-4.2# yum-builddep libguestfs -y
> 
>     -bash-4.2# git clone git://github.com/libguestfs/libguestfs.git
> 
> Build and test libguestfs in the container:
> 
>     -bash-4.2# cd libguestfs
> 
>     -bash-4.2# ./autogen.sh && time make 2>&1 \
>                 | tee /tmp/libguestfs-compile.log
> 
>      -bash-4.2# time make -k check \
>      LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 2>&1 \
>      | tee /tmp/libguestfs-test.log
> 
> 
> Notes
> -----
> 
> - If you need to build a container without networking (once all the
>   relevant dependencies are cloned/and network related operations
>   are done), the container can be invoked with no network
>   devices (except loopback):
> 
>     $ systemd-nspawn -bD /srv/testcontainer --private-network
>     [. . .]
>     -bash-4.2#
> 
> - Single `make` job timing to compile everything:
> 
>     real    31m9.792s
>     user    17m18.359s
>     sys     13m17.868s
For comparison, on the _host_, the same single `make` job timing:

    real    13m41.440s
    user    13m5.816s
    sys     1m9.911s
> 
> - `make -k check` is still running as I write this, albeit
>    a bit slow.
This just finished (in the container):

    [. . .]
    grep -v -E '^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/'
| \
    grep -v -E '/((guestfs|rc)_protocol\.c)$' | \
    LC_ALL=C sort > po/POTFILES
    cd .; \
    find builder mllib resize sparsify sysprep -name '*.ml' | \
    LC_ALL=C sort > po/POTFILES-ml
    make[1]: Leaving directory `/root/libguestfs'
    make: *** [check-recursive] Error 1
      GEN      public-submodule-commit
    make: Target `check' not remade because of errors.

    real    474m53.630s
    user    325m54.254s
    sys     205m58.032s

    -bash-4.2# git log | head -1
    commit c841d08d7084db69e81614d54423686cf0566ad6


Again, for comparison, `make -k check` on _host_:

    real    63m1.078s
    user    54m39.393s
    sys     12m8.130s


I think it's worth noting that this is all with Btrfs on the host. And,
the systemd[*] wiki page indeed warns: "YMMV if you use LVM or btrfs."

It needs to be examined what the result would be if I repeat this on
ext4 (and SSD).



[*] http://www.freedesktop.org/wiki/Software/systemd/VirtualizedTesting/

> 
> 
> 
>   [1] https://bugzilla.redhat.com/show_bug.cgi?id=966807
>   [2] https://www.redhat.com/archives/linux-audit/2013-May/msg00065.html
> 
> 

-- 
/kashyap

On Thu, Jan 30, 2014 at 11:50:35AM +0530, Kashyap Chamarthy
wrote:
> > - Single `make` job timing to compile everything:
> > 
> >     real    31m9.792s
> >     user    17m18.359s
> >     sys     13m17.868s
> 
> For comparison, on the _host_, the same single `make` job timing:
> 
>     real    13m41.440s
>     user    13m5.816s
>     sys     1m9.911s
This is timing the build only?

I'm surprised it is slower in the container.  Is memory or # CPUs
limited?
> > 
> > - `make -k check` is still running as I write this, albeit
> >    a bit slow.
> 
> This just finished (in the container):
> 
>     [. . .]
>     grep -v -E
'^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/' | \
>     grep -v -E '/((guestfs|rc)_protocol\.c)$' | \
>     LC_ALL=C sort > po/POTFILES
>     cd .; \
>     find builder mllib resize sparsify sysprep -name '*.ml' | \
>     LC_ALL=C sort > po/POTFILES-ml
>     make[1]: Leaving directory `/root/libguestfs'
>     make: *** [check-recursive] Error 1
>       GEN      public-submodule-commit
>     make: Target `check' not remade because of errors.
> 
>     real    474m53.630s
>     user    325m54.254s
>     sys     205m58.032s
> 
>     -bash-4.2# git log | head -1
>     commit c841d08d7084db69e81614d54423686cf0566ad6
> 
> 
> Again, for comparison, `make -k check` on _host_:
> 
>     real    63m1.078s
>     user    54m39.393s
>     sys     12m8.130s
Is KVM available in the container?  I've never tried that actually ..

I suppose the next step is to make LIBGUESTFS_BACKEND=libvirt:lxc:///
work!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)