Libguestfs - Jan 2014 - Re: Notes on building libguestfs in a systemd-nspawn container

On Thu, Jan 30, 2014 at 04:34:04PM +0530, Kashyap Chamarthy
wrote:
> On 01/30/2014 03:58 PM, Richard W.M. Jones wrote:
> >>> - `make -k check` is still running as I write this, albeit
> >>>    a bit slow.
> >>
> >> This just finished (in the container):
> >>
> >>     [. . .]
> >>     grep -v -E
'^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/' | \
> >>     grep -v -E '/((guestfs|rc)_protocol\.c)$' | \
> >>     LC_ALL=C sort > po/POTFILES
> >>     cd .; \
> >>     find builder mllib resize sparsify sysprep -name
'*.ml' | \
> >>     LC_ALL=C sort > po/POTFILES-ml
> >>     make[1]: Leaving directory `/root/libguestfs'
> >>     make: *** [check-recursive] Error 1
> >>       GEN      public-submodule-commit
> >>     make: Target `check' not remade because of errors.
> >>
> >>     real    474m53.630s
> >>     user    325m54.254s
> >>     sys     205m58.032s
> >>
> >>     -bash-4.2# git log | head -1
> >>     commit c841d08d7084db69e81614d54423686cf0566ad6
> >>
> >>
> >> Again, for comparison, `make -k check` on _host_:
> >>
> >>     real    63m1.078s
> >>     user    54m39.393s
> >>     sys     12m8.130s
> > 
> > Is KVM available in the container?  I've never tried that actually
..
> 
> No it isn't (as Dan noted in his next thread)
> 
> ========> -bash-4.2# file /dev/kvm
> /dev/kvm: ERROR: cannot open `/dev/kvm' (No such file or directory)
> ========> -bash-4.2# virt-host-validate
>   QEMU: Checking for hardware virtualization
>      : PASS
>   QEMU: Checking for device /dev/kvm
>      : FAIL (Check that the 'kvm-intel' or 'kvm-amd'
modules are loaded
> & the BIOS has enabled virtualization)
>   QEMU: Checking for device /dev/vhost-net
>      : WARN (Load the 'vhost_net' module to improve performance of
> virtio networking)
>   QEMU: Checking for device /dev/net/tun
>      : FAIL (Load the 'tun' module to enable networking for QEMU
guests)
>    LXC: Checking for Linux >= 2.6.26
>      : PASS
> ========> 
> Despite reading from the `systemd-nspawn` man page:
> 
>  ". . .kernel modules may not be loaded from within the
container."
> 
> I purposefully tried from inside the container:
With container based virt there is only one kernel image, so any
modules you want must be loaded in the host. Libvirt "passthrough"
of char/block devices simply involves libvirt doing mknod in the
/dev tmpfs it sets up. The container itself is blocked from doing
any 'mknod' calls since that'd be a security risk. Hence you must
list any desired device nodes in the XML config.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

On 01/30/2014 04:38 PM, Daniel P. Berrange wrote:

[. . .]
>>
>> Despite reading from the `systemd-nspawn` man page:
>>
>>  ". . .kernel modules may not be loaded from within the
container."
>>
>> I purposefully tried from inside the container:
> 
> With container based virt there is only one kernel image,
Noted, that's one of the main aspects, right, of containers: single
Kernel (also a single point of attack-surface; no custom Kernels, etc)[1]

But I see the use-case of systemd-nspawn: quick development/debugging
just like chroot, but better.
> so any
> modules you want must be loaded in the host. Libvirt
"passthrough"
> of char/block devices simply involves libvirt doing mknod in the
> /dev tmpfs it sets up. The container itself is blocked from doing
> any 'mknod' calls since that'd be a security risk. Hence you
must
> list any desired device nodes in the XML config.
Thanks for the explanation. I have to try libvirt-lxc tools next. Also
on my todo-list to try:

  $ virt-sandbox mock

  [Build a package]

I see that the above provides a default SELinux 'seclabel' element. Have
to test yet.

Meanwhile, I stumbled across an upstream thread[2][3] of yours this
morning & learnt re: a regression with user namespaces containers


  [1]
http://rwmj.wordpress.com/2013/06/19/the-boring-truth-full-virtualization-and-containerization-both-have-their-place/
  [2]
https://lists.linuxfoundation.org/pipermail/containers/2013-November/033635.html
  [3] https://bugzilla.redhat.com/show_bug.cgi?id=917708

-- 
/kashyap

On Thu, Jan 30, 2014 at 05:07:23PM +0530, Kashyap Chamarthy
wrote:
> On 01/30/2014 04:38 PM, Daniel P. Berrange wrote:
> 
> [. . .]
> 
> >>
> >> Despite reading from the `systemd-nspawn` man page:
> >>
> >>  ". . .kernel modules may not be loaded from within the
container."
> >>
> >> I purposefully tried from inside the container:
> > 
> > With container based virt there is only one kernel image,
> 
> Noted, that's one of the main aspects, right, of containers: single
> Kernel (also a single point of attack-surface; no custom Kernels, etc)[1]
> 
> But I see the use-case of systemd-nspawn: quick development/debugging
> just like chroot, but better.
> 
> > so any
> > modules you want must be loaded in the host. Libvirt
"passthrough"
> > of char/block devices simply involves libvirt doing mknod in the
> > /dev tmpfs it sets up. The container itself is blocked from doing
> > any 'mknod' calls since that'd be a security risk. Hence
you must
> > list any desired device nodes in the XML config.
> 
> Thanks for the explanation. I have to try libvirt-lxc tools next. Also
> on my todo-list to try:
> 
>   $ virt-sandbox mock
> 
>   [Build a package]
> 
> I see that the above provides a default SELinux 'seclabel' element.
Have
> to test yet.
> 
> Meanwhile, I stumbled across an upstream thread[2][3] of yours this
> morning & learnt re: a regression with user namespaces containers
Nb user namespaces aren't relevant here. Nothing you're using / trying
here involves user namespaces at all.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|