Richard W.M. Jones
2014-Jan-30 10:28 UTC
Re: [Libguestfs] Notes on building libguestfs in a systemd-nspawn container
On Thu, Jan 30, 2014 at 11:50:35AM +0530, Kashyap Chamarthy wrote:> > - Single `make` job timing to compile everything: > > > > real 31m9.792s > > user 17m18.359s > > sys 13m17.868s > > For comparison, on the _host_, the same single `make` job timing: > > real 13m41.440s > user 13m5.816s > sys 1m9.911sThis is timing the build only? I'm surprised it is slower in the container. Is memory or # CPUs limited?> > > > - `make -k check` is still running as I write this, albeit > > a bit slow. > > This just finished (in the container): > > [. . .] > grep -v -E '^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/' | \ > grep -v -E '/((guestfs|rc)_protocol\.c)$' | \ > LC_ALL=C sort > po/POTFILES > cd .; \ > find builder mllib resize sparsify sysprep -name '*.ml' | \ > LC_ALL=C sort > po/POTFILES-ml > make[1]: Leaving directory `/root/libguestfs' > make: *** [check-recursive] Error 1 > GEN public-submodule-commit > make: Target `check' not remade because of errors. > > real 474m53.630s > user 325m54.254s > sys 205m58.032s > > -bash-4.2# git log | head -1 > commit c841d08d7084db69e81614d54423686cf0566ad6 > > > Again, for comparison, `make -k check` on _host_: > > real 63m1.078s > user 54m39.393s > sys 12m8.130sIs KVM available in the container? I've never tried that actually .. I suppose the next step is to make LIBGUESTFS_BACKEND=libvirt:lxc:/// work! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
Daniel P. Berrange
2014-Jan-30 10:39 UTC
Re: [Libguestfs] Notes on building libguestfs in a systemd-nspawn container
On Thu, Jan 30, 2014 at 10:28:31AM +0000, Richard W.M. Jones wrote:> On Thu, Jan 30, 2014 at 11:50:35AM +0530, Kashyap Chamarthy wrote: > > > > Again, for comparison, `make -k check` on _host_: > > > > real 63m1.078s > > user 54m39.393s > > sys 12m8.130s > > Is KVM available in the container? I've never tried that actually ..I don't believe so. IIRC systemd-nspawn will populate a tmpfs on /dev with only zero, null, full, random, unrandom, tty nodes. If you use libvirt to setup the container then you can tell libvirt to add in the /dev/kvm node too. This shows how to setup the basic Fedora OS container https://www.berrange.com/posts/2013/08/12/running-a-full-fedora-os-inside-a-libvirt-lxc-guest/ Then update the XML to passthrough the /dev/kvm char device as a hostdev: http://libvirt.org/formatdomain.html#elementsHostDevCaps Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Kashyap Chamarthy
2014-Jan-30 11:04 UTC
Re: [Libguestfs] Notes on building libguestfs in a systemd-nspawn container
On 01/30/2014 03:58 PM, Richard W.M. Jones wrote:> On Thu, Jan 30, 2014 at 11:50:35AM +0530, Kashyap Chamarthy wrote: >>> - Single `make` job timing to compile everything: >>> >>> real 31m9.792s >>> user 17m18.359s >>> sys 13m17.868s >> >> For comparison, on the _host_, the same single `make` job timing: >> >> real 13m41.440s >> user 13m5.816s >> sys 1m9.911s > > This is timing the build only?Yes. (I wonder if Btrfs matters here.)> > I'm surprised it is slower in the container. Is memory or # CPUs > limited?I haven't done cgroups tuning or deep introspection or any such. It's just a default invocation of `systemd-nspawn`. That said, from inside the container: ========-bash-4.2# cat /proc/cpuinfo | grep processor | wc -l 48 -bash-4.2# free -m total used free shared buffers cached Mem: 64259 12461 51798 49 1 10757 -/+ buffers/cache: 1703 62556 Swap: 13996 0 13996 -bash-4.2# ======== The machine is Intel(R) Xeon(R) CPU E7- 4807 @ 1.87GHz, it has 48 processors and 60 GB memory> >>> >>> - `make -k check` is still running as I write this, albeit >>> a bit slow. >> >> This just finished (in the container): >> >> [. . .] >> grep -v -E '^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/' | \ >> grep -v -E '/((guestfs|rc)_protocol\.c)$' | \ >> LC_ALL=C sort > po/POTFILES >> cd .; \ >> find builder mllib resize sparsify sysprep -name '*.ml' | \ >> LC_ALL=C sort > po/POTFILES-ml >> make[1]: Leaving directory `/root/libguestfs' >> make: *** [check-recursive] Error 1 >> GEN public-submodule-commit >> make: Target `check' not remade because of errors. >> >> real 474m53.630s >> user 325m54.254s >> sys 205m58.032s >> >> -bash-4.2# git log | head -1 >> commit c841d08d7084db69e81614d54423686cf0566ad6 >> >> >> Again, for comparison, `make -k check` on _host_: >> >> real 63m1.078s >> user 54m39.393s >> sys 12m8.130s > > Is KVM available in the container? I've never tried that actually ..No it isn't (as Dan noted in his next thread) ========-bash-4.2# file /dev/kvm /dev/kvm: ERROR: cannot open `/dev/kvm' (No such file or directory) ========-bash-4.2# virt-host-validate QEMU: Checking for hardware virtualization : PASS QEMU: Checking for device /dev/kvm : FAIL (Check that the 'kvm-intel' or 'kvm-amd' modules are loaded & the BIOS has enabled virtualization) QEMU: Checking for device /dev/vhost-net : WARN (Load the 'vhost_net' module to improve performance of virtio networking) QEMU: Checking for device /dev/net/tun : FAIL (Load the 'tun' module to enable networking for QEMU guests) LXC: Checking for Linux >= 2.6.26 : PASS ======== Despite reading from the `systemd-nspawn` man page: ". . .kernel modules may not be loaded from within the container." I purposefully tried from inside the container: ========-bash-4.2# modprobe kvm_intel -bash-4.2# echo $? 1 -bash-4.2# file /dev/kvm /dev/kvm: ERROR: cannot open `/dev/kvm' (No such file or directory) -bash-4.2# ========> > I suppose the next step is to make LIBGUESTFS_BACKEND=libvirt:lxc:/// > work! > > Rich. >-- /kashyap
Daniel P. Berrange
2014-Jan-30 11:08 UTC
Re: [Libguestfs] Notes on building libguestfs in a systemd-nspawn container
On Thu, Jan 30, 2014 at 04:34:04PM +0530, Kashyap Chamarthy wrote:> On 01/30/2014 03:58 PM, Richard W.M. Jones wrote: > >>> - `make -k check` is still running as I write this, albeit > >>> a bit slow. > >> > >> This just finished (in the container): > >> > >> [. . .] > >> grep -v -E '^(examples|gnulib|perl/(blib|examples)|po-docs|tests)/' | \ > >> grep -v -E '/((guestfs|rc)_protocol\.c)$' | \ > >> LC_ALL=C sort > po/POTFILES > >> cd .; \ > >> find builder mllib resize sparsify sysprep -name '*.ml' | \ > >> LC_ALL=C sort > po/POTFILES-ml > >> make[1]: Leaving directory `/root/libguestfs' > >> make: *** [check-recursive] Error 1 > >> GEN public-submodule-commit > >> make: Target `check' not remade because of errors. > >> > >> real 474m53.630s > >> user 325m54.254s > >> sys 205m58.032s > >> > >> -bash-4.2# git log | head -1 > >> commit c841d08d7084db69e81614d54423686cf0566ad6 > >> > >> > >> Again, for comparison, `make -k check` on _host_: > >> > >> real 63m1.078s > >> user 54m39.393s > >> sys 12m8.130s > > > > Is KVM available in the container? I've never tried that actually .. > > No it isn't (as Dan noted in his next thread) > > ========> -bash-4.2# file /dev/kvm > /dev/kvm: ERROR: cannot open `/dev/kvm' (No such file or directory) > ========> -bash-4.2# virt-host-validate > QEMU: Checking for hardware virtualization > : PASS > QEMU: Checking for device /dev/kvm > : FAIL (Check that the 'kvm-intel' or 'kvm-amd' modules are loaded > & the BIOS has enabled virtualization) > QEMU: Checking for device /dev/vhost-net > : WARN (Load the 'vhost_net' module to improve performance of > virtio networking) > QEMU: Checking for device /dev/net/tun > : FAIL (Load the 'tun' module to enable networking for QEMU guests) > LXC: Checking for Linux >= 2.6.26 > : PASS > ========> > Despite reading from the `systemd-nspawn` man page: > > ". . .kernel modules may not be loaded from within the container." > > I purposefully tried from inside the container:With container based virt there is only one kernel image, so any modules you want must be loaded in the host. Libvirt "passthrough" of char/block devices simply involves libvirt doing mknod in the /dev tmpfs it sets up. The container itself is blocked from doing any 'mknod' calls since that'd be a security risk. Hence you must list any desired device nodes in the XML config. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Apparently Analagous Threads
- Re: Notes on building libguestfs in a systemd-nspawn container
- Re: Notes on building libguestfs in a systemd-nspawn container
- Re: Notes on building libguestfs in a systemd-nspawn container
- Re: Notes on building libguestfs in a systemd-nspawn container
- Notes on building libguestfs in a systemd-nspawn container