Libguestfs - Aug 2011 - Plans for libguestfs

On Thu, Aug 04, 2011 at 01:04:40AM +0200, Hilko Bengen
wrote:
> During last week, I finally got some packages for libguestfs and
> bindings done. The repository at
> git://anonscm.debian.org/pkg-libvirt/libguestfs.git has been updated to
> 1.10.6-2 which I have also uploaded to experimental. (The packages
> haven't appeared in the archive yet, though.) 
> 
> Before shifting my attention to the 1.12 release, I'd like to make sure
> that things work as expected. Please test the packages. I haven't had
> any opportunity to test anything so far because during this last week I
> lacked access to a KVM-capable machine.
> 
> An outstanding issue is how the appliance, the archive of files, mostly
> executables and libraries, needed to operate the VM, should be handled:
> Having it generated at build-time and shipping it as part of a .deb
> would be suboptimal both from a security and a maintenance perspective.
> For now, I have modified the build system a bit so that the build script
> and package lists are generated. I have added a README.Debian that tells
> the user how to call what script to manually generate the appliance
> after the package has been installed.
I'm interested to know what the security/maintenance objections are.
We ship the *supermin* appliance in Fedora, precisely because it
improves security and maintenance.

One specific issue is that your users will need to rebuild the
appliance after installing any security patch.  With the supermin
appliance, there's no need to do this -- updated binaries or libraries
are included automatically next time libguestfs is launched.
> I think that there should be a /usr/sbin/update-guestfs-appliance script
> that may be called from postinst if the user asks via Debconf.
> 
> Another issue has to do with the Python bindings: The
> autoconf/automake-based build system will only build the bindings for
> one Python version (which happens to be 2.6 at the moment). I would like
> to change that, but I haven't found an easy way short of rebuilding
> (almost) the wohle package for every Python version.
I guess we only have one version of Python at a time in Fedora.

Could we do something like adding a ./configure --disable-library
option, allowing you to disable everything (except Python) and thus
just rebuild Python bindings?
> Any comments and patches/contributions are welcome, of course.
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

On Thu, Aug 04, 2011 at 01:04:40AM +0200, Hilko Bengen
wrote:
> Before shifting my attention to the 1.12 release, I'd like to make sure
> that things work as expected. Please test the packages. I haven't had
> any opportunity to test anything so far because during this last week I
> lacked access to a KVM-capable machine.
On this point, testing shouldn't need a KVM-capable machine (although
for sure it's slower without!)

For Fedora we run the complete test suite ("make check") at build
time.  This takes about an hour.  It runs in the context of our
builders which don't have KVM [because the builders are Xen guests].

Another set of tests that I run myself after the build is to just make
sure the package functions on my laptop.  I do some combination of:

  libguestfs-test-tool
  guestfish -N fs -m /dev/sda1  # and type some interactive commands

This second set of informal testing doesn't need KVM either.
Eventually we plan to use our Autotest-based infrastructure to
formalize the second set of tests.  So post-build, the package will be
grabbed by Fedora AutoQA, installed, and these rudimentary tests run.

For RHEL we have a huge, manually-driven set of tests run by our QA
team.  It takes them 12 hours to run the whole thing.  These tests
aren't open source.  Red Hat open sourced the libvirt QA tests
recently, and I guess we might do the libguestfs tests at some point
in the future.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

On Thu, Aug 04, 2011 at 08:27:48AM +0100, Richard W.M. Jones
wrote:
> On Thu, Aug 04, 2011 at 01:04:40AM +0200, Hilko Bengen wrote:
> > I think that there should be a /usr/sbin/update-guestfs-appliance
script
> > that may be called from postinst if the user asks via Debconf.
> > 
> > Another issue has to do with the Python bindings: The
> > autoconf/automake-based build system will only build the bindings for
> > one Python version (which happens to be 2.6 at the moment). I would
like
> > to change that, but I haven't found an easy way short of
rebuilding
> > (almost) the wohle package for every Python version.
> 
> I guess we only have one version of Python at a time in Fedora.
Fedora in general has both Python 2.x and 3.x availabl, and the overall
goal is for all packages which include python modules to ship a version
for each python. So if libguestfs gains support to build both 2.x and 3.x
versions of its python binding, that'd be useful for Fedora as well as
Debian.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

* Richard W.M. Jones:
> I'm interested to know what the security/maintenance objections are.
> We ship the *supermin* appliance in Fedora, precisely because it
> improves security and maintenance.
Ah well, it seems that I misunderstood what base.img is for. For some
reason I believed that executables and libraries are extracted from the
downloaded binary packages and copied into the image.

What was I thinking? Sorry for the noise. :-)

With the question about security out of the way (for which I'm glad), I
still think that it would be best to build supermin.d/base.img and
supermin.d/hostfiles at installation time -- and to provide a documented
way for the user to rebuild them.
> I guess we only have one version of Python at a time in Fedora.
>
> Could we do something like adding a ./configure --disable-library
> option, allowing you to disable everything (except Python) and thus
> just rebuild Python bindings?
Is it even possible to build the Python bindings without the libguestfs
library being available through libtool?

Speaking of Python bindings... is it possible to convince libtool to
build just a .so file, without .0.0 extension and symlinks? (As I
reported a bug in dh_python2, it was pointed out to me that this was
wrong.)

Cheers,
-Hilko

On Thu, Aug 04, 2011 at 09:01:02PM +0200, Hilko Bengen
wrote:
> * Richard W.M. Jones:
> 
> > I'm interested to know what the security/maintenance objections
are.
> > We ship the *supermin* appliance in Fedora, precisely because it
> > improves security and maintenance.
> 
> Ah well, it seems that I misunderstood what base.img is for. For some
> reason I believed that executables and libraries are extracted from the
> downloaded binary packages and copied into the image.
> 
> What was I thinking? Sorry for the noise. :-)
> 
> With the question about security out of the way (for which I'm glad), I
> still think that it would be best to build supermin.d/base.img and
> supermin.d/hostfiles at installation time -- and to provide a documented
> way for the user to rebuild them.
Hmmm maybe rerunning febootstrap?  They will need a network connection
and it may be error prone, but it could work.
> > I guess we only have one version of Python at a time in Fedora.
> >
> > Could we do something like adding a ./configure --disable-library
> > option, allowing you to disable everything (except Python) and thus
> > just rebuild Python bindings?
> 
> Is it even possible to build the Python bindings without the libguestfs
> library being available through libtool?
Yes, you may be right about that.

OTOH if you disable the appliance and the daemon and all the language
bindings except Python, libguestfs builds are a bit quicker:

time sh -c './configure --disable-appliance --disable-daemon --disable-ocaml
--disable-perl --disable-fuse --disable-ruby --disable-haskell --disable-php
--with-java-home=no && make clean && make'
4m33s

That's still building the tools though, so we should have a
--disable-tools option.
> Speaking of Python bindings... is it possible to convince libtool to
> build just a .so file, without .0.0 extension and symlinks? (As I
> reported a bug in dh_python2, it was pointed out to me that this was
> wrong.)
No idea ..  We copied the code for building the Python bindings from
libvirt, so we probably inherited any bugs it has too.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora