Libguestfs - Mar 2015 - Re: [virt-tools-list] Statically compiling virt-rescue ?

[Sorry - I didn't see this email until now as virt-rescue is part of
the http://libguestfs.org project and so normally uses a different
mailing list]

On Tue, Mar 17, 2015 at 11:29:29AM +0000, Keith Roberts
wrote:
> Hi all.
>
> I’d like to create a statically-linked version of virt-rescue that I
> can compile on my laptop then deploy that binary locally onto a xen
> server hosting VM’s to run virt-rescue on the VM images from there
> locally.
Note that as well as the rather large list of shared libraries,
virt-rescue has a few other dependencies:

 - a non-Xen kernel
 - supermin
 - qemu (even required when the hypervisor is Xen)

[...]
> Is it possible to statically compile virt-rescue with all the above
> required libraries built into the executable, along the lines of the
> mysql statically-linked binary?
I made a pretty valiant attempt to build a static libguestfs +
virt-rescue:

  $ file rescue/virt-rescue
  rescue/virt-rescue: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux),
statically linked, for GNU/Linux 2.6.32,
BuildID[sha1]=c0151b6bc99684e970ebad993c4985ee90afb319, not stripped
  $ ll rescue/virt-rescue
  -rwxrwxr-x. 1 rjones rjones 8009264 Mar 27 22:50 rescue/virt-rescue

Unfortunately it fundamentally doesn't work, because none of the
`__attribute__((constructor))'s in the code are ever called[1].  This
means, for example, that no backends get registered with the main
code.

This is a bug, but not one that anyone has noticed until now.

Rich.

[1]
https://stackoverflow.com/questions/1202494/why-doesnt-attribute-constructor-work-in-a-static-library

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

I've added a few patches upstream to make static linking work, and it
is [surprisingly] possible to get it to work.

You'll have to check out libguestfs from git.  Read the README file
very carefully, and get familiar with building from source.  This is
easier on a recent Fedora machine or VM.

Once you are comfortable building libguestfs from source, configure it
like this:

  ./configure --without-libvirt --enable-static
  make clean
  make

To build a static virt-rescue you'll need to first build or install
the following static libraries:

         *-static in Fedora?
  file          no              self-built
  fuse          no              self-built
  glibc         yes
  libconfig     no              self-built
  pcre          yes
  xz (liblzma)  yes
  yajl          no              self-built
  zlib          yes

and then build a static virt-rescue binary like this:

  make -C rescue clean
  make -C rescue all CFLAGS=-static LDFLAGS=-all-static V=1

$ ll rescue/virt-rescue
-rwxrwxr-x. 1 rjones rjones 8170304 Mar 28 10:27 rescue/virt-rescue
$ file rescue/virt-rescue
rescue/virt-rescue: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux),
statically linked, for GNU/Linux 2.6.32,
BuildID[sha1]=7de92afbc9da27ea97d6312638b4f6f368dfb628, not stripped

Now build a fixed appliance[1]:

  mkdir /tmp/appliance
  ./run libguestfs-make-fixed-appliance /tmp/appliance

Now you have all the bits required for a static virt-rescue.

I'll assume that qemu exists on the Xen server, or you can just
install that from a package, or build a recent qemu from source on the
Xen server.  You'll need a recent qemu, so if the Xen server has some
old qemu then it's not going to work - build qemu from source in that
case.

Copy the virt-rescue binary and the fixed appliance directory to the
Xen server.

Note that the fixed appliance contains a large sparse file ('root'),
and if you don't use the appropriate method to copy it, then you'll
end up making it non-sparse and taking a large amount of space.

Run it like this:

  export LIBGUESTFS_QEMU=/path/to/qemu
  export LIBGUESTFS_PATH=/path/to/appliance
  ./virt-rescue [normal parameters ...]

If it doesn't work, then also set:

  export LIBGUESTFS_DEBUG=1
  export LIBGUESTFS_TRACE=1

Good luck!

Rich.

[1] http://libguestfs.org/libguestfs-make-fixed-appliance.1.html

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

I forgot one thing.  You have to apply the following patch/hack.

diff --git a/rescue/Makefile.am b/rescue/Makefile.am
index eab06e9..49568e7 100644
--- a/rescue/Makefile.am
+++ b/rescue/Makefile.am
@@ -60,6 +60,7 @@ virt_rescue_LDADD = \
 	$(LIBXML2_LIBS) \
 	$(LIBVIRT_LIBS) \
 	$(LTLIBINTL) \
+	-lyajl_s -lpcre -lmagic -lfuse -llzma -lz -ldl -lm \
 	../gnulib/lib/libgnu.la
 
 # Manual pages and HTML files for the website.


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/