Libguestfs - Oct 2010 - [PATCH] Make appliance-building work on systems with default library search paths differing from the appliance's

Systems such as Debian (and thus Ubuntu) operate with 64-bit programs in
/lib, with /lib64 a symlink to it: as a result, they apply a patch to
their glibc adjusting the default search path of the 64-bit dynamic
loader (only used if the library is not found in ld.so.cache) to look in
/lib, and instruct the 32-bit one to look in /lib32. Because fakechroot
is implemented via LD_PRELOAD, it ends up running the native dynamic
loader, and of course the appliance doesn't have an ld.so.cache early in
building. The effect of this is that, while building the appliance, you
are suddenly hit with an error like this:

,----
| Complete!
| ++ basename initramfs/lib/modules/2.6.33.3-85.fc13.x86_64
| + kversion=2.6.33.3-85.fc13.x86_64
| + febootstrap-run initramfs -- /sbin/depmod -a 2.6.33.3-85.fc13.x86_64
| + febootstrap-run initramfs -- mkdir -p --mode=0777 /sysroot
| mkdir: error while loading shared libraries: libselinux.so.1: wrong ELF class:
ELFCLASS32
`----

The 64-bit ld.so hunted in the initramfs's lib/ for 64-bit libraries to
link to... but in the appliance's lib/, 64-bit libraries are found in
lib64!

To work around this for good, explicitly set LD_LIBRARY_PATH
appropriately for the bitness of the current environment (and,
technically, for the distro of the appliance we're building, but that
just means a change to debirf is needed as well: this fix only handles
Fedora- appliance-on-Debian/Ubuntu-system, not vice versa).

(Note: it is quite possible that this is better implemented inside
febootstrap: but it works here.)
---
 appliance/make.sh.in |   63 +++++++++++++++++++++++++++++++++++--------------
 1 files changed, 45 insertions(+), 18 deletions(-)

diff --git a/appliance/make.sh.in b/appliance/make.sh.in
index efd1f46..d492ba5 100755
--- a/appliance/make.sh.in
+++ b/appliance/make.sh.in
@@ -24,6 +24,33 @@ set -e
 set -x
 
 if [ "@DIST@" = "REDHAT" ]; then
+
+  if [[ $(uname -m) =~ .*64 ]]; then
+     
BOOT_LD_LIBRARY_PATH="$(pwd)/../initramfs/lib64:$LD_LIBRARY_PATH"
+  else
+      BOOT_LD_LIBRARY_PATH="$(pwd)/../initramfs/lib:$LD_LIBRARY_PATH"
+  fi
+
+  bootstrap_run()
+  {
+    LD_LIBRARY_PATH=$BOOT_LD_LIBRARY_PATH @FEBOOTSTRAP_RUN@ "$@"
+  }
+
+  xargs0_bootstrap_run()
+  {
+    LD_LIBRARY_PATH=$BOOT_LD_LIBRARY_PATH xargs -0 @FEBOOTSTRAP_RUN@
"$@"
+  }
+
+  xargs_bootstrap_run()
+  {
+    LD_LIBRARY_PATH=$BOOT_LD_LIBRARY_PATH xargs @FEBOOTSTRAP_RUN@
"$@"
+  }
+
+  bootstrap_minimize()
+  {
+    LD_LIBRARY_PATH=$BOOT_LD_LIBRARY_PATH @FEBOOTSTRAP_MINIMIZE@ "$@"
+  }
+
   cd @top_builddir@
   # Decide on names for the final output.  These have to match Makefile.am.
   output=appliance/initramfs. at REPO@. at host_cpu@.img
@@ -44,45 +71,45 @@ if [ "@DIST@" = "REDHAT" ]; then
   # Create modules.dep.  This is only used in the normal appliance
   # (not supermin).
   kversion=$(basename initramfs/lib/modules/*)
-  @FEBOOTSTRAP_RUN@ initramfs -- /sbin/depmod -a $kversion
+  bootstrap_run initramfs -- /sbin/depmod -a $kversion
 
   # /sysroot is where the guest root filesystem will be mounted.
-  @FEBOOTSTRAP_RUN@ initramfs -- mkdir -p --mode=0777 /sysroot
+  bootstrap_run initramfs -- mkdir -p --mode=0777 /sysroot
 
   # Create /tmp if it is missing.
-  @FEBOOTSTRAP_RUN@ initramfs -- mkdir -p --mode=0777 /tmp
+  bootstrap_run initramfs -- mkdir -p --mode=0777 /tmp
 
   # Create /selinux if it is missing.
-  @FEBOOTSTRAP_RUN@ initramfs -- mkdir -p --mode=0755 /selinux
+  bootstrap_run initramfs -- mkdir -p --mode=0755 /selinux
 
   # Nuke some stuff.  The kernel pulls mkinitrd and plymouth which pulls in
   # all of Python.  Sheez.
   (cd initramfs && find -name '*python*' -print0) |
-    xargs -0 @FEBOOTSTRAP_RUN@ initramfs -- rm -rf
+    xargs0_bootstrap_run initramfs -- rm -rf
   (cd initramfs && find -name '*plymouth*' -print0) |
-    xargs -0 @FEBOOTSTRAP_RUN@ initramfs -- rm -rf
+    xargs0_bootstrap_run initramfs -- rm -rf
   (cd initramfs && find -name 'libply-*' -print0) |
-    xargs -0 @FEBOOTSTRAP_RUN@ initramfs -- rm -rf
+    xargs0_bootstrap_run initramfs -- rm -rf
 
   # In Fedora >= 11, it pulls in all of Perl from somewhere.  Nuke from
orbit.
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf /usr/lib/perl5 /usr/lib64/perl5
+  bootstrap_run initramfs -- rm -rf /usr/lib/perl5 /usr/lib64/perl5
 
   # Anaconda?  JPEG images?
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf /usr/lib/anaconda-runtime
+  bootstrap_run initramfs -- rm -rf /usr/lib/anaconda-runtime
 
   # Don't need any firmware.
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf /lib/firmware
+  bootstrap_run initramfs -- rm -rf /lib/firmware
 
   # Don't need any keyboard maps.
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf /lib/kbd
+  bootstrap_run initramfs -- rm -rf /lib/kbd
 
   # Remove anything in home directory.  Because of the potential for disaster
   # we don't put a slash before 'home'.
   (cd initramfs && echo home/*) |
-    xargs @FEBOOTSTRAP_RUN@ initramfs -- rm -rf
+    xargs_bootstrap_run initramfs -- rm -rf
 
   # Remove /var/lib/yum stuff.
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf /var/lib/yum
+  bootstrap_run initramfs -- rm -rf /var/lib/yum
 
   # Remove some unreadable binaries which are incompatible with
   # the supermin appliance.  Since these binaries can't be read
@@ -90,7 +117,7 @@ if [ "@DIST@" = "REDHAT" ]; then
   # appliance at run time.  XXX Need a better fix for this.
   # Probably we should change febootstrap-supermin-helper to just
   # ignore such files.
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -f \
+  bootstrap_run initramfs -- rm -f \
     /usr/bin/chfn \
     /usr/bin/chsh \
     /usr/libexec/pt_chown \
@@ -139,15 +166,15 @@ if [ "@DIST@" = "REDHAT" ]; then
 
   (cd initramfs && \
     find lib/modules/*/kernel -name '*.ko' $whitelist -a -print0 ) |
-    xargs -0 febootstrap-run initramfs -- rm
+    xargs0_bootstrap_run initramfs -- rm
 
   # Pull the kernel out into the current directory.  We don't want it in
   # the initramfs image.
   cp initramfs/boot/vmlinuz* $koutput
-  @FEBOOTSTRAP_RUN@ initramfs -- rm -rf boot
+  bootstrap_run initramfs -- rm -rf boot
 
   # Minimize the image.
-  @FEBOOTSTRAP_MINIMIZE@ initramfs
+  bootstrap_minimize initramfs
 
   # Add some missing configuration files.
   if [ ! -f initramfs/etc/hosts ]; then
@@ -160,7 +187,7 @@ __EOF__
   fi
 
   if [ ! -f initramfs/etc/fstab ]; then
-      @FEBOOTSTRAP_RUN@ initramfs -- touch /etc/fstab
+      bootstrap_run initramfs -- touch /etc/fstab
   fi
 
   echo nameserver 169.254.2.3 > resolv.conf.new
-- 
1.7.3.125.g963bc.dirty

This needs to be fixed in febootstrap.  I'll have a look at it
tomorrow.

Any reason not to be building with debootstrap + debirf on Debian?  It
produces a native Debian appliance, and modulo continual bugs in
debootstrap & debirf it does work.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v