Libguestfs - May 2010 - [PATCH febootstrap] Pull in febootstrap-supermin-helper (C version) from libguestfs.

This patch is FYI as it needs more testing, although feel free to take
a look at it if you want.

Background: We'd like to change libguestfs so that the appliance can
support optional features.  This would mean you could have (for
example) a 'libguestfs' base package and a 'libguestfs-xfs'
package
providing optional support for XFS filesystem tools.  This would only
apply to the supermin case.

Currently the supermin appliance is built out of 3 files:

  $ ls -1 /usr/lib64/guestfs/
  initramfs.fedora-12.x86_64.supermin.hostfiles
      # text file containing wildcards of files picked up from
      # the host at runtime

  initramfs.fedora-12.x86_64.supermin.img
      # base appliance data (cpio file)

  kmod.whitelist
      # text file containing wildcards matching kernel modules

There are a number of problems with this: (1) The files have fixed
names.  (2) The names are rather obscure.  (3) You can't have optional
appliance modules.

The idea would be to replace this with a scheme where the supermin
builder looks in a directory and picks out all the files it needs from
there.  It would look something like:

  $ ls -1 /usr/lib64/guestfs/supermin.d/
  base.img
  hostfiles
  optional-xfs.img     # installed by libguestfs-xfs package
  optional-zfs.img     # installed by libguestfs-zfs package

(Similarly the kernel module whitelist could point to a directory, but
we don't need that now and aren't implementing that right away).

At the same time we have for a while been maintaining some significant
duplicate functionality in febootstrap, cf:

http://git.annexia.org/?p=libguestfs.git;a=blob;f=appliance/supermin-split.sh.in;h=44cfe21723d94c7c927dfc5f8ae27f27dc8b8462;hb=HEAD#l50
http://git.annexia.org/?p=febootstrap.git;a=blob;f=febootstrap-to-supermin.sh;h=eccf18e290261485d0b241eb5558e38a566ee0ba;hb=HEAD#l76

and:

http://git.annexia.org/?p=libguestfs.git;a=blob;f=appliance/libguestfs-supermin-helper.c;h=127b5c0ebcfa41cbd7776440a8ba7bdabd1ff1c8;hb=HEAD
http://git.annexia.org/?p=febootstrap.git;a=blob;f=febootstrap-supermin-helper.sh;h=cd5cf19b096130a6d975830333555039b74b66b6;hb=HEAD

Since this is a major change to the way the appliance is built, it
makes sense at the same time to fold everything into febootstrap and
drop the custom libguestfs/appliance/*supermin* scripts.

This patch is the first part of this plan, porting the relevant
functionality from libguestfs into febootstrap.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
-------------- next part --------------
>From d7cae6f9d726ac693ee2b95592c14c8c9fc51582 Mon Sep 17 00:00:00 2001
From: Richard Jones <rjones at redhat.com>
Date: Thu, 13 May 2010 13:29:20 +0100
Subject: [PATCH] Pull in febootstrap-supermin-helper (C version) from
libguestfs.

This commit also pulls in the automake C dependencies and gnulib.

febootstrap-supermin-helper is modified so that it has the
--kmods option (to read the whitelist, from the old shell script),
and so that it can read supermin appliances composed of multiple
parts from out of a directory.
---
 .gitignore                      |   30 +-
 Makefile.am                     |   19 +-
 README                          |    2 +-
 autogen.sh                      |    4 +
 configure.ac                    |   23 +-
 febootstrap-supermin-helper.c   |  996 +++++++++++++++++++++++++++++++++++++++
 febootstrap-supermin-helper.pod |   14 +-
 febootstrap-supermin-helper.sh  |  143 ------
 febootstrap-to-supermin.pod     |   19 +-
 lib/.gitignore                  |  133 ++++++
 m4/.gitignore                   |   90 ++++
 m4/gnulib-cache.m4              |   41 ++
 12 files changed, 1346 insertions(+), 168 deletions(-)
 create mode 100644 febootstrap-supermin-helper.c
 delete mode 100755 febootstrap-supermin-helper.sh
 create mode 100644 lib/.gitignore
 create mode 100644 m4/.gitignore
 create mode 100644 m4/gnulib-cache.m4

diff --git a/.gitignore b/.gitignore
index a430f1f..310aa6e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,16 +1,22 @@
 *~
-febootstrap*.8
-febootstrap*.txt
-febootstrap-*.tar.gz
+*.o
+*.a
+.deps
 Makefile.in
 Makefile
 aclocal.m4
+/arg-nonnull.h
 autom4te.cache
+/c++defs.h
+compile
+config.guess
 config.h.in
 config.h
 config.log
 config.status
+config.sub
 configure
+depcomp
 febootstrap
 febootstrap-run
 febootstrap-install
@@ -18,9 +24,27 @@ febootstrap-minimize
 febootstrap-to-initramfs
 febootstrap-to-supermin
 febootstrap-supermin-helper
+febootstrap*.8
+febootstrap*.txt
+febootstrap-*.tar.gz
+lib/alloca.h
+lib/arg-nonnull.h
+lib/c++defs.h
+lib/dirent.h
+lib/fcntl.h
+lib/stdio.h
+lib/stdlib.h
+lib/string.h
+lib/sys/
+lib/time.h
+lib/unistd.h
+lib/warn-on-use.h
+lib/wchar.h
+INSTALL
 install-sh
 missing
 stamp-h1
+/warn-on-use.h
 examples/guestfs
 examples/*-initrd.img
 examples/guest-image
diff --git a/Makefile.am b/Makefile.am
index 861b5e0..bb3b505 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,5 +1,5 @@
 # febootstrap Makefile.am
-# (C) Copyright 2009 Red Hat Inc.
+# (C) Copyright 2009-2010 Red Hat Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -17,7 +17,9 @@
 #
 # Written by Richard W.M. Jones <rjones at redhat.com>
 
-SUBDIRS = examples
+ACLOCAL_AMFLAGS = -I m4
+
+SUBDIRS = lib examples
 
 bin_SCRIPTS = \
 	febootstrap \
@@ -25,7 +27,8 @@ bin_SCRIPTS = \
 	febootstrap-install \
 	febootstrap-minimize \
 	febootstrap-to-initramfs \
-	febootstrap-to-supermin \
+	febootstrap-to-supermin
+bin_PROGRAMS = \
 	febootstrap-supermin-helper
 DISTCLEANFILES = $(bin_SCRIPTS)
 
@@ -65,11 +68,9 @@ febootstrap-to-supermin: febootstrap-to-supermin.sh
 	chmod 0555 $@-t
 	mv $@-t $@
 
-febootstrap-supermin-helper: febootstrap-supermin-helper.sh
-	rm -f $@
-	cp $< $@-t
-	chmod 0555 $@-t
-	mv $@-t $@
+febootstrap_supermin_helper_SOURCES = febootstrap-supermin-helper.c
+febootstrap_supermin_helper_CFLAGS = -Wall -Ilib
+febootstrap_supermin_helper_LDADD = $(LTLIBINTL) -Llib -lgnu
 
 man_MANS = \
 	febootstrap.8 \
@@ -174,4 +175,4 @@ EXTRA_DIST = \
 	  febootstrap-to-supermin.sh \
 	febootstrap-supermin-helper.8 febootstrap-supermin-helper.txt \
 	  febootstrap-supermin-helper.pod \
-	  febootstrap-supermin-helper.sh
+	m4/gnulib-cache.m4
diff --git a/README b/README
index 38afbb3..53275ac 100644
--- a/README
+++ b/README
@@ -32,7 +32,7 @@ Requirements
 
   bash
 
-  MAKEDEV
+  gcc
 
   qemu
     - If you want to test-run your systems.
diff --git a/autogen.sh b/autogen.sh
index 32f085b..0ab1868 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -1,5 +1,9 @@
 #!/bin/sh -
 
+if [ -d ../gnulib ]; then
+    ../gnulib/gnulib-tool --update
+fi
+
 export AUTOMAKE='automake --foreign --add-missing'
 autoreconf
 ./configure "$@"
diff --git a/configure.ac b/configure.ac
index f564048..05597b1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,5 @@
 dnl febootstrap configure.ac
-dnl (C) Copyright 2009 Red Hat Inc.
+dnl (C) Copyright 2009-2010 Red Hat Inc.
 dnl
 dnl This program is free software; you can redistribute it and/or modify
 dnl it under the terms of the GNU General Public License as published by
@@ -17,9 +17,26 @@ dnl Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 dnl
 dnl Written by Richard W.M. Jones <rjones at redhat.com>
 
-AC_INIT(febootstrap,2.6)
+AC_INIT(febootstrap,2.7)
 AM_INIT_AUTOMAKE
 
+dnl Check for basic C environment.
+AC_PROG_CC_STDC
+gl_EARLY
+
+AC_PROG_INSTALL
+AC_PROG_CPP
+
+AC_C_PROTOTYPES
+test "x$U" != "x" && AC_MSG_ERROR([Compiler not
ANSI compliant])
+
+AM_PROG_CC_C_O
+
+dnl Check support for 64 bit file offsets.
+AC_SYS_LARGEFILE
+
+gl_INIT
+
 AC_CHECK_PROG(PERLDOC,[perldoc],[perldoc],[no])
 if test "x$PERLDOC" = "xno" ; then
   AC_MSG_WARN([perldoc not found - install perl to make man pages])
@@ -42,5 +59,5 @@ if test "x$YUM" = "xno" ; then
 fi
 
 AC_CONFIG_HEADERS([config.h])
-AC_CONFIG_FILES([Makefile examples/Makefile])
+AC_CONFIG_FILES([lib/Makefile Makefile examples/Makefile])
 AC_OUTPUT
diff --git a/febootstrap-supermin-helper.c b/febootstrap-supermin-helper.c
new file mode 100644
index 0000000..b7a19b2
--- /dev/null
+++ b/febootstrap-supermin-helper.c
@@ -0,0 +1,996 @@
+/* febootstrap-supermin-helper reimplementation in C.
+ * Copyright (C) 2009-2010 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/* This script builds the supermin appliance on the fly each
+ * time the appliance runs.
+ *
+ * *NOTE*: This program is designed to be very short-lived, and so we
+ * don't normally bother to free up any memory that we allocate.
+ * That's not completely true - we free up stuff if it's obvious and
+ * easy to free up, and ignore the rest.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <stdint.h>
+#include <string.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <getopt.h>
+#include <limits.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/time.h>
+#include <sys/stat.h>
+#include <assert.h>
+
+#include "error.h"
+#include "filevercmp.h"
+#include "fts_.h"
+#include "full-write.h"
+#include "hash.h"
+#include "hash-pjw.h"
+#include "xalloc.h"
+#include "xvasprintf.h"
+
+/* Directory containing candidate kernels.  We could make this
+ * configurable at some point.
+ */
+#define KERNELDIR "/boot"
+#define MODULESDIR "/lib/modules"
+
+/* Buffer size used in copy operations throughout.  Large for
+ * greatest efficiency.
+ */
+#define BUFFER_SIZE 65536
+
+static struct timeval start_t;
+static int verbose = 0;
+
+static void print_timestamped_message (const char *fs, ...);
+static const char *create_kernel (const char *hostcpu, const char *kernel);
+static void create_appliance (char **inputs, int nr_inputs, const char
*whitelist, const char *modpath, const char *initrd);
+
+enum { HELP_OPTION = CHAR_MAX + 1 };
+
+static const char *options = "k:vV";
+static const struct option long_options[] = {
+  { "help", 0, 0, HELP_OPTION },
+  { "kmods", 0, 0, 'k' },
+  { "verbose", 0, 0, 'v' },
+  { "version", 0, 0, 'V' },
+  { 0, 0, 0, 0 }
+};
+
+static void
+usage (const char *progname)
+{
+  printf ("%s: build the supermin appliance on the fly\n"
+          "\n"
+          "Usage:\n"
+          "  %s [-options] inputs [...] whitelist host_cpu kernel
initrd\n"
+          "  %s --help\n"
+          "  %s --version\n"
+          "\n"
+          "This script is used by febootstrap to build the supermin
appliance\n"
+          "(kernel and initrd output files).  You should NOT need to run
this\n"
+          "program directly except if you are debugging tricky
supermin\n"
+          "appliance problems.\n"
+          "\n"
+          "NB: The kernel and initrd parameters are OUTPUT parameters. 
If\n"
+          "those files exist, they are overwritten by the output.\n"
+          "\n"
+          "Options:\n"
+          "  --help\n"
+          "       Display this help text and exit.\n"
+          "  -k file | --kmods file\n"
+          "       Specify kernel module whitelist.\n"
+          "  --verbose | -v\n"
+          "       Enable verbose messages (give multiple times for more
verbosity).\n"
+          "  --version | -V\n"
+          "       Display version number and exit.\n",
+          progname, progname, progname, progname);
+}
+
+int
+main (int argc, char *argv[])
+{
+  /* First thing: start the clock. */
+  gettimeofday (&start_t, NULL);
+
+  const char *whitelist = NULL;
+
+  /* Command line arguments. */
+  for (;;) {
+    int c = getopt_long (argc, argv, options, long_options, NULL);
+    if (c == -1) break;
+
+    switch (c) {
+    case HELP_OPTION:
+      usage (argv[0]);
+      exit (EXIT_SUCCESS);
+
+    case 'k':
+      whitelist = optarg;
+      break;
+
+    case 'v':
+      verbose++;
+      break;
+
+    case 'V':
+      printf (PACKAGE_NAME " " PACKAGE_VERSION "\n");
+      exit (EXIT_SUCCESS);
+
+    default:
+      usage (argv[0]);
+      exit (EXIT_FAILURE);
+    }
+  }
+
+  char **inputs = &argv[optind];
+  int nr_inputs = argc - optind - 3;
+
+  if (nr_inputs < 1) {
+    usage (argv[0]);
+    exit (EXIT_FAILURE);
+  }
+
+  /* See: https://bugzilla.redhat.com/show_bug.cgi?id=558593 */
+  const char *hostcpu = argv[argc-3];
+
+  /* Output files. */
+  const char *kernel = argv[argc-2];
+  const char *initrd = argv[argc-1];
+
+  if (verbose) {
+    print_timestamped_message ("whitelist = %s, "
+                               "host_cpu = %s, "
+                               "kernel = %s, "
+                               "initrd = %s",
+                               whitelist ? : "(not specified)",
+                               hostcpu, kernel, initrd);
+    int i;
+    for (i = 0; i < nr_inputs; ++i)
+      print_timestamped_message ("inputs[%d] = %s", i, inputs[i]);
+  }
+
+  /* Remove the output files if they exist. */
+  unlink (kernel);
+  unlink (initrd);
+
+  /* Create kernel output file. */
+  const char *modpath;
+  modpath = create_kernel (hostcpu, kernel);
+
+  if (verbose)
+    print_timestamped_message ("finished creating kernel");
+
+  /* Create the appliance. */
+  create_appliance (inputs, nr_inputs, whitelist, modpath, initrd);
+
+  if (verbose)
+    print_timestamped_message ("finished creating appliance");
+
+  exit (EXIT_SUCCESS);
+}
+
+/* Compute Y - X and return the result in milliseconds.
+ * Approximately the same as this code:
+ * http://www.mpp.mpg.de/~huber/util/timevaldiff.c
+ */
+static int64_t
+timeval_diff (const struct timeval *x, const struct timeval *y)
+{
+  int64_t msec;
+
+  msec = (y->tv_sec - x->tv_sec) * 1000;
+  msec += (y->tv_usec - x->tv_usec) / 1000;
+  return msec;
+}
+
+static void
+print_timestamped_message (const char *fs, ...)
+{
+  struct timeval tv;
+  gettimeofday (&tv, NULL);
+
+  va_list args;
+  char *msg;
+  int err;
+
+  va_start (args, fs);
+  err = vasprintf (&msg, fs, args);
+  va_end (args);
+
+  if (err < 0) return;
+
+  fprintf (stderr, "supermin helper [%05" PRIi64 "ms]
%s\n",
+           timeval_diff (&start_t, &tv), msg);
+
+  free (msg);
+}
+
+static char **read_dir (const char *dir);
+static char **filter_fnmatch (char **strings, const char *patt, int flags);
+static char **filter_notmatching_substring (char **strings, const char *sub);
+static void sort (char **strings, int (*compare) (const void *, const void *));
+static int isdir (const char *path);
+
+static int
+reverse_filevercmp (const void *p1, const void *p2)
+{
+  const char *s1 = * (char * const *) p1;
+  const char *s2 = * (char * const *) p2;
+
+  /* Note, arguments are reversed to achieve a reverse sort. */
+  return filevercmp (s2, s1);
+}
+
+/* Create the kernel.  This chooses an appropriate kernel and makes a
+ * symlink to it.
+ *
+ * Look for the most recent kernel named vmlinuz-*.<arch>* which has a
+ * corresponding directory in /lib/modules/. If the architecture is
+ * x86, look for any x86 kernel.
+ *
+ * RHEL 5 didn't append the arch to the kernel name, so look for
+ * kernels without arch second.
+ *
+ * If no suitable kernel can be found, exit with an error.
+ *
+ * This function returns the module path (ie. /lib/modules/<version>).
+ */
+static const char *
+create_kernel (const char *hostcpu, const char *kernel)
+{
+  char **all_files = read_dir (KERNELDIR);
+
+  /* In original: ls -1dvr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen
*/
+  const char *patt;
+  if (hostcpu[0] == 'i' && hostcpu[2] == '8' &&
hostcpu[3] == '6' &&
+      hostcpu[4] == '\0')
+    patt = "vmlinuz-*.i?86*";
+  else
+    patt = xasprintf ("vmlinuz-*.%s*", hostcpu);
+
+  char **candidates;
+  candidates = filter_fnmatch (all_files, patt, FNM_NOESCAPE);
+  candidates = filter_notmatching_substring (candidates, "xen");
+
+  if (candidates[0] == NULL) {
+    /* In original: ls -1dvr /boot/vmlinuz-* 2>/dev/null | grep -v xen */
+    patt = "vmlinuz-*";
+    candidates = filter_fnmatch (all_files, patt, FNM_NOESCAPE);
+    candidates = filter_notmatching_substring (candidates, "xen");
+
+    if (candidates[0] == NULL)
+      goto no_kernels;
+  }
+
+  sort (candidates, reverse_filevercmp);
+
+  /* Choose the first candidate which has a corresponding /lib/modules
+   * directory.
+   */
+  int i;
+  for (i = 0; candidates[i] != NULL; ++i) {
+    if (verbose >= 2)
+      fprintf (stderr, "candidate kernel: " KERNELDIR
"/%s\n", candidates[i]);
+
+    /* Ignore "vmlinuz-" at the beginning of the kernel name. */
+    const char *version = &candidates[i][8];
+
+    /* /lib/modules/<version> */
+    char *modpath = xasprintf (MODULESDIR "/%s", version);
+
+    if (verbose >= 2)
+      fprintf (stderr, "checking modpath %s is a directory\n",
modpath);
+
+    if (isdir (modpath)) {
+      if (verbose >= 2)
+        fprintf (stderr, "picked %s because modpath %s exists\n",
+                 candidates[i], modpath);
+
+      char *tmp = xasprintf (KERNELDIR "/%s", candidates[i]);
+
+      if (verbose >= 2)
+        fprintf (stderr, "creating symlink %s -> %s\n", kernel,
tmp);
+
+      if (symlink (tmp, kernel) == -1)
+        error (EXIT_FAILURE, errno, "symlink kernel");
+
+      free (tmp);
+
+      return modpath;
+    }
+  }
+
+  /* Print more diagnostics here than the old script did. */
+ no_kernels:
+  fprintf (stderr,
+           "febootstrap-supermin-helper: failed to find a suitable
kernel.\n"
+           "I looked for kernels in " KERNELDIR " and modules in
" MODULESDIR
+           ".\n"
+           "If this is a Xen guest, and you only have Xen domU
kernels\n"
+           "installed, try installing a fullvirt kernel (only for\n"
+           "febootstrap use, you shouldn't boot the Xen guest with
it).\n");
+  exit (EXIT_FAILURE);
+}
+
+static void iterate_inputs (char **inputs, int nr_inputs);
+static void iterate_input_directory (const char *dirname, int dirfd);
+static void write_kernel_modules (const char *whitelist, const char *modpath);
+static void write_hostfiles (const char *hostfiles_file);
+static void write_to_fd (const void *buffer, size_t len);
+static void write_file_to_fd (const char *filename);
+static void write_file_len_to_fd (const char *filename, size_t len);
+static void write_padding (size_t len);
+static char **load_file (const char *filename);
+static void cpio_append_fts_entry (FTSENT *entry);
+static void cpio_append_stat (const char *filename, struct stat *);
+static void cpio_append (const char *filename);
+static void cpio_append_trailer (void);
+
+static int out_fd = -1;
+static off_t out_offset = 0;
+
+/* Create the appliance.
+ *
+ * The initrd consists of these components concatenated together:
+ *
+ * (1) The base skeleton appliance that we constructed at build time.
+ *     format = plain cpio
+ * (2) The host files which match wildcards in *.supermin.hostfiles.
+ *     input format = plain text, output format = plain cpio
+ * (3) The modules from modpath which are on the module whitelist.
+ *     output format = plain cpio
+ *
+ * The original shell scripted used the external cpio program to
+ * create parts (2) and (3), but we have decided it's going to be
+ * faster if we just write out the data outselves.  The reasons are
+ * that external cpio is slow (particularly when used with SELinux
+ * because it does 512 byte reads), and the format that we're writing
+ * is narrow and well understood, because we only care that the Linux
+ * kernel can read it.
+ *
+ * This version contains some improvements over the C version written
+ * for libguestfs, in that we can have multiple base images (or
+ * hostfiles) or use a directory to store these files.
+ */
+static void
+create_appliance (char **inputs, int nr_inputs,
+                  const char *whitelist,
+                  const char *modpath,
+                  const char *initrd)
+{
+  out_fd = open (initrd, O_WRONLY | O_CREAT | O_TRUNC | O_NOCTTY, 0644);
+  if (out_fd == -1)
+    error (EXIT_FAILURE, errno, "open: %s", initrd);
+  out_offset = 0;
+
+  iterate_inputs (inputs, nr_inputs);
+
+  /* Kernel modules (3). */
+  write_kernel_modules (whitelist, modpath);
+
+  cpio_append_trailer ();
+
+  /* Finish off and close output file. */
+  if (close (out_fd) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", initrd);
+}
+
+/* Iterate over the inputs to find out what they are, visiting
+ * directories if specified.
+ */
+static void
+iterate_inputs (char **inputs, int nr_inputs)
+{
+  int i;
+  for (i = 0; i < nr_inputs; ++i) {
+    if (verbose)
+      print_timestamped_message ("visiting %s", inputs[i]);
+
+    int fd = open (inputs[i], O_RDONLY);
+    if (fd == -1)
+      error (EXIT_FAILURE, errno, "open: %s", inputs[i]);
+
+    struct stat statbuf;
+    if (fstat (fd, &statbuf) == -1)
+      error (EXIT_FAILURE, errno, "fstat: %s", inputs[i]);
+
+    /* Directory? */
+    if (S_ISDIR (statbuf.st_mode))
+      iterate_input_directory (inputs[i], fd);
+    else if (S_ISREG (statbuf.st_mode)) {
+      /* Is it a cpio file? */
+      char buf[6];
+      if (read (fd, buf, 6) == 6 && memcmp (buf, "070701", 6)
== 0)
+        /* Yes, a cpio file.  This is a skeleton appliance, case (1). */
+        write_file_to_fd (inputs[i]);
+      else
+        /* No, must be hostfiles, case (2). */
+        write_hostfiles (inputs[i]);
+    }
+    else
+      error (EXIT_FAILURE, 0, "%s: input is not a regular file or
directory",
+             inputs[i]);
+
+    close (fd);
+  }
+}
+
+static void
+iterate_input_directory (const char *dirname, int dirfd)
+{
+  char path[PATH_MAX];
+  strcpy (path, dirname);
+  size_t len = strlen (dirname);
+  char *inputs[] = { path };
+
+  DIR *dir = fdopendir (dirfd);
+  if (dir == NULL)
+    error (EXIT_FAILURE, errno, "fdopendir: %s", dirname);
+
+  struct dirent *d;
+  errno = 0;
+  while ((d = readdir (dir)) != NULL) {
+    if (d->d_name[0] == '.') /* ignore ., .. and any hidden files.
*/
+      continue;
+
+    strcpy (&path[len], d->d_name);
+    iterate_inputs (inputs, 1);
+  }
+
+  if (errno != 0)
+    error (EXIT_FAILURE, errno, "readdir: %s", dirname);
+
+  if (closedir (dir) == -1)
+    error (EXIT_FAILURE, errno, "closedir: %s", dirname);
+}
+
+/* Copy kernel modules.
+ *
+ * Find every file under modpath.
+ *
+ * Exclude all *.ko files, *except* ones which match names in
+ * the whitelist (which may contain wildcards).  Include all
+ * other files.
+ *
+ * Add chosen files to the output.
+ *
+ * whitelist_file may be NULL, to include ALL kernel modules.
+ */
+static void
+write_kernel_modules (const char *whitelist_file, const char *modpath)
+{
+  char **whitelist = NULL;
+  if (whitelist_file != NULL)
+    whitelist = load_file (whitelist_file);
+
+  char *paths[2] = { (char *) modpath, NULL };
+  FTS *fts = fts_open (paths, FTS_COMFOLLOW|FTS_PHYSICAL, NULL);
+  if (fts == NULL)
+    error (EXIT_FAILURE, errno, "write_kernel_modules: fts_open: %s",
modpath);
+
+  for (;;) {
+    errno = 0;
+    FTSENT *entry = fts_read (fts);
+    if (entry == NULL && errno != 0)
+      error (EXIT_FAILURE, errno, "write_kernel_modules: fts_read:
%s", modpath);
+    if (entry == NULL)
+      break;
+
+    /* Ignore directories being visited in post-order. */
+    if (entry->fts_info & FTS_DP)
+      continue;
+
+    /* Is it a *.ko file? */
+    if (entry->fts_namelen >= 3 &&
+        entry->fts_name[entry->fts_namelen-3] == '.' &&
+        entry->fts_name[entry->fts_namelen-2] == 'k' &&
+        entry->fts_name[entry->fts_namelen-1] == 'o') {
+      if (whitelist) {
+        /* Is it a *.ko file which is on the whitelist? */
+        size_t j;
+        for (j = 0; whitelist[j] != NULL; ++j) {
+          int r;
+          r = fnmatch (whitelist[j], entry->fts_name, 0);
+          if (r == 0) {
+            /* It's on the whitelist, so include it. */
+            if (verbose >= 2)
+              fprintf (stderr, "including kernel module %s (matches
whitelist entry %s)\n",
+                       entry->fts_name, whitelist[j]);
+            cpio_append_fts_entry (entry);
+            break;
+          } else if (r != FNM_NOMATCH)
+            error (EXIT_FAILURE, 0, "internal error: fnmatch
('%s', '%s', %d) returned unexpected non-zero value %d\n",
+                   whitelist[j], entry->fts_name, 0, r);
+        } /* for (j) */
+      } else { /* whitelist == NULL, always include */
+        if (verbose >= 2)
+          fprintf (stderr, "including kernel module %s\n",
entry->fts_name);
+        cpio_append_fts_entry (entry);
+      }
+    } else
+      /* It's some other sort of file, or a directory, always include. */
+      cpio_append_fts_entry (entry);
+  }
+
+  if (fts_close (fts) == -1)
+    error (EXIT_FAILURE, errno, "write_kernel_modules: fts_close:
%s", modpath);
+}
+
+/* Copy the host files.
+ *
+ * Read the list of entries in hostfiles (which may contain
+ * wildcards).  Look them up in the filesystem, and add those files
+ * that exist.  Ignore any files that don't exist or are not readable.
+ */
+static void
+write_hostfiles (const char *hostfiles_file)
+{
+  char **hostfiles = load_file (hostfiles_file);
+
+  /* Hostfiles list can contain "." before each path - ignore it.
+   * It also contains each directory name before we enter it.  But
+   * we don't read that until we see a wildcard for that directory.
+   */
+  size_t i, j;
+  for (i = 0; hostfiles[i] != NULL; ++i) {
+    char *hostfile = hostfiles[i];
+    if (hostfile[0] == '.')
+      hostfile++;
+
+    struct stat statbuf;
+
+    /* Is it a wildcard? */
+    if (strchr (hostfile, '*') || strchr (hostfile, '?')) {
+      char *dirname = xstrdup (hostfile);
+      char *patt = strrchr (dirname, '/');
+      assert (patt);
+      *patt++ = '\0';
+
+      char **files = read_dir (dirname);
+      files = filter_fnmatch (files, patt, FNM_NOESCAPE);
+
+      /* Add matching files. */
+      for (j = 0; files[j] != NULL; ++j) {
+        char *tmp = xasprintf ("%s/%s", dirname, files[j]);
+
+        if (verbose >= 2)
+          fprintf (stderr, "including host file %s (matches %s)\n",
tmp, patt);
+
+        cpio_append (tmp);
+
+        free (tmp);
+      }
+    }
+    /* Else does this file/directory/whatever exist? */
+    else if (lstat (hostfile, &statbuf) == 0) {
+      if (verbose >= 2)
+        fprintf (stderr, "including host file %s (directly
referenced)\n",
+                 hostfile);
+
+      cpio_append_stat (hostfile, &statbuf);
+    } /* Ignore files that don't exist. */
+  }
+}
+
+/*----------*/
+/* Helper functions. */
+
+static void
+add_string (char ***argv, size_t *n_used, size_t *n_alloc, const char *str)
+{
+  char *new_str;
+
+  if (*n_used >= *n_alloc)
+    *argv = x2nrealloc (*argv, n_alloc, sizeof (char *));
+
+  if (str)
+    new_str = xstrdup (str);
+  else
+    new_str = NULL;
+
+  (*argv)[*n_used] = new_str;
+
+  (*n_used)++;
+}
+
+static size_t
+count_strings (char *const *argv)
+{
+  size_t argc;
+
+  for (argc = 0; argv[argc] != NULL; ++argc)
+    ;
+  return argc;
+}
+
+struct dir_cache {
+  char *path;
+  char **files;
+};
+
+static size_t
+dir_cache_hash (void const *x, size_t table_size)
+{
+  struct dir_cache const *p = x;
+  return hash_pjw (p->path, table_size);
+}
+
+static bool
+dir_cache_compare (void const *x, void const *y)
+{
+  struct dir_cache const *p = x;
+  struct dir_cache const *q = y;
+  return strcmp (p->path, q->path) == 0;
+}
+
+/* Read a directory into a list of strings.
+ *
+ * Previously looked up directories are cached and returned quickly,
+ * saving some considerable amount of time compared to reading the
+ * directory over again.  However this means you really must not
+ * alter the array of strings that are returned.
+ *
+ * Returns an empty list if the directory cannot be opened.
+ */
+static char **
+read_dir (const char *name)
+{
+  static Hash_table *ht = NULL;
+
+  if (!ht)
+    ht = hash_initialize (1024, NULL, dir_cache_hash, dir_cache_compare, NULL);
+
+  struct dir_cache key = { .path = (char *) name };
+  struct dir_cache *p = hash_lookup (ht, &key);
+  if (p)
+    return p->files;
+
+  char **files = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  DIR *dir = opendir (name);
+  if (!dir) {
+    /* If it fails to open, that's OK, skip to the end. */
+    /*perror (name);*/
+    goto done;
+  }
+
+  for (;;) {
+    errno = 0;
+    struct dirent *d = readdir (dir);
+    if (d == NULL) {
+      if (errno != 0)
+        /* But if it fails here, after opening and potentially reading
+         * part of the directory, that's a proper failure - inform the
+         * user and exit.
+         */
+        error (EXIT_FAILURE, errno, "%s", name);
+      break;
+    }
+
+    add_string (&files, &n_used, &n_alloc, d->d_name);
+  }
+
+  if (closedir (dir) == -1)
+    error (EXIT_FAILURE, errno, "closedir: %s", name);
+
+ done:
+  /* NULL-terminate the array. */
+  add_string (&files, &n_used, &n_alloc, NULL);
+
+  /* Add it to the hash for next time. */
+  p = xmalloc (sizeof *p);
+  p->path = (char *) name;
+  p->files = files;
+  p = hash_insert (ht, p);
+  assert (p != NULL);
+
+  return files;
+}
+
+/* Filter a list of strings and return only those matching the wildcard. */
+static char **
+filter_fnmatch (char **strings, const char *patt, int flags)
+{
+  char **out = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  int i, r;
+  for (i = 0; strings[i] != NULL; ++i) {
+    r = fnmatch (patt, strings[i], flags);
+    if (r == 0)
+      add_string (&out, &n_used, &n_alloc, strings[i]);
+    else if (r != FNM_NOMATCH)
+      error (EXIT_FAILURE, 0, "internal error: fnmatch ('%s',
'%s', %d) returned unexpected non-zero value %d\n",
+             patt, strings[i], flags, r);
+  }
+
+  add_string (&out, &n_used, &n_alloc, NULL);
+  return out;
+}
+
+/* Filter a list of strings and return only those which DON'T contain sub.
*/
+static char **
+filter_notmatching_substring (char **strings, const char *sub)
+{
+  char **out = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  int i;
+  for (i = 0; strings[i] != NULL; ++i) {
+    if (strstr (strings[i], sub) == NULL)
+      add_string (&out, &n_used, &n_alloc, strings[i]);
+  }
+
+  add_string (&out, &n_used, &n_alloc, NULL);
+  return out;
+}
+
+/* Sort a list of strings, in place, with the comparison function supplied. */
+static void
+sort (char **strings, int (*compare) (const void *, const void *))
+{
+  qsort (strings, count_strings (strings), sizeof (char *), compare);
+}
+
+/* Return true iff path exists and is a directory.  This version
+ * follows symlinks.
+ */
+static int
+isdir (const char *path)
+{
+  struct stat statbuf;
+
+  if (stat (path, &statbuf) == -1)
+    return 0;
+
+  return S_ISDIR (statbuf.st_mode);
+}
+
+/* Copy contents of buffer to out_fd and keep out_offset correct. */
+static void
+write_to_fd (const void *buffer, size_t len)
+{
+  if (full_write (out_fd, buffer, len) != len)
+    error (EXIT_FAILURE, errno, "write");
+  out_offset += len;
+}
+
+/* Copy contents of file to out_fd. */
+static void
+write_file_to_fd (const char *filename)
+{
+  char buffer[BUFFER_SIZE];
+  int fd2;
+  ssize_t r;
+
+  if (verbose >= 2)
+    fprintf (stderr, "write_file_to_fd %s -> %d\n", filename,
out_fd);
+
+  fd2 = open (filename, O_RDONLY);
+  if (fd2 == -1)
+    error (EXIT_FAILURE, errno, "open: %s", filename);
+  for (;;) {
+    r = read (fd2, buffer, sizeof buffer);
+    if (r == 0)
+      break;
+    if (r == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
+        continue;
+      error (EXIT_FAILURE, errno, "read: %s", filename);
+    }
+    write_to_fd (buffer, r);
+  }
+
+  if (close (fd2) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", filename);
+}
+
+/* Copy file of given length to output, and fail if the file has
+ * changed size.
+ */
+static void
+write_file_len_to_fd (const char *filename, size_t len)
+{
+  char buffer[BUFFER_SIZE];
+  size_t count = 0;
+
+  if (verbose >= 2)
+    fprintf (stderr, "write_file_to_fd %s -> %d\n", filename,
out_fd);
+
+  int fd2 = open (filename, O_RDONLY);
+  if (fd2 == -1)
+    error (EXIT_FAILURE, errno, "open: %s", filename);
+  for (;;) {
+    ssize_t r = read (fd2, buffer, sizeof buffer);
+    if (r == 0)
+      break;
+    if (r == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
+        continue;
+      error (EXIT_FAILURE, errno, "read: %s", filename);
+    }
+    write_to_fd (buffer, r);
+    count += r;
+    if (count > len)
+      error (EXIT_FAILURE, 0, "write_file_len_to_fd: %s: file has
increased in size\n", filename);
+  }
+
+  if (close (fd2) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", filename);
+
+  if (count != len)
+    error (EXIT_FAILURE, 0, "febootstrap-supermin-helper:
write_file_len_to_fd: %s: file has changed size\n", filename);
+}
+
+/* Load in a file, returning a list of lines. */
+static char **
+load_file (const char *filename)
+{
+  char **lines = 0;
+  size_t n_used = 0, n_alloc = 0;
+
+  FILE *fp;
+  fp = fopen (filename, "r");
+  if (fp == NULL)
+    error (EXIT_FAILURE, errno, "fopen: %s", filename);
+
+  char line[4096];
+  while (fgets (line, sizeof line, fp)) {
+    size_t len = strlen (line);
+    if (len > 0 && line[len-1] == '\n')
+      line[len-1] = '\0';
+    add_string (&lines, &n_used, &n_alloc, line);
+  }
+
+  add_string (&lines, &n_used, &n_alloc, NULL);
+  return lines;
+}
+
+/* Append the file pointed to by FTSENT to the cpio output. */
+static void
+cpio_append_fts_entry (FTSENT *entry)
+{
+  if (entry->fts_info & FTS_NS || entry->fts_info & FTS_NSOK)
+    cpio_append (entry->fts_path);
+  else
+    cpio_append_stat (entry->fts_path, entry->fts_statp);
+}
+
+/* Append the file named 'filename' to the cpio output. */
+static void
+cpio_append (const char *filename)
+{
+  struct stat statbuf;
+
+  if (lstat (filename, &statbuf) == -1)
+    error (EXIT_FAILURE, errno, "lstat: %s", filename);
+  cpio_append_stat (filename, &statbuf);
+}
+
+/* Append the file to the cpio output. */
+#define PADDING(len) ((((len) + 3) & ~3) - (len))
+
+#define CPIO_HEADER_LEN (6 + 13*8)
+
+static void
+cpio_append_stat (const char *filename, struct stat *statbuf)
+{
+  const char *orig_filename = filename;
+
+  if (*filename == '/')
+    filename++;
+  if (*filename == '\0')
+    filename = ".";
+
+  if (verbose >= 2)
+    fprintf (stderr, "cpio_append_stat %s 0%o -> %d\n",
+             orig_filename, statbuf->st_mode, out_fd);
+
+  /* Regular files and symlinks are the only ones that have a "body"
+   * in this cpio entry.
+   */
+  int has_body = S_ISREG (statbuf->st_mode) || S_ISLNK
(statbuf->st_mode);
+
+  size_t len = strlen (filename) + 1;
+
+  char header[CPIO_HEADER_LEN + 1];
+  snprintf (header, sizeof header,
+            "070701"            /* magic */
+            "%08X"              /* inode */
+            "%08X"              /* mode */
+            "%08X" "%08X"       /* uid, gid */
+            "%08X"              /* nlink */
+            "%08X"              /* mtime */
+            "%08X"              /* file length */
+            "%08X" "%08X"       /* device holding file
major, minor */
+            "%08X" "%08X"       /* for specials, device
major, minor */
+            "%08X"              /* name length (including \0 byte) */
+            "%08X",             /* checksum (not used by the kernel)
*/
+            (unsigned) statbuf->st_ino, statbuf->st_mode,
+            statbuf->st_uid, statbuf->st_gid,
+            (unsigned) statbuf->st_nlink, (unsigned) statbuf->st_mtime,
+            has_body ? (unsigned) statbuf->st_size : 0,
+            major (statbuf->st_dev), minor (statbuf->st_dev),
+            major (statbuf->st_rdev), minor (statbuf->st_rdev),
+            (unsigned) len, 0);
+
+  /* Write the header. */
+  write_to_fd (header, CPIO_HEADER_LEN);
+
+  /* Follow with the filename, and pad it. */
+  write_to_fd (filename, len);
+  size_t padding_len = PADDING (CPIO_HEADER_LEN + len);
+  write_padding (padding_len);
+
+  /* Follow with the file or symlink content, and pad it. */
+  if (has_body) {
+    if (S_ISREG (statbuf->st_mode))
+      write_file_len_to_fd (orig_filename, statbuf->st_size);
+    else if (S_ISLNK (statbuf->st_mode)) {
+      char tmp[PATH_MAX];
+      if (readlink (orig_filename, tmp, sizeof tmp) == -1)
+        error (EXIT_FAILURE, errno, "readlink: %s", orig_filename);
+      write_to_fd (tmp, statbuf->st_size);
+    }
+
+    padding_len = PADDING (statbuf->st_size);
+    write_padding (padding_len);
+  }
+}
+
+/* CPIO voodoo. */
+static void
+cpio_append_trailer (void)
+{
+  struct stat statbuf;
+  memset (&statbuf, 0, sizeof statbuf);
+  statbuf.st_nlink = 1;
+  cpio_append_stat ("TRAILER!!!", &statbuf);
+
+  /* CPIO seems to pad up to the next block boundary, ie. up to
+   * the next 512 bytes.
+   */
+  write_padding (((out_offset + 511) & ~511) - out_offset);
+  assert ((out_offset & 511) == 0);
+}
+
+/* Write 'len' bytes of zeroes out. */
+static void
+write_padding (size_t len)
+{
+  static const char buffer[512] = { 0 };
+
+  while (len > 0) {
+    size_t n = len < sizeof buffer ? len : sizeof buffer;
+    write_to_fd (buffer, n);
+    len -= n;
+  }
+}
diff --git a/febootstrap-supermin-helper.pod b/febootstrap-supermin-helper.pod
index 9ca7ca8..29f4b95 100644
--- a/febootstrap-supermin-helper.pod
+++ b/febootstrap-supermin-helper.pod
@@ -5,6 +5,7 @@ febootstrap-supermin-helper - Reconstruct initramfs from
supermin appliance.
 =head1 SYNOPSIS
 
  febootstrap-supermin-helper supermin.img hostfiles.txt host_cpu kernel initrd
+ febootstrap-supermin-helper input [...] host_cpu kernel initrd
 
 =head1 DESCRIPTION
 
@@ -15,11 +16,12 @@ L<febootstrap-to-supermin(8)>.
 
 =head1 PARAMETERS
 
-Of the five parameters, the first two are I<input> files, and the last
-two are I<output> files.
+Of the four or five required parameters, the first few are I<input>
+files, and the last two are I<output> files.
 
 C<supermin.img> and C<hostfiles.txt> are the input files which
-describe the supermin appliance.
+describe the supermin appliance.  (You can also use a directory name
+here which is searched for files).
 
 C<host_cpu> should be the host CPU, eg. C<x86_64> or C<i686>.
 
@@ -31,7 +33,7 @@ booting the appliance, and should be deleted straight
afterwards.
 
 =over 4
 
-=item B<--kmods file>
+=item B<-k file> | B<--kmods file>
 
 If this option is specified, then C<file> should be a list of
 wildcards matching kernel module names, eg:
@@ -89,8 +91,8 @@ Richard W.M. Jones <rjones @ redhat . com>
 
 =head1 COPYRIGHT
 
-(C) Copyright 2009 Red Hat Inc.,
-L<http://et.redhat.com/~rjones/febootstrap>.
+(C) Copyright 2009-2010 Red Hat Inc.,
+L<http://people.redhat.com/~rjones/febootstrap>.
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/febootstrap-supermin-helper.sh b/febootstrap-supermin-helper.sh
deleted file mode 100755
index cd5cf19..0000000
--- a/febootstrap-supermin-helper.sh
+++ /dev/null
@@ -1,143 +0,0 @@
-#!/bin/bash -
-# febootstrap-supermin-helper
-# (C) Copyright 2009 Red Hat Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-#
-# Written by Richard W.M. Jones <rjones at redhat.com>
-
-unset CDPATH
-
-TEMP=`getopt \
-        -o '' \
-        --long help,kmods: \
-        -n febootstrap-supermin-helper -- "$@"`
-if [ $? != 0 ]; then
-    echo "febootstrap-supermin-helper: problem parsing the command line
arguments"
-    exit 1
-fi
-eval set -- "$TEMP"
-
-usage ()
-{
-    echo "Usage: febootstrap-supermin-helper supermin.img hostfiles.txt
host_cpu kernel initrd"
-    echo "Please read febootstrap-supermin-helper(8) man page for more
information."
-}
-
-kmods=""
-
-while true; do
-    case "$1" in
-	--help)
-	    usage
-	    exit 0;;
-	--kmods)
-	    kmods=$2
-	    shift 2;;
-	--)
-	    shift
-	    break;;
-	*)
-	    echo "Internal error!"
-	    exit 1;;
-    esac
-done
-
-if [ $# -ne 5 ]; then
-    usage
-    exit 1
-fi
-
-set -e
-
-# Input files.
-supermin="$1"
-hostfiles="$2"
-
-host_cpu=$3
-
-# Output files.
-kernel="$4"
-initrd="$5"
-
-rm -f "$kernel" "$initrd"
-
-# Kernel:
-# Look for the most recent kernel named vmlinuz-*.<arch>* which has a
-# corresponding directory in /lib/modules/. If the architecture is x86, look
-# for any x86 kernel.
-#
-# RHEL 5 didn't append the arch to the kernel name, so look for kernels
-# without arch second.
-
-arch=$(echo $host_cpu | sed 's/^i.86$/i?86/')
-kernels=$(
-    ls -1dvr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen ||: ;
-    ls -1dvr /boot/vmlinuz-* 2>/dev/null | grep -v xen
-)
-
-if [ -z "$kernels" ]; then
-    echo "$0: failed to find a suitable kernel in /boot directory"
>&2
-    exit 1
-fi
-
-for f in $kernels; do
-    b=$(basename "$f")
-    b=$(echo "$b" | sed 's,vmlinuz-,,')
-    modpath="/lib/modules/$b"
-    if [ -d "$modpath" ]; then
-        ln -sf "$f" "$kernel"
-        break
-    fi
-    modpath-done
-
-if [ -z "$modpath" ]; then
-    echo "$0: failed to find a suitable kernel" >&2
-    exit 1
-fi
-
-# The initrd consists of these components:
-# (1) The base skeleton appliance that we constructed at build time.
-#     format = plain cpio (could be compressed cpio)
-# (2) The modules from modpath which are on the module whitelist.
-#     format = plain cpio
-# (3) The host files which match wildcards in hostfiles.
-#     format = plain cpio
-
-cp "$supermin" "$initrd" ;# (1)
-
-# Kernel modules (2).
-
-if [ -n "$kmods" ]; then
-    exec 5<"$kmods"
-    whitelist-    while read kmod 0<&5; do
-	whitelist="$whitelist -o -name $kmod"
-    done
-    exec 5<&-
-else
-    whitelist="-o -name *.ko"
-fi
-
-find "$modpath" \( -not -name '*.ko' $whitelist \) -a -print0
|
-  cpio --quiet -o -0 -H newc >> "$initrd"
-
-# Host files (3).
-
-hostfiles=$(readlink -f "$hostfiles")
-(cd / &&
-    ls -1df $(cat "$hostfiles") 2>/dev/null |
-    cpio -C 65536 --quiet -o -H newc ) >> "$initrd"
diff --git a/febootstrap-to-supermin.pod b/febootstrap-to-supermin.pod
index 6263f51..123327c 100644
--- a/febootstrap-to-supermin.pod
+++ b/febootstrap-to-supermin.pod
@@ -79,7 +79,8 @@ Use supermin appliances with caution.
 
 =head2 ANATOMY OF A SUPERMIN APPLIANCE
 
-A supermin appliance consists of two files:
+A supermin appliance consists usually of just two files, but can
+contain several files and directories from the list below:
 
 =over 4
 
@@ -90,6 +91,10 @@ call it anything you want) is the skeleton initramfs.  This
is like an
 initramfs built by L<febootstrap-to-initramfs(8)>, but all libraries
 and binaries are removed.
 
+Note that this file is a cpio file in cpio "newc" format, and is
+I<not> compressed (unlike initramfs files which are compressed cpio
+files).
+
 =item hostfiles.txt
 
 This plain text file contains a list of files that we need to add back
@@ -106,6 +111,14 @@ the file contains
 
  lib64/libreadline.so.6.*
 
+=item any directory
+
+You can specify a directory which should contain image file(s)
+and hostfile(s).
+
+Using a directory is useful either to keep the appliance-related files
+together, or to make more complex appliances containing optional bits.
+
 =back
 
 =head2 RECONSTRUCTING AN INITRAMFS FROM A SUPERMIN APPLIANCE
@@ -175,8 +188,8 @@ Richard W.M. Jones <rjones @ redhat . com>
 
 =head1 COPYRIGHT
 
-(C) Copyright 2009 Red Hat Inc.,
-L<http://et.redhat.com/~rjones/febootstrap>.
+(C) Copyright 2009-2010 Red Hat Inc.,
+L<http://people.redhat.com/~rjones/febootstrap>.
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/lib/.gitignore b/lib/.gitignore
new file mode 100644
index 0000000..048d811
--- /dev/null
+++ b/lib/.gitignore
@@ -0,0 +1,133 @@
+/Makefile.am
+/dummy.c
+/alloca.in.h
+/asnprintf.c
+/asprintf.c
+/at-func.c
+/basename-lgpl.c
+/bitrotate.h
+/c-ctype.c
+/c-ctype.h
+/chdir-long.c
+/chdir-long.h
+/chown.c
+/cloexec.c
+/cloexec.h
+/close-hook.c
+/close-hook.h
+/close.c
+/creat-safer.c
+/cycle-check.c
+/cycle-check.h
+/dev-ino.h
+/dirent--.h
+/dirent-safer.h
+/dirent.in.h
+/dirfd.c
+/dirname-lgpl.c
+/dirname.h
+/dup-safer.c
+/dup2.c
+/errno.in.h
+/error.c
+/error.h
+/exitfail.c
+/exitfail.h
+/fchdir.c
+/fchmodat.c
+/fchown-stub.c
+/fchownat.c
+/fclose.c
+/fcntl--.h
+/fcntl-safer.h
+/fcntl.c
+/fcntl.in.h
+/fd-safer.c
+/fdopendir.c
+/filevercmp.c
+/filevercmp.h
+/float+.h
+/float.in.h
+/fstatat.c
+/fts-cycle.c
+/fts.c
+/fts_.h
+/full-write.c
+/full-write.h
+/getcwd.c
+/getdtablesize.c
+/gettext.h
+/hash.c
+/hash.h
+/i-ring.c
+/i-ring.h
+/intprops.h
+/lchown.c
+/lstat.c
+/malloc.c
+/memchr.c
+/memchr.valgrind
+/mempcpy.c
+/memrchr.c
+/mkdir.c
+/mkdirat.c
+/open-safer.c
+/open.c
+/openat-die.c
+/openat-priv.h
+/openat-proc.c
+/openat-safer.c
+/openat.c
+/openat.h
+/opendir-safer.c
+/pipe-safer.c
+/printf-args.c
+/printf-args.h
+/printf-parse.c
+/printf-parse.h
+/realloc.c
+/rmdir.c
+/safe-read.c
+/safe-read.h
+/safe-write.c
+/safe-write.h
+/same-inode.h
+/save-cwd.c
+/save-cwd.h
+/size_max.h
+/stat.c
+/stdarg.in.h
+/stdbool.in.h
+/stddef.in.h
+/stdint.in.h
+/stdio-write.c
+/stdio.in.h
+/stdlib.in.h
+/strdup.c
+/strerror.c
+/string.in.h
+/stripslash.c
+/sys_stat.in.h
+/time.in.h
+/unistd--.h
+/unistd-safer.h
+/unistd.in.h
+/unlink.c
+/unlinkat.c
+/vasnprintf.c
+/vasnprintf.h
+/vasprintf.c
+/verify.h
+/wchar.in.h
+/write.c
+/xalloc-die.c
+/xalloc.h
+/xasprintf.c
+/xgetcwd.c
+/xgetcwd.h
+/xmalloc.c
+/xsize.h
+/xvasprintf.c
+/xvasprintf.h
+/hash-pjw.c
+/hash-pjw.h
diff --git a/m4/.gitignore b/m4/.gitignore
new file mode 100644
index 0000000..9601bdc
--- /dev/null
+++ b/m4/.gitignore
@@ -0,0 +1,90 @@
+/00gnulib.m4
+/gnulib-common.m4
+/gnulib-comp.m4
+/gnulib-tool.m4
+/onceonly.m4
+/alloca.m4
+/chdir-long.m4
+/chown.m4
+/cloexec.m4
+/close.m4
+/cycle-check.m4
+/d-ino.m4
+/d-type.m4
+/dirent-safer.m4
+/dirent_h.m4
+/dirfd.m4
+/dirname.m4
+/dos.m4
+/double-slash-root.m4
+/dup2.m4
+/errno_h.m4
+/error.m4
+/extensions.m4
+/fchdir.m4
+/fclose.m4
+/fcntl-o.m4
+/fcntl-safer.m4
+/fcntl.m4
+/fcntl_h.m4
+/fdopendir.m4
+/float_h.m4
+/fts.m4
+/getcwd-abort-bug.m4
+/getcwd-path-max.m4
+/getcwd.m4
+/getdtablesize.m4
+/hash.m4
+/i-ring.m4
+/include_next.m4
+/inline.m4
+/intmax_t.m4
+/inttypes_h.m4
+/lchown.m4
+/longlong.m4
+/lstat.m4
+/malloc.m4
+/memchr.m4
+/mempcpy.m4
+/memrchr.m4
+/mkdir.m4
+/mmap-anon.m4
+/mode_t.m4
+/multiarch.m4
+/open.m4
+/openat.m4
+/printf.m4
+/realloc.m4
+/rmdir.m4
+/safe-read.m4
+/safe-write.m4
+/save-cwd.m4
+/size_max.m4
+/ssize_t.m4
+/stat.m4
+/stdarg.m4
+/stdbool.m4
+/stddef_h.m4
+/stdint.m4
+/stdint_h.m4
+/stdio_h.m4
+/stdlib_h.m4
+/strdup.m4
+/strerror.m4
+/string_h.m4
+/sys_stat_h.m4
+/time_h.m4
+/unistd-safer.m4
+/unistd_h.m4
+/unlink.m4
+/vasnprintf.m4
+/vasprintf.m4
+/warn-on-use.m4
+/wchar_h.m4
+/wchar_t.m4
+/wint_t.m4
+/write.m4
+/xalloc.m4
+/xgetcwd.m4
+/xsize.m4
+/xvasprintf.m4
diff --git a/m4/gnulib-cache.m4 b/m4/gnulib-cache.m4
new file mode 100644
index 0000000..1f886cb
--- /dev/null
+++ b/m4/gnulib-cache.m4
@@ -0,0 +1,41 @@
+# Copyright (C) 2002-2010 Free Software Foundation, Inc.
+#
+# This file is free software, distributed under the terms of the GNU
+# General Public License.  As a special exception to the GNU General
+# Public License, this file may be distributed as part of a program
+# that contains a configuration script generated by Autoconf, under
+# the same distribution terms as the rest of that program.
+#
+# Generated by gnulib-tool.
+#
+# This file represents the specification of how gnulib-tool is used.
+# It acts as a cache: It is written and read by gnulib-tool.
+# In projects using CVS, this file is meant to be stored in CVS,
+# like the configure.ac and various Makefile.am files.
+
+
+# Specification in the form of a command-line invocation:
+#   gnulib-tool --import --dir=. --lib=libgnu --source-base=lib --m4-base=m4
--doc-base=doc --tests-base=tests --aux-dir=. --no-libtool --macro-prefix=gl
error filevercmp fts full-write hash hash-pjw xalloc xvasprintf
+
+# Specification in the form of a few gnulib-tool.m4 macro invocations:
+gl_LOCAL_DIR([])
+gl_MODULES([
+  error
+  filevercmp
+  fts
+  full-write
+  hash
+  hash-pjw
+  xalloc
+  xvasprintf
+])
+gl_AVOID([])
+gl_SOURCE_BASE([lib])
+gl_M4_BASE([m4])
+gl_PO_BASE([])
+gl_DOC_BASE([doc])
+gl_TESTS_BASE([tests])
+gl_LIB([libgnu])
+gl_MAKEFILE_NAME([])
+gl_MACRO_PREFIX([gl])
+gl_PO_DOMAIN([])
-- 
1.6.6.1

I made some minor changes to the patch, and I've tested it with a
patched libguestfs (that patch coming up) and it gets all the way
through the tests.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora
-------------- next part --------------
>From d0f3b9e13b51cb7be07483a98794673d60045b91 Mon Sep 17 00:00:00 2001
From: Richard Jones <rjones at redhat.com>
Date: Thu, 13 May 2010 13:29:20 +0100
Subject: [PATCH] Pull in febootstrap-supermin-helper (C version) from
libguestfs.

This commit also pulls in the automake C dependencies and gnulib.

febootstrap-supermin-helper is modified so that it has the
--kmods option (to read the whitelist, from the old shell script),
and so that it can read supermin appliances composed of multiple
parts from out of a directory.
---
 .gitignore                      |   30 +-
 Makefile.am                     |   19 +-
 README                          |    2 +-
 autogen.sh                      |    4 +
 configure.ac                    |   23 +-
 febootstrap-supermin-helper.c   |  997 +++++++++++++++++++++++++++++++++++++++
 febootstrap-supermin-helper.pod |   14 +-
 febootstrap-supermin-helper.sh  |  143 ------
 febootstrap-to-supermin.pod     |   19 +-
 febootstrap-to-supermin.sh      |    2 +-
 lib/.gitignore                  |  133 ++++++
 m4/.gitignore                   |   90 ++++
 m4/gnulib-cache.m4              |   41 ++
 13 files changed, 1348 insertions(+), 169 deletions(-)
 create mode 100644 febootstrap-supermin-helper.c
 delete mode 100755 febootstrap-supermin-helper.sh
 create mode 100644 lib/.gitignore
 create mode 100644 m4/.gitignore
 create mode 100644 m4/gnulib-cache.m4

diff --git a/.gitignore b/.gitignore
index a430f1f..310aa6e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,16 +1,22 @@
 *~
-febootstrap*.8
-febootstrap*.txt
-febootstrap-*.tar.gz
+*.o
+*.a
+.deps
 Makefile.in
 Makefile
 aclocal.m4
+/arg-nonnull.h
 autom4te.cache
+/c++defs.h
+compile
+config.guess
 config.h.in
 config.h
 config.log
 config.status
+config.sub
 configure
+depcomp
 febootstrap
 febootstrap-run
 febootstrap-install
@@ -18,9 +24,27 @@ febootstrap-minimize
 febootstrap-to-initramfs
 febootstrap-to-supermin
 febootstrap-supermin-helper
+febootstrap*.8
+febootstrap*.txt
+febootstrap-*.tar.gz
+lib/alloca.h
+lib/arg-nonnull.h
+lib/c++defs.h
+lib/dirent.h
+lib/fcntl.h
+lib/stdio.h
+lib/stdlib.h
+lib/string.h
+lib/sys/
+lib/time.h
+lib/unistd.h
+lib/warn-on-use.h
+lib/wchar.h
+INSTALL
 install-sh
 missing
 stamp-h1
+/warn-on-use.h
 examples/guestfs
 examples/*-initrd.img
 examples/guest-image
diff --git a/Makefile.am b/Makefile.am
index 861b5e0..bb3b505 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,5 +1,5 @@
 # febootstrap Makefile.am
-# (C) Copyright 2009 Red Hat Inc.
+# (C) Copyright 2009-2010 Red Hat Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -17,7 +17,9 @@
 #
 # Written by Richard W.M. Jones <rjones at redhat.com>
 
-SUBDIRS = examples
+ACLOCAL_AMFLAGS = -I m4
+
+SUBDIRS = lib examples
 
 bin_SCRIPTS = \
 	febootstrap \
@@ -25,7 +27,8 @@ bin_SCRIPTS = \
 	febootstrap-install \
 	febootstrap-minimize \
 	febootstrap-to-initramfs \
-	febootstrap-to-supermin \
+	febootstrap-to-supermin
+bin_PROGRAMS = \
 	febootstrap-supermin-helper
 DISTCLEANFILES = $(bin_SCRIPTS)
 
@@ -65,11 +68,9 @@ febootstrap-to-supermin: febootstrap-to-supermin.sh
 	chmod 0555 $@-t
 	mv $@-t $@
 
-febootstrap-supermin-helper: febootstrap-supermin-helper.sh
-	rm -f $@
-	cp $< $@-t
-	chmod 0555 $@-t
-	mv $@-t $@
+febootstrap_supermin_helper_SOURCES = febootstrap-supermin-helper.c
+febootstrap_supermin_helper_CFLAGS = -Wall -Ilib
+febootstrap_supermin_helper_LDADD = $(LTLIBINTL) -Llib -lgnu
 
 man_MANS = \
 	febootstrap.8 \
@@ -174,4 +175,4 @@ EXTRA_DIST = \
 	  febootstrap-to-supermin.sh \
 	febootstrap-supermin-helper.8 febootstrap-supermin-helper.txt \
 	  febootstrap-supermin-helper.pod \
-	  febootstrap-supermin-helper.sh
+	m4/gnulib-cache.m4
diff --git a/README b/README
index 38afbb3..53275ac 100644
--- a/README
+++ b/README
@@ -32,7 +32,7 @@ Requirements
 
   bash
 
-  MAKEDEV
+  gcc
 
   qemu
     - If you want to test-run your systems.
diff --git a/autogen.sh b/autogen.sh
index 32f085b..0ab1868 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -1,5 +1,9 @@
 #!/bin/sh -
 
+if [ -d ../gnulib ]; then
+    ../gnulib/gnulib-tool --update
+fi
+
 export AUTOMAKE='automake --foreign --add-missing'
 autoreconf
 ./configure "$@"
diff --git a/configure.ac b/configure.ac
index f564048..05597b1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,5 @@
 dnl febootstrap configure.ac
-dnl (C) Copyright 2009 Red Hat Inc.
+dnl (C) Copyright 2009-2010 Red Hat Inc.
 dnl
 dnl This program is free software; you can redistribute it and/or modify
 dnl it under the terms of the GNU General Public License as published by
@@ -17,9 +17,26 @@ dnl Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 dnl
 dnl Written by Richard W.M. Jones <rjones at redhat.com>
 
-AC_INIT(febootstrap,2.6)
+AC_INIT(febootstrap,2.7)
 AM_INIT_AUTOMAKE
 
+dnl Check for basic C environment.
+AC_PROG_CC_STDC
+gl_EARLY
+
+AC_PROG_INSTALL
+AC_PROG_CPP
+
+AC_C_PROTOTYPES
+test "x$U" != "x" && AC_MSG_ERROR([Compiler not
ANSI compliant])
+
+AM_PROG_CC_C_O
+
+dnl Check support for 64 bit file offsets.
+AC_SYS_LARGEFILE
+
+gl_INIT
+
 AC_CHECK_PROG(PERLDOC,[perldoc],[perldoc],[no])
 if test "x$PERLDOC" = "xno" ; then
   AC_MSG_WARN([perldoc not found - install perl to make man pages])
@@ -42,5 +59,5 @@ if test "x$YUM" = "xno" ; then
 fi
 
 AC_CONFIG_HEADERS([config.h])
-AC_CONFIG_FILES([Makefile examples/Makefile])
+AC_CONFIG_FILES([lib/Makefile Makefile examples/Makefile])
 AC_OUTPUT
diff --git a/febootstrap-supermin-helper.c b/febootstrap-supermin-helper.c
new file mode 100644
index 0000000..68645fe
--- /dev/null
+++ b/febootstrap-supermin-helper.c
@@ -0,0 +1,997 @@
+/* febootstrap-supermin-helper reimplementation in C.
+ * Copyright (C) 2009-2010 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/* This script builds the supermin appliance on the fly each
+ * time the appliance runs.
+ *
+ * *NOTE*: This program is designed to be very short-lived, and so we
+ * don't normally bother to free up any memory that we allocate.
+ * That's not completely true - we free up stuff if it's obvious and
+ * easy to free up, and ignore the rest.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <stdint.h>
+#include <string.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <getopt.h>
+#include <limits.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/time.h>
+#include <sys/stat.h>
+#include <assert.h>
+
+#include "error.h"
+#include "filevercmp.h"
+#include "fts_.h"
+#include "full-write.h"
+#include "hash.h"
+#include "hash-pjw.h"
+#include "xalloc.h"
+#include "xvasprintf.h"
+
+/* Directory containing candidate kernels.  We could make this
+ * configurable at some point.
+ */
+#define KERNELDIR "/boot"
+#define MODULESDIR "/lib/modules"
+
+/* Buffer size used in copy operations throughout.  Large for
+ * greatest efficiency.
+ */
+#define BUFFER_SIZE 65536
+
+static struct timeval start_t;
+static int verbose = 0;
+
+static void print_timestamped_message (const char *fs, ...);
+static const char *create_kernel (const char *hostcpu, const char *kernel);
+static void create_appliance (char **inputs, int nr_inputs, const char
*whitelist, const char *modpath, const char *initrd);
+
+enum { HELP_OPTION = CHAR_MAX + 1 };
+
+static const char *options = "k:vV";
+static const struct option long_options[] = {
+  { "help", 0, 0, HELP_OPTION },
+  { "kmods", required_argument, 0, 'k' },
+  { "verbose", 0, 0, 'v' },
+  { "version", 0, 0, 'V' },
+  { 0, 0, 0, 0 }
+};
+
+static void
+usage (const char *progname)
+{
+  printf ("%s: build the supermin appliance on the fly\n"
+          "\n"
+          "Usage:\n"
+          "  %s [-options] inputs [...] whitelist host_cpu kernel
initrd\n"
+          "  %s --help\n"
+          "  %s --version\n"
+          "\n"
+          "This script is used by febootstrap to build the supermin
appliance\n"
+          "(kernel and initrd output files).  You should NOT need to run
this\n"
+          "program directly except if you are debugging tricky
supermin\n"
+          "appliance problems.\n"
+          "\n"
+          "NB: The kernel and initrd parameters are OUTPUT parameters. 
If\n"
+          "those files exist, they are overwritten by the output.\n"
+          "\n"
+          "Options:\n"
+          "  --help\n"
+          "       Display this help text and exit.\n"
+          "  -k file | --kmods file\n"
+          "       Specify kernel module whitelist.\n"
+          "  --verbose | -v\n"
+          "       Enable verbose messages (give multiple times for more
verbosity).\n"
+          "  --version | -V\n"
+          "       Display version number and exit.\n",
+          progname, progname, progname, progname);
+}
+
+int
+main (int argc, char *argv[])
+{
+  /* First thing: start the clock. */
+  gettimeofday (&start_t, NULL);
+
+  const char *whitelist = NULL;
+
+  /* Command line arguments. */
+  for (;;) {
+    int c = getopt_long (argc, argv, options, long_options, NULL);
+    if (c == -1) break;
+
+    switch (c) {
+    case HELP_OPTION:
+      usage (argv[0]);
+      exit (EXIT_SUCCESS);
+
+    case 'k':
+      whitelist = optarg;
+      break;
+
+    case 'v':
+      verbose++;
+      break;
+
+    case 'V':
+      printf (PACKAGE_NAME " " PACKAGE_VERSION "\n");
+      exit (EXIT_SUCCESS);
+
+    default:
+      usage (argv[0]);
+      exit (EXIT_FAILURE);
+    }
+  }
+
+  char **inputs = &argv[optind];
+  int nr_inputs = argc - optind - 3;
+
+  if (nr_inputs < 1) {
+    usage (argv[0]);
+    exit (EXIT_FAILURE);
+  }
+
+  /* See: https://bugzilla.redhat.com/show_bug.cgi?id=558593 */
+  const char *hostcpu = argv[argc-3];
+
+  /* Output files. */
+  const char *kernel = argv[argc-2];
+  const char *initrd = argv[argc-1];
+
+  if (verbose) {
+    print_timestamped_message ("whitelist = %s, "
+                               "host_cpu = %s, "
+                               "kernel = %s, "
+                               "initrd = %s",
+                               whitelist ? : "(not specified)",
+                               hostcpu, kernel, initrd);
+    int i;
+    for (i = 0; i < nr_inputs; ++i)
+      print_timestamped_message ("inputs[%d] = %s", i, inputs[i]);
+  }
+
+  /* Remove the output files if they exist. */
+  unlink (kernel);
+  unlink (initrd);
+
+  /* Create kernel output file. */
+  const char *modpath;
+  modpath = create_kernel (hostcpu, kernel);
+
+  if (verbose)
+    print_timestamped_message ("finished creating kernel");
+
+  /* Create the appliance. */
+  create_appliance (inputs, nr_inputs, whitelist, modpath, initrd);
+
+  if (verbose)
+    print_timestamped_message ("finished creating appliance");
+
+  exit (EXIT_SUCCESS);
+}
+
+/* Compute Y - X and return the result in milliseconds.
+ * Approximately the same as this code:
+ * http://www.mpp.mpg.de/~huber/util/timevaldiff.c
+ */
+static int64_t
+timeval_diff (const struct timeval *x, const struct timeval *y)
+{
+  int64_t msec;
+
+  msec = (y->tv_sec - x->tv_sec) * 1000;
+  msec += (y->tv_usec - x->tv_usec) / 1000;
+  return msec;
+}
+
+static void
+print_timestamped_message (const char *fs, ...)
+{
+  struct timeval tv;
+  gettimeofday (&tv, NULL);
+
+  va_list args;
+  char *msg;
+  int err;
+
+  va_start (args, fs);
+  err = vasprintf (&msg, fs, args);
+  va_end (args);
+
+  if (err < 0) return;
+
+  fprintf (stderr, "supermin helper [%05" PRIi64 "ms]
%s\n",
+           timeval_diff (&start_t, &tv), msg);
+
+  free (msg);
+}
+
+static char **read_dir (const char *dir);
+static char **filter_fnmatch (char **strings, const char *patt, int flags);
+static char **filter_notmatching_substring (char **strings, const char *sub);
+static void sort (char **strings, int (*compare) (const void *, const void *));
+static int isdir (const char *path);
+
+static int
+reverse_filevercmp (const void *p1, const void *p2)
+{
+  const char *s1 = * (char * const *) p1;
+  const char *s2 = * (char * const *) p2;
+
+  /* Note, arguments are reversed to achieve a reverse sort. */
+  return filevercmp (s2, s1);
+}
+
+/* Create the kernel.  This chooses an appropriate kernel and makes a
+ * symlink to it.
+ *
+ * Look for the most recent kernel named vmlinuz-*.<arch>* which has a
+ * corresponding directory in /lib/modules/. If the architecture is
+ * x86, look for any x86 kernel.
+ *
+ * RHEL 5 didn't append the arch to the kernel name, so look for
+ * kernels without arch second.
+ *
+ * If no suitable kernel can be found, exit with an error.
+ *
+ * This function returns the module path (ie. /lib/modules/<version>).
+ */
+static const char *
+create_kernel (const char *hostcpu, const char *kernel)
+{
+  char **all_files = read_dir (KERNELDIR);
+
+  /* In original: ls -1dvr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen
*/
+  const char *patt;
+  if (hostcpu[0] == 'i' && hostcpu[2] == '8' &&
hostcpu[3] == '6' &&
+      hostcpu[4] == '\0')
+    patt = "vmlinuz-*.i?86*";
+  else
+    patt = xasprintf ("vmlinuz-*.%s*", hostcpu);
+
+  char **candidates;
+  candidates = filter_fnmatch (all_files, patt, FNM_NOESCAPE);
+  candidates = filter_notmatching_substring (candidates, "xen");
+
+  if (candidates[0] == NULL) {
+    /* In original: ls -1dvr /boot/vmlinuz-* 2>/dev/null | grep -v xen */
+    patt = "vmlinuz-*";
+    candidates = filter_fnmatch (all_files, patt, FNM_NOESCAPE);
+    candidates = filter_notmatching_substring (candidates, "xen");
+
+    if (candidates[0] == NULL)
+      goto no_kernels;
+  }
+
+  sort (candidates, reverse_filevercmp);
+
+  /* Choose the first candidate which has a corresponding /lib/modules
+   * directory.
+   */
+  int i;
+  for (i = 0; candidates[i] != NULL; ++i) {
+    if (verbose >= 2)
+      fprintf (stderr, "candidate kernel: " KERNELDIR
"/%s\n", candidates[i]);
+
+    /* Ignore "vmlinuz-" at the beginning of the kernel name. */
+    const char *version = &candidates[i][8];
+
+    /* /lib/modules/<version> */
+    char *modpath = xasprintf (MODULESDIR "/%s", version);
+
+    if (verbose >= 2)
+      fprintf (stderr, "checking modpath %s is a directory\n",
modpath);
+
+    if (isdir (modpath)) {
+      if (verbose >= 2)
+        fprintf (stderr, "picked %s because modpath %s exists\n",
+                 candidates[i], modpath);
+
+      char *tmp = xasprintf (KERNELDIR "/%s", candidates[i]);
+
+      if (verbose >= 2)
+        fprintf (stderr, "creating symlink %s -> %s\n", kernel,
tmp);
+
+      if (symlink (tmp, kernel) == -1)
+        error (EXIT_FAILURE, errno, "symlink kernel");
+
+      free (tmp);
+
+      return modpath;
+    }
+  }
+
+  /* Print more diagnostics here than the old script did. */
+ no_kernels:
+  fprintf (stderr,
+           "febootstrap-supermin-helper: failed to find a suitable
kernel.\n"
+           "I looked for kernels in " KERNELDIR " and modules in
" MODULESDIR
+           ".\n"
+           "If this is a Xen guest, and you only have Xen domU
kernels\n"
+           "installed, try installing a fullvirt kernel (only for\n"
+           "febootstrap use, you shouldn't boot the Xen guest with
it).\n");
+  exit (EXIT_FAILURE);
+}
+
+static void iterate_inputs (char **inputs, int nr_inputs);
+static void iterate_input_directory (const char *dirname, int dirfd);
+static void write_kernel_modules (const char *whitelist, const char *modpath);
+static void write_hostfiles (const char *hostfiles_file);
+static void write_to_fd (const void *buffer, size_t len);
+static void write_file_to_fd (const char *filename);
+static void write_file_len_to_fd (const char *filename, size_t len);
+static void write_padding (size_t len);
+static char **load_file (const char *filename);
+static void cpio_append_fts_entry (FTSENT *entry);
+static void cpio_append_stat (const char *filename, struct stat *);
+static void cpio_append (const char *filename);
+static void cpio_append_trailer (void);
+
+static int out_fd = -1;
+static off_t out_offset = 0;
+
+/* Create the appliance.
+ *
+ * The initrd consists of these components concatenated together:
+ *
+ * (1) The base skeleton appliance that we constructed at build time.
+ *     format = plain cpio
+ * (2) The host files which match wildcards in *.supermin.hostfiles.
+ *     input format = plain text, output format = plain cpio
+ * (3) The modules from modpath which are on the module whitelist.
+ *     output format = plain cpio
+ *
+ * The original shell scripted used the external cpio program to
+ * create parts (2) and (3), but we have decided it's going to be
+ * faster if we just write out the data outselves.  The reasons are
+ * that external cpio is slow (particularly when used with SELinux
+ * because it does 512 byte reads), and the format that we're writing
+ * is narrow and well understood, because we only care that the Linux
+ * kernel can read it.
+ *
+ * This version contains some improvements over the C version written
+ * for libguestfs, in that we can have multiple base images (or
+ * hostfiles) or use a directory to store these files.
+ */
+static void
+create_appliance (char **inputs, int nr_inputs,
+                  const char *whitelist,
+                  const char *modpath,
+                  const char *initrd)
+{
+  out_fd = open (initrd, O_WRONLY | O_CREAT | O_TRUNC | O_NOCTTY, 0644);
+  if (out_fd == -1)
+    error (EXIT_FAILURE, errno, "open: %s", initrd);
+  out_offset = 0;
+
+  iterate_inputs (inputs, nr_inputs);
+
+  /* Kernel modules (3). */
+  write_kernel_modules (whitelist, modpath);
+
+  cpio_append_trailer ();
+
+  /* Finish off and close output file. */
+  if (close (out_fd) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", initrd);
+}
+
+/* Iterate over the inputs to find out what they are, visiting
+ * directories if specified.
+ */
+static void
+iterate_inputs (char **inputs, int nr_inputs)
+{
+  int i;
+  for (i = 0; i < nr_inputs; ++i) {
+    if (verbose)
+      print_timestamped_message ("visiting %s", inputs[i]);
+
+    int fd = open (inputs[i], O_RDONLY);
+    if (fd == -1)
+      error (EXIT_FAILURE, errno, "open: %s", inputs[i]);
+
+    struct stat statbuf;
+    if (fstat (fd, &statbuf) == -1)
+      error (EXIT_FAILURE, errno, "fstat: %s", inputs[i]);
+
+    /* Directory? */
+    if (S_ISDIR (statbuf.st_mode))
+      iterate_input_directory (inputs[i], fd);
+    else if (S_ISREG (statbuf.st_mode)) {
+      /* Is it a cpio file? */
+      char buf[6];
+      if (read (fd, buf, 6) == 6 && memcmp (buf, "070701", 6)
== 0)
+        /* Yes, a cpio file.  This is a skeleton appliance, case (1). */
+        write_file_to_fd (inputs[i]);
+      else
+        /* No, must be hostfiles, case (2). */
+        write_hostfiles (inputs[i]);
+    }
+    else
+      error (EXIT_FAILURE, 0, "%s: input is not a regular file or
directory",
+             inputs[i]);
+
+    close (fd);
+  }
+}
+
+static void
+iterate_input_directory (const char *dirname, int dirfd)
+{
+  char path[PATH_MAX];
+  strcpy (path, dirname);
+  size_t len = strlen (dirname);
+  path[len++] = '/';
+
+  char *inputs[] = { path };
+
+  DIR *dir = fdopendir (dirfd);
+  if (dir == NULL)
+    error (EXIT_FAILURE, errno, "fdopendir: %s", dirname);
+
+  struct dirent *d;
+  while ((errno = 0, d = readdir (dir)) != NULL) {
+    if (d->d_name[0] == '.') /* ignore ., .. and any hidden files.
*/
+      continue;
+
+    strcpy (&path[len], d->d_name);
+    iterate_inputs (inputs, 1);
+  }
+
+  if (errno != 0)
+    error (EXIT_FAILURE, errno, "readdir: %s", dirname);
+
+  if (closedir (dir) == -1)
+    error (EXIT_FAILURE, errno, "closedir: %s", dirname);
+}
+
+/* Copy kernel modules.
+ *
+ * Find every file under modpath.
+ *
+ * Exclude all *.ko files, *except* ones which match names in
+ * the whitelist (which may contain wildcards).  Include all
+ * other files.
+ *
+ * Add chosen files to the output.
+ *
+ * whitelist_file may be NULL, to include ALL kernel modules.
+ */
+static void
+write_kernel_modules (const char *whitelist_file, const char *modpath)
+{
+  char **whitelist = NULL;
+  if (whitelist_file != NULL)
+    whitelist = load_file (whitelist_file);
+
+  char *paths[2] = { (char *) modpath, NULL };
+  FTS *fts = fts_open (paths, FTS_COMFOLLOW|FTS_PHYSICAL, NULL);
+  if (fts == NULL)
+    error (EXIT_FAILURE, errno, "write_kernel_modules: fts_open: %s",
modpath);
+
+  for (;;) {
+    errno = 0;
+    FTSENT *entry = fts_read (fts);
+    if (entry == NULL && errno != 0)
+      error (EXIT_FAILURE, errno, "write_kernel_modules: fts_read:
%s", modpath);
+    if (entry == NULL)
+      break;
+
+    /* Ignore directories being visited in post-order. */
+    if (entry->fts_info & FTS_DP)
+      continue;
+
+    /* Is it a *.ko file? */
+    if (entry->fts_namelen >= 3 &&
+        entry->fts_name[entry->fts_namelen-3] == '.' &&
+        entry->fts_name[entry->fts_namelen-2] == 'k' &&
+        entry->fts_name[entry->fts_namelen-1] == 'o') {
+      if (whitelist) {
+        /* Is it a *.ko file which is on the whitelist? */
+        size_t j;
+        for (j = 0; whitelist[j] != NULL; ++j) {
+          int r;
+          r = fnmatch (whitelist[j], entry->fts_name, 0);
+          if (r == 0) {
+            /* It's on the whitelist, so include it. */
+            if (verbose >= 2)
+              fprintf (stderr, "including kernel module %s (matches
whitelist entry %s)\n",
+                       entry->fts_name, whitelist[j]);
+            cpio_append_fts_entry (entry);
+            break;
+          } else if (r != FNM_NOMATCH)
+            error (EXIT_FAILURE, 0, "internal error: fnmatch
('%s', '%s', %d) returned unexpected non-zero value %d\n",
+                   whitelist[j], entry->fts_name, 0, r);
+        } /* for (j) */
+      } else { /* whitelist == NULL, always include */
+        if (verbose >= 2)
+          fprintf (stderr, "including kernel module %s\n",
entry->fts_name);
+        cpio_append_fts_entry (entry);
+      }
+    } else
+      /* It's some other sort of file, or a directory, always include. */
+      cpio_append_fts_entry (entry);
+  }
+
+  if (fts_close (fts) == -1)
+    error (EXIT_FAILURE, errno, "write_kernel_modules: fts_close:
%s", modpath);
+}
+
+/* Copy the host files.
+ *
+ * Read the list of entries in hostfiles (which may contain
+ * wildcards).  Look them up in the filesystem, and add those files
+ * that exist.  Ignore any files that don't exist or are not readable.
+ */
+static void
+write_hostfiles (const char *hostfiles_file)
+{
+  char **hostfiles = load_file (hostfiles_file);
+
+  /* Hostfiles list can contain "." before each path - ignore it.
+   * It also contains each directory name before we enter it.  But
+   * we don't read that until we see a wildcard for that directory.
+   */
+  size_t i, j;
+  for (i = 0; hostfiles[i] != NULL; ++i) {
+    char *hostfile = hostfiles[i];
+    if (hostfile[0] == '.')
+      hostfile++;
+
+    struct stat statbuf;
+
+    /* Is it a wildcard? */
+    if (strchr (hostfile, '*') || strchr (hostfile, '?')) {
+      char *dirname = xstrdup (hostfile);
+      char *patt = strrchr (dirname, '/');
+      assert (patt);
+      *patt++ = '\0';
+
+      char **files = read_dir (dirname);
+      files = filter_fnmatch (files, patt, FNM_NOESCAPE);
+
+      /* Add matching files. */
+      for (j = 0; files[j] != NULL; ++j) {
+        char *tmp = xasprintf ("%s/%s", dirname, files[j]);
+
+        if (verbose >= 2)
+          fprintf (stderr, "including host file %s (matches %s)\n",
tmp, patt);
+
+        cpio_append (tmp);
+
+        free (tmp);
+      }
+    }
+    /* Else does this file/directory/whatever exist? */
+    else if (lstat (hostfile, &statbuf) == 0) {
+      if (verbose >= 2)
+        fprintf (stderr, "including host file %s (directly
referenced)\n",
+                 hostfile);
+
+      cpio_append_stat (hostfile, &statbuf);
+    } /* Ignore files that don't exist. */
+  }
+}
+
+/*----------*/
+/* Helper functions. */
+
+static void
+add_string (char ***argv, size_t *n_used, size_t *n_alloc, const char *str)
+{
+  char *new_str;
+
+  if (*n_used >= *n_alloc)
+    *argv = x2nrealloc (*argv, n_alloc, sizeof (char *));
+
+  if (str)
+    new_str = xstrdup (str);
+  else
+    new_str = NULL;
+
+  (*argv)[*n_used] = new_str;
+
+  (*n_used)++;
+}
+
+static size_t
+count_strings (char *const *argv)
+{
+  size_t argc;
+
+  for (argc = 0; argv[argc] != NULL; ++argc)
+    ;
+  return argc;
+}
+
+struct dir_cache {
+  char *path;
+  char **files;
+};
+
+static size_t
+dir_cache_hash (void const *x, size_t table_size)
+{
+  struct dir_cache const *p = x;
+  return hash_pjw (p->path, table_size);
+}
+
+static bool
+dir_cache_compare (void const *x, void const *y)
+{
+  struct dir_cache const *p = x;
+  struct dir_cache const *q = y;
+  return strcmp (p->path, q->path) == 0;
+}
+
+/* Read a directory into a list of strings.
+ *
+ * Previously looked up directories are cached and returned quickly,
+ * saving some considerable amount of time compared to reading the
+ * directory over again.  However this means you really must not
+ * alter the array of strings that are returned.
+ *
+ * Returns an empty list if the directory cannot be opened.
+ */
+static char **
+read_dir (const char *name)
+{
+  static Hash_table *ht = NULL;
+
+  if (!ht)
+    ht = hash_initialize (1024, NULL, dir_cache_hash, dir_cache_compare, NULL);
+
+  struct dir_cache key = { .path = (char *) name };
+  struct dir_cache *p = hash_lookup (ht, &key);
+  if (p)
+    return p->files;
+
+  char **files = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  DIR *dir = opendir (name);
+  if (!dir) {
+    /* If it fails to open, that's OK, skip to the end. */
+    /*perror (name);*/
+    goto done;
+  }
+
+  for (;;) {
+    errno = 0;
+    struct dirent *d = readdir (dir);
+    if (d == NULL) {
+      if (errno != 0)
+        /* But if it fails here, after opening and potentially reading
+         * part of the directory, that's a proper failure - inform the
+         * user and exit.
+         */
+        error (EXIT_FAILURE, errno, "%s", name);
+      break;
+    }
+
+    add_string (&files, &n_used, &n_alloc, d->d_name);
+  }
+
+  if (closedir (dir) == -1)
+    error (EXIT_FAILURE, errno, "closedir: %s", name);
+
+ done:
+  /* NULL-terminate the array. */
+  add_string (&files, &n_used, &n_alloc, NULL);
+
+  /* Add it to the hash for next time. */
+  p = xmalloc (sizeof *p);
+  p->path = (char *) name;
+  p->files = files;
+  p = hash_insert (ht, p);
+  assert (p != NULL);
+
+  return files;
+}
+
+/* Filter a list of strings and return only those matching the wildcard. */
+static char **
+filter_fnmatch (char **strings, const char *patt, int flags)
+{
+  char **out = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  int i, r;
+  for (i = 0; strings[i] != NULL; ++i) {
+    r = fnmatch (patt, strings[i], flags);
+    if (r == 0)
+      add_string (&out, &n_used, &n_alloc, strings[i]);
+    else if (r != FNM_NOMATCH)
+      error (EXIT_FAILURE, 0, "internal error: fnmatch ('%s',
'%s', %d) returned unexpected non-zero value %d\n",
+             patt, strings[i], flags, r);
+  }
+
+  add_string (&out, &n_used, &n_alloc, NULL);
+  return out;
+}
+
+/* Filter a list of strings and return only those which DON'T contain sub.
*/
+static char **
+filter_notmatching_substring (char **strings, const char *sub)
+{
+  char **out = NULL;
+  size_t n_used = 0, n_alloc = 0;
+
+  int i;
+  for (i = 0; strings[i] != NULL; ++i) {
+    if (strstr (strings[i], sub) == NULL)
+      add_string (&out, &n_used, &n_alloc, strings[i]);
+  }
+
+  add_string (&out, &n_used, &n_alloc, NULL);
+  return out;
+}
+
+/* Sort a list of strings, in place, with the comparison function supplied. */
+static void
+sort (char **strings, int (*compare) (const void *, const void *))
+{
+  qsort (strings, count_strings (strings), sizeof (char *), compare);
+}
+
+/* Return true iff path exists and is a directory.  This version
+ * follows symlinks.
+ */
+static int
+isdir (const char *path)
+{
+  struct stat statbuf;
+
+  if (stat (path, &statbuf) == -1)
+    return 0;
+
+  return S_ISDIR (statbuf.st_mode);
+}
+
+/* Copy contents of buffer to out_fd and keep out_offset correct. */
+static void
+write_to_fd (const void *buffer, size_t len)
+{
+  if (full_write (out_fd, buffer, len) != len)
+    error (EXIT_FAILURE, errno, "write");
+  out_offset += len;
+}
+
+/* Copy contents of file to out_fd. */
+static void
+write_file_to_fd (const char *filename)
+{
+  char buffer[BUFFER_SIZE];
+  int fd2;
+  ssize_t r;
+
+  if (verbose >= 2)
+    fprintf (stderr, "write_file_to_fd %s -> %d\n", filename,
out_fd);
+
+  fd2 = open (filename, O_RDONLY);
+  if (fd2 == -1)
+    error (EXIT_FAILURE, errno, "open: %s", filename);
+  for (;;) {
+    r = read (fd2, buffer, sizeof buffer);
+    if (r == 0)
+      break;
+    if (r == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
+        continue;
+      error (EXIT_FAILURE, errno, "read: %s", filename);
+    }
+    write_to_fd (buffer, r);
+  }
+
+  if (close (fd2) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", filename);
+}
+
+/* Copy file of given length to output, and fail if the file has
+ * changed size.
+ */
+static void
+write_file_len_to_fd (const char *filename, size_t len)
+{
+  char buffer[BUFFER_SIZE];
+  size_t count = 0;
+
+  if (verbose >= 2)
+    fprintf (stderr, "write_file_to_fd %s -> %d\n", filename,
out_fd);
+
+  int fd2 = open (filename, O_RDONLY);
+  if (fd2 == -1)
+    error (EXIT_FAILURE, errno, "open: %s", filename);
+  for (;;) {
+    ssize_t r = read (fd2, buffer, sizeof buffer);
+    if (r == 0)
+      break;
+    if (r == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR)
+        continue;
+      error (EXIT_FAILURE, errno, "read: %s", filename);
+    }
+    write_to_fd (buffer, r);
+    count += r;
+    if (count > len)
+      error (EXIT_FAILURE, 0, "write_file_len_to_fd: %s: file has
increased in size\n", filename);
+  }
+
+  if (close (fd2) == -1)
+    error (EXIT_FAILURE, errno, "close: %s", filename);
+
+  if (count != len)
+    error (EXIT_FAILURE, 0, "febootstrap-supermin-helper:
write_file_len_to_fd: %s: file has changed size\n", filename);
+}
+
+/* Load in a file, returning a list of lines. */
+static char **
+load_file (const char *filename)
+{
+  char **lines = 0;
+  size_t n_used = 0, n_alloc = 0;
+
+  FILE *fp;
+  fp = fopen (filename, "r");
+  if (fp == NULL)
+    error (EXIT_FAILURE, errno, "fopen: %s", filename);
+
+  char line[4096];
+  while (fgets (line, sizeof line, fp)) {
+    size_t len = strlen (line);
+    if (len > 0 && line[len-1] == '\n')
+      line[len-1] = '\0';
+    add_string (&lines, &n_used, &n_alloc, line);
+  }
+
+  add_string (&lines, &n_used, &n_alloc, NULL);
+  return lines;
+}
+
+/* Append the file pointed to by FTSENT to the cpio output. */
+static void
+cpio_append_fts_entry (FTSENT *entry)
+{
+  if (entry->fts_info & FTS_NS || entry->fts_info & FTS_NSOK)
+    cpio_append (entry->fts_path);
+  else
+    cpio_append_stat (entry->fts_path, entry->fts_statp);
+}
+
+/* Append the file named 'filename' to the cpio output. */
+static void
+cpio_append (const char *filename)
+{
+  struct stat statbuf;
+
+  if (lstat (filename, &statbuf) == -1)
+    error (EXIT_FAILURE, errno, "lstat: %s", filename);
+  cpio_append_stat (filename, &statbuf);
+}
+
+/* Append the file to the cpio output. */
+#define PADDING(len) ((((len) + 3) & ~3) - (len))
+
+#define CPIO_HEADER_LEN (6 + 13*8)
+
+static void
+cpio_append_stat (const char *filename, struct stat *statbuf)
+{
+  const char *orig_filename = filename;
+
+  if (*filename == '/')
+    filename++;
+  if (*filename == '\0')
+    filename = ".";
+
+  if (verbose >= 2)
+    fprintf (stderr, "cpio_append_stat %s 0%o -> %d\n",
+             orig_filename, statbuf->st_mode, out_fd);
+
+  /* Regular files and symlinks are the only ones that have a "body"
+   * in this cpio entry.
+   */
+  int has_body = S_ISREG (statbuf->st_mode) || S_ISLNK
(statbuf->st_mode);
+
+  size_t len = strlen (filename) + 1;
+
+  char header[CPIO_HEADER_LEN + 1];
+  snprintf (header, sizeof header,
+            "070701"            /* magic */
+            "%08X"              /* inode */
+            "%08X"              /* mode */
+            "%08X" "%08X"       /* uid, gid */
+            "%08X"              /* nlink */
+            "%08X"              /* mtime */
+            "%08X"              /* file length */
+            "%08X" "%08X"       /* device holding file
major, minor */
+            "%08X" "%08X"       /* for specials, device
major, minor */
+            "%08X"              /* name length (including \0 byte) */
+            "%08X",             /* checksum (not used by the kernel)
*/
+            (unsigned) statbuf->st_ino, statbuf->st_mode,
+            statbuf->st_uid, statbuf->st_gid,
+            (unsigned) statbuf->st_nlink, (unsigned) statbuf->st_mtime,
+            has_body ? (unsigned) statbuf->st_size : 0,
+            major (statbuf->st_dev), minor (statbuf->st_dev),
+            major (statbuf->st_rdev), minor (statbuf->st_rdev),
+            (unsigned) len, 0);
+
+  /* Write the header. */
+  write_to_fd (header, CPIO_HEADER_LEN);
+
+  /* Follow with the filename, and pad it. */
+  write_to_fd (filename, len);
+  size_t padding_len = PADDING (CPIO_HEADER_LEN + len);
+  write_padding (padding_len);
+
+  /* Follow with the file or symlink content, and pad it. */
+  if (has_body) {
+    if (S_ISREG (statbuf->st_mode))
+      write_file_len_to_fd (orig_filename, statbuf->st_size);
+    else if (S_ISLNK (statbuf->st_mode)) {
+      char tmp[PATH_MAX];
+      if (readlink (orig_filename, tmp, sizeof tmp) == -1)
+        error (EXIT_FAILURE, errno, "readlink: %s", orig_filename);
+      write_to_fd (tmp, statbuf->st_size);
+    }
+
+    padding_len = PADDING (statbuf->st_size);
+    write_padding (padding_len);
+  }
+}
+
+/* CPIO voodoo. */
+static void
+cpio_append_trailer (void)
+{
+  struct stat statbuf;
+  memset (&statbuf, 0, sizeof statbuf);
+  statbuf.st_nlink = 1;
+  cpio_append_stat ("TRAILER!!!", &statbuf);
+
+  /* CPIO seems to pad up to the next block boundary, ie. up to
+   * the next 512 bytes.
+   */
+  write_padding (((out_offset + 511) & ~511) - out_offset);
+  assert ((out_offset & 511) == 0);
+}
+
+/* Write 'len' bytes of zeroes out. */
+static void
+write_padding (size_t len)
+{
+  static const char buffer[512] = { 0 };
+
+  while (len > 0) {
+    size_t n = len < sizeof buffer ? len : sizeof buffer;
+    write_to_fd (buffer, n);
+    len -= n;
+  }
+}
diff --git a/febootstrap-supermin-helper.pod b/febootstrap-supermin-helper.pod
index 9ca7ca8..29f4b95 100644
--- a/febootstrap-supermin-helper.pod
+++ b/febootstrap-supermin-helper.pod
@@ -5,6 +5,7 @@ febootstrap-supermin-helper - Reconstruct initramfs from
supermin appliance.
 =head1 SYNOPSIS
 
  febootstrap-supermin-helper supermin.img hostfiles.txt host_cpu kernel initrd
+ febootstrap-supermin-helper input [...] host_cpu kernel initrd
 
 =head1 DESCRIPTION
 
@@ -15,11 +16,12 @@ L<febootstrap-to-supermin(8)>.
 
 =head1 PARAMETERS
 
-Of the five parameters, the first two are I<input> files, and the last
-two are I<output> files.
+Of the four or five required parameters, the first few are I<input>
+files, and the last two are I<output> files.
 
 C<supermin.img> and C<hostfiles.txt> are the input files which
-describe the supermin appliance.
+describe the supermin appliance.  (You can also use a directory name
+here which is searched for files).
 
 C<host_cpu> should be the host CPU, eg. C<x86_64> or C<i686>.
 
@@ -31,7 +33,7 @@ booting the appliance, and should be deleted straight
afterwards.
 
 =over 4
 
-=item B<--kmods file>
+=item B<-k file> | B<--kmods file>
 
 If this option is specified, then C<file> should be a list of
 wildcards matching kernel module names, eg:
@@ -89,8 +91,8 @@ Richard W.M. Jones <rjones @ redhat . com>
 
 =head1 COPYRIGHT
 
-(C) Copyright 2009 Red Hat Inc.,
-L<http://et.redhat.com/~rjones/febootstrap>.
+(C) Copyright 2009-2010 Red Hat Inc.,
+L<http://people.redhat.com/~rjones/febootstrap>.
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/febootstrap-supermin-helper.sh b/febootstrap-supermin-helper.sh
deleted file mode 100755
index cd5cf19..0000000
--- a/febootstrap-supermin-helper.sh
+++ /dev/null
@@ -1,143 +0,0 @@
-#!/bin/bash -
-# febootstrap-supermin-helper
-# (C) Copyright 2009 Red Hat Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-#
-# Written by Richard W.M. Jones <rjones at redhat.com>
-
-unset CDPATH
-
-TEMP=`getopt \
-        -o '' \
-        --long help,kmods: \
-        -n febootstrap-supermin-helper -- "$@"`
-if [ $? != 0 ]; then
-    echo "febootstrap-supermin-helper: problem parsing the command line
arguments"
-    exit 1
-fi
-eval set -- "$TEMP"
-
-usage ()
-{
-    echo "Usage: febootstrap-supermin-helper supermin.img hostfiles.txt
host_cpu kernel initrd"
-    echo "Please read febootstrap-supermin-helper(8) man page for more
information."
-}
-
-kmods=""
-
-while true; do
-    case "$1" in
-	--help)
-	    usage
-	    exit 0;;
-	--kmods)
-	    kmods=$2
-	    shift 2;;
-	--)
-	    shift
-	    break;;
-	*)
-	    echo "Internal error!"
-	    exit 1;;
-    esac
-done
-
-if [ $# -ne 5 ]; then
-    usage
-    exit 1
-fi
-
-set -e
-
-# Input files.
-supermin="$1"
-hostfiles="$2"
-
-host_cpu=$3
-
-# Output files.
-kernel="$4"
-initrd="$5"
-
-rm -f "$kernel" "$initrd"
-
-# Kernel:
-# Look for the most recent kernel named vmlinuz-*.<arch>* which has a
-# corresponding directory in /lib/modules/. If the architecture is x86, look
-# for any x86 kernel.
-#
-# RHEL 5 didn't append the arch to the kernel name, so look for kernels
-# without arch second.
-
-arch=$(echo $host_cpu | sed 's/^i.86$/i?86/')
-kernels=$(
-    ls -1dvr /boot/vmlinuz-*.$arch* 2>/dev/null | grep -v xen ||: ;
-    ls -1dvr /boot/vmlinuz-* 2>/dev/null | grep -v xen
-)
-
-if [ -z "$kernels" ]; then
-    echo "$0: failed to find a suitable kernel in /boot directory"
>&2
-    exit 1
-fi
-
-for f in $kernels; do
-    b=$(basename "$f")
-    b=$(echo "$b" | sed 's,vmlinuz-,,')
-    modpath="/lib/modules/$b"
-    if [ -d "$modpath" ]; then
-        ln -sf "$f" "$kernel"
-        break
-    fi
-    modpath-done
-
-if [ -z "$modpath" ]; then
-    echo "$0: failed to find a suitable kernel" >&2
-    exit 1
-fi
-
-# The initrd consists of these components:
-# (1) The base skeleton appliance that we constructed at build time.
-#     format = plain cpio (could be compressed cpio)
-# (2) The modules from modpath which are on the module whitelist.
-#     format = plain cpio
-# (3) The host files which match wildcards in hostfiles.
-#     format = plain cpio
-
-cp "$supermin" "$initrd" ;# (1)
-
-# Kernel modules (2).
-
-if [ -n "$kmods" ]; then
-    exec 5<"$kmods"
-    whitelist-    while read kmod 0<&5; do
-	whitelist="$whitelist -o -name $kmod"
-    done
-    exec 5<&-
-else
-    whitelist="-o -name *.ko"
-fi
-
-find "$modpath" \( -not -name '*.ko' $whitelist \) -a -print0
|
-  cpio --quiet -o -0 -H newc >> "$initrd"
-
-# Host files (3).
-
-hostfiles=$(readlink -f "$hostfiles")
-(cd / &&
-    ls -1df $(cat "$hostfiles") 2>/dev/null |
-    cpio -C 65536 --quiet -o -H newc ) >> "$initrd"
diff --git a/febootstrap-to-supermin.pod b/febootstrap-to-supermin.pod
index 6263f51..123327c 100644
--- a/febootstrap-to-supermin.pod
+++ b/febootstrap-to-supermin.pod
@@ -79,7 +79,8 @@ Use supermin appliances with caution.
 
 =head2 ANATOMY OF A SUPERMIN APPLIANCE
 
-A supermin appliance consists of two files:
+A supermin appliance consists usually of just two files, but can
+contain several files and directories from the list below:
 
 =over 4
 
@@ -90,6 +91,10 @@ call it anything you want) is the skeleton initramfs.  This
is like an
 initramfs built by L<febootstrap-to-initramfs(8)>, but all libraries
 and binaries are removed.
 
+Note that this file is a cpio file in cpio "newc" format, and is
+I<not> compressed (unlike initramfs files which are compressed cpio
+files).
+
 =item hostfiles.txt
 
 This plain text file contains a list of files that we need to add back
@@ -106,6 +111,14 @@ the file contains
 
  lib64/libreadline.so.6.*
 
+=item any directory
+
+You can specify a directory which should contain image file(s)
+and hostfile(s).
+
+Using a directory is useful either to keep the appliance-related files
+together, or to make more complex appliances containing optional bits.
+
 =back
 
 =head2 RECONSTRUCTING AN INITRAMFS FROM A SUPERMIN APPLIANCE
@@ -175,8 +188,8 @@ Richard W.M. Jones <rjones @ redhat . com>
 
 =head1 COPYRIGHT
 
-(C) Copyright 2009 Red Hat Inc.,
-L<http://et.redhat.com/~rjones/febootstrap>.
+(C) Copyright 2009-2010 Red Hat Inc.,
+L<http://people.redhat.com/~rjones/febootstrap>.
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/febootstrap-to-supermin.sh b/febootstrap-to-supermin.sh
index eccf18e..b6a2fd9 100755
--- a/febootstrap-to-supermin.sh
+++ b/febootstrap-to-supermin.sh
@@ -98,7 +98,7 @@ while read path <&7; do
     if [ "$path" = "./fakeroot.log" ]; then
 	:
 
-    # All we're going to keep are the special files /init, the daemon,
+    # All we're going to keep are the special files /init,
     # configuration files (/etc), devices and modifiable stuff (/var).
     elif [ "$path" = "./init" ]; then
         echo "$path" >&5
diff --git a/lib/.gitignore b/lib/.gitignore
new file mode 100644
index 0000000..048d811
--- /dev/null
+++ b/lib/.gitignore
@@ -0,0 +1,133 @@
+/Makefile.am
+/dummy.c
+/alloca.in.h
+/asnprintf.c
+/asprintf.c
+/at-func.c
+/basename-lgpl.c
+/bitrotate.h
+/c-ctype.c
+/c-ctype.h
+/chdir-long.c
+/chdir-long.h
+/chown.c
+/cloexec.c
+/cloexec.h
+/close-hook.c
+/close-hook.h
+/close.c
+/creat-safer.c
+/cycle-check.c
+/cycle-check.h
+/dev-ino.h
+/dirent--.h
+/dirent-safer.h
+/dirent.in.h
+/dirfd.c
+/dirname-lgpl.c
+/dirname.h
+/dup-safer.c
+/dup2.c
+/errno.in.h
+/error.c
+/error.h
+/exitfail.c
+/exitfail.h
+/fchdir.c
+/fchmodat.c
+/fchown-stub.c
+/fchownat.c
+/fclose.c
+/fcntl--.h
+/fcntl-safer.h
+/fcntl.c
+/fcntl.in.h
+/fd-safer.c
+/fdopendir.c
+/filevercmp.c
+/filevercmp.h
+/float+.h
+/float.in.h
+/fstatat.c
+/fts-cycle.c
+/fts.c
+/fts_.h
+/full-write.c
+/full-write.h
+/getcwd.c
+/getdtablesize.c
+/gettext.h
+/hash.c
+/hash.h
+/i-ring.c
+/i-ring.h
+/intprops.h
+/lchown.c
+/lstat.c
+/malloc.c
+/memchr.c
+/memchr.valgrind
+/mempcpy.c
+/memrchr.c
+/mkdir.c
+/mkdirat.c
+/open-safer.c
+/open.c
+/openat-die.c
+/openat-priv.h
+/openat-proc.c
+/openat-safer.c
+/openat.c
+/openat.h
+/opendir-safer.c
+/pipe-safer.c
+/printf-args.c
+/printf-args.h
+/printf-parse.c
+/printf-parse.h
+/realloc.c
+/rmdir.c
+/safe-read.c
+/safe-read.h
+/safe-write.c
+/safe-write.h
+/same-inode.h
+/save-cwd.c
+/save-cwd.h
+/size_max.h
+/stat.c
+/stdarg.in.h
+/stdbool.in.h
+/stddef.in.h
+/stdint.in.h
+/stdio-write.c
+/stdio.in.h
+/stdlib.in.h
+/strdup.c
+/strerror.c
+/string.in.h
+/stripslash.c
+/sys_stat.in.h
+/time.in.h
+/unistd--.h
+/unistd-safer.h
+/unistd.in.h
+/unlink.c
+/unlinkat.c
+/vasnprintf.c
+/vasnprintf.h
+/vasprintf.c
+/verify.h
+/wchar.in.h
+/write.c
+/xalloc-die.c
+/xalloc.h
+/xasprintf.c
+/xgetcwd.c
+/xgetcwd.h
+/xmalloc.c
+/xsize.h
+/xvasprintf.c
+/xvasprintf.h
+/hash-pjw.c
+/hash-pjw.h
diff --git a/m4/.gitignore b/m4/.gitignore
new file mode 100644
index 0000000..9601bdc
--- /dev/null
+++ b/m4/.gitignore
@@ -0,0 +1,90 @@
+/00gnulib.m4
+/gnulib-common.m4
+/gnulib-comp.m4
+/gnulib-tool.m4
+/onceonly.m4
+/alloca.m4
+/chdir-long.m4
+/chown.m4
+/cloexec.m4
+/close.m4
+/cycle-check.m4
+/d-ino.m4
+/d-type.m4
+/dirent-safer.m4
+/dirent_h.m4
+/dirfd.m4
+/dirname.m4
+/dos.m4
+/double-slash-root.m4
+/dup2.m4
+/errno_h.m4
+/error.m4
+/extensions.m4
+/fchdir.m4
+/fclose.m4
+/fcntl-o.m4
+/fcntl-safer.m4
+/fcntl.m4
+/fcntl_h.m4
+/fdopendir.m4
+/float_h.m4
+/fts.m4
+/getcwd-abort-bug.m4
+/getcwd-path-max.m4
+/getcwd.m4
+/getdtablesize.m4
+/hash.m4
+/i-ring.m4
+/include_next.m4
+/inline.m4
+/intmax_t.m4
+/inttypes_h.m4
+/lchown.m4
+/longlong.m4
+/lstat.m4
+/malloc.m4
+/memchr.m4
+/mempcpy.m4
+/memrchr.m4
+/mkdir.m4
+/mmap-anon.m4
+/mode_t.m4
+/multiarch.m4
+/open.m4
+/openat.m4
+/printf.m4
+/realloc.m4
+/rmdir.m4
+/safe-read.m4
+/safe-write.m4
+/save-cwd.m4
+/size_max.m4
+/ssize_t.m4
+/stat.m4
+/stdarg.m4
+/stdbool.m4
+/stddef_h.m4
+/stdint.m4
+/stdint_h.m4
+/stdio_h.m4
+/stdlib_h.m4
+/strdup.m4
+/strerror.m4
+/string_h.m4
+/sys_stat_h.m4
+/time_h.m4
+/unistd-safer.m4
+/unistd_h.m4
+/unlink.m4
+/vasnprintf.m4
+/vasprintf.m4
+/warn-on-use.m4
+/wchar_h.m4
+/wchar_t.m4
+/wint_t.m4
+/write.m4
+/xalloc.m4
+/xgetcwd.m4
+/xsize.m4
+/xvasprintf.m4
diff --git a/m4/gnulib-cache.m4 b/m4/gnulib-cache.m4
new file mode 100644
index 0000000..1f886cb
--- /dev/null
+++ b/m4/gnulib-cache.m4
@@ -0,0 +1,41 @@
+# Copyright (C) 2002-2010 Free Software Foundation, Inc.
+#
+# This file is free software, distributed under the terms of the GNU
+# General Public License.  As a special exception to the GNU General
+# Public License, this file may be distributed as part of a program
+# that contains a configuration script generated by Autoconf, under
+# the same distribution terms as the rest of that program.
+#
+# Generated by gnulib-tool.
+#
+# This file represents the specification of how gnulib-tool is used.
+# It acts as a cache: It is written and read by gnulib-tool.
+# In projects using CVS, this file is meant to be stored in CVS,
+# like the configure.ac and various Makefile.am files.
+
+
+# Specification in the form of a command-line invocation:
+#   gnulib-tool --import --dir=. --lib=libgnu --source-base=lib --m4-base=m4
--doc-base=doc --tests-base=tests --aux-dir=. --no-libtool --macro-prefix=gl
error filevercmp fts full-write hash hash-pjw xalloc xvasprintf
+
+# Specification in the form of a few gnulib-tool.m4 macro invocations:
+gl_LOCAL_DIR([])
+gl_MODULES([
+  error
+  filevercmp
+  fts
+  full-write
+  hash
+  hash-pjw
+  xalloc
+  xvasprintf
+])
+gl_AVOID([])
+gl_SOURCE_BASE([lib])
+gl_M4_BASE([m4])
+gl_PO_BASE([])
+gl_DOC_BASE([doc])
+gl_TESTS_BASE([tests])
+gl_LIB([libgnu])
+gl_MAKEFILE_NAME([])
+gl_MACRO_PREFIX([gl])
+gl_PO_DOMAIN([])
-- 
1.6.6.1