thr3ads.net - Libguestfs - Re: [Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init. [Feb 2016]

If this information is useful, please help other people find it:
Share via:

Richard W.M. Jones

2016-Feb-17 14:22 UTC

[Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

v1 -> v2:

- If we split out the init program into a separate init/ directory,
  that makes it much easier to build against an alternate libc.

I tried to build against uClibc, but uClibc requires an entire build
chain, which looked like it was going to be a massive ballache.

Rich.

Richard W.M. Jones

2016-Feb-17 14:22 UTC

head link

[Libguestfs] [PATCH supermin v2 1/4] init: Uncompress modules before adding them to the mini initrd.

When building the mini initrd, previously we copied the modules into
the initrd as-is, so for example if the module was xz-compressed, we
copied the foo.ko.xz file to the initrd.  This requires that the mini
init binary is linked to zlib & lzma, so that it knows how to
uncompress these modules when insmoding them at boot time.  Also since
the init is statically linked, it required _static_ versions of these
libraries.

This changes things so that the modules are uncompressed in the mini
initrd, so they are a little bit larger, but the init binary no longer
needs to be statically linked to zlib & lzma.

The init binary is smaller (966K -> 837K), but because we are storing
uncompressed modules in the mini initrd, the initrd as a whole becomes
larger (1.4M -> 2.6M)

However there are benefits to this change:

 - The code in the init binary is much simpler.

 - Removes the dependency on static zlib & lzma.

 - We can use an alternate libc to make a much smaller init binary
   (see following commits).
---
 README             |   4 +-
 configure.ac       |  74 ++-------------------------
 src/Makefile.am    |   1 -
 src/config.ml.in   |   2 +
 src/ext2_initrd.ml |  28 ++++++++++-
 src/init.c         | 145 -----------------------------------------------------
 6 files changed, 34 insertions(+), 220 deletions(-)

diff --git a/README b/README
index 34949a2..56a28f6 100644
--- a/README
+++ b/README
@@ -97,9 +97,9 @@ are building:
   qemu >= 0.13
   kernel >= 2.6.36
 
-  zlib (statically linked) - if your kernel uses gzipped modules
+  gunzip (command) - if your kernel uses gzipped modules
 
-  xz (statically linked) - if your kernel uses xz-compressed modules
+  unxz (command) - if your kernel uses xz-compressed modules
 
 Building and installing
 -----------------------
diff --git a/configure.ac b/configure.ac
index 126366b..0fe88c7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -127,77 +127,11 @@ dnl Check for fakeroot, only used a few drivers where the
host package
 dnl manager contains broken/unnecessary tests for root privs.
 AC_PATH_PROG(FAKEROOT,[fakeroot],[no])
 
-dnl Support for compressed input files, gzipped kernel modules.
-AC_CHECK_HEADER([zlib.h],[
-    AC_CHECK_LIB([z],[gzopen],[
-        zlib=yes
-        ZLIB_LIBS=-lz
+dnl Check for gunzip, only needed if you have gzip-compressed kernel modules.
+AC_PATH_PROG(GUNZIP,[gunzip],[no])
 
-        AC_MSG_CHECKING([for gzip static library])
-        old_CFLAGS="$CFLAGS"
-        old_LDFLAGS="$LDFLAGS"
-        old_LIBS="$LIBS"
-        CFLAGS="$CFLAGS -static"
-        LDFLAGS="$LDFLAGS -static"
-        LIBS="$LIBS -lz"
-        AC_LINK_IFELSE([
-            #include <stdio.h>
-            #include <stdlib.h>
-            #include <zlib.h>
-            int main () { gzFile g = gzopen ("test", "rb");
exit (g ? 1 : 0); }
-        ],[
-            zlib_static=yes
-            ZLIB_STATIC_LIBS="$ZLIB_LIBS"
-            AC_MSG_RESULT([yes])
-        ],[
-            AC_MSG_RESULT([no])
-        ])
-        CFLAGS="$old_CFLAGS"
-        LDFLAGS="$old_LDFLAGS"
-        LIBS="$old_LIBS"
-    ])
-])
-if test "x$zlib" = "xyes"; then
-    AC_DEFINE([HAVE_ZLIB],[1],[Define if you have zlib])
-    AC_SUBST([ZLIB_LIBS])
-fi
-if test "x$zlib_static" = "xyes"; then
-    AC_DEFINE([HAVE_ZLIB_STATIC],[1],[Define if you have static zlib])
-    AC_SUBST([ZLIB_STATIC_LIBS])
-fi
-
-dnl Support for xzed kernel modules.
-AC_CHECK_HEADER([lzma.h],[
-    AC_CHECK_LIB([lzma],[lzma_code],[
-        AC_MSG_CHECKING([for xz static library])
-        old_CFLAGS="$CFLAGS"
-        old_LDFLAGS="$LDFLAGS"
-        old_LIBS="$LIBS"
-        CFLAGS="$CFLAGS -static"
-        LDFLAGS="$LDFLAGS -static"
-        LIBS="$LIBS -llzma"
-        AC_LINK_IFELSE([
-            #include <stdio.h>
-            #include <stdlib.h>
-            #include <lzma.h>
-            int main () { lzma_stream s = LZMA_STREAM_INIT;
-                 exit (s.next_in == NULL ? 1 : 0); }
-        ],[
-            lzma_static=yes
-            LZMA_STATIC_LIBS="-llzma"
-            AC_MSG_RESULT([yes])
-        ],[
-            AC_MSG_RESULT([no])
-        ])
-        CFLAGS="$old_CFLAGS"
-        LDFLAGS="$old_LDFLAGS"
-        LIBS="$old_LIBS"
-    ])
-])
-if test "x$lzma_static" = "xyes"; then
-    AC_DEFINE([HAVE_LZMA_STATIC],[1],[Define if you have static lzma])
-    AC_SUBST([LZMA_STATIC_LIBS])
-fi
+dnl Check for unxz, only needed if you have xz-compressed kernel modules.
+AC_PATH_PROG(UNXZ,[unxz],[no])
 
 dnl mke2fs.
 AC_PATH_PROG([MKE2FS],[mke2fs],[no],
diff --git a/src/Makefile.am b/src/Makefile.am
index 6261c86..5a601fe 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -143,7 +143,6 @@ noinst_PROGRAMS = init
 init_SOURCES = init.c
 init_CFLAGS = -static
 init_LDFLAGS = -static
-init_LDADD = $(ZLIB_STATIC_LIBS) $(LZMA_STATIC_LIBS)
 
 CLEANFILES += ext2init-bin.S
 
diff --git a/src/config.ml.in b/src/config.ml.in
index 42cf833..19545b6 100644
--- a/src/config.ml.in
+++ b/src/config.ml.in
@@ -29,12 +29,14 @@ let dpkg_deb = "@DPKG_DEB@"
 let dpkg_query = "@DPKG_QUERY@"
 let dpkg_divert = "@DPKG_DIVERT@"
 let fakeroot = "@FAKEROOT@"
+let gunzip = "@GUNZIP@"
 let makepkg = "@MAKEPKG@"
 let pacman = "@PACMAN@"
 let pactree = "@PACTREE@"
 let pacman_g2 = "@PACMAN_G2@"
 let rpm = "@RPM@"
 let rpm2cpio = "@RPM2CPIO@"
+let unxz = "@UNXZ@"
 let urpmi = "@URPMI@"
 let yumdownloader = "@YUMDOWNLOADER@"
 let zypper = "@ZYPPER@"
diff --git a/src/ext2_initrd.ml b/src/ext2_initrd.ml
index b34f0e6..e49a19d 100644
--- a/src/ext2_initrd.ml
+++ b/src/ext2_initrd.ml
@@ -1,5 +1,5 @@
 (* supermin 5
- * Copyright (C) 2009-2014 Red Hat Inc.
+ * Copyright (C) 2009-2016 Red Hat Inc.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -105,8 +105,32 @@ let rec build_initrd debug tmpdir modpath initrd           
sprintf "cp -t %s %s" (quote initdir) (quote (modpath // modl)) in
           run_command cmd;
 
+          (* Uncompress the module, if the name ends in .xz or .gz. *)
+          let basename = Filename.basename modl in
+          let basename +            let len = String.length basename in
+            if Config.unxz <> "no" &&
Filename.check_suffix basename ".xz"
+            then (
+              let cmd = sprintf "%s %s"
+                                (quote Config.unxz)
+                                (quote (initdir // basename)) in
+              run_command cmd;
+              String.sub basename 0 (len-3)
+            )
+            else if Config.gunzip <> "no" &&
+                      Filename.check_suffix basename ".gz"
+            then (
+              let cmd = sprintf "%s %s"
+                                (quote Config.gunzip)
+                                (quote (initdir // basename)) in
+              run_command cmd;
+              String.sub basename 0 (len-3)
+            )
+            else
+              basename in
+
           (* Write module name to 'modules' file. *)
-          fprintf chan "%s\n" (Filename.basename modl);
+          fprintf chan "%s\n" basename;
           incr loaded
         )
     ) set
diff --git a/src/init.c b/src/init.c
index 814243a..25d6bc6 100644
--- a/src/init.c
+++ b/src/init.c
@@ -42,14 +42,6 @@
 
 #include <asm/unistd.h>
 
-#ifdef HAVE_ZLIB_STATIC
-#include <zlib.h>
-#endif
-
-#ifdef HAVE_LZMA_STATIC
-#include <lzma.h>
-#endif
-
 /* Maximum time to wait for the root device to appear (seconds).
  *
  * On slow machines with lots of disks (Koji running the 255 disk test
@@ -102,12 +94,6 @@ main ()
   print_uptime ();
   fprintf (stderr, "supermin: ext2 mini initrd starting up: "
            PACKAGE_VERSION
-#ifdef HAVE_ZLIB_STATIC
-           " zlib"
-#endif
-#ifdef HAVE_LZMA_STATIC
-           " xz"
-#endif
            "\n");
 
   read_cmdline ();
@@ -283,20 +269,6 @@ main ()
   exit (EXIT_FAILURE);
 }
 
-#if HAVE_LZMA_STATIC
-static int
-ends_with (const char *str, const char *suffix)
-{
-  if (!str || !suffix)
-    return 0;
-  size_t lenstr = strlen (str);
-  size_t lensuffix = strlen (suffix);
-  if (lensuffix >  lenstr)
-    return 0;
-  return strncmp (str + lenstr - lensuffix, suffix, lensuffix) == 0;
-}
-#endif
-
 static void
 insmod (const char *filename)
 {
@@ -305,118 +277,6 @@ insmod (const char *filename)
   if (verbose)
     fprintf (stderr, "supermin: internal insmod %s\n", filename);
 
-#ifdef HAVE_ZLIB_STATIC
-  int capacity = 64*1024;
-  char *buf = (char *) malloc (capacity);
-  int tmpsize = 8 * 1024;
-  char tmp[tmpsize];
-  int num;
-
-  errno = 0;
-  size = 0;
-
-  if (!buf) {
-    perror("malloc");
-    exit (EXIT_FAILURE);
-  }
-
-#ifdef HAVE_LZMA_STATIC
-  if (ends_with(filename, ".xz")) {
-    lzma_stream strm = LZMA_STREAM_INIT;
-    lzma_ret ret = lzma_stream_decoder(&strm, UINT64_MAX,
-          LZMA_CONCATENATED);
-    if (verbose)
-      fprintf (stderr, "supermin: running xz\n");
-    FILE *fd = fopen (filename, "r");
-    if (!fd) {
-      perror("popen failed");
-      exit (EXIT_FAILURE);
-    }
-    char tmp_out[tmpsize];
-    strm.avail_in = 0;
-    strm.next_out = tmp_out;
-    strm.avail_out = tmpsize;
-
-    lzma_action action = LZMA_RUN;
-
-    while (1) {
-      if (strm.avail_in == 0) {
-       strm.next_in = tmp;
-       strm.avail_in = fread(tmp, 1, tmpsize, fd);
-
-       if (ferror(fd)) {
-         // POSIX says that fread() sets errno if
-         // an error occurred. ferror() doesn't
-         // touch errno.
-         perror("Error reading input file");
-         exit (EXIT_FAILURE);
-       }
-       if (feof(fd)) action = LZMA_FINISH;
-      }
-
-      ret = lzma_code(&strm, action);
-
-      // Write and check write error before checking decoder error.
-      // This way as much data as possible gets written to output
-      // even if decoder detected an error.
-      if (strm.avail_out == 0 || ret != LZMA_OK) {
-          const size_t num =  tmpsize - strm.avail_out;
-          if (num > capacity) {
-               buf = (char*) realloc (buf, size*2);
-               if (!buf) {
-                    perror("realloc");
-                    exit (EXIT_FAILURE);
-               }
-               capacity = size;
-          }
-          memcpy (buf+size, tmp_out, num);
-          capacity -= num;
-          size += num;
-          strm.next_out = tmp_out;
-          strm.avail_out = tmpsize;
-      }
-      if (ret != LZMA_OK) {
-       if (ret == LZMA_STREAM_END) {
-           break;
-       } else {
-        perror("internal error");
-        exit(EXIT_FAILURE);
-       }
-     }
-    }
-    fclose (fd);
-    if (verbose)
-      fprintf (stderr, "done with xz %d read\n", size);
-  } else {
-#endif
-  gzFile gzfp = gzopen (filename, "rb");
-  if (gzfp == NULL) {
-    fprintf (stderr, "insmod: gzopen failed: %s", filename);
-    exit (EXIT_FAILURE);
-  }
-  while ((num = gzread (gzfp, tmp, tmpsize)) > 0) {
-    if (num > capacity) {
-      buf = (char*) realloc (buf, size*2);
-      if (!buf) {
-        perror("realloc");
-        exit (EXIT_FAILURE);
-      }
-      capacity = size;
-    }
-    memcpy (buf+size, tmp, num);
-    capacity -= num;
-    size += num;
-  }
-  if (num == -1) {
-    perror ("insmod: gzread");
-    exit (EXIT_FAILURE);
-  }
-  gzclose (gzfp);
-#ifdef HAVE_LZMA_STATIC
-}
-#endif
-
-#else
   int fd = open (filename, O_RDONLY);
   if (fd == -1) {
     fprintf (stderr, "insmod: open: %s: %m\n", filename);
@@ -439,7 +299,6 @@ insmod (const char *filename)
     offset += rc;
   } while (offset < size);
   close (fd);
-#endif
 
   if (init_module (buf, size, "") != 0) {
     fprintf (stderr, "insmod: init_module: %s: %s\n", filename,
moderror (errno));
@@ -447,10 +306,6 @@ insmod (const char *filename)
      * of a missing device.
      */
   }
-
-#ifdef HAVE_ZLIB_STATIC
-  free (buf);
-#endif
 }
 
 /* Mount /proc unless it's mounted already. */
-- 
2.5.0

Richard W.M. Jones

2016-Feb-17 14:22 UTC

head link

[Libguestfs] [PATCH supermin v2 2/4] init: Move init program into separate init directory.

This is just code motion.
---
 .gitignore       |   6 +-
 Makefile.am      |   6 +-
 configure.ac     |   1 +
 init/Makefile.am |  25 ++++
 init/init.c      | 428 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/Makefile.am  |  12 +-
 src/bin2s.pl     |  11 +-
 src/init.c       | 428 -------------------------------------------------------
 8 files changed, 466 insertions(+), 451 deletions(-)
 create mode 100644 init/Makefile.am
 create mode 100644 init/init.c
 delete mode 100644 src/init.c

diff --git a/.gitignore b/.gitignore
index 2163b37..ff3680e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -29,12 +29,9 @@ pod2htm?.tmp
 /depcomp
 /examples/basic-full-appliance
 /examples/basic-supermin.d
-/helper/ext2init.S
-/helper/supermin-helper
-/helper/supermin-helper.1
-/helper/init
 /html/supermin-helper.1.html
 /html/supermin.1.html
+/init/init
 /local*
 /INSTALL
 /install-sh
@@ -43,7 +40,6 @@ pod2htm?.tmp
 /src/.depend
 /src/config.ml
 /src/ext2init-bin.S
-/src/init
 /src/supermin
 /src/supermin.1
 /src/supermin-link.sh
diff --git a/Makefile.am b/Makefile.am
index 9679da9..681c52a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,5 +1,5 @@
 # supermin Makefile.am
-# (C) Copyright 2009-2013 Red Hat Inc.
+# (C) Copyright 2009-2016 Red Hat Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -14,12 +14,10 @@
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-#
-# Written by Richard W.M. Jones <rjones@redhat.com>
 
 ACLOCAL_AMFLAGS = -I m4
 
-SUBDIRS = lib src examples tests
+SUBDIRS = lib init src examples tests
 
 EXTRA_DIST = \
 	.gitignore \
diff --git a/configure.ac b/configure.ac
index 0fe88c7..15f6537 100644
--- a/configure.ac
+++ b/configure.ac
@@ -192,6 +192,7 @@ AC_CONFIG_FILES([src/supermin-link.sh],
 		[chmod +x,-w src/supermin-link.sh])
 AC_CONFIG_FILES([Makefile
         examples/Makefile
+        init/Makefile
         lib/Makefile
         src/config.ml
         src/Makefile
diff --git a/init/Makefile.am b/init/Makefile.am
new file mode 100644
index 0000000..075cf46
--- /dev/null
+++ b/init/Makefile.am
@@ -0,0 +1,25 @@
+# supermin Makefile.am
+# (C) Copyright 2009-2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+
+# init "script" used by ext2 initrd.
+
+CLEANFILES = *~
+
+noinst_PROGRAMS = init
+init_SOURCES = init.c
+init_CFLAGS = -static
+init_LDFLAGS = -static
diff --git a/init/init.c b/init/init.c
new file mode 100644
index 0000000..25d6bc6
--- /dev/null
+++ b/init/init.c
@@ -0,0 +1,428 @@
+/* supermin-helper reimplementation in C.
+ * Copyright (C) 2009-2014 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/* This very minimal init "script" goes in the mini-initrd used to
+ * boot the ext2-based appliance.  Note we have no shell, so we cannot
+ * use system(3) to run external commands.  In fact, we don't have
+ * very much at all, except this program, and some kernel modules.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <dirent.h>
+#include <time.h>
+#include <termios.h>
+#include <sys/types.h>
+#include <sys/mount.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+
+#include <asm/unistd.h>
+
+/* Maximum time to wait for the root device to appear (seconds).
+ *
+ * On slow machines with lots of disks (Koji running the 255 disk test
+ * in libguestfs) this really can take several minutes.
+ *
+ * Note that the actual wait time is approximately double the number
+ * given here because there is a delay which doubles until it reaches
+ * this value.
+ */
+#define MAX_ROOT_WAIT 300
+
+extern long init_module (void *, unsigned long, const char *);
+
+/* translation taken from module-init-tools/insmod.c  */
+static const char *moderror(int err)
+{
+  switch (err) {
+  case ENOEXEC:
+    return "Invalid module format";
+  case ENOENT:
+    return "Unknown symbol in module";
+  case ESRCH:
+    return "Module has wrong symbol version";
+  case EINVAL:
+    return "Invalid parameters";
+  default:
+    return strerror(err);
+  }
+}
+
+/* Leave this enabled for now.  When we get more confident in the boot
+ * process we can turn this off or make it configurable.
+ */
+#define verbose 1
+
+static void mount_proc (void);
+static void print_uptime (void);
+static void read_cmdline (void);
+static void insmod (const char *filename);
+static void show_directory (const char *dir);
+
+static char cmdline[1024];
+static char line[1024];
+
+int
+main ()
+{
+  mount_proc ();
+
+  print_uptime ();
+  fprintf (stderr, "supermin: ext2 mini initrd starting up: "
+           PACKAGE_VERSION
+           "\n");
+
+  read_cmdline ();
+
+  /* Create some fixed directories. */
+  mkdir ("/dev", 0755);
+  mkdir ("/root", 0755);
+  mkdir ("/sys", 0755);
+
+  /* Mount /sys. */
+  if (verbose)
+    fprintf (stderr, "supermin: mounting /sys\n");
+  if (mount ("sysfs", "/sys", "sysfs", 0,
"") == -1) {
+    perror ("mount: /sys");
+    exit (EXIT_FAILURE);
+  }
+
+  FILE *fp = fopen ("/modules", "r");
+  if (fp == NULL) {
+    perror ("fopen: /modules");
+    exit (EXIT_FAILURE);
+  }
+  while (fgets (line, sizeof line, fp)) {
+    size_t n = strlen (line);
+    if (n > 0 && line[n-1] == '\n')
+      line[--n] = '\0';
+
+    /* XXX Because of the way we construct the module list, the
+     * "modules" file can contain non-existent modules.  Ignore those
+     * for now.  Really we should add them as missing dependencies.
+     * See ext2initrd.c:ext2_make_initrd().
+     */
+    if (access (line, R_OK) == 0)
+      insmod (line);
+    else
+      fprintf (stderr, "skipped %s, module is missing\n", line);
+  }
+  fclose (fp);
+
+  /* Look for the ext2 filesystem device.  It's always the last one
+   * that was added.  Modern versions of libguestfs supply the
+   * expected name of the root device on the command line
+   * ("root=/dev/...").  For virtio-scsi this is required, because we
+   * must wait for the device to appear after the module is loaded.
+   */
+  char *root, *path;
+  size_t len;
+  root = strstr (cmdline, "root=");
+  if (root) {
+    root += 5;
+    if (strncmp (root, "/dev/", 5) == 0)
+      root += 5;
+    len = strcspn (root, " ");
+    root[len] = '\0';
+
+    asprintf (&path, "/sys/block/%s/dev", root);
+
+    uint64_t delay_ns = 250000;
+    int virtio_message = 0;
+    while (delay_ns <= MAX_ROOT_WAIT * UINT64_C(1000000000)) {
+      fp = fopen (path, "r");
+      if (fp != NULL)
+        goto found;
+
+      if (delay_ns > 1000000000) {
+	fprintf (stderr,
+		 "supermin: waiting another %" PRIu64 " ns for %s to
appear\n",
+		 delay_ns, path);
+        if (!virtio_message) {
+          fprintf (stderr,
+                   "This usually means your kernel doesn't support
virtio, or supermin was unable\n"
+                   "to load some kernel modules (see module loading
messages above).\n");
+          virtio_message = 1;
+        }
+      }
+
+      struct timespec t;
+      t.tv_sec = delay_ns / 1000000000;
+      t.tv_nsec = delay_ns % 1000000000;
+      nanosleep (&t, NULL);
+      delay_ns *= 2;
+    }
+  }
+  else {
+    path = strdup ("/sys/block/xdx/dev");
+
+    char class[3] = { 'v', 's', 'h' };
+    size_t i, j;
+    fp = NULL;
+    for (i = 0; i < sizeof class; ++i) {
+      for (j = 'z'; j >= 'a'; --j) {
+        path[11] = class[i];
+        path[13] = j;
+        fp = fopen (path, "r");
+        if (fp != NULL)
+          goto found;
+      }
+    }
+  }
+
+  fprintf (stderr,
+           "supermin: no ext2 root device found\n"
+           "Please include FULL verbose output in your bug
report.\n");
+  exit (EXIT_FAILURE);
+
+ found:
+  if (verbose)
+    fprintf (stderr, "supermin: picked %s as root device\n", path);
+
+  fgets (line, sizeof line, fp);
+  int major = atoi (line);
+  char *p = line + strcspn (line, ":") + 1;
+  int minor = atoi (p);
+
+  fclose (fp);
+  if (umount ("/sys") == -1) {
+    perror ("umount: /sys");
+    exit (EXIT_FAILURE);
+  }
+
+  /* Make current process the controlling process of the tty. */
+  setsid ();
+#ifdef TIOCSCTTY
+  if (ioctl (0, TIOCSCTTY, 1) == -1)
+    perror ("ioctl: TIOCSCTTY");
+#endif
+
+  if (verbose)
+    fprintf (stderr, "supermin: creating /dev/root as block special
%d:%d\n",
+             major, minor);
+
+  if (mknod ("/dev/root", S_IFBLK|0700, makedev (major, minor)) ==
-1) {
+    perror ("mknod: /dev/root");
+    exit (EXIT_FAILURE);
+  }
+
+  /* Mount new root and chroot to it. */
+  if (verbose)
+    fprintf (stderr, "supermin: mounting new root on /root\n");
+  if (mount ("/dev/root", "/root", "ext2",
MS_NOATIME, "") == -1) {
+    perror ("mount: /root");
+    exit (EXIT_FAILURE);
+  }
+
+  /* Note that pivot_root won't work.  See the note in
+   * Documentation/filesystems/ramfs-rootfs-initramfs.txt
+   * We could remove the old initramfs files, but let's not bother.
+   */
+  if (verbose)
+    fprintf (stderr, "supermin: chroot\n");
+
+  if (chroot ("/root") == -1) {
+    perror ("chroot: /root");
+    exit (EXIT_FAILURE);
+  }
+
+  chdir ("/");
+
+  /* Run /init from ext2 filesystem. */
+  execl ("/init", "init", NULL);
+  perror ("execl: /init");
+
+  /* /init failed to execute, but why?  Before we ditch, print some
+   * debug.  Although we have a full appliance, the fact that /init
+   * failed to run means we may not be able to run any commands.
+   */
+  show_directory ("/");
+  show_directory ("/bin");
+  show_directory ("/lib");
+  show_directory ("/lib64");
+  fflush (stderr);
+
+  exit (EXIT_FAILURE);
+}
+
+static void
+insmod (const char *filename)
+{
+  size_t size;
+
+  if (verbose)
+    fprintf (stderr, "supermin: internal insmod %s\n", filename);
+
+  int fd = open (filename, O_RDONLY);
+  if (fd == -1) {
+    fprintf (stderr, "insmod: open: %s: %m\n", filename);
+    exit (EXIT_FAILURE);
+  }
+  struct stat st;
+  if (fstat (fd, &st) == -1) {
+    perror ("insmod: fstat");
+    exit (EXIT_FAILURE);
+  }
+  size = st.st_size;
+  char buf[size];
+  size_t offset = 0;
+  do {
+    ssize_t rc = read (fd, buf + offset, size - offset);
+    if (rc == -1) {
+      perror ("insmod: read");
+      exit (EXIT_FAILURE);
+    }
+    offset += rc;
+  } while (offset < size);
+  close (fd);
+
+  if (init_module (buf, size, "") != 0) {
+    fprintf (stderr, "insmod: init_module: %s: %s\n", filename,
moderror (errno));
+    /* However ignore the error because this can just happen because
+     * of a missing device.
+     */
+  }
+}
+
+/* Mount /proc unless it's mounted already. */
+static void
+mount_proc (void)
+{
+  if (access ("/proc/uptime", R_OK) == -1) {
+    mkdir ("/proc", 0755);
+
+    if (verbose)
+      fprintf (stderr, "supermin: mounting /proc\n");
+
+    if (mount ("proc", "/proc", "proc", 0,
"") == -1) {
+      perror ("mount: /proc");
+      /* Non-fatal. */
+    }
+  }
+}
+
+/* Print contents of /proc/uptime. */
+static void
+print_uptime (void)
+{
+  FILE *fp = fopen ("/proc/uptime", "r");
+  if (fp == NULL) {
+    perror ("/proc/uptime");
+    return;
+  }
+
+  fgets (line, sizeof line, fp);
+  fclose (fp);
+
+  fprintf (stderr, "supermin: uptime: %s", line);
+}
+
+/* Read /proc/cmdline into cmdline global (or at least the first 1024
+ * bytes of it).
+ */
+static void
+read_cmdline (void)
+{
+  FILE *fp;
+  size_t len;
+
+  fp = fopen ("/proc/cmdline", "r");
+  if (fp == NULL) {
+    perror ("/proc/cmdline");
+    return;
+  }
+
+  fgets (cmdline, sizeof cmdline, fp);
+  fclose (fp);
+
+  len = strlen (cmdline);
+  if (len >= 1 && cmdline[len-1] == '\n')
+    cmdline[len-1] = '\0';
+
+  fprintf (stderr, "supermin: cmdline: %s\n", cmdline);
+}
+
+/* Display a directory on stderr.  This is used for debugging only. */
+static char
+dirtype (int dt)
+{
+  switch (dt) {
+  case DT_BLK: return 'b';
+  case DT_CHR: return 'c';
+  case DT_DIR: return 'd';
+  case DT_FIFO: return 'p';
+  case DT_LNK: return 'l';
+  case DT_REG: return '-';
+  case DT_SOCK: return 's';
+  case DT_UNKNOWN: return 'u';
+  default: return '?';
+  }
+}
+
+static void
+show_directory (const char *dirname)
+{
+  DIR *dir;
+  struct dirent *d;
+  struct stat statbuf;
+  char link[PATH_MAX+1];
+  ssize_t n;
+
+  fprintf (stderr, "supermin: debug: listing directory %s\n",
dirname);
+
+  if (chdir (dirname) == -1) {
+    perror (dirname);
+    return;
+  }
+
+  dir = opendir (".");
+  if (!dir) {
+    perror (dirname);
+    chdir ("/");
+    return;
+  }
+
+  while ((d = readdir (dir)) != NULL) {
+    fprintf (stderr, "%5lu %c %-16s", d->d_ino, dirtype
(d->d_type), d->d_name);
+    if (lstat (d->d_name, &statbuf) >= 0) {
+      fprintf (stderr, " %06o %ld %d:%d",
+               statbuf.st_mode, statbuf.st_size,
+               statbuf.st_uid, statbuf.st_gid);
+      if (S_ISLNK (statbuf.st_mode)) {
+        n = readlink (d->d_name, link, PATH_MAX);
+        if (n >= 0) {
+          link[n] = '\0';
+          fprintf (stderr, " -> %s", link);
+        }
+      }
+    }
+    fprintf (stderr, "\n");
+  }
+
+  closedir (dir);
+  chdir ("/");
+}
diff --git a/src/Makefile.am b/src/Makefile.am
index 5a601fe..443e25d 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -1,5 +1,5 @@
 # supermin Makefile.am
-# (C) Copyright 2009-2014 Red Hat Inc.
+# (C) Copyright 2009-2016 Red Hat Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -14,8 +14,6 @@
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-#
-# Written by Richard W.M. Jones <rjones@redhat.com>
 
 EXTRA_DIST = \
 	.depend \
@@ -138,18 +136,12 @@ supermin_LINK = \
 .ml.cmx:
 	$(OCAMLFIND) ocamlopt $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
 
-# init "script" used by ext2 initrd.
-noinst_PROGRAMS = init
-init_SOURCES = init.c
-init_CFLAGS = -static
-init_LDFLAGS = -static
-
 CLEANFILES += ext2init-bin.S
 
 ext2init-bin.o: ext2init-bin.S
 	$(CC) -o $@ -c $<
 
-ext2init-bin.S: init $(srcdir)/bin2s.pl
+ext2init-bin.S: ../init/init $(srcdir)/bin2s.pl
 	strip --strip-all $<
 	@file $< | grep -isq static || \
 	  (echo "*** error: init is not statically linked"; exit 1)
diff --git a/src/bin2s.pl b/src/bin2s.pl
index 8558126..6c70446 100755
--- a/src/bin2s.pl
+++ b/src/bin2s.pl
@@ -30,6 +30,9 @@ my ($buf, $i, $sz);
 open my $ifh, '<', $infile or die "open $infile: $!";
 open my $ofh, '>', $outfile or die "open $outfile: $!";
 
+my $infile_basename = $infile;
+$infile_basename =~ s{.*/}{};
+
 print $ofh <<"EOF";
 /* This file has been automatically generated from $infile by $0 */
 
@@ -37,11 +40,11 @@ print $ofh <<"EOF";
 \t.section .note.GNU-stack,"",%progbits
 \t.previous
 
-\t.globl\t_binary_${infile}_start
-\t.globl\t_binary_${infile}_end
+\t.globl\t_binary_${infile_basename}_start
+\t.globl\t_binary_${infile_basename}_end
 
 \t.section\t.rodata
-_binary_${infile}_start:
+_binary_${infile_basename}_start:
 EOF
 
 $sz = 0;
@@ -54,7 +57,7 @@ die "read $infile (at offset $sz): $!\n" if not
defined $i;
 close $ifh;
 
 print $ofh <<"EOF";
-_binary_${infile}_end:
+_binary_${infile_basename}_end:
 EOF
 
 close $ofh;
diff --git a/src/init.c b/src/init.c
deleted file mode 100644
index 25d6bc6..0000000
--- a/src/init.c
+++ /dev/null
@@ -1,428 +0,0 @@
-/* supermin-helper reimplementation in C.
- * Copyright (C) 2009-2014 Red Hat Inc.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
- */
-
-/* This very minimal init "script" goes in the mini-initrd used to
- * boot the ext2-based appliance.  Note we have no shell, so we cannot
- * use system(3) to run external commands.  In fact, we don't have
- * very much at all, except this program, and some kernel modules.
- */
-
-#include <config.h>
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <stdint.h>
-#include <string.h>
-#include <inttypes.h>
-#include <unistd.h>
-#include <errno.h>
-#include <fcntl.h>
-#include <dirent.h>
-#include <time.h>
-#include <termios.h>
-#include <sys/types.h>
-#include <sys/mount.h>
-#include <sys/stat.h>
-#include <sys/wait.h>
-
-#include <asm/unistd.h>
-
-/* Maximum time to wait for the root device to appear (seconds).
- *
- * On slow machines with lots of disks (Koji running the 255 disk test
- * in libguestfs) this really can take several minutes.
- *
- * Note that the actual wait time is approximately double the number
- * given here because there is a delay which doubles until it reaches
- * this value.
- */
-#define MAX_ROOT_WAIT 300
-
-extern long init_module (void *, unsigned long, const char *);
-
-/* translation taken from module-init-tools/insmod.c  */
-static const char *moderror(int err)
-{
-  switch (err) {
-  case ENOEXEC:
-    return "Invalid module format";
-  case ENOENT:
-    return "Unknown symbol in module";
-  case ESRCH:
-    return "Module has wrong symbol version";
-  case EINVAL:
-    return "Invalid parameters";
-  default:
-    return strerror(err);
-  }
-}
-
-/* Leave this enabled for now.  When we get more confident in the boot
- * process we can turn this off or make it configurable.
- */
-#define verbose 1
-
-static void mount_proc (void);
-static void print_uptime (void);
-static void read_cmdline (void);
-static void insmod (const char *filename);
-static void show_directory (const char *dir);
-
-static char cmdline[1024];
-static char line[1024];
-
-int
-main ()
-{
-  mount_proc ();
-
-  print_uptime ();
-  fprintf (stderr, "supermin: ext2 mini initrd starting up: "
-           PACKAGE_VERSION
-           "\n");
-
-  read_cmdline ();
-
-  /* Create some fixed directories. */
-  mkdir ("/dev", 0755);
-  mkdir ("/root", 0755);
-  mkdir ("/sys", 0755);
-
-  /* Mount /sys. */
-  if (verbose)
-    fprintf (stderr, "supermin: mounting /sys\n");
-  if (mount ("sysfs", "/sys", "sysfs", 0,
"") == -1) {
-    perror ("mount: /sys");
-    exit (EXIT_FAILURE);
-  }
-
-  FILE *fp = fopen ("/modules", "r");
-  if (fp == NULL) {
-    perror ("fopen: /modules");
-    exit (EXIT_FAILURE);
-  }
-  while (fgets (line, sizeof line, fp)) {
-    size_t n = strlen (line);
-    if (n > 0 && line[n-1] == '\n')
-      line[--n] = '\0';
-
-    /* XXX Because of the way we construct the module list, the
-     * "modules" file can contain non-existent modules.  Ignore those
-     * for now.  Really we should add them as missing dependencies.
-     * See ext2initrd.c:ext2_make_initrd().
-     */
-    if (access (line, R_OK) == 0)
-      insmod (line);
-    else
-      fprintf (stderr, "skipped %s, module is missing\n", line);
-  }
-  fclose (fp);
-
-  /* Look for the ext2 filesystem device.  It's always the last one
-   * that was added.  Modern versions of libguestfs supply the
-   * expected name of the root device on the command line
-   * ("root=/dev/...").  For virtio-scsi this is required, because we
-   * must wait for the device to appear after the module is loaded.
-   */
-  char *root, *path;
-  size_t len;
-  root = strstr (cmdline, "root=");
-  if (root) {
-    root += 5;
-    if (strncmp (root, "/dev/", 5) == 0)
-      root += 5;
-    len = strcspn (root, " ");
-    root[len] = '\0';
-
-    asprintf (&path, "/sys/block/%s/dev", root);
-
-    uint64_t delay_ns = 250000;
-    int virtio_message = 0;
-    while (delay_ns <= MAX_ROOT_WAIT * UINT64_C(1000000000)) {
-      fp = fopen (path, "r");
-      if (fp != NULL)
-        goto found;
-
-      if (delay_ns > 1000000000) {
-	fprintf (stderr,
-		 "supermin: waiting another %" PRIu64 " ns for %s to
appear\n",
-		 delay_ns, path);
-        if (!virtio_message) {
-          fprintf (stderr,
-                   "This usually means your kernel doesn't support
virtio, or supermin was unable\n"
-                   "to load some kernel modules (see module loading
messages above).\n");
-          virtio_message = 1;
-        }
-      }
-
-      struct timespec t;
-      t.tv_sec = delay_ns / 1000000000;
-      t.tv_nsec = delay_ns % 1000000000;
-      nanosleep (&t, NULL);
-      delay_ns *= 2;
-    }
-  }
-  else {
-    path = strdup ("/sys/block/xdx/dev");
-
-    char class[3] = { 'v', 's', 'h' };
-    size_t i, j;
-    fp = NULL;
-    for (i = 0; i < sizeof class; ++i) {
-      for (j = 'z'; j >= 'a'; --j) {
-        path[11] = class[i];
-        path[13] = j;
-        fp = fopen (path, "r");
-        if (fp != NULL)
-          goto found;
-      }
-    }
-  }
-
-  fprintf (stderr,
-           "supermin: no ext2 root device found\n"
-           "Please include FULL verbose output in your bug
report.\n");
-  exit (EXIT_FAILURE);
-
- found:
-  if (verbose)
-    fprintf (stderr, "supermin: picked %s as root device\n", path);
-
-  fgets (line, sizeof line, fp);
-  int major = atoi (line);
-  char *p = line + strcspn (line, ":") + 1;
-  int minor = atoi (p);
-
-  fclose (fp);
-  if (umount ("/sys") == -1) {
-    perror ("umount: /sys");
-    exit (EXIT_FAILURE);
-  }
-
-  /* Make current process the controlling process of the tty. */
-  setsid ();
-#ifdef TIOCSCTTY
-  if (ioctl (0, TIOCSCTTY, 1) == -1)
-    perror ("ioctl: TIOCSCTTY");
-#endif
-
-  if (verbose)
-    fprintf (stderr, "supermin: creating /dev/root as block special
%d:%d\n",
-             major, minor);
-
-  if (mknod ("/dev/root", S_IFBLK|0700, makedev (major, minor)) ==
-1) {
-    perror ("mknod: /dev/root");
-    exit (EXIT_FAILURE);
-  }
-
-  /* Mount new root and chroot to it. */
-  if (verbose)
-    fprintf (stderr, "supermin: mounting new root on /root\n");
-  if (mount ("/dev/root", "/root", "ext2",
MS_NOATIME, "") == -1) {
-    perror ("mount: /root");
-    exit (EXIT_FAILURE);
-  }
-
-  /* Note that pivot_root won't work.  See the note in
-   * Documentation/filesystems/ramfs-rootfs-initramfs.txt
-   * We could remove the old initramfs files, but let's not bother.
-   */
-  if (verbose)
-    fprintf (stderr, "supermin: chroot\n");
-
-  if (chroot ("/root") == -1) {
-    perror ("chroot: /root");
-    exit (EXIT_FAILURE);
-  }
-
-  chdir ("/");
-
-  /* Run /init from ext2 filesystem. */
-  execl ("/init", "init", NULL);
-  perror ("execl: /init");
-
-  /* /init failed to execute, but why?  Before we ditch, print some
-   * debug.  Although we have a full appliance, the fact that /init
-   * failed to run means we may not be able to run any commands.
-   */
-  show_directory ("/");
-  show_directory ("/bin");
-  show_directory ("/lib");
-  show_directory ("/lib64");
-  fflush (stderr);
-
-  exit (EXIT_FAILURE);
-}
-
-static void
-insmod (const char *filename)
-{
-  size_t size;
-
-  if (verbose)
-    fprintf (stderr, "supermin: internal insmod %s\n", filename);
-
-  int fd = open (filename, O_RDONLY);
-  if (fd == -1) {
-    fprintf (stderr, "insmod: open: %s: %m\n", filename);
-    exit (EXIT_FAILURE);
-  }
-  struct stat st;
-  if (fstat (fd, &st) == -1) {
-    perror ("insmod: fstat");
-    exit (EXIT_FAILURE);
-  }
-  size = st.st_size;
-  char buf[size];
-  size_t offset = 0;
-  do {
-    ssize_t rc = read (fd, buf + offset, size - offset);
-    if (rc == -1) {
-      perror ("insmod: read");
-      exit (EXIT_FAILURE);
-    }
-    offset += rc;
-  } while (offset < size);
-  close (fd);
-
-  if (init_module (buf, size, "") != 0) {
-    fprintf (stderr, "insmod: init_module: %s: %s\n", filename,
moderror (errno));
-    /* However ignore the error because this can just happen because
-     * of a missing device.
-     */
-  }
-}
-
-/* Mount /proc unless it's mounted already. */
-static void
-mount_proc (void)
-{
-  if (access ("/proc/uptime", R_OK) == -1) {
-    mkdir ("/proc", 0755);
-
-    if (verbose)
-      fprintf (stderr, "supermin: mounting /proc\n");
-
-    if (mount ("proc", "/proc", "proc", 0,
"") == -1) {
-      perror ("mount: /proc");
-      /* Non-fatal. */
-    }
-  }
-}
-
-/* Print contents of /proc/uptime. */
-static void
-print_uptime (void)
-{
-  FILE *fp = fopen ("/proc/uptime", "r");
-  if (fp == NULL) {
-    perror ("/proc/uptime");
-    return;
-  }
-
-  fgets (line, sizeof line, fp);
-  fclose (fp);
-
-  fprintf (stderr, "supermin: uptime: %s", line);
-}
-
-/* Read /proc/cmdline into cmdline global (or at least the first 1024
- * bytes of it).
- */
-static void
-read_cmdline (void)
-{
-  FILE *fp;
-  size_t len;
-
-  fp = fopen ("/proc/cmdline", "r");
-  if (fp == NULL) {
-    perror ("/proc/cmdline");
-    return;
-  }
-
-  fgets (cmdline, sizeof cmdline, fp);
-  fclose (fp);
-
-  len = strlen (cmdline);
-  if (len >= 1 && cmdline[len-1] == '\n')
-    cmdline[len-1] = '\0';
-
-  fprintf (stderr, "supermin: cmdline: %s\n", cmdline);
-}
-
-/* Display a directory on stderr.  This is used for debugging only. */
-static char
-dirtype (int dt)
-{
-  switch (dt) {
-  case DT_BLK: return 'b';
-  case DT_CHR: return 'c';
-  case DT_DIR: return 'd';
-  case DT_FIFO: return 'p';
-  case DT_LNK: return 'l';
-  case DT_REG: return '-';
-  case DT_SOCK: return 's';
-  case DT_UNKNOWN: return 'u';
-  default: return '?';
-  }
-}
-
-static void
-show_directory (const char *dirname)
-{
-  DIR *dir;
-  struct dirent *d;
-  struct stat statbuf;
-  char link[PATH_MAX+1];
-  ssize_t n;
-
-  fprintf (stderr, "supermin: debug: listing directory %s\n",
dirname);
-
-  if (chdir (dirname) == -1) {
-    perror (dirname);
-    return;
-  }
-
-  dir = opendir (".");
-  if (!dir) {
-    perror (dirname);
-    chdir ("/");
-    return;
-  }
-
-  while ((d = readdir (dir)) != NULL) {
-    fprintf (stderr, "%5lu %c %-16s", d->d_ino, dirtype
(d->d_type), d->d_name);
-    if (lstat (d->d_name, &statbuf) >= 0) {
-      fprintf (stderr, " %06o %ld %d:%d",
-               statbuf.st_mode, statbuf.st_size,
-               statbuf.st_uid, statbuf.st_gid);
-      if (S_ISLNK (statbuf.st_mode)) {
-        n = readlink (d->d_name, link, PATH_MAX);
-        if (n >= 0) {
-          link[n] = '\0';
-          fprintf (stderr, " -> %s", link);
-        }
-      }
-    }
-    fprintf (stderr, "\n");
-  }
-
-  closedir (dir);
-  chdir ("/");
-}
-- 
2.5.0

Richard W.M. Jones

2016-Feb-17 14:22 UTC

head link

[Libguestfs] [PATCH supermin v2 3/4] docs: Document how to link the init binary with an alternate libc.

Because of the previous commits, it is now possible to link the init
binary using alternate libc's.  This updates the documentation to
describe how to link it using dietlibc.

Using dietlibc reduces the overall size of the initrd substantially.
For libguestfs, 2.6M -> 1.8M.  Though this is still slightly larger
than the initrd using compressed modules (1.4M).
---
 README           | 24 ++++++++++++++++++++++++
 init/Makefile.am |  3 +++
 2 files changed, 27 insertions(+)

diff --git a/README b/README
index 56a28f6..ee2ec0e 100644
--- a/README
+++ b/README
@@ -40,6 +40,8 @@ Requirements
     - This is just used to generate the manpage.
 
   static libc
+    - Can be replaced with dietlibc (and maybe other alternate libc).
+      See section ``Alternate libc'' below.
 
   bash
 
@@ -144,3 +146,25 @@ Feedback and bugs
 Send feedback to libguestfs@redhat.com.  You can file bugs in
 https://bugzilla.redhat.com/ (under "Fedora", "supermin")
 
+Alternate libc
+--------------
+
+Supermin uses a small, statically linked "init" binary.  Normally
this
+is linked to static glibc, but static glibc produces enormous binaries
+(800KB+).  You can use an alternate libc if you prefer.  For example,
+using dietlibc, I can build a 22K init, about 1/40th of the size.
+
+  $ ls -l init/init
+  -rwxrwxr-x. 1 rjones rjones 21736 Feb 17 14:03 init/init
+
+- Dietlibc
+
+For dietlibc, build supermin like this:
+
+  ./configure
+  make clean
+  make -C init init CC="diet gcc"
+  make
+
+which builds the init using dietlibc, and then builds the rest of
+supermin with the dietlibc-using init binary.
diff --git a/init/Makefile.am b/init/Makefile.am
index 075cf46..5e02870 100644
--- a/init/Makefile.am
+++ b/init/Makefile.am
@@ -19,6 +19,9 @@
 
 CLEANFILES = *~
 
+# You can build this using an alternate libc if you want.  See
+# README ``Alternate libc''.
+
 noinst_PROGRAMS = init
 init_SOURCES = init.c
 init_CFLAGS = -static
-- 
2.5.0

Richard W.M. Jones

2016-Feb-17 14:22 UTC

head link

[Libguestfs] [PATCH supermin v2 4/4] init: Debug which libc is in use.

---
 init/init.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/init/init.c b/init/init.c
index 25d6bc6..38392c4 100644
--- a/init/init.c
+++ b/init/init.c
@@ -94,6 +94,15 @@ main ()
   print_uptime ();
   fprintf (stderr, "supermin: ext2 mini initrd starting up: "
            PACKAGE_VERSION
+#if defined(__dietlibc__)
+           " dietlibc"
+#elif defined(__GLIBC__)
+           " glibc"
+#elif defined(__NEWLIB_H__)
+           " newlib"
+#elif defined(__UCLIBC__)
+           " uClibc"
+#endif
            "\n");
 
   read_cmdline ();
-- 
2.5.0

Pino Toscano

2016-Feb-17 15:54 UTC

head link

Re: [Libguestfs] [PATCH supermin v2 1/4] init: Uncompress modules before adding them to the mini initrd.

On Wednesday 17 February 2016 14:22:31 Richard W.M. Jones
wrote:> When building the mini initrd, previously we copied the modules into
> the initrd as-is, so for example if the module was xz-compressed, we
> copied the foo.ko.xz file to the initrd.  This requires that the mini
> init binary is linked to zlib & lzma, so that it knows how to
> uncompress these modules when insmoding them at boot time.  Also since
> the init is statically linked, it required _static_ versions of these
> libraries.
> 
> This changes things so that the modules are uncompressed in the mini
> initrd, so they are a little bit larger, but the init binary no longer
> needs to be statically linked to zlib & lzma.
> 
> The init binary is smaller (966K -> 837K), but because we are storing
> uncompressed modules in the mini initrd, the initrd as a whole becomes
> larger (1.4M -> 2.6M)
> 
> However there are benefits to this change:
> 
>  - The code in the init binary is much simpler.
> 
>  - Removes the dependency on static zlib & lzma.
> 
>  - We can use an alternate libc to make a much smaller init binary
>    (see following commits).
LGTM, just one note below.
> ---
>  README             |   4 +-
>  configure.ac       |  74 ++-------------------------
>  src/Makefile.am    |   1 -
>  src/config.ml.in   |   2 +
>  src/ext2_initrd.ml |  28 ++++++++++-
>  src/init.c         | 145
-----------------------------------------------------
>  6 files changed, 34 insertions(+), 220 deletions(-)
> 
> diff --git a/README b/README
> index 34949a2..56a28f6 100644
> --- a/README
> +++ b/README
> @@ -97,9 +97,9 @@ are building:
>    qemu >= 0.13
>    kernel >= 2.6.36
>  
> -  zlib (statically linked) - if your kernel uses gzipped modules
> +  gunzip (command) - if your kernel uses gzipped modules
>  
> -  xz (statically linked) - if your kernel uses xz-compressed modules
> +  unxz (command) - if your kernel uses xz-compressed modules
>  
>  Building and installing
>  -----------------------
> diff --git a/configure.ac b/configure.ac
> index 126366b..0fe88c7 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -127,77 +127,11 @@ dnl Check for fakeroot, only used a few drivers where
the host package
>  dnl manager contains broken/unnecessary tests for root privs.
>  AC_PATH_PROG(FAKEROOT,[fakeroot],[no])
>  
> -dnl Support for compressed input files, gzipped kernel modules.
> -AC_CHECK_HEADER([zlib.h],[
> -    AC_CHECK_LIB([z],[gzopen],[
> -        zlib=yes
> -        ZLIB_LIBS=-lz
> +dnl Check for gunzip, only needed if you have gzip-compressed kernel
modules.
> +AC_PATH_PROG(GUNZIP,[gunzip],[no])
>  
> -        AC_MSG_CHECKING([for gzip static library])
> -        old_CFLAGS="$CFLAGS"
> -        old_LDFLAGS="$LDFLAGS"
> -        old_LIBS="$LIBS"
> -        CFLAGS="$CFLAGS -static"
> -        LDFLAGS="$LDFLAGS -static"
> -        LIBS="$LIBS -lz"
> -        AC_LINK_IFELSE([
> -            #include <stdio.h>
> -            #include <stdlib.h>
> -            #include <zlib.h>
> -            int main () { gzFile g = gzopen ("test",
"rb"); exit (g ? 1 : 0); }
> -        ],[
> -            zlib_static=yes
> -            ZLIB_STATIC_LIBS="$ZLIB_LIBS"
> -            AC_MSG_RESULT([yes])
> -        ],[
> -            AC_MSG_RESULT([no])
> -        ])
> -        CFLAGS="$old_CFLAGS"
> -        LDFLAGS="$old_LDFLAGS"
> -        LIBS="$old_LIBS"
> -    ])
> -])
> -if test "x$zlib" = "xyes"; then
> -    AC_DEFINE([HAVE_ZLIB],[1],[Define if you have zlib])
> -    AC_SUBST([ZLIB_LIBS])
> -fi
> -if test "x$zlib_static" = "xyes"; then
> -    AC_DEFINE([HAVE_ZLIB_STATIC],[1],[Define if you have static zlib])
> -    AC_SUBST([ZLIB_STATIC_LIBS])
> -fi
> -
> -dnl Support for xzed kernel modules.
> -AC_CHECK_HEADER([lzma.h],[
> -    AC_CHECK_LIB([lzma],[lzma_code],[
> -        AC_MSG_CHECKING([for xz static library])
> -        old_CFLAGS="$CFLAGS"
> -        old_LDFLAGS="$LDFLAGS"
> -        old_LIBS="$LIBS"
> -        CFLAGS="$CFLAGS -static"
> -        LDFLAGS="$LDFLAGS -static"
> -        LIBS="$LIBS -llzma"
> -        AC_LINK_IFELSE([
> -            #include <stdio.h>
> -            #include <stdlib.h>
> -            #include <lzma.h>
> -            int main () { lzma_stream s = LZMA_STREAM_INIT;
> -                 exit (s.next_in == NULL ? 1 : 0); }
> -        ],[
> -            lzma_static=yes
> -            LZMA_STATIC_LIBS="-llzma"
> -            AC_MSG_RESULT([yes])
> -        ],[
> -            AC_MSG_RESULT([no])
> -        ])
> -        CFLAGS="$old_CFLAGS"
> -        LDFLAGS="$old_LDFLAGS"
> -        LIBS="$old_LIBS"
> -    ])
> -])
> -if test "x$lzma_static" = "xyes"; then
> -    AC_DEFINE([HAVE_LZMA_STATIC],[1],[Define if you have static lzma])
> -    AC_SUBST([LZMA_STATIC_LIBS])
> -fi
> +dnl Check for unxz, only needed if you have xz-compressed kernel modules.
> +AC_PATH_PROG(UNXZ,[unxz],[no])
>  
>  dnl mke2fs.
>  AC_PATH_PROG([MKE2FS],[mke2fs],[no],
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 6261c86..5a601fe 100644
> --- a/src/Makefile.am
> +++ b/src/Makefile.am
> @@ -143,7 +143,6 @@ noinst_PROGRAMS = init
>  init_SOURCES = init.c
>  init_CFLAGS = -static
>  init_LDFLAGS = -static
> -init_LDADD = $(ZLIB_STATIC_LIBS) $(LZMA_STATIC_LIBS)
>  
>  CLEANFILES += ext2init-bin.S
>  
> diff --git a/src/config.ml.in b/src/config.ml.in
> index 42cf833..19545b6 100644
> --- a/src/config.ml.in
> +++ b/src/config.ml.in
> @@ -29,12 +29,14 @@ let dpkg_deb = "@DPKG_DEB@"
>  let dpkg_query = "@DPKG_QUERY@"
>  let dpkg_divert = "@DPKG_DIVERT@"
>  let fakeroot = "@FAKEROOT@"
> +let gunzip = "@GUNZIP@"
>  let makepkg = "@MAKEPKG@"
>  let pacman = "@PACMAN@"
>  let pactree = "@PACTREE@"
>  let pacman_g2 = "@PACMAN_G2@"
>  let rpm = "@RPM@"
>  let rpm2cpio = "@RPM2CPIO@"
> +let unxz = "@UNXZ@"
>  let urpmi = "@URPMI@"
>  let yumdownloader = "@YUMDOWNLOADER@"
>  let zypper = "@ZYPPER@"
> diff --git a/src/ext2_initrd.ml b/src/ext2_initrd.ml
> index b34f0e6..e49a19d 100644
> --- a/src/ext2_initrd.ml
> +++ b/src/ext2_initrd.ml
> @@ -1,5 +1,5 @@
>  (* supermin 5
> - * Copyright (C) 2009-2014 Red Hat Inc.
> + * Copyright (C) 2009-2016 Red Hat Inc.
>   *
>   * This program is free software; you can redistribute it and/or modify
>   * it under the terms of the GNU General Public License as published by
> @@ -105,8 +105,32 @@ let rec build_initrd debug tmpdir modpath initrd > 
sprintf "cp -t %s %s" (quote initdir) (quote (modpath // modl)) in
>            run_command cmd;
>  
> +          (* Uncompress the module, if the name ends in .xz or .gz. *)
> +          let basename = Filename.basename modl in
> +          let basename > +            let len = String.length basename
in
> +            if Config.unxz <> "no" &&
Filename.check_suffix basename ".xz"
> +            then (
> +              let cmd = sprintf "%s %s"
> +                                (quote Config.unxz)
> +                                (quote (initdir // basename)) in
> +              run_command cmd;
> +              String.sub basename 0 (len-3)
> +            )
> +            else if Config.gunzip <> "no" &&
> +                      Filename.check_suffix basename ".gz"
> +            then (
> +              let cmd = sprintf "%s %s"
> +                                (quote Config.gunzip)
> +                                (quote (initdir // basename)) in
> +              run_command cmd;
> +              String.sub basename 0 (len-3)
> +            )
> +            else
> +              basename in
An optimization here could be use xz/gunzip -c instead of cp+xz/gunzip,
which would avoid I/O during the appliance (re)build.
> +
>            (* Write module name to 'modules' file. *)
> -          fprintf chan "%s\n" (Filename.basename modl);
> +          fprintf chan "%s\n" basename;
>            incr loaded
>          )
>      ) set
> diff --git a/src/init.c b/src/init.c
> index 814243a..25d6bc6 100644
> --- a/src/init.c
> +++ b/src/init.c
> @@ -42,14 +42,6 @@
>  
>  #include <asm/unistd.h>
>  
> -#ifdef HAVE_ZLIB_STATIC
> -#include <zlib.h>
> -#endif
> -
> -#ifdef HAVE_LZMA_STATIC
> -#include <lzma.h>
> -#endif
> -
>  /* Maximum time to wait for the root device to appear (seconds).
>   *
>   * On slow machines with lots of disks (Koji running the 255 disk test
> @@ -102,12 +94,6 @@ main ()
>    print_uptime ();
>    fprintf (stderr, "supermin: ext2 mini initrd starting up: "
>             PACKAGE_VERSION
> -#ifdef HAVE_ZLIB_STATIC
> -           " zlib"
> -#endif
> -#ifdef HAVE_LZMA_STATIC
> -           " xz"
> -#endif
>             "\n");
>  
>    read_cmdline ();
> @@ -283,20 +269,6 @@ main ()
>    exit (EXIT_FAILURE);
>  }
>  
> -#if HAVE_LZMA_STATIC
> -static int
> -ends_with (const char *str, const char *suffix)
> -{
> -  if (!str || !suffix)
> -    return 0;
> -  size_t lenstr = strlen (str);
> -  size_t lensuffix = strlen (suffix);
> -  if (lensuffix >  lenstr)
> -    return 0;
> -  return strncmp (str + lenstr - lensuffix, suffix, lensuffix) == 0;
> -}
> -#endif
> -
>  static void
>  insmod (const char *filename)
>  {
> @@ -305,118 +277,6 @@ insmod (const char *filename)
>    if (verbose)
>      fprintf (stderr, "supermin: internal insmod %s\n",
filename);
>  
> -#ifdef HAVE_ZLIB_STATIC
> -  int capacity = 64*1024;
> -  char *buf = (char *) malloc (capacity);
> -  int tmpsize = 8 * 1024;
> -  char tmp[tmpsize];
> -  int num;
> -
> -  errno = 0;
> -  size = 0;
> -
> -  if (!buf) {
> -    perror("malloc");
> -    exit (EXIT_FAILURE);
> -  }
> -
> -#ifdef HAVE_LZMA_STATIC
> -  if (ends_with(filename, ".xz")) {
> -    lzma_stream strm = LZMA_STREAM_INIT;
> -    lzma_ret ret = lzma_stream_decoder(&strm, UINT64_MAX,
> -          LZMA_CONCATENATED);
> -    if (verbose)
> -      fprintf (stderr, "supermin: running xz\n");
> -    FILE *fd = fopen (filename, "r");
> -    if (!fd) {
> -      perror("popen failed");
> -      exit (EXIT_FAILURE);
> -    }
> -    char tmp_out[tmpsize];
> -    strm.avail_in = 0;
> -    strm.next_out = tmp_out;
> -    strm.avail_out = tmpsize;
> -
> -    lzma_action action = LZMA_RUN;
> -
> -    while (1) {
> -      if (strm.avail_in == 0) {
> -       strm.next_in = tmp;
> -       strm.avail_in = fread(tmp, 1, tmpsize, fd);
> -
> -       if (ferror(fd)) {
> -         // POSIX says that fread() sets errno if
> -         // an error occurred. ferror() doesn't
> -         // touch errno.
> -         perror("Error reading input file");
> -         exit (EXIT_FAILURE);
> -       }
> -       if (feof(fd)) action = LZMA_FINISH;
> -      }
> -
> -      ret = lzma_code(&strm, action);
> -
> -      // Write and check write error before checking decoder error.
> -      // This way as much data as possible gets written to output
> -      // even if decoder detected an error.
> -      if (strm.avail_out == 0 || ret != LZMA_OK) {
> -          const size_t num =  tmpsize - strm.avail_out;
> -          if (num > capacity) {
> -               buf = (char*) realloc (buf, size*2);
> -               if (!buf) {
> -                    perror("realloc");
> -                    exit (EXIT_FAILURE);
> -               }
> -               capacity = size;
> -          }
> -          memcpy (buf+size, tmp_out, num);
> -          capacity -= num;
> -          size += num;
> -          strm.next_out = tmp_out;
> -          strm.avail_out = tmpsize;
> -      }
> -      if (ret != LZMA_OK) {
> -       if (ret == LZMA_STREAM_END) {
> -           break;
> -       } else {
> -        perror("internal error");
> -        exit(EXIT_FAILURE);
> -       }
> -     }
> -    }
> -    fclose (fd);
> -    if (verbose)
> -      fprintf (stderr, "done with xz %d read\n", size);
> -  } else {
> -#endif
> -  gzFile gzfp = gzopen (filename, "rb");
> -  if (gzfp == NULL) {
> -    fprintf (stderr, "insmod: gzopen failed: %s", filename);
> -    exit (EXIT_FAILURE);
> -  }
> -  while ((num = gzread (gzfp, tmp, tmpsize)) > 0) {
> -    if (num > capacity) {
> -      buf = (char*) realloc (buf, size*2);
> -      if (!buf) {
> -        perror("realloc");
> -        exit (EXIT_FAILURE);
> -      }
> -      capacity = size;
> -    }
> -    memcpy (buf+size, tmp, num);
> -    capacity -= num;
> -    size += num;
> -  }
> -  if (num == -1) {
> -    perror ("insmod: gzread");
> -    exit (EXIT_FAILURE);
> -  }
> -  gzclose (gzfp);
> -#ifdef HAVE_LZMA_STATIC
> -}
> -#endif
> -
> -#else
>    int fd = open (filename, O_RDONLY);
>    if (fd == -1) {
>      fprintf (stderr, "insmod: open: %s: %m\n", filename);
> @@ -439,7 +299,6 @@ insmod (const char *filename)
>      offset += rc;
>    } while (offset < size);
>    close (fd);
> -#endif
>  
>    if (init_module (buf, size, "") != 0) {
>      fprintf (stderr, "insmod: init_module: %s: %s\n", filename,
moderror (errno));
> @@ -447,10 +306,6 @@ insmod (const char *filename)
>       * of a missing device.
>       */
>    }
> -
> -#ifdef HAVE_ZLIB_STATIC
> -  free (buf);
> -#endif
>  }
>  
>  /* Mount /proc unless it's mounted already. */
> 
-- 
Pino Toscano

Pino Toscano

2016-Feb-17 15:59 UTC

head link

Re: [Libguestfs] [PATCH supermin v2 4/4] init: Debug which libc is in use.

On Wednesday 17 February 2016 14:22:34 Richard W.M. Jones
wrote:> ---
>  init/init.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/init/init.c b/init/init.c
> index 25d6bc6..38392c4 100644
> --- a/init/init.c
> +++ b/init/init.c
> @@ -94,6 +94,15 @@ main ()
>    print_uptime ();
>    fprintf (stderr, "supermin: ext2 mini initrd starting up: "
>             PACKAGE_VERSION
> +#if defined(__dietlibc__)
> +           " dietlibc"
> +#elif defined(__GLIBC__)
> +           " glibc"
This needs to be put as last, as libc implementations that copied the
glibc headers (such as uclibc) define __GLIBC__...
> +#elif defined(__NEWLIB_H__)
> +           " newlib"
> +#elif defined(__UCLIBC__)
> +           " uClibc"
  #else
  # error unrecognized libc

While not having an identification string is not an issue, requiring it
might ease check whether supermin has actually been tested with a libc.

Thanks,
-- 
Pino Toscano

Pino Toscano

2016-Feb-17 16:03 UTC

head link

Re: [Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

On Wednesday 17 February 2016 14:22:30 Richard W.M. Jones
wrote:> v1 -> v2:
> 
> - If we split out the init program into a separate init/ directory,
>   that makes it much easier to build against an alternate libc.
> 
> I tried to build against uClibc, but uClibc requires an entire build
> chain, which looked like it was going to be a massive ballache.
Patches #1/2/3 LGTM, and #4 needs just a couple of easy changes.

-- 
Pino Toscano

Richard W.M. Jones

2016-Feb-17 16:13 UTC

head link

Re: [Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

On Wed, Feb 17, 2016 at 05:03:14PM +0100, Pino Toscano
wrote:> On Wednesday 17 February 2016 14:22:30 Richard W.M. Jones wrote:
> > v1 -> v2:
> > 
> > - If we split out the init program into a separate init/ directory,
> >   that makes it much easier to build against an alternate libc.
> > 
> > I tried to build against uClibc, but uClibc requires an entire build
> > chain, which looked like it was going to be a massive ballache.
> 
> Patches #1/2/3 LGTM, and #4 needs just a couple of easy changes.
Thanks - I've pushed those, with all the changes, except for
the no-libc one.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Maybe Matching Threads

Search for more maybe matching threads

Libguestfs - Feb 2016 - Re: [PATCH supermin 0/2] Allow an alternate libc to be used for init.

[Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

[Libguestfs] [PATCH supermin v2 1/4] init: Uncompress modules before adding them to the mini initrd.

[Libguestfs] [PATCH supermin v2 2/4] init: Move init program into separate init directory.

[Libguestfs] [PATCH supermin v2 3/4] docs: Document how to link the init binary with an alternate libc.

[Libguestfs] [PATCH supermin v2 4/4] init: Debug which libc is in use.

Re: [Libguestfs] [PATCH supermin v2 1/4] init: Uncompress modules before adding them to the mini initrd.

Re: [Libguestfs] [PATCH supermin v2 4/4] init: Debug which libc is in use.

Re: [Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

Re: [Libguestfs] [PATCH supermin 0/2] Allow an alternate libc to be used for init.

Maybe Matching Threads