klibc - Aug 2002 - initramfs scripts + cpio archive stuff

--32u276st3Jlj2kUU
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Hi all (two) klibc hackers! 8)

Here's the stuff I've been using to build an initramfs image and run it
on one of my machines.  Unfortunately, the "init" script contains hard
coded information about where to find the root, etc, but it should give
people a working base to start from.

It requires at least klibc 0.51, and is loosely based on what the kernel
does today, minus the initrd support.  This has been tested here using
an ext2 initrd image (I've not merged the initramfs unpacker stuff into
my kernel yet) but should work with very little modification to mount
IDE drives, etc.

Issues that need to be addressed at some point:

1. how do we get kernel command line parameters into initramfs?

   my favoured option is to remove the ones we care about from
   the kernel, they will then appear in the environment for the
   init script

2. load_ramdisk currently does not perform any detection of gzip vs
   romfs vs ext2 vs minix.  If this feature is required, we may have
   to look at file(1).  I've already looked at file(1) around klibc
   0.17 time, and it wasn't anywhere clean enough for klibc.

3. do we need to worry about initrd support?

   initrd has this nasty habbit of wanting to change the root device
   that should be mounted by poking a device number into some random
   /proc/sys file (yuck).  Obviously this isn't going to be portable
   to initramfs.  I'd argue that initrd support is obsolete with
   initramfs in the kernel.

4. running other scripts so initramfs can be extended easily by
   concatenating images.

   my idea here is to do something like:
	 for script in /addon/*; do $script; done
   so we run any scripts found in /addon which other cpio archives
   can add at unpack time.

Note that I made a cock up in klibc/ash/expand.c, which is fixed by
this:

diff -ur klibc/ash/expand.c klibc-0.51/ash/expand.c
--- klibc/ash/expand.c	Sun Aug 18 01:39:42 2002
+++ klibc-0.51/ash/expand.c	Sun Aug 18 23:34:10 2002
@@ -435,7 +435,7 @@
 	} else if (name == '@' || name == '*') {
 		if (*shellparam.p == NULL)
 			return 0;
-	} else if (name >= '1' || name <= '9') {
+	} else if (name >= '1' && name <= '9') {
 		int idx = name - '1';
 		ap = shellparam.p;
 		do {


The initrd I used to test (this contains ARM rather than Thumb
executables):

-rw-r--r--    1 src      src        225280 Aug 18 21:11 initramfs.ext2
-rw-r--r--    1 src      src         57645 Aug 19 00:07 initramfs.ext2.gz

Finally, here's the boot messages from my successful boot:

...
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
RAMDISK: Compressed image found at block 0
Freeing initrd memory: 3072K
VFS: Mounted root (ext2 filesystem).
Freeing init memory: 76K
INITRAMFS: Loading root image into /dev/ram1: done
INITRAMFS: Mounting /dev/ram1: done
INITRAMFS: Switching root: done
INIT: version 2.74 booting
Loading modules: 

INIT: Entering runlevel: 3
Starting system logger:  syslogd
Starting kernel logger:  klogd
Starting INET services:  inetd
Starting pcmcia Starting PCMCIA services: cardmgr.

Starting local Setting up network: lo eth0 

SA1100 Linux (experimental)
Kernel 2.5.31 on an armv4l
assabet login: 


Enjoy.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html


--32u276st3Jlj2kUU
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="initramfs.diff"

diff -ur -x CVS -x *.[oa] klibc/Makefile klibc-0.51/Makefile
--- klibc/Makefile	Sun Aug 18 02:00:41 2002
+++ klibc-0.51/Makefile	Sun Aug 18 22:27:28 2002
@@ -1,6 +1,47 @@
 SUBDIRS = klibc ash utils gzip
+KLIBSRC = klibc
+include MCONFIG
+
+PROGS   = ash/sh \
+          gzip/gzip \
+          utils/chroot utils/mkdir utils/mkfifo utils/mount \
+          utils/pivot_root utils/umount
+
+SCRIPTS = scripts/init scripts/load_ramdisk
+
+.PHONY: all clean spotless size initramfs
 
 all:
+	@set -e; for d in $(SUBDIRS); do $(MAKE) -C $$d $@; done
 
-%:
+clean spotless:
 	@set -e; for d in $(SUBDIRS); do $(MAKE) -C $$d $@; done
+	rm -rf initramfs*
+
+size:
+	@size $(PROGS)
+
+# Build the initramfs directory structure
+initramfs: all
+	@if [ `id -u` != 0 ]; then echo You must be root 1>&2; exit 1; fi
+	rm -rf initramfs/{bin,dev,scripts}
+	mkdir -p initramfs/{bin,dev,scripts}
+	set -e; for f in $(PROGS); do $(STRIP) -o initramfs/bin/`basename $$f` $$f;
done
+	set -e; for f in $(SCRIPTS); do cp $$f initramfs/$$f; chmod 755 initramfs/$$f;
done
+	mv initramfs/scripts/init initramfs/bin
+	mknod initramfs/dev/console c 5 1
+	mknod initramfs/dev/ram0 b 1 0
+	mknod initramfs/dev/ram1 b 1 1
+	mknod initramfs/dev/ram2 b 1 2
+	mknod initramfs/dev/mtdblock0 b 31 0
+	mknod initramfs/dev/mtdblock1 b 31 1
+	mknod initramfs/dev/mtdblock2 b 31 2
+	mknod initramfs/dev/mtdblock3 b 31 3
+
+# Create initramfs cpio archive
+initramfs.cpio: initramfs
+	cd initramfs; find . -print -depth | cpio -ovH crc > ../initramfs.cpio
+
+# gzip cpio arcive
+%.cpio.gz: %.cpio
+	gzip -c9 < $< > $@
diff -urN -x CVS -x *.[oa] klibc/scripts/init klibc-0.51/scripts/init
--- klibc/scripts/init	Thu Jan  1 01:00:00 1970
+++ klibc-0.51/scripts/init	Mon Aug 19 00:07:36 2002
@@ -0,0 +1,59 @@
+#!/bin/sh -e
+export COL=""
+
+# These should be passed from the kernel via the environment
+# (because eventually the kernel won't recognise them)  For
+# now, we hard code them here.
+devfs+#init=/bin/sh
+load_ramdisk=1
+prompt_ramdisk=0
+root=/dev/mtdblock3
+rootfstype=ext2
+root_mount_data+
+# Make directory for root filesystem mount
+mkdir -p /root
+
+if [ -n "$load_ramdisk" ]; then
+  export prompt_ramdisk
+  /scripts/load_ramdisk $root /dev/ram1 "root " &&
root=/dev/ram1
+fi
+
+if [ -n "$root" ]; then
+  # Mount the root filesystem
+  echo -n "INITRAMFS: Mounting $root: "
+  mount -o "$root_mount_data" -t "$rootfstype"
"$root" /root
+  cd /root
+  echo -e ${COL}done
+fi
+
+# Switch things around and leave us in /initramfs
+# The current directory is the new root.
+echo -n "INITRAMFS: Switching root: "
+mkdir -p -m 700 initramfs
+pivot_root . initramfs
+echo -e ${COL}done
+
+# Mount devfs (FIXME)
+if [ "$devfs" = "mount" ]; then
+  mount -t devfs devfs /dev
+fi
+
+# Pick up the new console device from the root filesystem
+exec </dev/console >/dev/console 2>&1
+
+# Unmount original devfs if any (FIXME)
+if [ "$devfs" = "mount" ]; then
+  umount -f /initramfs/dev
+fi
+
+# This is the bit at the end of init/main.c:init()
+set +e
+if [ -n "$init" -a -x "$init" ]; then
+  exec "$init"
+fi
+for iprog in /sbin/init /etc/init /bin/init /bin/sh; do
+  test -x $iprog && exec $iprog
+done
+echo PANIC: No init found.  Try passing init= option to kernel.
diff -urN -x CVS -x *.[oa] klibc/scripts/load_ramdisk
klibc-0.51/scripts/load_ramdisk
--- klibc/scripts/load_ramdisk	Thu Jan  1 01:00:00 1970
+++ klibc-0.51/scripts/load_ramdisk	Mon Aug 19 00:06:08 2002
@@ -0,0 +1,11 @@
+#!/bin/sh
+# We probably want something more complex here (FIXME)
+# eg, detect type of $1, decompress it or copy a certain
+# length into the device $2
+echo -n -e "INITRAMFS: Loading ${3}image into $2: "
+gzip -qdc < $1 > $2
+if [ "$?" = "0" -o "$?" = "2" ]; then
+  echo -e ${COL}done
+else
+  exit 1
+fi

--32u276st3Jlj2kUU--

Russell King wrote:
> 
> Issues that need to be addressed at some point:
> 
> 1. how do we get kernel command line parameters into initramfs?
> 
>    my favoured option is to remove the ones we care about from
>    the kernel, they will then appear in the environment for the
>    init script
> 
That's certainly an option (and will handle the stuff we remove from the 
kernel nicely.)  There is always /proc/cmdline, of course; however, 
going forward we may want to consider if the kernel command line really 
is going to be enough (255 characters maximum.)
> 2. load_ramdisk currently does not perform any detection of gzip vs
>    romfs vs ext2 vs minix.  If this feature is required, we may have
>    to look at file(1).  I've already looked at file(1) around klibc
>    0.17 time, and it wasn't anywhere clean enough for klibc.
Looking for major numbers hardly needs file(1).  A small set of 
hardcoded majors is just a matter of reading the beginning into a buffer 
and looking for the magic numbers in known locations.
> 3. do we need to worry about initrd support?
> 
>    initrd has this nasty habbit of wanting to change the root device
>    that should be mounted by poking a device number into some random
>    /proc/sys file (yuck).  Obviously this isn't going to be portable
>    to initramfs.  I'd argue that initrd support is obsolete with
>    initramfs in the kernel.
I agree; it's fundamentally braindamaged, and has been obsolete for 
quite a while even so; the currently favoured method of initrd uses 
pivot_root().
> 4. running other scripts so initramfs can be extended easily by
>    concatenating images.
> 
>    my idea here is to do something like:
> 	 for script in /addon/*; do $script; done
>    so we run any scripts found in /addon which other cpio archives
>    can add at unpack time.
How about calling it /rc.d/ instead :)

But yes, I like that idea.

For what it's worth, I actually got shared-but-not-dynamic klibc to work 
today (on i386; it requires some crt0.S changes although they are 
relatively minor -- rather than expanding the feature set of crt0.S I 
have now convinced myself that once you have a pointer and the stack is 
set up the rest of the work can be done generically, so I've moved it to 
C.)  It saved about 2-3K per binary, which was less than I'd expected; 
less than I would have expected, although it certainly makes blowing 4K 
on supporting real error messages a heck of a lot more defensible.

klibc itself, once stripped, is currently 23956 bytes in size, so it's 
only a win if we expect to have a dozen or more binaries on the typical 
initramfs...

Since klibc is shared but not dynamic, I have it set up to automatically 
generate a hash from the symbol addresses, and makes that part of the 
interpreter name.  You can only run against the exact same version of 
klibc as you compiled a binary against.

I'm not really sure if the dynamic klibc is actually worthwhile.  At 
least we have the technology now...

	-hpa