freebsd stable - Mar 2009 - Installing FreeBSD from USB flash drive - some experiments

It seems like questions about installing FreeBSD from a USB drive
come up at regular intervals.  With the low price of flash drives these days, it
would
be sort of nice to be able to offer a way for users to conveniently
roll their own USB installer images, or even to offer them for download
as well as the ISOs.

  I spent some time working on it this weekend, initially because I was
trying to get the current pfSense 1.2.2 (based on FreeBSD 7.0) to
install on the Geode-based fit-PC box.  (I eventually got there with
pfSense 1.2.3, based on 7.1.)

  While troubleshooting problems with installing pfSense from USB, I
ended up also building USB images from the 7.1-RELEASE bootonly and the
7.1-RELEASE disc1 ISOs.  To do this I've been hacking on Dario Freni's
fbsd-installiso2img.sh script from 2006, to convert FreeBSD or
FreeBSD-based ISOs to image files.  Initially I thought everything was
working fine, because the bootonly image came right up into sysinstall,
but when I dug a little deeper and tried with a USB image built from
the disc1 liveCD/install ISO, I found that it actually seems not to be
workable as it stands.

  I think I have found some limitations of the current sysinstall which
currently make it currently difficult to convert a working FreeBSD ISO
to an installable or live-disk flash image.   Booting the kernel is
easy.  Getting the rest of the way is harder.

  Background: AFAICT the release ISOs don't currently mount the media
they were booted from (da0a in this case.) Instead they load
boot/mfsroot.gz (compressed memory image) as the root fs, which is fine
and makes good sense, but they don't then mount the boot media anywhere
under it.  (My guess is that this was probably originally done to
support the floppy boot case.)

  So how can you direct it to install from the USB media?

  1) If you try to select media via the menu options, you can't select
USB specifically, nor specify an arbitrary device as the source
(/dev/da0a in this case), so you can't get the booted medium mounted to
install from, so you can't get the distributions accessible, so install
doesn't work.  Picking CD as the media seems to mean specifically an
cd or acd device.

  3) "Aha!" you say, "No problem.  If you're using a USB
system, you
can instead choose install from filesystem." Good point, but the file
system on the USB flash is not mounted, and you can't mount it via the
menu.

  4) "So, the LiveCD, then?" The LiveCD function in sysinstall seems
to
also currently require the install/liveCD image to be on a cd or acd
device.  With no such device, it won't mount anything and remains stuck
at the "Please insert a LiveCD or DVD" prompt until you cancel.

  5) "The emergency holographic shell, then?"  Yes, you can get into
the emergency holographic shell, with the very limited set of commands
from the mfsroot bin/sbin/stand.  However, if you try using the
emergency holographic shell to mount da0a - as far as I can tell, you
can't do so, because there is no mount in mfsboot's stand, only
mount_nfs.  (I'm guessing the "mount" operation for acd0 or cd0 is
coded into sysinstall?)  The usual mount command of course exists on the
media, but it's on the LiveCD portion which isn't mounted at this
point...

  As far as I could tell, from the state you're in after the system
boots, it's not possible to bootstrap up to where you can access the
contents of the flash drive.  Perhaps I'm just not creative enough.

  I'm looking for the simplest way to make this workable, so that one
could automatically generate a bootable flash drive image from a given
FreeBSD ISO.  Would adding an /etc/fstab mount entry for da0a (inside a
modified version of mfsroot.gz) be honored after loading the root, to
get the install medium premounted for bootstrapping?

  I'm attaching my hacked-up fbsd-installiso2img.sh in case anyone else
is interested in playing with it further.

  -- Clifton
----- cut here ----
#!/bin/sh
# fbsd-install-iso2img.sh
# Original version by Dario Freni 9/2006
# Enhancements by Clifton Royston 3/2009.
# License: Beerware

# You can set some variables here. Edit them to fit your needs.

# Set serial variable to 0 if you don't want serial console at all,
# 1 if you want comconsole and 2 if you want comconsole and vidconsole
serial=0

# Set nofstab=1 to not create any initial fstab on the USB drive;
# this makes the next two settings largely irrelevant.
nofstab=0

# Set rootperm=rw for root fs to mount r/w from the USB drive
#  (Should be unnecessary.)
rootperm=ro

# Set USBLABEL here or with -L label to label the image file system, 
#  to help the loader find the root file system when booting; 
#  otherwise the USB must come up as da0 to finish loading successfully.
USBLABELlbparams
# Set dopause=1 here or with -p to pause and allow review or editing of
#  the flash image before finalizing the image.
dopause=0

pause() {
 echo "Press enter to continue"
 read foo
}


set -u

if [ $# -ge 3 ]; then
  flag=$1
  if [ ${flag} = "-p" ]; then
    dopause=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-n" ]; then
    nofstab=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-L" ]; then
    shift;
    USBLABEL=$1; shift
    lbparams="-L ${USBLABEL}"
  fi
fi
if [ $# -lt 2 ]; then
  echo "Usage: $0 [-p] [-n] [-L vollabel] source-iso-path
output-img-path"
  echo "	-p	pause for review before finalizing image"
  echo "	-n	don't update the /etc/fstab within the image"
  echo "	-L	set file system label on image, to help loader find it"
  exit 1
fi

isoimage=$1; shift
imgoutfile=$1; shift

export tmpdir=$(mktemp -d -t fbsdmount)

# Temp file and directory to be used later
export tmpfile=$(mktemp -t bsdmount)

export isodev=$(mdconfig -a -t vnode -f ${isoimage})

echo "#### Building bootable UFS image ####"

ISOSIZE=$(du -k ${isoimage} | awk '{print $1}')
SECTS=$((($ISOSIZE + ($ISOSIZE/5))*2))

# Root partition size

echo "Initializing image..."
dd if=/dev/zero of=${imgoutfile} count=${SECTS}
ls -l ${imgoutfile}
export imgdev=$(mdconfig -a -t vnode -f ${imgoutfile})

bsdlabel -w -B ${imgdev}
newfs -O1 ${lbparams} /dev/${imgdev}a

mkdir -p ${tmpdir}/iso ${tmpdir}/img

mount -r -t cd9660 /dev/${isodev} ${tmpdir}/iso
mount /dev/${imgdev}a ${tmpdir}/img

echo "Copying files to the image via cpio"
( cd ${tmpdir}/iso && find . -print -depth | cpio -dump ${tmpdir}/img )
# Dump doesn't work from an ISO file system, too bad.
# echo "Copying files to the image via dump/restore..."
## dump -0f - /dev/${isodev} | (cd ${tmpdir}/img && restore -r -f - ) 

#bzcat ${tmpdir}/iso/dist/root.dist.bz2 | mtree -PUr -p ${tmpdir}/img
2>&1 > /dev/null

if [ ${nofstab} -ne 1 ]; then
  echo "Saving original /etc/fstab as /etc/fstab.orig"
  mv ${tmpdir}/img/etc/fstab  ${tmpdir}/img/etc/fstab.orig
  echo "Replacing /etc/fstab, so loader can find root filesystem on
flash!"
  if [ "${USBLABEL}" != "" ]; then
    echo "/dev/ufs/${USBLABEL} / ufs ${rootperm} 0 0" >
${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  else 
    echo "/dev/da0a / ufs ${rootperm} 0 0" >
${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  fi
else
  echo "Skipping write of image /etc/fstab"

fi

if [ ${serial} -eq 2 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-D" > ${tmpdir}/img/boot.config
        echo 'console="comconsole, vidconsole"' >>
${tmpdir}/img/boot/loader.conf
elif [ ${serial} -eq 1 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-h" > ${tmpdir}/img/boot.config
        echo 'console="comconsole"' >>
${tmpdir}/img/boot/loader.conf
fi

if [ ${dopause} -eq 1 ]; then
  echo "Pausing to allow manual review and modification of image
file:"
  echo "Image is located in ${tmpdir}/img"
  echo "If you need to fix up ${tmpdir}/img/etc/fstab, now is the
time."
  pause
fi


cleanup() {
    umount ${tmpdir}/iso
    mdconfig -d -u ${isodev}
    umount ${tmpdir}/img
    mdconfig -d -u ${imgdev}
    rm -rf ${tmpdir} ${tmpfile}
}

cleanup

ls -lh ${imgoutfile}

echo "To write the image to flash, use dd, for example:"
echo "   dd if=${imgoutfile} of=/dev/da0 bs=4M"
----- cut here ----
-- 
    Clifton Royston  --  cliftonr@iandicomputing.com / cliftonr@lava.net
          I and I Computing * http://www.iandicomputing.com/
 Custom programming, network design, systems and network consulting services

On Wednesday 11 March 2009 13:55:58 Clifton Royston
wrote:
>   I think I have found some limitations of the current sysinstall which
> currently make it currently difficult to convert a working FreeBSD ISO
> to an installable or live-disk flash image.   Booting the kernel is
> easy.  Getting the rest of the way is harder.
I have a working USB installer for 7.1 but it is not terribly straightfoward.
>   Background: AFAICT the release ISOs don't currently mount the media
> they were booted from (da0a in this case.) Instead they load
> boot/mfsroot.gz (compressed memory image) as the root fs, which is fine
> and makes good sense, but they don't then mount the boot media anywhere
> under it.  (My guess is that this was probably originally done to
> support the floppy boot case.)
Yes, I think so.
>   So how can you direct it to install from the USB media?
>
>   1) If you try to select media via the menu options, you can't select
> USB specifically, nor specify an arbitrary device as the source
> (/dev/da0a in this case), so you can't get the booted medium mounted to
> install from, so you can't get the distributions accessible, so install
> doesn't work.  Picking CD as the media seems to mean specifically an
> cd or acd device.
For some reason the "Install from UFS" thing doesn't let you mount
anything, I
think this would be fairly easy to fix.

I didn't actually attempt to fix this, instead I split my USB stick in 2.
The
first partition was 2.5Gb FAT32, the last 1.5Gb was UFS. I put the CD image on 
there and did boot0cfg etc..

The reason FAT goes first is that otherwise when you put it into a Windows PC 
(as I'd like people to be able to do to edit the install.cfg) Windows gets 
confused and wants to format the disk.. Putting UFS last gets around this and 
it still works fine.
>   5) "The emergency holographic shell, then?"  Yes, you can get
into
> the emergency holographic shell, with the very limited set of commands
> from the mfsroot bin/sbin/stand.  However, if you try using the
> emergency holographic shell to mount da0a - as far as I can tell, you
> can't do so, because there is no mount in mfsboot's stand, only
> mount_nfs.  (I'm guessing the "mount" operation for acd0 or
cd0 is
> coded into sysinstall?)  The usual mount command of course exists on the
> media, but it's on the LiveCD portion which isn't mounted at this
> point...
Yeah, the lack of mount in /stand on the install disk is a big caveat.

I suspect you could go into label and add /dev/da0a (also, argh, DD is bad, 
use an MBR) and tell it not to newfs it and then sysinstall will mount it for 
you.

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20090311/71b77682/attachment.pgp

Here's the steps I use to create a 1GB USB image:

# dd if=/dev/zero of=bootable.image bs=1m count=1 oseek=1000 conv=sparse
# mdconfig -a -t vnode -f bootable.image -u 0
# newfs -m 0 -o space -n /dev/md0
# mount /dev/md0 /mnt

# cd /usr/src
# make installworld DESTDIR=/mnt
# make distribution DESTDIR=/mnt
# make installkernel DESTDIR=/mnt
# umount /mnt

At this point you have a file "bootable.image" but instead of actually
making that a bootable dd image, I choose to create a dump file which is 
a bit more flexible as you can restore it to a USB stick of any size.

# dump -0 -C 8 -f - /dev/md0 | gzip -9 > bootable.dump.gz
# mdconfig -d -u 0


At this point, you have a dump file which you can use to create a 
bootable USB as follows:

Assuming the USB stick is /dev/da0 !

# dd if=/dev/zero of=/dev/da0 bs=16k
# fdisk -BI /dev/da0
# disklabel -B -w /dev/da0s1
# newfs -m 0 -o space -n /dev/da0s1a
# mount -o noatime,async /dev/da0s1a /mnt
# gzcat bootable.dump.gz | ( cd /mnt ; restore -rvf - )
# umount /mnt



Hope that helps


- Andrew