Syslinux - Oct 2017 - Unable to find linux kernel on ext4, filesystem-related root cause assumed

Dear all,

I'm working on a shell script which is producing a bootable image file. It
contains a populated MBR and a single file system (ext4) which again contains a
debootstrabed debian stable distribution.

Unfortunately, I'm experiencing problems with the bootability of the
produced image. When selecting the installed system at the boot prompt, Syslinux
responds with something like "couldnt load vmlinuz: File or directory not
found".

The syslinux.cfg is located at the root of the said file system. The linux
kernel and the initrd are relocated from their initial location at /boot to the
file system root as well and are correctly referenced in syslinux.cfg.

Now the strange thing:
The problem only occurs when said relocation happens through `mv` command:
`rm /tmp/generate-image/mount/initrd.img
/tmp/generate-image/mount/initrd.img.old /tmp/generate-image/mount/vmlinuz
/tmp/generate-image/mount/vmlinuz.old`
`mv /tmp/generate-image/mount/boot/initrd.img-4.9.0-4-amd64
/tmp/generate-image/mount/initrd`
`mv /tmp/generate-image/mount/boot/vmlinuz-4.9.0-4-amd64
/tmp/generate-image/mount/vmlinuz`

The problem does NOT occur when relocation is implemented using `cp`:
`rm /tmp/generate-image/mount/initrd.img
/tmp/generate-image/mount/initrd.img.old /tmp/generate-image/mount/vmlinuz
/tmp/generate-image/mount/vmlinuz.old`
`cp /tmp/generate-image/mount/boot/initrd.img-4.9.0-4-amd64
/tmp/generate-image/mount/initrd`
`cp /tmp/generate-image/mount/boot/vmlinuz-4.9.0-4-amd64
/tmp/generate-image/mount/vmlinuz`

This was hard to believe for me, but it was actually reproducible using qemu.
The fact that all steps are executed through shell script rule out any
user-introduced errors.

I'm posting my bash shell script (which contains syslinux.cfg) below. You
are invited to execute it yourself.  I'm also eager to share my generated
1024MiB-sized image which can be mounted through losetup or booted-from using
qemu.

In the syslinux-wiki, it was mentioned that there are known bugs with the
current syslinux version. Is a list available? Are there any known limitations
regarding ext4 support? Is the underlying code of ldlinux.sys available? I
wasn't able to find it in the syslinux source tree.

Used versions:
extlinux 6.03
QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6+deb9u2)

Help or advice is much appreciated,
best regards,
Manuel


#######################

#!/bin/bash
set -o xtrace


BASEDIR=/tmp/generate-image
IMAGE=${BASEDIR}/image
MOUNT=${BASEDIR}/mount
PARTITIONOFFSET=$[ 2 ** 20 ]
PARTITIONOFFSET_512B_SECTORS=$[ ${PARTITIONOFFSET} / 512 ]
PARTITIONLAYOUT="start=${PARTITIONOFFSET_512B_SECTORS}, bootable, type=83,
name=system-part"


echo "Creating directory ${BASEDIR}"
mkdir --parents "${BASEDIR}"

echo "Creating sparse file ${IMAGE}"
fallocate --length 1024M "${IMAGE}"

echo "Creating partition on ${IMAGE} with specs: ${PARTITIONLAYOUT}"
echo "${PARTITIONLAYOUT}" | sfdisk "${IMAGE}" --no-reread
--no-tell-kernel

echo "Creating file system on ${IMAGE} with offset ${PARTITIONOFFSET}"
mkfs.ext4 -F -L system-fs -E
offset=${PARTITIONOFFSET},lazy_itable_init=0,lazy_journal_init=0
"${IMAGE}"

echo "Loopback-mounting ${IMAGE}"
DEVICE=`losetup --find --partscan --show "${IMAGE}"`
echo "File ${IMAGE} is now ${DEVICE}"

echo "Creating dir ${MOUNT}"
mkdir "${MOUNT}"

echo "Mounting partition 1 of ${DEVICE} to ${MOUNT}"
mount "${DEVICE}p1" "${MOUNT}"

echo "Press any key to continue"
read -n 1 -s

echo "Debootstrapping"
debootstrap --include linux-image-amd64 stretch "${MOUNT}"

echo "Setting up syslinux in chroot"

dd \
    if=/usr/lib/syslinux/mbr/mbr.bin \
    of="${DEVICE}" \
    conv=notrunc bs=440 count=1


cp /usr/lib/syslinux/modules/bios/menu.c32 \
	/usr/lib/syslinux/modules/bios/hdt.c32 \
	/usr/lib/syslinux/modules/bios/ldlinux.c32 \
	/usr/lib/syslinux/modules/bios/libutil.c32 \
	/usr/lib/syslinux/modules/bios/libmenu.c32 \
	/usr/lib/syslinux/modules/bios/libcom32.c32 \
	/usr/lib/syslinux/modules/bios/libgpl.c32 \
	/usr/share/misc/pci.ids \
	"${MOUNT}"

FSUUID=`blkid -s UUID -o value "${DEVICE}p1"`

SYSLINUXCONF="UI menu.c32

prompt 0
menu title Debian Live

timeout 300

label Debian Live
menu label ^Debian Live
menu default
kernel /vmlinuz
append initrd=/initrd root=UUID=${FSUUID}"
echo "${SYSLINUXCONF}" > "${MOUNT}/syslinux.cfg"

rm "${MOUNT}/initrd.img"* "${MOUNT}/vmlinuz"*
mv "${MOUNT}/boot/initrd.img"* "${MOUNT}/initrd"
mv "${MOUNT}/boot/vmlinuz"* "${MOUNT}/vmlinuz"

cp "$(readlink -f $0)" "${MOUNT}"

sync
extlinux -i "${MOUNT}"

echo "Press any key to continue and clean up"
read -n 1 -s

umount ${MOUNT}
rm -r ${MOUNT}
losetup -d ${DEVICE}

On 10/10/17 12:28, Manuel Wagesreither via Syslinux
wrote:
> Dear all,
> 
> I'm working on a shell script which is producing a bootable image file.
It contains a populated MBR and a single file system (ext4) which again contains
a debootstrabed debian stable distribution.
> 
> Unfortunately, I'm experiencing problems with the bootability of the
produced image. When selecting the installed system at the boot prompt, Syslinux
responds with something like "couldnt load vmlinuz: File or directory not
found".
> 
> The syslinux.cfg is located at the root of the said file system. The linux
kernel and the initrd are relocated from their initial location at /boot to the
file system root as well and are correctly referenced in syslinux.cfg.
> 
> Now the strange thing:
> The problem only occurs when said relocation happens through `mv` command:
> `rm /tmp/generate-image/mount/initrd.img
/tmp/generate-image/mount/initrd.img.old /tmp/generate-image/mount/vmlinuz
/tmp/generate-image/mount/vmlinuz.old`
> `mv /tmp/generate-image/mount/boot/initrd.img-4.9.0-4-amd64
/tmp/generate-image/mount/initrd`
> `mv /tmp/generate-image/mount/boot/vmlinuz-4.9.0-4-amd64
/tmp/generate-image/mount/vmlinuz`
> 
> The problem does NOT occur when relocation is implemented using `cp`:
> `rm /tmp/generate-image/mount/initrd.img
/tmp/generate-image/mount/initrd.img.old /tmp/generate-image/mount/vmlinuz
/tmp/generate-image/mount/vmlinuz.old`
> `cp /tmp/generate-image/mount/boot/initrd.img-4.9.0-4-amd64
/tmp/generate-image/mount/initrd`
> `cp /tmp/generate-image/mount/boot/vmlinuz-4.9.0-4-amd64
/tmp/generate-image/mount/vmlinuz`
> 
> This was hard to believe for me, but it was actually reproducible using
qemu. The fact that all steps are executed through shell script rule out any
user-introduced errors.
> 
> I'm posting my bash shell script (which contains syslinux.cfg) below.
You are invited to execute it yourself.  I'm also eager to share my
generated 1024MiB-sized image which can be mounted through losetup or
booted-from using qemu.
> 
> In the syslinux-wiki, it was mentioned that there are known bugs with the
current syslinux version. Is a list available? Are there any known limitations
regarding ext4 support? Is the underlying code of ldlinux.sys available? I
wasn't able to find it in the syslinux source tree.
> 
> Used versions:
> extlinux 6.03
> QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6+deb9u2)
> 
> Help or advice is much appreciated,
> best regards,
> Manuel
> 
Please try the latest 6.04-preX or even better, if you have the ability,
build it from the git tree and test.

	-hpa

> On 10/10/17 12:28, Manuel Wagesreither via Syslinux wrote:
> > Dear all,
> > 
> > I'm working on a shell script which is producing a bootable image
file. It contains a populated MBR and a single file system (ext4) which again
contains a debootstrabed debian stable distribution.

> > Used versions:
> > extlinux 6.03
> > QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6+deb9u2)
> > 
> > Help or advice is much appreciated,
> > best regards,
> > Manuel
> > 
> 
> Please try the latest 6.04-preX or even better, if you have the ability,
> build it from the git tree and test.
> 
> 	-hpa
> 
 
 
The problem with ext4 (e2fsprogs 1.43+) is explicitly mentioned in the 
wiki, with workarounds.

A patch for Syslinux was proposed by Paulo. It was ignored.

Regards,
Ady.