Syslinux - Mar 2013 - isohybrid --uefi - resulting EF00 partition too small

Hello everyone,

I am maintaining a distribution loosely based on LFS and am now trying 
to add UEFI support. Integrating a FAT formatted partition image 
containing, preloader, gummiboot, kernel and initramfs correctly results 
in a ISO image that is bootable on BIOS and UEFI systems if booted from 
the optical drive.

However, when I run

isohybrid --uefi name-of.iso

the resulting GPT entry for the EF00 partition is just 19MB. It should 
be around 53MB:

root at barium:/tmp# gdisk -l 
lesslinux-search-and-rescue-uluru-20130312-110347.iso
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: present

Found valid MBR and GPT. Which do you want to use?
1 - MBR
2 - GPT
3 - Create blank GPT

Your answer: 2
Using GPT and creating fresh protective MBR.
Warning! Main partition table overlaps the first partition by 34 blocks!
You will need to delete this partition or resize it in another utility.
Disk lesslinux-search-and-rescue-uluru-20130312-110347.iso: 1413120 
sectors, 690.0 MiB
Logical sector size: 512 bytes
Disk identifier (GUID): 3F6E0DF3-466E-4B62-9BCC-14303CE24ADF
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1413086
Partitions will be aligned on 8-sector boundaries
Total free space is 514 sectors (257.0 KiB)

Number Start (sector) End (sector) Size Code Name
2 212648 251559 19.0 MiB 0700 ????d??e??


I have to note that all kernel modules and quite some binaries are part 
of the initramfs, which is responsible for the large size. This has 
never been a problem since the distribution is not intended for low 
memory applications. Thus I assume the chinese characters say "Mattias, 
you fucked it up!".

The ISO image was built using xorriso 1.1.8 (from Ubuntu 12.04) with the 
command line:

xorriso -as mkisofs -graft-points -b boot/isolinux/isolinux.bin 
-no-emul-boot -boot-info-table -boot-load-size 4 -c 
boot/isolinux/boot.cat --efi-boot boot/efi.img -V 
LESSLINUX-SEARCH-AND-RESCUE -o 
lesslinux-search-and-rescue-uluru-20130312-110347.iso -r cdmaster 
--sort-weight 0 / --sort-weight 1 /boot

Isohybrid capability was added by using a statically linked isohybrid 
binary built from the sources of syslinux 5.02pre3:

/mnt/archiv/LessLinux/llbuild/stage01/chroot/usr/bin/isohybrid --uefi 
/mnt/archiv/LessLinux/llbuild/stage03/lesslinux-search-and-rescue-uluru-20130312-110347.iso

If you want to examine the resulting ISO image, it can be downloaded here:

http://download.lesslinux.org/testing/lesslinux-search-and-rescue/lesslinux-search-and-rescue-uluru-20130312-110347.iso

Am I wrong with the assumption that it should be possible to create a 
valid ISO image that boots on all four boot systems: BIOS-El-Torito, 
UEFI-El-Torito, BIOS from MBR (respectively compatibility MBR) and UEFI 
from EF00 partition mapped to the El-Torito image? Or did just my large 
FAT image make me run into some side effects? If there will be no 
possibility of supporting all four methods in one image I will move to 
creating separate images for USB drives, something I outlined here 
http://blog.lesslinux.org/massive-changes-ahead-uefi-secure-boot-soon-to-be-supported/
Of course in this case I would neither blame Thomas Schmitt nor Matthew 
Garrett nor anyone else who did a great job on making isohybrid work for 
so many use cases, neither would I blame my large initramfs - I think in 
some cases it is just fate that jigsaw puzzle of four boot standards 
does not fit together.

Regards,
Mattias

-- 
Mattias Schlenker - Redaktion + EDV-Beratung + Linux-CD/DVD-Konzepte
Dietrich-Bonhoeffer-Str. 3 - 40667 MEERBUSCH - GERMANY

Telefon (VoIP "ueberall"), geschaeftlich: +49 341 39290767
Telefon (Festnetz), privat und Fax:       +49 2132 9952906
Mobil:                                    +49 163  6953657
Mobil (SIM in Testgeraeten):              +49 1578 3499550

Bitte fuer geschaeftliche Telefonate vorzugsweise die VoIP-Telefonnummer
+49 341 39290767 verwenden, da ich diese aufs Mobiltelefon routen kann!

http://blog.rootserverexperiment.de/ http://news.mattiasschlenker.de/

Hello everyone,

I am maintaining a distribution loosely based on LFS and am now trying 
to add UEFI support. Integrating a FAT formatted partition image 
containing, preloader, gummiboot, kernel and initramfs correctly results 
in a ISO image that is bootable on BIOS and UEFI systems if booted from 
the optical drive.

However, when I run

isohybrid --uefi name-of.iso

the resulting GPT entry for the EF00 partition is just 19MB. It should 
be around 53MB:

root at barium:/tmp# gdisk -l 
lesslinux-search-and-rescue-uluru-20130312-110347.iso
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: present

Found valid MBR and GPT. Which do you want to use?
1 - MBR
2 - GPT
3 - Create blank GPT

Your answer: 2
Using GPT and creating fresh protective MBR.
Warning! Main partition table overlaps the first partition by 34 blocks!
You will need to delete this partition or resize it in another utility.
Disk lesslinux-search-and-rescue-uluru-20130312-110347.iso: 1413120 
sectors, 690.0 MiB
Logical sector size: 512 bytes
Disk identifier (GUID): 3F6E0DF3-466E-4B62-9BCC-14303CE24ADF
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1413086
Partitions will be aligned on 8-sector boundaries
Total free space is 514 sectors (257.0 KiB)

Number Start (sector) End (sector) Size Code Name
2 212648 251559 19.0 MiB 0700 ????d??e??


I have to note that all kernel modules and quite some binaries are part 
of the initramfs, which is responsible for the large size. This has 
never been a problem since the distribution is not intended for low 
memory applications. Thus I assume the chinese characters say "Mattias, 
you fucked it up!".

The ISO image was built using xorriso 1.1.8 (from Ubuntu 12.04) with the 
command line:

xorriso -as mkisofs -graft-points -b boot/isolinux/isolinux.bin 
-no-emul-boot -boot-info-table -boot-load-size 4 -c 
boot/isolinux/boot.cat --efi-boot boot/efi.img -V 
LESSLINUX-SEARCH-AND-RESCUE -o 
lesslinux-search-and-rescue-uluru-20130312-110347.iso -r cdmaster 
--sort-weight 0 / --sort-weight 1 /boot

Isohybrid capability was added by using a statically linked isohybrid 
binary built from the sources of syslinux 5.02pre3:

/mnt/archiv/LessLinux/llbuild/stage01/chroot/usr/bin/isohybrid --uefi 
/mnt/archiv/LessLinux/llbuild/stage03/lesslinux-search-and-rescue-uluru-20130312-110347.iso

If you want to examine the resulting ISO image, it can be downloaded here:

http://download.lesslinux.org/testing/lesslinux-search-and-rescue/lesslinux-search-and-rescue-uluru-20130312-110347.iso

Am I wrong with the assumption that it should be possible to create a 
valid ISO image that boots on all four boot systems: BIOS-El-Torito, 
UEFI-El-Torito, BIOS from MBR (respectively compatibility MBR) and UEFI 
from EF00 partition mapped to the El-Torito image? Or did just my large 
FAT image make me run into some side effects? If there will be no 
possibility of supporting all four methods in one image I will move to 
creating separate images for USB drives, something I outlined here 
http://blog.lesslinux.org/massive-changes-ahead-uefi-secure-boot-soon-to-be-supported/
Of course in this case I would neither blame Thomas Schmitt nor Matthew 
Garrett nor anyone else who did a great job on making isohybrid work for 
so many use cases, neither would I blame my large initramfs - I think in 
some cases it is just fate that jigsaw puzzle of four boot standards 
does not fit together.

Regards,
Mattias



-- 
Mattias Schlenker - Redaktion + EDV-Beratung + Linux-CD/DVD-Konzepte
Dietrich-Bonhoeffer-Str. 3 - 40667 MEERBUSCH - GERMANY

Telefon (VoIP "ueberall"), geschaeftlich: +49 341 39290767
Telefon (Festnetz), privat und Fax:       +49 2132 9952906
Mobil:                                    +49 163  6953657
Mobil (SIM in Testgeraeten):              +49 1578 3499550

Bitte fuer geschaeftliche Telefonate vorzugsweise die VoIP-Telefonnummer
+49 341 39290767 verwenden, da ich diese aufs Mobiltelefon routen kann!

http://blog.rootserverexperiment.de/ http://news.mattiasschlenker.de/

Hello everyone,

I am maintaining a distribution loosely based on LFS and am now trying 
to add UEFI support. Integrating a FAT formatted partition image 
containing, preloader, gummiboot, kernel and initramfs correctly results 
in a ISO image that is bootable on BIOS and UEFI systems if booted from 
the optical drive.

However, when I run

isohybrid --uefi name-of.iso

the resulting GPT entry for the EF00 partition is just 19MB. It should 
be around 53MB:

root at barium:/tmp# gdisk -l 
lesslinux-search-and-rescue-uluru-20130312-110347.iso
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: present

Found valid MBR and GPT. Which do you want to use?
1 - MBR
2 - GPT
3 - Create blank GPT

Your answer: 2
Using GPT and creating fresh protective MBR.
Warning! Main partition table overlaps the first partition by 34 blocks!
You will need to delete this partition or resize it in another utility.
Disk lesslinux-search-and-rescue-uluru-20130312-110347.iso: 1413120 
sectors, 690.0 MiB
Logical sector size: 512 bytes
Disk identifier (GUID): 3F6E0DF3-466E-4B62-9BCC-14303CE24ADF
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1413086
Partitions will be aligned on 8-sector boundaries
Total free space is 514 sectors (257.0 KiB)

Number Start (sector) End (sector) Size Code Name
2 212648 251559 19.0 MiB 0700 ????d??e??


I have to note that all kernel modules and quite some binaries are part 
of the initramfs, which is responsible for the large size. This has 
never been a problem since the distribution is not intended for low 
memory applications. Thus I assume the chinese characters say "Mattias, 
you fucked it up!".

The ISO image was built using xorriso 1.1.8 (from Ubuntu 12.04) with the 
command line:

xorriso -as mkisofs -graft-points -b boot/isolinux/isolinux.bin 
-no-emul-boot -boot-info-table -boot-load-size 4 -c 
boot/isolinux/boot.cat --efi-boot boot/efi.img -V 
LESSLINUX-SEARCH-AND-RESCUE -o 
lesslinux-search-and-rescue-uluru-20130312-110347.iso -r cdmaster 
--sort-weight 0 / --sort-weight 1 /boot

Isohybrid capability was added by using a statically linked isohybrid 
binary built from the sources of syslinux 5.02pre3:

/mnt/archiv/LessLinux/llbuild/stage01/chroot/usr/bin/isohybrid --uefi 
/mnt/archiv/LessLinux/llbuild/stage03/lesslinux-search-and-rescue-uluru-20130312-110347.iso

If you want to examine the resulting ISO image, it can be downloaded here:

http://download.lesslinux.org/testing/lesslinux-search-and-rescue/lesslinux-search-and-rescue-uluru-20130312-110347.iso

Am I wrong with the assumption that it should be possible to create a 
valid ISO image that boots on all four boot systems: BIOS-El-Torito, 
UEFI-El-Torito, BIOS from MBR (respectively compatibility MBR) and UEFI 
from EF00 partition mapped to the El-Torito image? Or did just my large 
FAT image make me run into some side effects? If there will be no 
possibility of supporting all four methods in one image I will move to 
creating separate images for USB drives, something I outlined here 
http://blog.lesslinux.org/massive-changes-ahead-uefi-secure-boot-soon-to-be-supported/
Of course in this case I would neither blame Thomas Schmitt nor Matthew 
Garrett nor anyone else who did a great job on making isohybrid work for 
so many use cases, neither would I blame my large initramfs - I think in 
some cases it is just fate that jigsaw puzzle of four boot standards 
does not fit together.

Regards,
Mattias

PS: Sorry for posting twice... Is the list initially moderated until 
someone's first post is accepted? I sent this mail immediately after 
subscribing and did neither receive a copy or find it in the archives. 
Let's see...

-- 
Mattias Schlenker - Redaktion + EDV-Beratung + Linux-CD/DVD-Konzepte
Dietrich-Bonhoeffer-Str. 3 - 40667 MEERBUSCH - GERMANY

Telefon (VoIP "ueberall"), geschaeftlich: +49 341 39290767
Telefon (Festnetz), privat und Fax:       +49 2132 9952906
Mobil:                                    +49 163  6953657
Mobil (SIM in Testgeraeten):              +49 1578 3499550

Bitte fuer geschaeftliche Telefonate vorzugsweise die VoIP-Telefonnummer
+49 341 39290767 verwenden, da ich diese aufs Mobiltelefon routen kann!

http://blog.rootserverexperiment.de/ http://news.mattiasschlenker.de/

Argh. I am a bit tired, sorry. Of course. 51 % 32 =19. Yes, it would be helpful
if xorriso (and every other ISO creator) gracefully failed for too large el
torito images. I was not aware oft this limit.

Regards, Mattias

Hi,
> I did not yet look on the code of isohybrid.c, but it would seem to make
> sense to change the calculation to use a higher level library like e.g.
> libisofs to retrieve the correct size inforamtion.
libisofs is not very lean in terms of code size and runtime
memory consumption. It loads the whole ISO directory tree.
libcdio would probably be leaner at runtime.

But it would be more attractive then to let isohybrid.c traverse the
ISO directory tree by own means and to look for the start address
of the image file which was obtained from El Torito. El Torito
does not tell the name or path of the image file.

Nevertheless, the simpler solution would be to give isohybrid.c
an option by which you can override the questionable size info
from the El Torito catalog. (Quite everybody has more clue about
that size than El Torito has.)

> However, I think there are quite some people creating bootable ISOs
sticking
> to genisoimage or mkisofs combined with isohybrid because it is proven and
> they do not want to change a running system and they fear breaking
> bootability especially with older buggy BIOSes.
isohybrid.c seems quite unmaintained currently. At least for the
GPT part. genisoimage is nearly unmaintained, too. I could help
with both, but will not take over responsibility for them.

xorriso offers a migration path by its mkisofs emulation.
There is not much room for individual solutions with the oldfashioned
ways of booting via ISO 9660. So in that aspect it should be equivalent
to mkisofs and genisoimage.
The only boot related mkisofs feature that xorriso cannot do for
Debian is option -hfs for old PowerPC machines. In exchange it can
offer -hfsplus for newer Macs (contribution of Vladimir Serbinenko).

> The thumb drive now correctly boots from BIOS and UEFI (at least up to
> kernel and initramfs on UEFI, trouble in userland should be my problem for
> now,
I live in happy ignorance towards the further boot stages.
All my tests are done with existing bootable images which i repack
like i did with yours yesterday. Regrettably i have no machine
that boots via EFI.

> trouble on ISO/MBR with not found com32 files in syslinux 5.10pre1 is
> another story).
If USB stick and CD show different bugs, then i cannot rule out a
xorriso bug. But on the other hand, the typical problem indication
is no booting at all. I just provide pointers to the first step.
Give me a note if particular suspicion arises towards xorriso.


Have a nice day :)

Thomas