Syslinux - Mar 2017 - "isolinux.bin missing or corrupt" when booting USB flash drive in old PC

Hi,

Ady wrote:
> so, why the original isohybrid image from Debian and many other 
> isohybrid images have mostly (but not all) "00s" in place of the
"90s"?
That's a fake Block0 of an Apple Partition Map. Part of mjg's layout.
Caused by isohybrid option --mac or xorrisofs option -isohybrid-apm-hfsplus.

* Fedora has it with EFI FAT filesystem in one partition and a HFS filesystem
  in another partition.
* Debian and Ubuntu have it only for the EFI FAT partition.
* ArchLinux 2016 has no APM.
* Gentoo 2016 varies from BIOS-only to full Fedora BIOS+EFI+HFS equipment.
* OpenSuSE last time i looked had no APM and no GPT, but self made EFI
  partition entry in MBR partition table. isohybrid without --uefi and
  then custom post processing.

debian-8.7.1-i386-xfce-CD-1.iso has:

00000000 :    45  52  08  00  00  00  90  90  00  00  00  00  00  00  00  00
               E   R                                                        
       0 :    69  82   8   0   0   0 144 144   0   0   0   0   0   0   0   0

00000010 :    00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
                                                                            
      16 :     0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0

My own cheat sheet:

  https://dev.lovelyhq.com/libburnia/libisofs/blob/master/doc/boot_sectors.txt

says:
---------------------------------------------------------------------
...
                           Apple Partition Map (APM)
...
Sources:
    http://mjg59.dreamwidth.org/11285.html
   
http://opensource.apple.com/source/IOStorageFamily/IOStorageFamily-116/IOApplePartitionScheme.h
(typedef struct Block0)
    http://www.informit.com/articles/article.aspx?p=376123&seqNum=3
    syslinux-4.05/utils/isohybrid.c
...
The layout of a Block0 of an APM is:

Byte Range | Value      | Meaning (all numbers are stored big endian)
---------- | ---------- | ----------------------------------------------------
   0 -   1 | sig        | Signature 0x45 = 'E' , 0x52 = 'R'
   2 -   3 | block_size | 0x0800 = 2048
   4 -   7 | block_count| Number of blocks covered by APM
           |            | Often some x86-harmless dummy. E.g. 0x9090 = 37008
           |            | or 0xeb02ffff = 3,942,842,367
   8 -   9 | dev_type   | obscure: "device type"
  10 -  11 | dev_id     | obscure: "device id"
  12 -  15 | drv_data   | obscure: "driver data"
  16 -  17 | drv_count  | obscure: "driver descriptor count"
  18 -  81 | drv_map    | obscure: "driver descriptor table"
           |            |          with 8 entries of 16 bytes each
  82 - 511 | reserved   |
---------- | ---------- | ----------------------------------------------------

The SYSLINUX program isohybrid.c overwrites the first 32 bytes of this
layout by its dummy values. It uses the small block_count 0x00009090 and
sets all bytes up to 31 to 0.
...
---------------------------------------------------------------------

It is a lucky incident that the signature "ER" of an APM is x86
program
code with no undesirable effects. Else one could not combine APM and MBR.

A decisive trick is to set APM block size to 2048.
Thus the first 2048 bytes are considered to be part of Block0.
We need the bytes from 32 to 512 as MBR program code and partition table.
At 512 to 1023 is the GPT header block which points to the GPT entries
array located after the end of the APM entries array.


Have a nice day :)

Thomas

> Hi,
> 
> Ady wrote:
> > so, why the original isohybrid image from Debian and many other 
> > isohybrid images have mostly (but not all) "00s" in place of
the "90s"?
> 
> That's a fake Block0 of an Apple Partition Map. Part of mjg's
layout.
> Caused by isohybrid option --mac or xorrisofs option
-isohybrid-apm-hfsplus.
> 
> * Fedora has it with EFI FAT filesystem in one partition and a HFS
filesystem
>   in another partition.
> * Debian and Ubuntu have it only for the EFI FAT partition.
> * ArchLinux 2016 has no APM.
> * Gentoo 2016 varies from BIOS-only to full Fedora BIOS+EFI+HFS equipment.
> * OpenSuSE last time i looked had no APM and no GPT, but self made EFI
>   partition entry in MBR partition table. isohybrid without --uefi and
>   then custom post processing.
> 
> debian-8.7.1-i386-xfce-CD-1.iso has:
> 
> 00000000 :    45  52  08  00  00  00  90  90  00  00  00  00  00  00  00 
00
>                E   R
>        0 :    69  82   8   0   0   0 144 144   0   0   0   0   0   0   0  
0
> 
> 00000010 :    00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
00
>
>       16 :     0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  
0
> 
> My own cheat sheet:
> 
>  
https://dev.lovelyhq.com/libburnia/libisofs/blob/master/doc/boot_sectors.txt
> 
> says:
> ---------------------------------------------------------------------
> ...
>                            Apple Partition Map (APM)
> ...
> Sources:
>     http://mjg59.dreamwidth.org/11285.html
>    
http://opensource.apple.com/source/IOStorageFamily/IOStorageFamily-116/IOApplePartitionScheme.h
(typedef struct Block0)
>     http://www.informit.com/articles/article.aspx?p=376123&seqNum=3
>     syslinux-4.05/utils/isohybrid.c
> ...
> The layout of a Block0 of an APM is:
> 
> Byte Range | Value      | Meaning (all numbers are stored big endian)
> ---------- | ---------- |
----------------------------------------------------
>    0 -   1 | sig        | Signature 0x45 = 'E' , 0x52 = 'R'
>    2 -   3 | block_size | 0x0800 = 2048
>    4 -   7 | block_count| Number of blocks covered by APM
>            |            | Often some x86-harmless dummy. E.g. 0x9090 =
37008
>            |            | or 0xeb02ffff = 3,942,842,367
>    8 -   9 | dev_type   | obscure: "device type"
>   10 -  11 | dev_id     | obscure: "device id"
>   12 -  15 | drv_data   | obscure: "driver data"
>   16 -  17 | drv_count  | obscure: "driver descriptor count"
>   18 -  81 | drv_map    | obscure: "driver descriptor table"
>            |            |          with 8 entries of 16 bytes each
>   82 - 511 | reserved   |
> ---------- | ---------- |
----------------------------------------------------
> 
> The SYSLINUX program isohybrid.c overwrites the first 32 bytes of this
> layout by its dummy values. It uses the small block_count 0x00009090 and
> sets all bytes up to 31 to 0.
> ...
> ---------------------------------------------------------------------
> 
> It is a lucky incident that the signature "ER" of an APM is x86
program
> code with no undesirable effects. Else one could not combine APM and MBR.
> 
> A decisive trick is to set APM block size to 2048.
> Thus the first 2048 bytes are considered to be part of Block0.
> We need the bytes from 32 to 512 as MBR program code and partition table.
> At 512 to 1023 is the GPT header block which points to the GPT entries
> array located after the end of the APM entries array.
> 
> 
> Have a nice day :)
> 
> Thomas
> 
 
@Thomas,

With the variety of potential combinations, is there really one generic 
"receipt" that can be provided so as to help users (other than those 
using the specific Debian ISO image that was tested by David)?

We seem to be suggesting to just dd' the first 432 bytes of their USB 
devices with Martin's isohybrid .bin20170324 (overwriting their first 
432 bytes that were previously written when they used dd with their 
original isohybrid image on the same USB device). Would this be a 
generic receipt, independently of:

_ the original size of the original isohybrid image?

_ whichever CHS geometry that was used for the isohybrid tool (or 
xorriso's equivalent) when the original isohybrid image was created?

_ whichever combination of boot entries in the El Torito boot catalog?

Or, in other words, which would be the step-by-step instructions for 
users of other isohybrid images? Would they differ according to 
some/any parameter?

The reason I am asking is because we should try to avoid going throw 
dozens of emails the next time we have a similar issue (also for the 
sake of common users and their sanity).

Additionally, improving the documentation regarding the existence, 
usage and goals of the "diag/*.img" images should be part of potential
generic instructions.

TIA,
Ady.

PS:
@Thomas, in "CHS", the "S" is not really "Sectors per
head" but rather
"Sectors per Track". The term "Sector per head" can
potentially be
confused with a different calculation, and we should want to avoid such 
misunderstandings.

Hi,

Ady wrote:
> We seem to be suggesting to just dd' the first 432 bytes of their USB 
> devices with Martin's isohybrid
Yes. As long as it is not about the old Macs which the APM of Fedora
and some Gentoo ISOs wants to make boot, and not about ISOs made
with isohybrid options --partok -ctrlhd0 --forcehd0.

> which would be the step-by-step instructions for users of other
> isohybrid images?
Copy at most 432 bytes to the start of USB stick or ISO image file:
  
  iso=/dev/sdc
or
  iso=image.iso

  dd conv=notrunc if=isohdpfx.bin.170324 bs=1 count=432 of="$iso"

  
> Would they differ according to some/any parameter?
We do not yet have downloadable binaries of the other five isohdp*
variants as listed in
  http://www.syslinux.org/wiki/index.php?title=Isohybrid#MBR_selection
with --partok -ctrlhd0 --forcehd0.
I doubt that any distro uses them.


If one cares for Macs which were old in 2010, then one can keep
the first 32 bytes of the ISO:

  dd conv=notrunc if=isohdpfx.bin.170324 bs=1 skip=32 seek=32 count=400
of="$iso"

but only after having verified that this is really a pseudo APM:

  dd if="$iso" bs=1 count=8 | od -t x1

must report

  0000000 45 52 08 00 00 00 90 90

(... or the start bytes of isohdpfx.bin without APM patched in:
  0000000 33 ed 90 90 90 90 90 90
But then one would not need to preserve them.
)

Else the old MBR is supposed to be older than the preparation for
mjg layout on 2011-08-24:
 
https://git.kernel.org/pub/scm/boot/syslinux/syslinux.git/commit/mbr/isohdpfx.S?id=2c3a24e5f4b807ec31595227afa59a818c060ca9
Thus it would be dangerous to let its first 32 bytes survive.
After all they get executed as x86 instructions when BIOS starts
the MBR.

As long as no old Mac is around, one should not take that risk.


To my knowledge the APM in the Debian and Ubuntu ISOs is useless.
The APM of Fedora and some Gentoo ISOs points to a HFS+ filesystem
and thus might be of interest for the Macs.

In any case the APM Block0 is a dummy with always the same content,
regardless of the size of the ISO or the position of the HFS+ partition.
Its presence without further APM blocks would not harm BIOS booting
but could mislead the old Macs for which APM matters.

> With the variety of potential combinations, is there really one generic 
> "receipt" that can be provided so as to help users (other than
those
> using the specific Debian ISO image that was tested by David)?
Above dd run should match any isohybrid which was made with SYSLINUX
version 3.82 or later.

The MBR has to properly work in itself, which Martin's hopefully does.

And it has to properly prepare the run of isolinux.bin, of which it
finds the block address in byte 432 to 435.
Martin's MBR prepares better for the expectations of isolinux.bin of
version 3.82 or later, than the old isohdpfx.bin did.

> independently of:
> _ the original size of the original isohybrid image?
Yes. This is mentioned only in the MBR partition table. Bytes 446 to 510.
> _ whichever CHS geometry that was used for the isohybrid tool (or 
> xorriso's equivalent) when the original isohybrid image was created?
Yes. C/H/S addresses are only recorded in the MBR partition table.

(Anyways, only the heads/cylinder and sectors/head ideas of the BIOS
 matter. One should never believe the C/H/S addresses in the partition
 table, but rather convert the LBAs of the partition table to C/H/S by
 the numbers inquired from the BIOS.)
> _ whichever combination of boot entries in the El Torito boot catalog?
Yes. El Torito does not matter if the MBR gets started.

(That's why i do not generally propose to patch in Martin's MBR by
 program isohybrid and its option --mbr.
 The isohybrid program reads El Torito with some special expectations.
 If all works well it puts the LBA of isolinux.bin to byte 432 to 435
 of the MBR.
 Above dd run stops overwriting exactly before byte 432.)

> @Thomas, in "CHS", the "S" is not really "Sectors
per head" but rather
> "Sectors per Track".
C/H/S = Cylinder/Head/Sector.

The two conversion factors HpC and SpH are needed for the formula
to convert C/H/S to LBA or vice versa:

  LBA = C * HpC * SpH + H * SpH + S - 1

Introducing the term "track" here seems surplus.
It would become confusing as soon as we speak of optical media too.
There "track" is a consequtive sequence of some normally quite large
number of blocks.


Have a nice day :)

Thomas