Syslinux - Jan 2018 - isolinux.bin checksum

Hi,

Ady wrote:
> This is a quote from Wonko/Jaclaz, who has also been investigation this 
> issue and deserves credit for it:
> Ok, not that I understand the code, but what *somehow* happens is that 
> the "embedded" checksum in Isolinux.bin (starting from 4.00) is
the
> checksum of the WHOLE file (i.e. starting from offset 0 instead of 64) 
> at a time when:
> 1) the dword at offset 0x10 (bi_length) is still 0xDEAFBEEF
> 2) the dword at offset 0x14 (bi_csum) is still 0xDEADBEEF
> [...] Credits also go to Erwan.L.
Slapping hand against forehead: I should have come to the idea to restore
the initial byte pattern before testing with offset 0 instead of 64.

The observations match my theory about the missing addition of "start"
when beginning the checksum loop, if "start" is 64.

> Since I am not a developer, I cannot avoid wondering whether patching 
> prepcore.c would have additional consequences.
The change should only influence the checksum, which can hardly get worse.

It comes to me that a run of old checksumiso.pl should have the same
effect as putting isolinux.bin into an ISO. I.e. test would be whether
the four bytes at offset 20 of a freshly built isolinux change by a run
of checksumiso.pl.
If they don't change, then we have a high probability of correct code
in prepcore.c.


Have a nice day :)

Thomas

> Ady wrote:
> > This is a quote from Wonko/Jaclaz, who has also been investigation
this
> > issue and deserves credit for it:
> > Ok, not that I understand the code, but what *somehow* happens is that
> > the "embedded" checksum in Isolinux.bin (starting from 4.00)
is the
> > checksum of the WHOLE file (i.e. starting from offset 0 instead of 64)
> > at a time when:
> > 1) the dword at offset 0x10 (bi_length) is still 0xDEAFBEEF
> > 2) the dword at offset 0x14 (bi_csum) is still 0xDEADBEEF
> > [...] Credits also go to Erwan.L.
> 
> Slapping hand against forehead: I should have come to the idea to restore
> the initial byte pattern before testing with offset 0 instead of 64.
> 
> The observations match my theory about the missing addition of
"start"
> when beginning the checksum loop, if "start" is 64.
> 
> 
> > Since I am not a developer, I cannot avoid wondering whether patching 
> > prepcore.c would have additional consequences.
> 
> The change should only influence the checksum, which can hardly get worse.
> 
> It comes to me that a run of old checksumiso.pl should have the same
> effect as putting isolinux.bin into an ISO. I.e. test would be whether
> the four bytes at offset 20 of a freshly built isolinux change by a run
> of checksumiso.pl.
> If they don't change, then we have a high probability of correct code
> in prepcore.c.
 
 
I was able to test isolinux.bin + ldlinux.c32 built with the patch 
suggested by Thomas.

A_ The default checksum included in the tested isolinux.bin (offset 20, 
4-bytes-long) is the "correct" one (as oppose to the current
situation,
since version 4.00).

B_ The aforementioned checksum is corroborated by several ISO-building 
tools, by using their respective "patch info table" options.

C_ When building ISO images without using the "patch info table" 
option, the checksum was unaltered, and the ISO image still 
successfully boots a VM. This is in contrast to the current situation 
(since version 4.00), in which the default checksum is incorrect and 
the resulting ISO images (without patching the boot info table) fail to 
boot, showing instead a "checksum error".

D_ Considering that mkisofs patches the boot info table on the source 
isolinux.bin file (not just the file within the resulting ISO image, as 
xorriso does), the content of every ISO image generated for these tests 
was then extracted, and the extracted isolinux.bin file was compared, 
not just against the source of the ISO image but also against the 
original/initial isolinux.bin (built with Thomas' patch).

E_ In the tested isolinux.bin, after the 4-bytes-long checksum (offsets 
20 to 23), there is a series of 0xDEADBEEF, repeated 7 times (i.e. 
28-bytes-long series of "EF BE AD DE"). Under some circumstances (e.g.
using, or not using, specific options), some ISO-building tools replace 
these bytes with zeroes. This is not new, and should not affect the 
boot behavior.

F_ In the tested isolinux.bin file, there are two additional bytes that 
I found to be changed/affected by some ISO-building tools (e.g. 
mkisofs), at offset 12 and offset 13. I haven't checked the 
meaning/usage of these particular offsets in isolinux.bin, so I don't 
know the reason for this change. Perhaps Thomas has this (background) 
info more "at hand" than I?

G_ I have not tested isolinux-debug.bin nor any isohybrid features.

>From the point of view of the isolinux.bin checksum, Thomas's patch 
seems to provide the correct effect: successfully booting isolinux.bin 
is possible even when not patching the boot info table (under 
simplified conditions), which should help support less-knowledgeable 
users. This was the intention of the improvements originally introduced 
in Syslinux 2.12, and broken since version 4.00.

Thoughts? Comments? Feedback?

Regards,
Ady.

H,

Ady wrote:
> A_ The default checksum included in the tested isolinux.bin (offset 20, 
> 4-bytes-long) is the "correct" one (as oppose to the current
situation,
> since version 4.00).
That's good news.

> F_ In the tested isolinux.bin file, there are two additional bytes that 
> I found to be changed/affected by some ISO-building tools (e.g. 
> mkisofs), at offset 12 and offset 13.
This is the block number where the file content of isolinux.bin starts in
the ISO filesystem. It is the main reason to have a boot info table.

By mkisofs option -boot-load-size 4 the El Torito Boot Catalog tells BIOS
to load only the first 2048 bytes of isolinux.bin, because some BIOSes are
said to fail if they shall load all of isolinux.bin.
So the program must find and load the rest of itself. At that stage it
does not understand ISO 9660 filesystems. A simple block address helps
a lot.

man mkisofs paragraph "EL TORITO BOOT INFORMATION TABLE" says:
-----------------------------------------------------------------------------
   The  format  of  this  table is as follows; all integers are in section
   7.3.1 ("little endian") format.

     Offset    Name           Size      Meaning
      8        bi_pvd         4 bytes   LBA of primary volume descriptor
     12        bi_file        4 bytes   LBA of boot file
     16        bi_length      4 bytes   Boot file length in bytes
     20        bi_csum        4 bytes   32-bit checksum
     24        bi_reserved    40 bytes  Reserved

          The 32-bit checksum is the sum of all the 32-bit  words  in  the
          boot  file  starting  at  byte  offset  64.   All  linear  block
          addresses (LBAs) are given in CD sectors (normally 2048 bytes).
-----------------------------------------------------------------------------

The "primary volume descriptor" of a first session on medium is always
at
Logical Block Address 16 = 0x10 0x00 0x00 0x00.
"Boot file length" and "32-bit checksum" get filled in by
prepcore and do
not change by the -boot-info-table changes. Nevertheless the ISO producer
programs set these values too.

> G_ I have not tested isolinux-debug.bin nor any isohybrid features.
I understand that the isohybrid MBR loads and runs the first 2048 bytes of
isolinux.bin like BIOS would do with CD/DVD/BD media. From there on
everything should work fine. (Well, we know that a bug with handing over
of C/H/S geometry made it fail on old BIOS until last year ...)


Have a nice day :)

Thomas