> Hi, > > Ady wrote: > > During May 2009, a commit by Peter deleted the checksumiso.pl file. The > > commit is: > > core: LZO compress the PM part of the core > > repo.or.cz/syslinux.git/commit/0d82b71304d596d80f3c4520f9dcf90048ca50b7 > > And so, since version 4.00, the 'code/checksumiso.pl' file is no longer > > included. > > How is the checksum of isolinux.bin calculated since then? > > By the ISO 9660 production program which patches in the Boot Info Table. > > The computation of checksumiso.pl can be seen in libisofs function > make_boot_info_table(): > > https://dev.lovelyhq.com/libburnia/libisofs/blob/master/libisofs/eltorito.c#L1252 > (without Javascript > https://dev.lovelyhq.com/libburnia/libisofs/raw/master/libisofs/eltorito.c > you have to hop to the line yourself. Sorry. GitLab. Urghhh ...) > > libisofs does not pad to full blocks of 2048 bytes, like checksumiso.pl > did. But the isolinux.bin binary from SYSLINUX installations is already > aligned to that block size. It works for libisofs since about 10 years. > > > > What was/is the purpose of this range change? > > I'm not sure whether it is due to a range change, but it really looks like > the checksum in the file isolinux.bin from SYSLINUX installation differs > from the checksum after the file was put with Boot Info Table into an > ISO 9660 filesystem. > > I ran > > xorriso -as mkisofs -o test.iso -b isolinux.bin \ > -no-emul-boot -boot-load-size 4 -boot-info-table \ > /usr/lib/ISOLINUX/isolinux.bin > > and the checksum changed from 0x09e675b8 to 0x8cc6977a. > > But i never had to change the algorithm in libisofs in order to keep the > result bootable. The isolinux,bin binaries do boot and in > http://repo.or.cz/syslinux.git/blob/refs/heads/master:/core/isolinux.asm#l262 > i see a checksum computation and later a test which would lead to label > "kaboom" if not matching. > > > So where did the checksum 0x09e675b8 in Sid's installed isolinux.bin come > from ? > > The source code sets it to 4-byte value > 0xdeadbeef > See the comments from > http://repo.or.cz/syslinux.git/blob/f1aa00224b23e2b4c71f204c1417c7b6e5ea8e51:/core/isolinux.asm#l213 > up to line 220. > > So i assume that checksumming still is applied somehow during building of > isolinux.bin. > > In the diff of core/Makefile i see a new program ../lzo/prepcore applied. > In its source, one can see the Boot Info Table checksum calculation: > http://repo.or.cz/syslinux.git/blob/0d82b71304d596d80f3c4520f9dcf90048ca50b7:/lzo/prepcore.c#l370 > It looks to me as if it computes the checksum of payload and block padding. > (This explains why the installed isolinux.bin are always block aligned.) > > Nevertheless, something must be different there, compared to libisofs and > the check in the running isolinux.bin boot image program. > > For now i fail to see that difference. Maybe others are smarter. > I will later chew a bit more on this riddle. > > > Have a nice day :) > > Thomas >I very much appreciate the reply. From it, either I haven't been as clear as I thought, or things are not clear from the starting point. AFAIK, the default checksum (at least the one I am talking about) in official isolinux.bin was supposed to help ISO-building programs. Until ISOLINUX (Syslinux) version 2.11, mkisofs was patching the "boot info table" by means of an option in its command line. But, what happened when a user didn't know that ISOLINUX needed that option, or when using other ISO-building tools? One improvement was introduced in version 2.12 of Syslinux. Since version 2.12, isolinux.bin included a default checksum that would attempt to help ISO-building tools and users that, for whichever reason, would not patch the boot info table when building bootable ISO images. Until version 3.86, the checksum in isolinux.bin (at least the one I am talking about) was successfully created by checksumiso.pl. An ISO image that was created without patching the boot info table would at least have a chance of successfully booting anyway, under certain (simplified) conditions. But then, by version 4.00 the checksumiso.pl file was gone (by means of the commit I posted in my prior email) and instead "_some kind of_ checksum" is introduced in isolinux.bin when building it from source. Apparently, one of the consequences seems to be that the default checksum included in isolinux.bin doesn't seem to help those users/programs that, for whichever reason (e.g. user lacking knowledge), do not patch the boot info table. So, my question is whether there is some intentional reason for these changes in isolinux.bin. Perhaps the "new" content (since 4.00) provides other improvements, or it is compatible with other features (e.g. isohybrid perhaps?) but the "old" potentially-helpful checksum can no longer be calculated or saved (within isolinux.bin) in the same way as it was before (i.e. 2.12-3.86). Is that the issue? So, to be clear, I am not asking how mkisofs or xorriso are patching the boot info table or how ISO-building tools are patching isolinux.bin with their own calculated checksum. What I'd like to know is what happened with the "default" checksum in isolinux.bin, that was previously correctly calculated by checksumiso.pl (and now, it seems to be incorrect). Summing up, in upstream isolinux.bin (with no relation to specific ISO-building tools, nor their command lines at the time of building ISO images), how is/are the checksum(s) calculated now? Are this/these calculation(s) / results "correct", or "expected"? Is it possible that the current calculation is incorrect / a bug / not ideal / not taking the adequate range of values as input? Is there any way to improve this situation? TIA, Ady.
Hi, Ady wrote:> Apparently, one of the consequences seems to be that the default > checksum included in isolinux.bin doesn't seem to help those > users/programs that, for whichever reason (e.g. user lacking > knowledge), do not patch the boot info table.I agree and still riddle about the reason.> So, my question is whether there is some intentional reason for these > changes in isolinux.bin.I assume that hpa intended to move the functionality of checksumiso.pl into the "prepcore" helper program which compresses all or parts of the ./core/*.bin files.> What I'd like to know is what > happened with the "default" checksum in isolinux.bin,To my theory it fell victim to a bug.> how is/are the checksum(s) calculated now?To my best knowledge by helper program "prepcore". At least the Debian build log looks like no other program changes isolinux.bin afterwards.> Is it possible that the current calculation is incorrect / a bugMost likely.> not taking the adequate range of values as input?Quite probably it is some other bug. But as long as i can show no good suspect, everything is possible. ----------------------------------------------------------------------- Unsuccessful theories so far: The only potential problem i found in lzo/prepcore.c up to now, is that the buffer for the compressed isolinux.bin is created by malloc(3), which does not guarantee to hand out the buffer will all zeros as content. prepcore nevertheless rounds up the outfile_len to finally yield full blocks of 2048 bytes size. So up to 2047 bytes of arbitrary discarded memory content could end up in the compressed isolinux.bin. But those bytes would also be part of prepcore's checksum computation. So the checksum should match the one from libisofs. I further tried to provoke the initial checksum of isolinux.bin by spoiling the byte order of the 4-byte unsigned integers which get added. But prepcode's function get_32() in both variants yields on amd64 the same integer values as libisofs sees. Willfully reverting byte order of the integers does not yield the initial checksum 0x09e675b8. Another suspect would be a difference between checksum computation and effective file writing. But the code looks good in that aspect. Checksumming and file writing is just 20 lines apart in the same function. Unless the number of uncompressed bytes at the file start is not divisible by 4, the checksummed byte range and the written byte range match. (The first 64 bytes get not checksummed but written. That's normal.) The number "offset" of uncompressed bytes stems from the .raw file that serves as input for prepcore. I assume it is larger than 400 bytes, because i saw a comment that the LZO decompressor code has 400+ bytes. It must be in the uncompressed part. Regrettably those parameters are not written into isolinux.bin. So i can only guess. Next i wondered whether Debian maybe does not treat isolinux.raw by prepcore. But the build log https://buildd.debian.org/status/fetch.php?pkg=syslinux&arch=amd64&ver=3%3A6.03%2Bdfsg1-2&stamp=1508286838&raw=0 shows /<<BUILDDIR>>/syslinux-6.03+dfsg1/bios/core/../lzo/prepcore ldlinux.raw ldlinux.bin and isolinux.bin seems not to be manipulated afterwards. ----------------------------------------------------------------------- Have a nice day :) Thomas
Hi,
i think i found a suspect in lzo/prepcore.c and it would indeed be a
wrong range of checksumming (speculative congratulations to Ady).
Looking at
http://repo.or.cz/syslinux.git/blob/0d82b71304d596d80f3c4520f9dcf90048ca50b7:/lzo/prepcore.c
it seems that this change in line 374 could yield correct checksums:
unsigned int ptr;
- for (ptr = 64; ptr < offset; ptr += 4)
+ for (ptr = start+64; ptr < offset; ptr += 4)
csum += get_32((uint32_t *)(infile+ptr));
A test whether it works would be to produce isolinux.bin, read the 4 checksum
bytes from it at offset 20, put isolinux.bin into an ISO with mkisofs option
-boot-info-table, and check whether the 4 bytes at offset 20 are still the
same.
(If you use mkisofs or genisoimage, isolinux.bin on hard disk will change.
With xorrisofs the change will only appear in the file isolinux.bin in the
ISO filesystem.)
--------------------------------------------------------------------------
Reasoning:
- The prescription for Boot Info Table says that checksumming begins
at byte 64 of isolinux.bin. (See for example man mkisofs paragraph
"EL TORITO BOOT INFORMATION TABLE".)
- prepcore.c constructs isolinux.bin from an unencrypted piece out of
the input file isolinux.raw (which stems from an objcopy -S run) and
from a compressed part.
So it begins to compute the checksum on the input file bytes
for (ptr = 64; ptr < offset; ptr += 4)
csum += get_32((uint32_t *)(infile+ptr));
and then continues on the compressed part
for (ptr = 0; ptr < outfile_len; ptr += 4)
csum += get_32((uint32_t *)(out+ptr));
- But prepcore.c copies infile data to isolinux.bin starting with an
offset named "start" which i strongly believe is not zero:
if (fwrite(infile+start,1,offset-start,f) != offset-start ||
If "start" is not zero, then this is not the same byte range as with
the checksum loop over (infile+ptr).
- "start" obviously is used to skip at least the 32 byte header which
prepcore.c interprets at the beginning of the input file:
struct prefix {
uint32_t pfx_start;
uint32_t pfx_compressed;
uint32_t pfx_cdatalen;
uint32_t pfx_checksum;
};
...
struct prefix *prefix;
...
infile_len = (lzo_uint) fread(infile,1,infile_len,f);
...
prefix = (struct prefix *)infile;
start = get_32(&prefix->pfx_start);
offset = get_32(&prefix->pfx_compressed);
Such header bytes with plausible values are not to see at the beginning of
isolinux.bin:
fa ea 6c 7c 00 00 90 90 10 00 00 00 00 00 00 00
The struct members pfx_start and pfx_compressed are read as little endian
unsigned integers and used as offsets in the isolinux.raw file.
0x7c6ceafa and 0x90900000 are much too large to have served for that
purpose.
So the original prefix header was skipped, and thus "start" was not
zero.
Have a nice day :)
Thomas