Syslinux - Feb 2015 - isohybrid and ISO images whose size is not a multiple of 2048 bytes vs. VirtualBox

Hi,

at [Tails] we're in the process of shipping hybrid ISO images by
default again. We're using `isohybrid -h 255 -s 63' to do that (thanks
to the advice we got on this mailing-list a few months ago!).

And then, we've discovered that sometimes, isohybrid produces ISO
images whose size is not a multiple of 2048 bytes.

Who cares? ... may you ask. Well, apparently, VirtualBox does: it
refuses to add such an ISO image as a virtual DVD. The [details] have
been researched by my team-mate anonym (cc'd), for the curious.
According to this research, it might even be that VirtualBox is
correct in this respect.

Now, anonym also discovered that one can trivially "fix" such an ISO
image by padding it with zeroes until its size is a multiple of 2048
bytes (`truncate -s %2048 $ISO'). And then, VirtualBox is happy again.
That's what we're going to do on the short term on our side, but we
like it better when things are improved upstream as a result of our
work, hence this email. So:

What do you think shall be done on the isohybrid side? Shall it do
this round-up-to-multiple-of-2048-bytes operation itself?

[Tails] https://tails.boum.org/
[details] https://labs.riseup.net/code/issues/8891#note-9

Cheers,
-- 
intrigeri

> Hi,
> 
> at [Tails] we're in the process of shipping hybrid ISO images by
> default again. We're using `isohybrid -h 255 -s 63' to do that
(thanks
> to the advice we got on this mailing-list a few months ago!).
> 
> And then, we've discovered that sometimes, isohybrid produces ISO
> images whose size is not a multiple of 2048 bytes.
> 
> Who cares? ... may you ask. Well, apparently, VirtualBox does: it
> refuses to add such an ISO image as a virtual DVD. The [details] have
> been researched by my team-mate anonym (cc'd), for the curious.
> According to this research, it might even be that VirtualBox is
> correct in this respect.
> 
> Now, anonym also discovered that one can trivially "fix" such an
ISO
> image by padding it with zeroes until its size is a multiple of 2048
> bytes (`truncate -s %2048 $ISO'). And then, VirtualBox is happy again.
> That's what we're going to do on the short term on our side, but we
> like it better when things are improved upstream as a result of our
> work, hence this email. So:
> 
> What do you think shall be done on the isohybrid side? Shall it do
> this round-up-to-multiple-of-2048-bytes operation itself?
> 
> [Tails] https://tails.boum.org/
> [details] https://labs.riseup.net/code/issues/8891#note-9
> 
> Cheers,
> -- 
> intrigeri
 
Any (common) ISO image size should be a multiple of 2048 bytes, as this 
is the normal bytes_per_sector value.

For isohybrid images, the result should be a multiple of 1MiB (1048576 
bytes), which is itself a multiple of 2048 bytes.

So, in addition to adding the corresponding MBR and partition table to 
the original ISO image, the isohybrid tool should also check that the 
ISO image is a multiple of 1MiB. If it is not, then the isohybrid tool 
should first add a "tail" of zeroes to the ISO image up to the next 
multiple of 1MiB, and then add the MBR and partition table.

I do not know the exact complete procedure that the TAILS team is using 
to build the original ISO image, and how it is transformed into an 
isohybrid image (there are multiple tools for each, sometimes 
combined).

If the isohybrid method (whichever tool you are using) is not applied 
to your original ISO image, is such ISO image' size a multiple of 2048 
bytes?

What's the difference, in bytes, between the non-hybrid ISO image and 
the isohybrid one? Could you please provide both exact sizes? It is 
possible that some "limit" (e.g. around the 1GiB) might be triggering 
some condition (I am not saying that this is a bug in the isohybrid 
tools, or that such case might trigger a problem; just assuming that 
there might be some potential chance that some case might have not been 
considered in the source code).

I don't know whether there is some bug in any one of the isohybrid 
tools included in Syslinux (there might be). Which one are you using? 
Is it part of the same exact version of the ISOLINUX bootloader? Or, 
perhaps, are you using an external tool (i.e. not using any of the ones 
included in Syslinux)?

Have you tried not using the Syslinux-related packages, and instead 
using the upstream official *pre-built* binaries (downloaded from 
kernel.org) to build a new ISO image and then to make it isohybrid 
(both pre-built binaries, the isolinux.bin bootloader and the isohybrid 
tool, from the upstream official archives)?

Additionally, IIRC, the "isolinux-debug.bin" file cannot be used for 
isohybrid images.

Regards,
Ady. 
> _______________________________________________
> Syslinux mailing list
> Submissions to Syslinux at zytor.com
> Unsubscribe or set options at:
> http://www.zytor.com/mailman/listinfo/syslinux
>

Hi,

intrigeri wrote:
> We're using `isohybrid -h 255 -s 63' [...]
> isohybrid produces ISO
> images whose size is not a multiple of 2048 bytes.
Seems to be intentional.

http://git.kernel.org/cgit/boot/syslinux/syslinux.git/tree/utils/isohybrid.in
  # Target image size: round up to a multiple of $h*$s*512

http://git.kernel.org/cgit/boot/syslinux/syslinux.git/tree/utils/isohybrid.c
    cylsize = head * sector * 512;
    frac = isostat.st_size % cylsize;
    padding = (frac > 0) ? cylsize - frac : 0;

With -h 255 -s 63, both pad to an integer number of cylinders
with an impair multiple of 512 bytes. So if the number of
cylinders is not divisible by 4, then the number of bytes is
not divisible by 2048.

An ISO image may have a fuzzy end, but a virtual CD-ROM
should indeed provide an integer number of 2048-byte blocks.

> truncate -s %2048 $ISO
This makes the image larger than the partition.
Not overly unhealthy, i'd say, because the ISO filesystem
still fits into the partition.
Nevertheless not totally clean.
Does fdisk -lu complain ?


An alternative would be to request a cylinder size which
is divisible by 2048. E.g.

  isohybrid -h 252 -s 63

The cylinder sizes 64x32 and 255x63 emerged as best
candidates when we compared our collections of questionable
experience and rumors. More substance is not to see behind
them.
(Actually Cylinder/Head/Sector addressing is so brain
 damaged that only ill firmware can insist in it having
 a particular form. All sane firmwares will simply use
 LBA numbers for addressing.)


Ady wrote:
> For isohybrid images, the result should be a multiple of 1MiB (1048576 
> bytes), which is itself a multiple of 2048 bytes.
This is possible only up to an image size of 1024 MiB,
because the cylinder field allows numbers from 0 to 1023.
For larger images the current isohybrid advise is
indeed -h 255 -s 63 (ends shortly before 8 GiB).

The found problem should make us rethink and trigger
massive testing.

> I don't know whether there is some bug in any one of the isohybrid 
> tools included in Syslinux (there might be).
No. This time it is a conceptional problem of isohybrid.
It wants to be an MBR partitioned hard disk and a CD-ROM
at the same time. I am quite sure that VirtualBox does
not complain about the ISO filesystem but about the
virtual DVD-ROM.

If isohybrid pads to a cylinder size, then it would be
odd to add further 1, 2, or 3 blocks of 512 bytes.
I think the workaround with truncate is a valid way to
override the valid decision of the isohybrid program
... if one cannot affort a cylinder size that is
divisible by 2048.


Have a nice day :)

Thomas