Syslinux - Feb 2015 - isohybrid and ISO images whose size is not a multiple of 2048 bytes vs. VirtualBox

Hi,
> Hopefully I am explaining it better this time.
Our ideas of implementation are supposed to yield the
same result.

As said, i am willing to implement my proposal in
a copy of isohybrid.c, if somebody wants to test it.
Advantages and disadvantages should be obvious.

Interested users please send util/isohybrid.c from
your local SYSLINUX source code installation to me.
Just to be sure that the modified copy fits into
that installation.

----------------------------------------------------------
Reasoning for my proposal (math, not computer science):
> > You want to align the
> > resulting ISO to 4 full cylinders in order to avoid
> > the need for a cylinder size divisible by 4.
 
> No :). I want to align to resulting isohybrid image size to 4 full 
> cylinders *only when necessary*.
That's what my proposed computation does.
It predicts the minimum number of cylinders which you need
as padding target in isohybrid.c (or .in).

Sorry for being sloppy with the natural language description
of the concept. Let me try it more formal:

The size of an image file which is aligned to full
cylinder size is:

  number_of_cylinders * h * s * 512

If we divide by 2048 = 2 exp 11, we can cancel
512 = 2 exp 9 and get 

  number_of_cylinders * h * s / 4

In order to cancel the remaining 4, we need two prime
factors 2 among those of number_of_cylinders, h, and s.

We find at least two 2s in h and s if and only if h * s is
divisible by 4. All is fine then. number_of_cylinders can
have any value.

We find exactly one 2 in h and s, if and only if h * s is
divisible by 2 but not by 4. So we need one factor of 2 in
number_of_cylinders. This is equivalent to padding up to
pairs of blocks. I.e. a double granularity of padding.

We find no 2 in h and s if and only if h * s is not
divisible by 2. Then we need two factors 2 in
number_of_cylinders. Thus padding up to full quatruples
of cylinders.

This does not mean that two or four cylinders get
added unconditionally. If the input has 131 cylinders
plus a few sectors, then it gets padded up to 132 cylinders.
But if it has 132 cylinders plus 4 sectors, then it must
get padded up to 136 cylinders. (Assumed -h 255 -s 63.)

This is the theoretical minimum. Math-wise.

> Now, say we have a "good" set of potential CHS values such as 
> 2048*255*63 so to calculate the resulting size of the isohybrid image. 
> That's good for the _size_, but such amount of Cylinders is too big for
> the partition table. This condition should already be part of the 
> source code
Both isohybrid programs check this already. After
padding is added to size.

The calculated padding cannot exceed cylinder 1024,
as long as the input ISO does not exceed 1024 cylinders.
That's because 1024 is a padding target in all three
granularities.


Have a nice day :)

Thomas

> Hi,
> 
> > Hopefully I am explaining it better this time.
> 
> Our ideas of implementation are supposed to yield the
> same result.
> 
 
Hmm, I am not sure. Or perhaps I am misunderstanding your code.

Let me try with an example (or two). For this example, let's assume:
 -h 255
 -s 63
 ISO size: 1'085'736'960 bytes ( > 1GiB)
This size happens to be a multiple of:
 63, 255 and 2048

So, using these values in my suggested loop.

1_ Calculate a potential Cylinders (amount) value;

Cylinders  trunc(roundup( ISO_size / (Heads * Sectors_per_track * 512))

Cylinders (1st attempt)  trunc(roundup(1085736960/(255*63*512))

Cylinders (1st attempt)  trunc(roundup(132))

Cylinders (1st attempt)=132 

2_ Calculate 'Cylinders * Heads * Sectors_per_track';

 132 * 255 * 63 = 2120580

3._ Is the result of the calculation (as explained in the previous step 
#2) a multiple of '4'?

 2120580 / 4 = 530145, so the answer is "yes".

3.a_ If it is, continue (to step #4).

Since the answer was "yes", we end the loop and we have found the 
minimal Cylinders amount that achieves our conditions.


Now, if I take your code:
  if (Heads * Sectors_per_track) is divisible by 4
     # Pad up to next full cylinder
     align_factor = 1
  else if (Heads * Sectors_per_track) is divisible by 2
     # Pad up to next full double cylinder 
     align_factor = 2
  else
     # Pad up to next full quatruple cylinder (nearly 32 MB)
     align_factor = 4

For the example I am presenting here, "Heads*Sectors_per_track" is 
255*63, so it is not a multiple of 4 nor of 2. Would you add anyway 
nearly 32MiB? I assume you wouldn't. Instead, I guess you would first 
evaluate whether it is already a full "valid" cylinder and that no 
additional zero-padding would be required.

When comparing your code with my suggested calculation, I see that your 
conditionals do not account for the ISO image size, but only for 
"Heads" and "Sectors_per_track".

So let's assume a new example. Let's add a few bytes to the size of the 
ISO image:
 -h 255
 -s 63
 ISO size: 1'085'736'960 bytes+2048 bytes
 ISO size: 1'085'739'008 bytes  ( > 1GiB)

Now let's perform the calculations for this new example:

Cylinders (1st attempt)  trunc(roundup(1085739008/(255*63*512))

Cylinders (1st attempt)  trunc(roundup(132.0002489884842826...)

Cylinders (1st attempt)=133

 133 * 255 * 63 = 2136645

Is it multiple of '4'?

 2136645 / 4 = 534161.25, so the answer is "no".

3.b_ If it is not, then add '+1' to the attempted Cylinders (amount) 
value and loop.

 134 * 255 * 63 = 2152710

Is it multiple of '4'?

 2152710 / 4 = 538177.5 so the answer is "no".

 135 * 255 * 63 = 2168775

Is it multiple of '4'?

 2168775 / 4 = 542193.75 so the answer is "no".

 136 * 255 * 63 = 2184840

 2184840 / 4 = 546210 so the answer is "yes".

Since the answer was "yes", we end the loop and we have found the 
minimal Cylinders amount that achieves our conditions.

In this second example, we have started with a first attempt of 133 for 
the amount of complete Cylinders, and we would had to add 3 more, up to 
136.

Other values of "ISO_size", "Heads" and
"Sectors_per_track" might lead
my suggested loop to add either 0, 1, 2, or 3 complete Cylinders. The 
zero-padding could be between 0 bytes and "almost 4 complete
Cylinders"
(4 complete Cylinders minus 2048 bytes).


Now, considering our mutual slight misunderstandings throughout this 
email thread, I have to wonder whether the wording you (Thomas) used 
for your 3 variants of zero-padding, together with the 3 conditionals, 
are equivalent to the 4 possible loops I am suggesting.

Although the size of the input image doesn't seem to be considered in 
the conditionals you (Thomas) presented, I could assume that this is 
just "implied" in the pseudo-code, similarly to what happens in my 
suggested loop (the size of the image is used in the calculations 
before the loop, not part of it). If it is not, then we have a first 
difference in our approach.

The second difference could be the 3 conditionals vs the 4 loops. As 
seen in my second example above, depending on the size of the input 
image the required padding varies, even with a "fixed" pair of
"Heads"
and "Sectors_per_track" values. Although I have not tried to find 
specific examples for having to add between 1 and 2 complete Cylinders 
(of zero-padding bytes), the second example above should show that the 
possibility indeed exists.

So, unless I am misinterpreting the wording in your code:
 "# Pad up to next full ... cylinder" ,
my second example suggests that having 4 ranges (coded as 4 loops or 
using conditionals or however it could be done) might be able to 
achieve better optimal values (less zero-padding) at least for some 
cases.

I must apologize for the long emails. Certainly if I would know how to 
code it myself it would be shorter to explain.

Thank you for your attention.

Regards,
Ady.

Hi,

my proposal is based on a mathematical proof (see mail
before) and yours is correct too. They cannot but match.

Let's inspect your examples:

Ady wrote:
>  ISO size: 1'085'736'960 bytes ( > 1GiB)
This are exactly 132 cylinders of 255x63.
132 is divisible by 4. So there is no padding needed.
> Cylinders (1st attempt)=132 
> [...]
>  2120580 / 4 = 530145, so the answer is "yes".
My calculation gets align_factor = 4.
So isohybrid would pad up to quadruple cylinders.
Well, 132 is already such a quadruple.

> When comparing your code with my suggested calculation, I see that your 
> conditionals do not account for the ISO image size, but only for 
> "Heads" and "Sectors_per_track".
The existing code in isohybrid already handles the size
for padding up to a target granularity. Currently the
target is always a single cylinder.
My new code only decides about the granularity of 1, 2,
or 4 cylinders.

>  ISO size: 1'085'736'960 bytes+2048 bytes
> [...]
> 136 * 255 * 63 = 2184840
> 2184840 / 4 = 546210 so the answer is "yes".
My proposal:
Granularity is still 4 cylinders.
Image size is 132.0002489884843 cylinders.
So the target size is the next multiple of 4: 136 cylinders.

Your algorithm is more complicated than mine, because it
tries to do the part of the job which is already
implemented in isohybrid:

    cylsize = head * sector * 512;
    frac = isostat.st_size % cylsize;
    padding = (frac > 0) ? cylsize - frac : 0;

isostat.st_size is the ISO image size in bytes.
cylsize is the current granularity of 1 cylinder.
My proposal has 1, 2, or 4 cylinders instead.

    cylsize = head * sector * 512;
    align_cylsize = cylsize * align_factor;
    frac = isostat.st_size % align_cylsize;
    padding = (frac > 0) ? align_cylsize - frac : 0;

Numbers inserted:

    cylsize = 255 * 63 * 512 = 8225280
    align_cylsize = 8225280 * 4 = 32901120 
    frac = (1085736960 + 2048) % 32901120 = 2048
    padding = 32901120 - 2048 = 32899072

This means that the image will be padded up by nearly 32 MB
to 1085736960 + 2048 + 32899072 = 1118638080 bytes.
This are  1118638080 / 255 / 63 / 512 = 136 cylinders.

Same result as yours.


Have a nice day :)

Thomas