Libguestfs - Mar 2021 - Consider to replace isoinfo by own code rather than by xorriso

Hi,

Richard W.M. Jones wrote:
> "megabytes" may not be a problem - and in fact hasn't been in
my
> testing.  Could it grow unbounded?
It depends on the number of files, the lengths of the file names,
and the amount of some other meta data in the tree.
So if malice is involved: Yes, it can become nasty.

In a backup ISO with 78478 files with conservative names i see about 17 MB
of directory tree (because the first data content starts at LBA 8613).
The memory representation of the tree in libisofs underneath xorriso will
need some extra bytes per file as overhead.
ps -o rsz,vsz of the process shows the numbers 47880  67136 with a
dynamically linked Debian binary of xorriso, and 73836  99844 with my
statically linked development version of GNU xorriso.

The run time of -indev with this ISO from RAM cache is about 0.5 seconds
on a not really young 3.5 GHz Xeon system.

Direct reading of the PVD would just need to load 2048 bytes from LBA 16
and to pick from byte fields at well documented locations the strings (with
trailing blanks) and numbers (with both endianesses).


Have a nice day :)

Thomas

On Wed, Mar 31, 2021 at 12:47:21AM +0200, Thomas Schmitt
wrote:
> Hi,
> 
> Richard W.M. Jones wrote:
> > "megabytes" may not be a problem - and in fact hasn't
been in my
> > testing.  Could it grow unbounded?
> 
> It depends on the number of files, the lengths of the file names,
> and the amount of some other meta data in the tree.
> So if malice is involved: Yes, it can become nasty.
> 
> In a backup ISO with 78478 files with conservative names i see about 17 MB
> of directory tree (because the first data content starts at LBA 8613).
> The memory representation of the tree in libisofs underneath xorriso will
> need some extra bytes per file as overhead.
> ps -o rsz,vsz of the process shows the numbers 47880  67136 with a
> dynamically linked Debian binary of xorriso, and 73836  99844 with my
> statically linked development version of GNU xorriso.
> 
> The run time of -indev with this ISO from RAM cache is about 0.5 seconds
> on a not really young 3.5 GHz Xeon system.
> 
> Direct reading of the PVD would just need to load 2048 bytes from LBA 16
> and to pick from byte fields at well documented locations the strings (with
> trailing blanks) and numbers (with both endianesses).
Thanks - I pushed this commit which reads directly from the PVD:

https://github.com/libguestfs/libguestfs/commit/2f587bbaec718e414e46c7e6f2a3e2662c3a1c2a

We are still using xorriso for some things outside the appliance
like creating the test ISO.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Hi,

regarding
 
https://github.com/libguestfs/libguestfs/commit/2f587bbaec718e414e46c7e6f2a3e2662c3a1c2a
i have to report that the OSDev wiki had a bug about the start address
of ISO 9660 property Abstract File Identifier.

I guess that libguestfs' daemon/isoinfo.ml should be changed like this

  -  let iso_copyright_file_id = sub 702 38 |> iso_parse_strD in
  -  let iso_abstract_file_id = sub 740 36 |> iso_parse_strD in
  +  let iso_copyright_file_id = sub 702 37 |> iso_parse_strD in
  +  let iso_abstract_file_id = sub 739 37 |> iso_parse_strD in

Reasoning:

According to ECMA-119 the field starts at byte offset 739 rather than 740:
  "8.4.24 Abstract File Identifier (BP 740 to 776)"
"BP" means Byte Position and begins its counting at 1. So the byte
offset
is one less. The length of the field is 37 bytes, like the length of its
neighbors.

For more arguments why the OSDev wiki was wrong, see:
  https://lists.gnu.org/archive/html/bug-xorriso/2021-04/msg00005.html
I felt confident enough to make this change in the wiki:
  https://wiki.osdev.org/index.php?title=ISO_9660&diff=25811&oldid=23724


Have a nice day :)

Thomas