Libguestfs - May 2023 - [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

... And while I'm scrawling my throughts into this thread ...

What we intend here are two slightly different operations:

(A) Free every unreachable object.  That's what we want in this
specific place in the code.

(B) Provide a soft test that the OCaml heap hasn't been screwed
up because of bugs in our bindings.  That's what we use eg here:
https://gitlab.com/nbdkit/libnbd/-/blob/master/ocaml/tests/test_100_handle.ml#L24

In OCaml 4 Gc.compact provides (A) and (B), in particular compaction
is quite an aggressive & costly operation that requires examining and
moving every object on the heap.  In practice we found this quickly
reveals bugs in bindings.

OCaml 5 provides (A) through Gc.full_major--albeit not documented--and
doesn't seem to provide any way to do (B) as far as I can tell.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

On 5/31/23 13:23, Richard W.M. Jones wrote:
> 
> ... And while I'm scrawling my throughts into this thread ...
> 
> What we intend here are two slightly different operations:
> 
> (A) Free every unreachable object.  That's what we want in this
> specific place in the code.
> 
> (B) Provide a soft test that the OCaml heap hasn't been screwed
> up because of bugs in our bindings.  That's what we use eg here:
>
https://gitlab.com/nbdkit/libnbd/-/blob/master/ocaml/tests/test_100_handle.ml#L24
> 
> In OCaml 4 Gc.compact provides (A) and (B), in particular compaction
> is quite an aggressive & costly operation that requires examining and
> moving every object on the heap.  In practice we found this quickly
> reveals bugs in bindings.
> 
> OCaml 5 provides (A) through Gc.full_major--albeit not documented--and
> doesn't seem to provide any way to do (B) as far as I can tell.
Re: (A) -- I didn't expect to see
<https://github.com/rpm-software-management/dnf/commit/5ce5ed1ea08ad>
again, just in different clothing. :/

At least we have a way to force (A), going forward.

Re: (B) -- can we keep testing our OCaml bindings on OCaml 4, kind of as
a "requirement", until goal (B) becomes possible on OCaml 5 too? That
wouldn't test (A)+(B) on OCaml 5, but we'd still cover (A)+(B) in total,
across multiple OCaml versions.

Meanwhile we should probably chime in on one of these upstream
discussions. For that, how would we we "formalize" goal (B)? Call it
"heap fsck" (cf. "git-fsck", which is an existent utility)?

Ticket <https://github.com/ocaml/ocaml/issues/11812> mentions, "OCaml
5
does not implement heap compaction. It is not yet clear whether we will
need one." -- a heap consistency check (for the sake of C language
extensions) would be one reason.

Laszlo