Libguestfs - May 2023 - [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

On 5/31/23 13:23, Richard W.M. Jones wrote:
> 
> ... And while I'm scrawling my throughts into this thread ...
> 
> What we intend here are two slightly different operations:
> 
> (A) Free every unreachable object.  That's what we want in this
> specific place in the code.
> 
> (B) Provide a soft test that the OCaml heap hasn't been screwed
> up because of bugs in our bindings.  That's what we use eg here:
>
https://gitlab.com/nbdkit/libnbd/-/blob/master/ocaml/tests/test_100_handle.ml#L24
> 
> In OCaml 4 Gc.compact provides (A) and (B), in particular compaction
> is quite an aggressive & costly operation that requires examining and
> moving every object on the heap.  In practice we found this quickly
> reveals bugs in bindings.
> 
> OCaml 5 provides (A) through Gc.full_major--albeit not documented--and
> doesn't seem to provide any way to do (B) as far as I can tell.
Re: (A) -- I didn't expect to see
<https://github.com/rpm-software-management/dnf/commit/5ce5ed1ea08ad>
again, just in different clothing. :/

At least we have a way to force (A), going forward.

Re: (B) -- can we keep testing our OCaml bindings on OCaml 4, kind of as
a "requirement", until goal (B) becomes possible on OCaml 5 too? That
wouldn't test (A)+(B) on OCaml 5, but we'd still cover (A)+(B) in total,
across multiple OCaml versions.

Meanwhile we should probably chime in on one of these upstream
discussions. For that, how would we we "formalize" goal (B)? Call it
"heap fsck" (cf. "git-fsck", which is an existent utility)?

Ticket <https://github.com/ocaml/ocaml/issues/11812> mentions, "OCaml
5
does not implement heap compaction. It is not yet clear whether we will
need one." -- a heap consistency check (for the sake of C language
extensions) would be one reason.

Laszlo

On Wed, May 31, 2023 at 05:33:13PM +0200, Laszlo Ersek
wrote:
> On 5/31/23 13:23, Richard W.M. Jones wrote:
> > 
> > ... And while I'm scrawling my throughts into this thread ...
> > 
> > What we intend here are two slightly different operations:
> > 
> > (A) Free every unreachable object.  That's what we want in this
> > specific place in the code.
> > 
> > (B) Provide a soft test that the OCaml heap hasn't been screwed
> > up because of bugs in our bindings.  That's what we use eg here:
> >
https://gitlab.com/nbdkit/libnbd/-/blob/master/ocaml/tests/test_100_handle.ml#L24
> > 
> > In OCaml 4 Gc.compact provides (A) and (B), in particular compaction
> > is quite an aggressive & costly operation that requires examining
and
> > moving every object on the heap.  In practice we found this quickly
> > reveals bugs in bindings.
> > 
> > OCaml 5 provides (A) through Gc.full_major--albeit not documented--and
> > doesn't seem to provide any way to do (B) as far as I can tell.
> 
> Re: (A) -- I didn't expect to see
> <https://github.com/rpm-software-management/dnf/commit/5ce5ed1ea08ad>
> again, just in different clothing. :/
> 
> At least we have a way to force (A), going forward.
> 
> Re: (B) -- can we keep testing our OCaml bindings on OCaml 4, kind of as
> a "requirement", until goal (B) becomes possible on OCaml 5 too?
That
> wouldn't test (A)+(B) on OCaml 5, but we'd still cover (A)+(B) in
total,
> across multiple OCaml versions.
> 
> Meanwhile we should probably chime in on one of these upstream
> discussions. For that, how would we we "formalize" goal (B)? Call
it
> "heap fsck" (cf. "git-fsck", which is an existent
utility)?
> 
> Ticket <https://github.com/ocaml/ocaml/issues/11812> mentions,
"OCaml 5
> does not implement heap compaction. It is not yet clear whether we will
> need one." -- a heap consistency check (for the sake of C language
> extensions) would be one reason.
To start with I have opened a discussion topic.  We can think about
opening bugs later.

https://discuss.ocaml.org/t/ocaml-heap-fsck-and-forcing-collection-of-unreachable-objects/12281

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v