Richard W.M. Jones
2023-May-31 16:13 UTC
[Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks
On Wed, May 31, 2023 at 05:33:13PM +0200, Laszlo Ersek wrote:> On 5/31/23 13:23, Richard W.M. Jones wrote: > > > > ... And while I'm scrawling my throughts into this thread ... > > > > What we intend here are two slightly different operations: > > > > (A) Free every unreachable object. That's what we want in this > > specific place in the code. > > > > (B) Provide a soft test that the OCaml heap hasn't been screwed > > up because of bugs in our bindings. That's what we use eg here: > > https://gitlab.com/nbdkit/libnbd/-/blob/master/ocaml/tests/test_100_handle.ml#L24 > > > > In OCaml 4 Gc.compact provides (A) and (B), in particular compaction > > is quite an aggressive & costly operation that requires examining and > > moving every object on the heap. In practice we found this quickly > > reveals bugs in bindings. > > > > OCaml 5 provides (A) through Gc.full_major--albeit not documented--and > > doesn't seem to provide any way to do (B) as far as I can tell. > > Re: (A) -- I didn't expect to see > <https://github.com/rpm-software-management/dnf/commit/5ce5ed1ea08ad> > again, just in different clothing. :/ > > At least we have a way to force (A), going forward. > > Re: (B) -- can we keep testing our OCaml bindings on OCaml 4, kind of as > a "requirement", until goal (B) becomes possible on OCaml 5 too? That > wouldn't test (A)+(B) on OCaml 5, but we'd still cover (A)+(B) in total, > across multiple OCaml versions. > > Meanwhile we should probably chime in on one of these upstream > discussions. For that, how would we we "formalize" goal (B)? Call it > "heap fsck" (cf. "git-fsck", which is an existent utility)? > > Ticket <https://github.com/ocaml/ocaml/issues/11812> mentions, "OCaml 5 > does not implement heap compaction. It is not yet clear whether we will > need one." -- a heap consistency check (for the sake of C language > extensions) would be one reason.To start with I have opened a discussion topic. We can think about opening bugs later. https://discuss.ocaml.org/t/ocaml-heap-fsck-and-forcing-collection-of-unreachable-objects/12281 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Richard W.M. Jones
2023-Jun-01 11:18 UTC
[Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks
[Sorry, not read the other replies yet, in meetings all day today] On Wed, May 31, 2023 at 05:13:58PM +0100, Richard W.M. Jones wrote:> To start with I have opened a discussion topic. We can think about > opening bugs later. > > https://discuss.ocaml.org/t/ocaml-heap-fsck-and-forcing-collection-of-unreachable-objects/12281There were some replies to this thread. One open merge request is notable - it adds compaction back: https://github.com/ocaml/ocaml/pull/12193 although only for "small" blocks, less than 128 * 8-byte words, which probably covers everything we do. It seems like full_major is the way to force freeing of unreachable objects, in all versions of OCaml. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org