Libguestfs - Mar 2015 - [PATCH] generator: small optimization of pod2text cache memoization

Instead of save every time there's a new element in the cache, batch the
saving to disk every 100 changes, saving the unsaved remainder at the
exit.

While not a big optimization, this reduces a bit the disk usage during
generator run.
---
 generator/utils.ml | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/generator/utils.ml b/generator/utils.ml
index 3a62084..1b00ce5 100644
--- a/generator/utils.ml
+++ b/generator/utils.ml
@@ -291,10 +291,22 @@ let pod2text_memo : (memo_key, memo_value) Hashtbl.t     
v
   with
     _ -> Hashtbl.create 13
-let pod2text_memo_updated () +let pod2text_memo_unsaved_count = ref 0
+let pod2text_memo_atexit = ref false
+let pod2text_memo_save ()    let chan = open_out pod2text_memo_filename in
   output_value chan pod2text_memo;
   close_out chan
+let pod2text_memo_updated () +  if not (!pod2text_memo_atexit) then (
+    at_exit pod2text_memo_save;
+    pod2text_memo_atexit := true;
+  );
+  pod2text_memo_unsaved_count := !pod2text_memo_unsaved_count + 1;
+  if !pod2text_memo_unsaved_count >= 100 then (
+    pod2text_memo_save ();
+    pod2text_memo_unsaved_count := 0;
+  )
 
 (* Useful if you need the longdesc POD text as plain text.  Returns a
  * list of lines.
-- 
2.1.0

On Thu, Mar 12, 2015 at 06:09:40PM +0100, Pino Toscano
wrote:
> Instead of save every time there's a new element in the cache, batch
the
> saving to disk every 100 changes, saving the unsaved remainder at the
> exit.
> 
> While not a big optimization, this reduces a bit the disk usage during
> generator run.
> ---
>  generator/utils.ml | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/generator/utils.ml b/generator/utils.ml
> index 3a62084..1b00ce5 100644
> --- a/generator/utils.ml
> +++ b/generator/utils.ml
> @@ -291,10 +291,22 @@ let pod2text_memo : (memo_key, memo_value) Hashtbl.t
>      v
>    with
>      _ -> Hashtbl.create 13
> -let pod2text_memo_updated () > +let pod2text_memo_unsaved_count = ref 0
> +let pod2text_memo_atexit = ref false
> +let pod2text_memo_save () >    let chan = open_out
pod2text_memo_filename in
>    output_value chan pod2text_memo;
>    close_out chan
> +let pod2text_memo_updated () > +  if not (!pod2text_memo_atexit) then (
> +    at_exit pod2text_memo_save;
> +    pod2text_memo_atexit := true;
> +  );
> +  pod2text_memo_unsaved_count := !pod2text_memo_unsaved_count + 1;
> +  if !pod2text_memo_unsaved_count >= 100 then (
> +    pod2text_memo_save ();
> +    pod2text_memo_unsaved_count := 0;
> +  )
>  
>  (* Useful if you need the longdesc POD text as plain text.  Returns a
>   * list of lines.
> -- 
> 2.1.0
Looks sensible, ACK.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org