thr3ads.net - Libguestfs - [Libguestfs] [PATCH 0/4] mltools: JSON unification [Aug 2018]

If this information is useful, please help other people find it:
Share via:

Richard W.M. Jones

2018-Aug-20 16:02 UTC

[Libguestfs] [PATCH 0/4] mltools: JSON unification

An evolution of:

https://www.redhat.com/archives/libguestfs/2018-August/msg00155.html

Richard W.M. Jones

2018-Aug-20 16:02 UTC

head link

[Libguestfs] [PATCH 1/4] mltools: JSON: Rename Yajl module as JSON_parser and move to common/mltools.

Commit bd1c5c9f4dcf38458099db8a0bf4659a07ef055d changed all the code
to use Jansson instead of yajl.  However it didn't change the OCaml
module name (still Yajl).

This commit changes the module to a neutral name ("JSON_parser") and
moves it into common/mltools so it can be used by other tools.

This leaves us in a slightly awkward situation of having two JSON-ish
OCaml modules (JSON for creating trees and JSON_parser for parsing
them) with incompatible types.  That is left for future work to
resolve.  (It should be easier to do now that both modules live in the
same directory.)

This is just renaming and general refactoring.  There should be no
change in functionality.
---
 .gitignore                                    |  2 +-
 builder/Makefile.am                           | 57 ++++---------------
 builder/index_parser.ml                       |  2 +-
 builder/repository_main.ml                    |  2 +-
 builder/simplestreams_parser.ml               |  6 +-
 builder/utils.ml                              |  2 +-
 builder/utils.mli                             |  4 +-
 .../mltools/JSON_parser-c.c                   |  8 +--
 .../yajl.ml => common/mltools/JSON_parser.ml  | 38 ++++++-------
 .../mltools/JSON_parser.mli                   | 42 +++++++-------
 .../mltools/JSON_parser_tests.ml              | 56 +++++++++---------
 common/mltools/Makefile.am                    | 39 ++++++++++++-
 docs/C_SOURCE_FILES                           |  4 +-
 po/POTFILES                                   |  3 +-
 po/POTFILES-ml                                |  4 +-
 15 files changed, 135 insertions(+), 134 deletions(-)

diff --git a/.gitignore b/.gitignore
index 41219f4a1..b714b1a43 100644
--- a/.gitignore
+++ b/.gitignore
@@ -113,7 +113,6 @@ Makefile.in
 /builder/virt-index-validate
 /builder/virt-index-validate.1
 /builder/*.xz
-/builder/yajl_tests
 /cat/stamp-virt-*.pod
 /cat/virt-cat
 /cat/virt-cat.1
@@ -146,6 +145,7 @@ Makefile.in
 /common/mltools/.depend
 /common/mltools/getopt_tests
 /common/mltools/JSON_tests
+/common/mltools/JSON_parser_tests
 /common/mltools/tools_utils_tests
 /common/mltools/oUnit-*
 /common/mlutils/.depend
diff --git a/builder/Makefile.am b/builder/Makefile.am
index 7ede544b7..f64750c7f 100644
--- a/builder/Makefile.am
+++ b/builder/Makefile.am
@@ -45,8 +45,7 @@ EXTRA_DIST = \
 	test-virt-index-validate-good-4 \
 	virt-builder.pod \
 	virt-builder-repository.pod \
-	virt-index-validate.pod \
-	yajl_tests.ml
+	virt-index-validate.pod
 
 SOURCES_MLI = \
 	builder.mli \
@@ -67,11 +66,9 @@ SOURCES_MLI = \
 	sigchecker.mli \
 	simplestreams_parser.mli \
 	sources.mli \
-	utils.mli \
-	yajl.mli
+	utils.mli
 
 SOURCES_ML = \
-	yajl.ml \
 	utils.ml \
 	osinfo_config.ml \
 	osinfo.ml \
@@ -97,11 +94,9 @@ SOURCES_C = \
 	index-parse.c \
 	index-parser-c.c \
 	pxzcat-c.c \
-	setlocale-c.c \
-	yajl-c.c
+	setlocale-c.c
 
 REPOSITORY_SOURCES_ML = \
-	yajl.ml \
 	utils.ml \
 	index.ml \
 	cache.ml \
@@ -122,15 +117,13 @@ REPOSITORY_SOURCES_MLI = \
 	index_parser.mli \
 	ini_reader.mli \
 	sigchecker.mli \
-	sources.mli \
-	yajl.mli
+	sources.mli
 
 REPOSITORY_SOURCES_C = \
 	index-scan.c \
 	index-struct.c \
 	index-parse.c \
-	index-parser-c.c \
-	yajl-c.c
+	index-parser-c.c
 
 man_MANS  noinst_DATA @@ -156,8 +149,7 @@ virt_builder_CFLAGS = \
 	-Wno-unused-macros \
 	$(LIBLZMA_CFLAGS) \
 	$(LIBTINFO_CFLAGS) \
-	$(LIBXML2_CFLAGS) \
-	$(JANSSON_CFLAGS)
+	$(LIBXML2_CFLAGS)
 
 BOBJECTS = $(SOURCES_ML:.ml=.cmo)
 XOBJECTS = $(BOBJECTS:.cmo=.cmx)
@@ -175,8 +167,7 @@ virt_builder_repository_CFLAGS = \
 	$(WARN_CFLAGS) $(WERROR_CFLAGS) \
 	-Wno-unused-macros \
 	$(LIBTINFO_CFLAGS) \
-	$(LIBXML2_CFLAGS) \
-	$(YAJL_CFLAGS)
+	$(LIBXML2_CFLAGS)
 REPOSITORY_BOBJECTS = $(REPOSITORY_SOURCES_ML:.ml=.cmo)
 REPOSITORY_XOBJECTS = $(REPOSITORY_BOBJECTS:.cmo=.cmx)
 
@@ -323,22 +314,13 @@ fedora.qcow2.xz: fedora.qcow2
 	xz --best -c $< > $@-t
 	mv $@-t $@
 
-yajl_tests_SOURCES = yajl-c.c
-yajl_tests_CPPFLAGS = $(virt_builder_CPPFLAGS)
-yajl_tests_BOBJECTS = \
-	yajl.cmo \
-	yajl_tests.cmo
-yajl_tests_XOBJECTS = $(yajl_tests_BOBJECTS:.cmo=.cmx)
-
 index_parser_tests_SOURCES = \
 	index-scan.c \
 	index-struct.c \
 	index-parser-c.c \
-	index-parse.c \
-	yajl-c.c
+	index-parse.c
 index_parser_tests_CPPFLAGS = $(virt_builder_CPPFLAGS)
 index_parser_tests_BOBJECTS = \
-	yajl.cmo \
 	utils.cmo \
 	index.cmo \
 	cache.cmo \
@@ -351,32 +333,13 @@ index_parser_tests_XOBJECTS =
$(index_parser_tests_BOBJECTS:.cmo=.cmx)
 
 # Can't call the following as <test>_OBJECTS because automake gets
confused.
 if HAVE_OCAMLOPT
-yajl_tests_THEOBJECTS = $(yajl_tests_XOBJECTS)
-yajl_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
-
 index_parser_tests_THEOBJECTS = $(index_parser_tests_XOBJECTS)
 index_parser_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 else
-yajl_tests_THEOBJECTS = $(yajl_tests_BOBJECTS)
-yajl_tests.cmo: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
-
 index_parser_tests_THEOBJECTS = $(index_parser_tests_BOBJECTS)
 index_parser_tests.cmo: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 endif
 
-yajl_tests_DEPENDENCIES = \
-	$(yajl_tests_THEOBJECTS) \
-	../common/mlpcre/mlpcre.$(MLARCHIVE) \
-	../common/mlstdutils/mlstdutils.$(MLARCHIVE) \
-	../common/mlutils/mlcutils.$(MLARCHIVE) \
-	../common/mltools/mltools.$(MLARCHIVE) \
-	../customize/customize.$(MLARCHIVE) \
-	$(top_srcdir)/ocaml-link.sh
-yajl_tests_LINK = \
-	$(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \
-	  $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLPACKAGES_TESTS)
$(OCAMLLINKFLAGS) \
-	  $(yajl_tests_THEOBJECTS) -o $@
-
 index_parser_tests_DEPENDENCIES = \
 	$(index_parser_tests_THEOBJECTS) \
 	../common/mltools/mltools.$(MLARCHIVE) \
@@ -400,8 +363,8 @@ if ENABLE_APPLIANCE
 TESTS += test-virt-builder.sh
 endif ENABLE_APPLIANCE
 if HAVE_OCAML_PKG_OUNIT
-check_PROGRAMS += yajl_tests index_parser_tests
-TESTS += yajl_tests index_parser_tests
+check_PROGRAMS += index_parser_tests
+TESTS += index_parser_tests
 endif
 
 check-valgrind:
diff --git a/builder/index_parser.ml b/builder/index_parser.ml
index 79653a91b..59fa5097d 100644
--- a/builder/index_parser.ml
+++ b/builder/index_parser.ml
@@ -140,7 +140,7 @@ let get_index ~downloader ~sigchecker ?(template = false) {
Sources.uri; proxy }
               match detect_file_type filepath with
               | `Unknown ->
                 let infos = Utils.get_image_infos filepath in
-                Yajl.object_get_number "virtual-size" infos
+                JSON_parser.object_get_number "virtual-size" infos
               | `XZ | `GZip | `Tar | ` Zip ->
                 eprintf (f_"%s: cannot determine the virtual size of %s
due to compression")
                         prog filepath;
diff --git a/builder/repository_main.ml b/builder/repository_main.ml
index 03d6ed13e..4ec434e57 100644
--- a/builder/repository_main.ml
+++ b/builder/repository_main.ml
@@ -22,7 +22,7 @@ open Tools_utils
 open Unix_utils
 open Getopt.OptionName
 open Utils
-open Yajl
+open JSON_parser
 open Xpath_helpers
 
 open Printf
diff --git a/builder/simplestreams_parser.ml b/builder/simplestreams_parser.ml
index 18f45a6fd..fa5b887ac 100644
--- a/builder/simplestreams_parser.ml
+++ b/builder/simplestreams_parser.ml
@@ -20,7 +20,7 @@ open Std_utils
 open Tools_utils
 open Common_gettext.Gettext
 
-open Yajl
+open JSON_parser
 open Utils
 
 open Printf
@@ -44,7 +44,7 @@ let get_index ~downloader ~sigchecker { Sources.uri; proxy }  
| Some f -> f
       ) else
         tmpfile in
-    yajl_tree_parse (read_whole_file file) in
+    json_parser_tree_parse (read_whole_file file) in
 
   let downloads      let uri_index @@ -106,7 +106,7 @@ let get_index
~downloader ~sigchecker { Sources.uri; proxy }                     * the ones
related to checksums, explicitly filter
                    * the supported checksums.
                    *)
-                  | ("sha256"|"sha512" as t, Yajl_string c)
->
+                  | ("sha256"|"sha512" as t,
JSON_parser_string c) ->
                     Some (Checksums.of_string t c)
                   | _ -> None
                 ) disk_item in
diff --git a/builder/utils.ml b/builder/utils.ml
index 1446561b0..538a43be9 100644
--- a/builder/utils.ml
+++ b/builder/utils.ml
@@ -43,4 +43,4 @@ let get_image_infos filepath    let qemuimg_cmd =
"qemu-img info --output json " ^ quote filepath in
   let lines = external_command qemuimg_cmd in
   let line = String.concat "\n" lines in
-  Yajl.yajl_tree_parse line
+  JSON_parser.json_parser_tree_parse line
diff --git a/builder/utils.mli b/builder/utils.mli
index 4249fd956..5dde43a01 100644
--- a/builder/utils.mli
+++ b/builder/utils.mli
@@ -29,9 +29,9 @@ and revision  val string_of_revision : revision -> string
 (** Convert a {!revision} into a string. *)
 
-val get_image_infos : string -> Yajl.yajl_val
+val get_image_infos : string -> JSON_parser.json_parser_val
 (** [get_image_infos path] Run qemu-img info on the image pointed at
-    path as YAJL tree. *)
+    path as JSON tree. *)
 
 val increment_revision : revision -> revision
 (** Add one to the revision number *)
diff --git a/builder/yajl-c.c b/common/mltools/JSON_parser-c.c
similarity index 94%
rename from builder/yajl-c.c
rename to common/mltools/JSON_parser-c.c
index e53755f55..32432dc5b 100644
--- a/builder/yajl-c.c
+++ b/common/mltools/JSON_parser-c.c
@@ -1,5 +1,5 @@
-/* virt-builder
- * Copyright (C) 2015 Red Hat Inc.
+/* JSON parser
+ * Copyright (C) 2015-2018 Red Hat Inc.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -30,7 +30,7 @@
 
 #define Val_none (Val_int (0))
 
-value virt_builder_yajl_tree_parse (value stringv);
+value virt_builder_json_parser_tree_parse (value stringv);
 
 static value
 convert_json_t (json_t *val, int level)
@@ -95,7 +95,7 @@ convert_json_t (json_t *val, int level)
 }
 
 value
-virt_builder_yajl_tree_parse (value stringv)
+virt_builder_json_parser_tree_parse (value stringv)
 {
   CAMLparam1 (stringv);
   CAMLlocal1 (rv);
diff --git a/builder/yajl.ml b/common/mltools/JSON_parser.ml
similarity index 75%
rename from builder/yajl.ml
rename to common/mltools/JSON_parser.ml
index d95f3932a..a82127454 100644
--- a/builder/yajl.ml
+++ b/common/mltools/JSON_parser.ml
@@ -1,5 +1,5 @@
-(* virt-builder
- * Copyright (C) 2015 Red Hat Inc.
+(* JSON parser
+ * Copyright (C) 2015-2018 Red Hat Inc.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -20,19 +20,19 @@ open Std_utils
 open Tools_utils
 open Common_gettext.Gettext
 
-type yajl_val -| Yajl_null
-| Yajl_string of string
-| Yajl_number of int64
-| Yajl_double of float
-| Yajl_object of (string * yajl_val) array
-| Yajl_array of yajl_val array
-| Yajl_bool of bool
+type json_parser_val +| JSON_parser_null
+| JSON_parser_string of string
+| JSON_parser_number of int64
+| JSON_parser_double of float
+| JSON_parser_object of (string * json_parser_val) array
+| JSON_parser_array of json_parser_val array
+| JSON_parser_bool of bool
 
-external yajl_tree_parse : string -> yajl_val =
"virt_builder_yajl_tree_parse"
+external json_parser_tree_parse : string -> json_parser_val =
"virt_builder_json_parser_tree_parse"
 
 let object_find_optional key = function
-  | Yajl_object o ->
+  | JSON_parser_object o ->
     (match List.filter (fun (k, _) -> k = key) (Array.to_list o) with
     | [(k, v)] -> Some v
     | [] -> None
@@ -46,27 +46,27 @@ let object_find key yv  
 let object_get_string key yv    match object_find key yv with
-  | Yajl_string s -> s
+  | JSON_parser_string s -> s
   | _ -> error (f_"the value for the key ‘%s’ is not a string")
key
 
 let object_find_object key yv    match object_find key yv with
-  | Yajl_object _ as o -> o
+  | JSON_parser_object _ as o -> o
   | _ -> error (f_"the value for the key ‘%s’ is not an object")
key
 
 let object_find_objects fn = function
-  | Yajl_object o -> List.filter_map fn (Array.to_list o)
+  | JSON_parser_object o -> List.filter_map fn (Array.to_list o)
   | _ -> error (f_"the value is not an object")
 
 let object_get_object key yv    match object_find_object key yv with
-  | Yajl_object o -> o
+  | JSON_parser_object o -> o
   | _ -> assert false (* object_find_object already errors out. *)
 
 let object_get_number key yv    match object_find key yv with
-  | Yajl_number n -> n
-  | Yajl_double d -> Int64.of_float d
+  | JSON_parser_number n -> n
+  | JSON_parser_double d -> Int64.of_float d
   | _ -> error (f_"the value for the key ‘%s’ is not an integer")
key
 
 let objects_get_string key yvs @@ -74,7 +74,7 @@ let objects_get_string key yvs
| [] -> None
     | x :: xs ->
       (match object_find_optional key x with
-      | Some (Yajl_string s) -> Some s
+      | Some (JSON_parser_string s) -> Some s
       | Some _ -> error (f_"the value for key ‘%s’ is not a string as
expected") key
       | None -> loop xs
       )
diff --git a/builder/yajl.mli b/common/mltools/JSON_parser.mli
similarity index 60%
rename from builder/yajl.mli
rename to common/mltools/JSON_parser.mli
index ca0eb92f4..f505953f2 100644
--- a/builder/yajl.mli
+++ b/common/mltools/JSON_parser.mli
@@ -1,5 +1,5 @@
-(* virt-builder
- * Copyright (C) 2015 Red Hat Inc.
+(* JSON parser
+ * Copyright (C) 2015-2018 Red Hat Inc.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -16,43 +16,43 @@
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *)
 
-type yajl_val -| Yajl_null
-| Yajl_string of string
-| Yajl_number of int64
-| Yajl_double of float
-| Yajl_object of (string * yajl_val) array
-| Yajl_array of yajl_val array
-| Yajl_bool of bool
+type json_parser_val +| JSON_parser_null
+| JSON_parser_string of string
+| JSON_parser_number of int64
+| JSON_parser_double of float
+| JSON_parser_object of (string * json_parser_val) array
+| JSON_parser_array of json_parser_val array
+| JSON_parser_bool of bool
 
-val yajl_tree_parse : string -> yajl_val
+val json_parser_tree_parse : string -> json_parser_val
 (** Parse the JSON string. *)
 
-val object_get_string : string -> yajl_val -> string
+val object_get_string : string -> json_parser_val -> string
 (** [object_get_string key yv] gets the value of the [key] field as a string
     in the [yv] structure *)
 
-val object_find_object : string -> yajl_val -> yajl_val
-(** [object_get_object key yv] gets the value of the [key] field as a yajl
+val object_find_object : string -> json_parser_val -> json_parser_val
+(** [object_get_object key yv] gets the value of the [key] field as a JSON
     value in the [yv] structure.
 
     Mind the returned type is different from [object_get_object] *)
 
-val object_get_object : string -> yajl_val -> (string * yajl_val) array
-(** [object_get_object key yv] gets the value of the [key] field as a Yajl
+val object_get_object : string -> json_parser_val -> (string *
json_parser_val) array
+(** [object_get_object key yv] gets the value of the [key] field as a JSON
     object in the [yv] structure *)
 
-val object_get_number : string -> yajl_val -> int64
+val object_get_number : string -> json_parser_val -> int64
 (** [object_get_number key yv] gets the value of the [key] field as an
     integer in the [yv] structure *)
 
-val objects_get_string : string -> yajl_val list -> string
+val objects_get_string : string -> json_parser_val list -> string
 (** [objects_get_string key yvs] gets the value of the [key] field as a string
-    in an [yvs] list of yajl_val structure.
+    in an [yvs] list of json_parser_val structure.
 
     The key may not be found at all in the list, in which case an error
     is raised *)
 
-val object_find_objects : ((string * yajl_val) -> 'a option) ->
yajl_val -> 'a list
-(** [object_find_objects fn obj] returns all the Yajl objects matching the [fn]
+val object_find_objects : ((string * json_parser_val) -> 'a option)
-> json_parser_val -> 'a list
+(** [object_find_objects fn obj] returns all the JSON objects matching the [fn]
     function in [obj] list. *)
diff --git a/builder/yajl_tests.ml b/common/mltools/JSON_parser_tests.ml
similarity index 73%
rename from builder/yajl_tests.ml
rename to common/mltools/JSON_parser_tests.ml
index f5a44f2fa..42045122d 100644
--- a/builder/yajl_tests.ml
+++ b/common/mltools/JSON_parser_tests.ml
@@ -16,10 +16,10 @@
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *)
 
-(* This file tests the Yajl module. *)
+(* This file tests the JSON_parser module. *)
 
 open OUnit2
-open Yajl
+open JSON_parser
 
 (* Utils. *)
 let assert_equal_string = assert_equal ~printer:(fun x -> x)
@@ -27,16 +27,16 @@ let assert_equal_int = assert_equal ~printer:(fun x ->
string_of_int x)
 let assert_equal_int64 = assert_equal ~printer:(fun x -> Int64.to_string x)
 let assert_equal_bool = assert_equal ~printer:(fun x -> string_of_bool x)
 
-let string_of_yajl_val_type = function
-  | Yajl_null -> "null"
-  | Yajl_string _ -> "string"
-  | Yajl_number _ -> "number"
-  | Yajl_double _ -> "float"
-  | Yajl_object _ -> "object"
-  | Yajl_array _ -> "array"
-  | Yajl_bool _ -> "bool"
+let string_of_json_parser_val_type = function
+  | JSON_parser_null -> "null"
+  | JSON_parser_string _ -> "string"
+  | JSON_parser_number _ -> "number"
+  | JSON_parser_double _ -> "float"
+  | JSON_parser_object _ -> "object"
+  | JSON_parser_array _ -> "array"
+  | JSON_parser_bool _ -> "bool"
 let type_mismatch_string exp value -  Printf.sprintf "value is not %s but
%s" exp (string_of_yajl_val_type value)
+  Printf.sprintf "value is not %s but %s" exp
(string_of_json_parser_val_type value)
 
 let assert_raises_invalid_argument str    (* Replace the Invalid_argument
string with a fixed one, just to check
@@ -44,37 +44,37 @@ let assert_raises_invalid_argument str     *)
   let mock = "parse_error" in
   let wrapped_tree_parse str -    try yajl_tree_parse str
+    try json_parser_tree_parse str
     with Invalid_argument _ -> raise (Invalid_argument mock) in
   assert_raises (Invalid_argument mock) (fun () -> wrapped_tree_parse str)
 let assert_raises_nested str    let err = "too many levels of object/array
nesting" in
-  assert_raises (Invalid_argument err) (fun () -> yajl_tree_parse str)
+  assert_raises (Invalid_argument err) (fun () -> json_parser_tree_parse
str)
 
 let assert_is_object value    assert_bool
     (type_mismatch_string "object" value)
-    (match value with | Yajl_object _ -> true | _ -> false)
+    (match value with | JSON_parser_object _ -> true | _ -> false)
 let assert_is_string exp = function
-  | Yajl_string s -> assert_equal_string exp s
+  | JSON_parser_string s -> assert_equal_string exp s
   | _ as v -> assert_failure (type_mismatch_string "string" v)
 let assert_is_number exp = function
-  | Yajl_number n -> assert_equal_int64 exp n
-  | Yajl_double d -> assert_equal_int64 exp (Int64.of_float d)
+  | JSON_parser_number n -> assert_equal_int64 exp n
+  | JSON_parser_double d -> assert_equal_int64 exp (Int64.of_float d)
   | _ as v -> assert_failure (type_mismatch_string "number/double"
v)
 let assert_is_array value    assert_bool
     (type_mismatch_string "array" value)
-    (match value with | Yajl_array _ -> true | _ -> false)
+    (match value with | JSON_parser_array _ -> true | _ -> false)
 let assert_is_bool exp = function
-  | Yajl_bool b -> assert_equal_bool exp b
+  | JSON_parser_bool b -> assert_equal_bool exp b
   | _ as v -> assert_failure (type_mismatch_string "bool" v)
 
 let get_object_list = function
-  | Yajl_object x -> x
+  | JSON_parser_object x -> x
   | _ as v -> assert_failure (type_mismatch_string "object" v)
 let get_array = function
-  | Yajl_array x -> x
+  | JSON_parser_array x -> x
   | _ as v -> assert_failure (type_mismatch_string "array" v)
 
 
@@ -90,29 +90,29 @@ let test_tree_parse_invalid ctx    assert_raises_nested str
 
 let test_tree_parse_basic ctx -  let value = yajl_tree_parse "{}" in
+  let value = json_parser_tree_parse "{}" in
   assert_is_object value;
 
-  let value = yajl_tree_parse "\"foo\"" in
+  let value = json_parser_tree_parse "\"foo\"" in
   assert_is_string "foo" value;
 
-  let value = yajl_tree_parse "[]" in
+  let value = json_parser_tree_parse "[]" in
   assert_is_array value
 
 let test_tree_parse_inspect ctx -  let value = yajl_tree_parse
"{\"foo\":5}" in
+  let value = json_parser_tree_parse "{\"foo\":5}" in
   let l = get_object_list value in
   assert_equal_int 1 (Array.length l);
   assert_equal_string "foo" (fst (l.(0)));
   assert_is_number 5_L (snd (l.(0)));
 
-  let value = yajl_tree_parse "[\"foo\", true]" in
+  let value = json_parser_tree_parse "[\"foo\", true]" in
   let a = get_array value in
   assert_equal_int 2 (Array.length a);
   assert_is_string "foo" (a.(0));
   assert_is_bool true (a.(1));
 
-  let value = yajl_tree_parse "{\"foo\":[false, {}, 10],
\"second\":2}" in
+  let value = json_parser_tree_parse "{\"foo\":[false, {}, 10],
\"second\":2}" in
   let l = get_object_list value in
   assert_equal_int 2 (Array.length l);
   assert_equal_string "foo" (fst (l.(0)));
@@ -126,7 +126,7 @@ let test_tree_parse_inspect ctx  
 (* Suites declaration. *)
 let suite -  "builder Yajl" >:::
+  "mltools JSON_parser" >:::
     [
       "tree_parse.invalid" >:: test_tree_parse_invalid;
       "tree_parse.basic" >:: test_tree_parse_basic;
diff --git a/common/mltools/Makefile.am b/common/mltools/Makefile.am
index 66b18f5de..ac5f53651 100644
--- a/common/mltools/Makefile.am
+++ b/common/mltools/Makefile.am
@@ -23,6 +23,7 @@ EXTRA_DIST = \
 	$(SOURCES_C) \
 	getopt_tests.ml \
 	JSON_tests.ml \
+	JSON_parser_tests.ml \
 	test-getopt.sh \
 	tools_utils_tests.ml
 
@@ -31,6 +32,7 @@ SOURCES_MLI = \
 	curl.mli \
 	getopt.mli \
 	JSON.mli \
+	JSON_parser.mli \
 	planner.mli \
 	regedit.mli \
 	registry.mli \
@@ -47,6 +49,7 @@ SOURCES_ML = \
 	registry.ml \
 	regedit.ml \
 	JSON.ml \
+	JSON_parser.ml \
 	curl.ml \
 	checksums.ml \
 	xpath_helpers.ml \
@@ -57,6 +60,7 @@ SOURCES_C = \
 	../options/keys.c \
 	../options/uri.c \
 	getopt-c.c \
+	JSON_parser-c.c \
 	tools_utils-c.c \
 	uri-c.c
 
@@ -94,6 +98,7 @@ libmltools_a_CPPFLAGS = \
 libmltools_a_CFLAGS = \
 	$(WARN_CFLAGS) $(WERROR_CFLAGS) \
 	$(LIBVIRT_CFLAGS) $(LIBXML2_CFLAGS) \
+	$(JANSSON_CFLAGS) \
 	-fPIC
 
 BOBJECTS = $(SOURCES_ML:.ml=.cmo)
@@ -128,6 +133,7 @@ OCAMLCLIBS = \
 	$(LIBCRYPT_LIBS) \
 	$(LIBVIRT_LIBS) \
 	$(LIBXML2_LIBS) \
+	$(JANSSON_LIBS) \
 	$(LIBINTL) \
 	-lgnu
 
@@ -169,6 +175,16 @@ JSON_tests_SOURCES = dummy.c
 JSON_tests_BOBJECTS = JSON_tests.cmo
 JSON_tests_XOBJECTS = $(JSON_tests_BOBJECTS:.cmo=.cmx)
 
+JSON_parser_tests_SOURCES = dummy.c
+JSON_parser_tests_CPPFLAGS = \
+	-I . \
+	-I$(top_builddir) \
+	-I$(shell $(OCAMLC) -where) \
+	-I$(top_srcdir)/lib
+JSON_parser_tests_BOBJECTS = \
+	JSON_parser_tests.cmo
+JSON_parser_tests_XOBJECTS = $(JSON_parser_tests_BOBJECTS:.cmo=.cmx)
+
 # Can't call the following as <test>_OBJECTS because automake gets
confused.
 if !HAVE_OCAMLOPT
 tools_utils_tests_THEOBJECTS = $(tools_utils_tests_BOBJECTS)
@@ -179,6 +195,9 @@ getopt_tests.cmo: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 
 JSON_tests_THEOBJECTS = $(JSON_tests_BOBJECTS)
 JSON_tests.cmo: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
+
+JSON_parser_tests_THEOBJECTS = $(JSON_parser_tests_BOBJECTS)
+JSON_parser_tests.cmo: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 else
 tools_utils_tests_THEOBJECTS = $(tools_utils_tests_XOBJECTS)
 tools_utils_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
@@ -188,6 +207,9 @@ getopt_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 
 JSON_tests_THEOBJECTS = $(JSON_tests_XOBJECTS)
 JSON_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
+
+JSON_parser_tests_THEOBJECTS = $(JSON_parser_tests_XOBJECTS)
+JSON_parser_tests.cmx: OCAMLPACKAGES += $(OCAMLPACKAGES_TESTS)
 endif
 
 OCAMLLINKFLAGS = \
@@ -237,6 +259,19 @@ JSON_tests_LINK = \
 	  $(OCAMLPACKAGES) $(OCAMLPACKAGES_TESTS) \
 	  $(JSON_tests_THEOBJECTS) -o $@
 
+JSON_parser_tests_DEPENDENCIES = \
+	$(JSON_parser_tests_THEOBJECTS) \
+	../mlstdutils/mlstdutils.$(MLARCHIVE) \
+	../mlgettext/mlgettext.$(MLARCHIVE) \
+	../mlpcre/mlpcre.$(MLARCHIVE) \
+	$(MLTOOLS_CMA) \
+	$(top_srcdir)/ocaml-link.sh
+JSON_parser_tests_LINK = \
+	$(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \
+	  $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLLINKFLAGS) \
+	  $(OCAMLPACKAGES) $(OCAMLPACKAGES_TESTS) \
+	  $(JSON_parser_tests_THEOBJECTS) -o $@
+
 TESTS_ENVIRONMENT = $(top_builddir)/run --test
 
 TESTS = \
@@ -245,8 +280,8 @@ check_PROGRAMS = \
 	getopt_tests
 
 if HAVE_OCAML_PKG_OUNIT
-check_PROGRAMS += JSON_tests tools_utils_tests
-TESTS += JSON_tests tools_utils_tests
+check_PROGRAMS += JSON_tests JSON_parser_tests tools_utils_tests
+TESTS += JSON_tests JSON_parser_tests tools_utils_tests
 endif
 
 check-valgrind:
diff --git a/docs/C_SOURCE_FILES b/docs/C_SOURCE_FILES
index f0d1f6c9d..b1db59cf5 100644
--- a/docs/C_SOURCE_FILES
+++ b/docs/C_SOURCE_FILES
@@ -5,7 +5,6 @@ builder/index-struct.h
 builder/index-validate.c
 builder/pxzcat-c.c
 builder/setlocale-c.c
-builder/yajl-c.c
 cat/cat.c
 cat/filesystems.c
 cat/log.c
@@ -21,6 +20,7 @@ common/mlpcre/dummy.c
 common/mlpcre/pcre-c.c
 common/mlprogress/progress-c.c
 common/mlstdutils/dummy.c
+common/mltools/JSON_parser-c.c
 common/mltools/dummy.c
 common/mltools/getopt-c.c
 common/mltools/tools_utils-c.c
@@ -62,6 +62,8 @@ common/utils/cleanups.h
 common/utils/gnulib-cleanups.c
 common/utils/guestfs-utils.h
 common/utils/libxml2-cleanups.c
+common/utils/libxml2-utils.c
+common/utils/libxml2-utils.h
 common/utils/utils.c
 common/visit/visit.c
 common/visit/visit.h
diff --git a/po/POTFILES b/po/POTFILES
index 55d7e6894..4e53b694f 100644
--- a/po/POTFILES
+++ b/po/POTFILES
@@ -6,7 +6,6 @@ builder/index-struct.c
 builder/index-validate.c
 builder/pxzcat-c.c
 builder/setlocale-c.c
-builder/yajl-c.c
 cat/cat.c
 cat/filesystems.c
 cat/log.c
@@ -22,6 +21,7 @@ common/mlpcre/dummy.c
 common/mlpcre/pcre-c.c
 common/mlprogress/progress-c.c
 common/mlstdutils/dummy.c
+common/mltools/JSON_parser-c.c
 common/mltools/dummy.c
 common/mltools/getopt-c.c
 common/mltools/tools_utils-c.c
@@ -51,6 +51,7 @@ common/structs/structs-print.c
 common/utils/cleanups.c
 common/utils/gnulib-cleanups.c
 common/utils/libxml2-cleanups.c
+common/utils/libxml2-utils.c
 common/utils/utils.c
 common/visit/visit.c
 common/windows/windows.c
diff --git a/po/POTFILES-ml b/po/POTFILES-ml
index b18ff7410..de07e37d4 100644
--- a/po/POTFILES-ml
+++ b/po/POTFILES-ml
@@ -18,8 +18,6 @@ builder/sigchecker.ml
 builder/simplestreams_parser.ml
 builder/sources.ml
 builder/utils.ml
-builder/yajl.ml
-builder/yajl_tests.ml
 common/mlaugeas/augeas.ml
 common/mlgettext/common_gettext.ml
 common/mlpcre/PCRE.ml
@@ -31,6 +29,8 @@ common/mlstdutils/std_utils_tests.ml
 common/mlstdutils/stringMap.ml
 common/mlstdutils/stringSet.ml
 common/mltools/JSON.ml
+common/mltools/JSON_parser.ml
+common/mltools/JSON_parser_tests.ml
 common/mltools/JSON_tests.ml
 common/mltools/URI.ml
 common/mltools/checksums.ml
-- 
2.18.0

Richard W.M. Jones

2018-Aug-20 16:02 UTC

head link

[Libguestfs] [PATCH 2/4] mltools: JSON: Implement JSON.Float.

This is not actually used anywhere, but it completes the JSON
implementation and it will be useful for unifying JSON with
JSON_parser.
---
 common/mltools/JSON.ml       |  6 ++++++
 common/mltools/JSON.mli      |  1 +
 common/mltools/JSON_tests.ml | 20 ++++++++++++++++++++
 3 files changed, 27 insertions(+)

diff --git a/common/mltools/JSON.ml b/common/mltools/JSON.ml
index 1a88f8b1f..a51037ab4 100644
--- a/common/mltools/JSON.ml
+++ b/common/mltools/JSON.ml
@@ -23,6 +23,7 @@ and json_t    | String of string
   | Int of int
   | Int64 of int64
+  | Float of float
   | Bool of bool
   | List of json_t list
   | Dict of field list
@@ -110,6 +111,11 @@ and output_field ~indent ~fmt = function
   | Int i -> string_of_int i
   | Bool b -> if b then "true" else "false"
   | Int64 i -> Int64.to_string i
+  (* The JSON standard permits either "1" or "1.0" but not
"1.".
+   * OCaml string_of_float will generate "1.", but the %g formatter
+   * will only generate the valid JSON values.
+   *)
+  | Float f -> Printf.sprintf "%g" f
   | List l -> output_list ~indent:(indent + 1) ~fmt l
   | Dict d -> output_dict ~indent:(indent + 1) ~fmt d
 
diff --git a/common/mltools/JSON.mli b/common/mltools/JSON.mli
index 5b487af8e..06a777c20 100644
--- a/common/mltools/JSON.mli
+++ b/common/mltools/JSON.mli
@@ -23,6 +23,7 @@ and json_t =                    (** JSON value. *)
   | String of string            (** string value, eg. ["string"] *)
   | Int of int                  (** int value, eg. [99] *)
   | Int64 of int64              (** int64 value, eg. [99] *)
+  | Float of float              (** floating point value, eg. [9.9] *)
   | Bool of bool                (** boolean value, [true] or [false] *)
   | List of json_t list         (** array value, eg. [[1,2,3]] *)
   | Dict of field list          (** object, eg. [{ "a": 1,
"b": "c" }] *)
diff --git a/common/mltools/JSON_tests.ml b/common/mltools/JSON_tests.ml
index 55a474474..2f3998e6e 100644
--- a/common/mltools/JSON_tests.ml
+++ b/common/mltools/JSON_tests.ml
@@ -71,6 +71,25 @@ let test_int ctx  }"
     (JSON.string_of_doc ~fmt:JSON.Indented doc)
 
+let test_float ctx +  let doc = [ "test_zero", JSON.Float 0.;
+              "test_one", JSON.Float 1.;
+              "test_frac", JSON.Float 1.5;
+              "test_neg_frac", JSON.Float (-1.5);
+              "test_exp", JSON.Float 1e100 ] in
+  assert_equal_string
+    "{ \"test_zero\": 0, \"test_one\": 1,
\"test_frac\": 1.5, \"test_neg_frac\": -1.5,
\"test_exp\": 1e+100 }"
+    (JSON.string_of_doc doc);
+  assert_equal_string
+    "{
+  \"test_zero\": 0,
+  \"test_one\": 1,
+  \"test_frac\": 1.5,
+  \"test_neg_frac\": -1.5,
+  \"test_exp\": 1e+100
+}"
+    (JSON.string_of_doc ~fmt:JSON.Indented doc)
+
 let test_list ctx    let doc = [ "item", JSON.List [ JSON.String
"foo"; JSON.Int 10; JSON.Bool true ] ] in
   assert_equal_string
@@ -227,6 +246,7 @@ let suite        "basic.string" >::
test_string;
       "basic.bool" >:: test_bool;
       "basic.int" >:: test_int;
+      "basic.float" >:: test_float;
       "basic.list" >:: test_list;
       "basic.nested_dict" >:: test_nested_dict;
       "basic.nested_nested dict" >:: test_nested_nested_dict;
-- 
2.18.0

Richard W.M. Jones

2018-Aug-20 16:02 UTC

head link

[Libguestfs] [PATCH 3/4] mltools: JSON: combine JSON.Int and JSON.Int64 into a single variant.

It was convenient to have these as separate variants when we were only
using this type to generate JSON.  However if we also use it to parse
JSON documents then integers in the document should only map to a
single variant.
---
 builder/list_entries.ml      |  4 ++--
 common/mltools/JSON.ml       |  8 +++-----
 common/mltools/JSON.mli      |  3 +--
 common/mltools/JSON_tests.ml | 40 +++++++++++++++++++-----------------
 v2v/input_libvirt_xen_ssh.ml |  2 +-
 v2v/input_ova.ml             |  4 ++--
 v2v/input_vmx.ml             |  2 +-
 v2v/output_rhv_upload.ml     |  2 +-
 v2v/parse_libvirt_xml.ml     |  4 ++--
 v2v/utils.ml                 |  4 ++--
 v2v/vCenter.ml               |  4 ++--
 11 files changed, 38 insertions(+), 39 deletions(-)

diff --git a/builder/list_entries.ml b/builder/list_entries.ml
index c64d554a2..f1f67290c 100644
--- a/builder/list_entries.ml
+++ b/builder/list_entries.ml
@@ -117,7 +117,7 @@ and list_entries_json ~sources index            | None ->
item
           | Some str -> ("full-name", JSON.String str) :: item in
         let item = ("arch", JSON.String (Index.string_of_arch arch))
:: item in
-        let item = ("size", JSON.Int64 size) :: item in
+        let item = ("size", JSON.Int size) :: item in
         let item            match compressed_size with
           | None -> item
@@ -148,7 +148,7 @@ and list_entries_json ~sources index          JSON.Dict
(List.rev item)
     ) index in
   let doc = [
-    "version", JSON.Int 1;
+    "version", JSON.Int 1L;
     "sources", JSON.List json_sources;
     "templates", JSON.List json_templates;
   ] in
diff --git a/common/mltools/JSON.ml b/common/mltools/JSON.ml
index a51037ab4..8c2e695e2 100644
--- a/common/mltools/JSON.ml
+++ b/common/mltools/JSON.ml
@@ -21,8 +21,7 @@
 type field = string * json_t
 and json_t    | String of string
-  | Int of int
-  | Int64 of int64
+  | Int of int64
   | Float of float
   | Bool of bool
   | List of json_t list
@@ -108,14 +107,13 @@ and output_list fields ~fmt ~indent  
 and output_field ~indent ~fmt = function
   | String s -> json_quote_string s
-  | Int i -> string_of_int i
-  | Bool b -> if b then "true" else "false"
-  | Int64 i -> Int64.to_string i
+  | Int i -> Int64.to_string i
   (* The JSON standard permits either "1" or "1.0" but not
"1.".
    * OCaml string_of_float will generate "1.", but the %g formatter
    * will only generate the valid JSON values.
    *)
   | Float f -> Printf.sprintf "%g" f
+  | Bool b -> if b then "true" else "false"
   | List l -> output_list ~indent:(indent + 1) ~fmt l
   | Dict d -> output_dict ~indent:(indent + 1) ~fmt d
 
diff --git a/common/mltools/JSON.mli b/common/mltools/JSON.mli
index 06a777c20..c85b786ff 100644
--- a/common/mltools/JSON.mli
+++ b/common/mltools/JSON.mli
@@ -21,8 +21,7 @@
 type field = string * json_t    (** ["field": "value"] *)
 and json_t =                    (** JSON value. *)
   | String of string            (** string value, eg. ["string"] *)
-  | Int of int                  (** int value, eg. [99] *)
-  | Int64 of int64              (** int64 value, eg. [99] *)
+  | Int of int64                (** int value, eg. [99] *)
   | Float of float              (** floating point value, eg. [9.9] *)
   | Bool of bool                (** boolean value, [true] or [false] *)
   | List of json_t list         (** array value, eg. [[1,2,3]] *)
diff --git a/common/mltools/JSON_tests.ml b/common/mltools/JSON_tests.ml
index 2f3998e6e..6bd98af26 100644
--- a/common/mltools/JSON_tests.ml
+++ b/common/mltools/JSON_tests.ml
@@ -18,6 +18,8 @@
 
 (* This file tests the JSON module. *)
 
+open Std_utils
+
 open OUnit2
 
 (* Utils. *)
@@ -53,21 +55,21 @@ let test_bool ctx      (JSON.string_of_doc
~fmt:JSON.Indented doc)
 
 let test_int ctx -  let doc = [ "test_zero", JSON.Int 0;
-              "test_pos", JSON.Int 5;
-              "test_neg", JSON.Int (-5);
-              "test_pos64", JSON.Int64 (Int64.of_int 10);
-              "test_neg64", JSON.Int64 (Int64.of_int (-10)); ] in
+  let doc = [ "test_zero", JSON.Int 0L;
+              "test_pos", JSON.Int 5L;
+              "test_neg", JSON.Int (-5L);
+              "test_pos64", JSON.Int 1_000_000_000_000L;
+              "test_neg64", JSON.Int (-1_000_000_000_000L); ] in
   assert_equal_string
-    "{ \"test_zero\": 0, \"test_pos\": 5,
\"test_neg\": -5, \"test_pos64\": 10,
\"test_neg64\": -10 }"
+    "{ \"test_zero\": 0, \"test_pos\": 5,
\"test_neg\": -5, \"test_pos64\": 1000000000000,
\"test_neg64\": -1000000000000 }"
     (JSON.string_of_doc doc);
   assert_equal_string
     "{
   \"test_zero\": 0,
   \"test_pos\": 5,
   \"test_neg\": -5,
-  \"test_pos64\": 10,
-  \"test_neg64\": -10
+  \"test_pos64\": 1000000000000,
+  \"test_neg64\": -1000000000000
 }"
     (JSON.string_of_doc ~fmt:JSON.Indented doc)
 
@@ -91,7 +93,7 @@ let test_float ctx      (JSON.string_of_doc ~fmt:JSON.Indented
doc)
 
 let test_list ctx -  let doc = [ "item", JSON.List [ JSON.String
"foo"; JSON.Int 10; JSON.Bool true ] ] in
+  let doc = [ "item", JSON.List [ JSON.String "foo";
JSON.Int 10L; JSON.Bool true ] ] in
   assert_equal_string
     "{ \"item\": [ \"foo\", 10, true ] }"
     (JSON.string_of_doc doc);
@@ -107,8 +109,8 @@ let test_list ctx  
 let test_nested_dict ctx    let doc = [
-      "item", JSON.Dict [ "int", JSON.Int 5;
"string", JSON.String "foo"; ];
-      "last", JSON.Int 10;
+      "item", JSON.Dict [ "int", JSON.Int 5L;
"string", JSON.String "foo"; ];
+      "last", JSON.Int 10L;
     ] in
   assert_equal_string
     "{ \"item\": { \"int\": 5, \"string\":
\"foo\" }, \"last\": 10 }"
@@ -125,10 +127,10 @@ let test_nested_dict ctx  
 let test_nested_nested_dict ctx    let doc = [
-      "item", JSON.Dict [ "int", JSON.Int 5;
-        "item2", JSON.Dict [ "int", JSON.Int 0; ];
+      "item", JSON.Dict [ "int", JSON.Int 5L;
+        "item2", JSON.Dict [ "int", JSON.Int 0L; ];
       ];
-      "last", JSON.Int 10;
+      "last", JSON.Int 10L;
     ] in
   assert_equal_string
     "{ \"item\": { \"int\": 5, \"item2\": {
\"int\": 0 } }, \"last\": 10 }"
@@ -159,8 +161,8 @@ let test_qemu ctx    let doc = [
     "file.driver", JSON.String "https";
     "file.url", JSON.String "https://libguestfs.org";
-    "file.timeout", JSON.Int 60;
-    "file.readahead", JSON.Int (64 * 1024 * 1024);
+    "file.timeout", JSON.Int 60L;
+    "file.readahead", JSON.Int (64L *^ 1024L *^ 1024L);
   ] in
   assert_equal_string
     "{ \"file.driver\": \"https\",
\"file.url\": \"https://libguestfs.org\",
\"file.timeout\": 60, \"file.readahead\": 67108864 }"
@@ -176,7 +178,7 @@ let test_qemu ctx  
 let test_builder ctx    let doc = [
-    "version", JSON.Int 1;
+    "version", JSON.Int 1L;
     "sources", JSON.List [
       JSON.Dict [
         "uri", JSON.String "http://libguestfs.org/index";
@@ -187,7 +189,7 @@ let test_builder ctx          "os-version",
JSON.String "phony-debian";
         "full-name", JSON.String "Phony Debian";
         "arch", JSON.String "x86_64";
-        "size", JSON.Int64 536870912_L;
+        "size", JSON.Int 536870912_L;
         "notes", JSON.Dict [
           "C", JSON.String "Phony Debian look-alike used for
testing.";
         ];
@@ -197,7 +199,7 @@ let test_builder ctx          "os-version",
JSON.String "phony-fedora";
         "full-name", JSON.String "Phony Fedora";
         "arch", JSON.String "x86_64";
-        "size", JSON.Int64 1073741824_L;
+        "size", JSON.Int 1073741824_L;
         "notes", JSON.Dict [
           "C", JSON.String "Phony Fedora look-alike used for
testing.";
         ];
diff --git a/v2v/input_libvirt_xen_ssh.ml b/v2v/input_libvirt_xen_ssh.ml
index 597957f92..c4b671490 100644
--- a/v2v/input_libvirt_xen_ssh.ml
+++ b/v2v/input_libvirt_xen_ssh.ml
@@ -83,7 +83,7 @@ object
           (* qemu will actually assert-fail if you send the port
            * number as a string ...
            *)
-          | i -> ("file.port", JSON.Int i) :: json_params in
+          | i -> ("file.port", JSON.Int (Int64.of_int i)) ::
json_params in
 
         let json_params            match parsed_uri.uri_user with
diff --git a/v2v/input_ova.ml b/v2v/input_ova.ml
index bfd72dee4..d5c9f1203 100644
--- a/v2v/input_ova.ml
+++ b/v2v/input_ova.ml
@@ -161,8 +161,8 @@ class input_ova ova = object
            let doc = [
                "file", JSON.Dict [
                            "driver", JSON.String "raw";
-                           "offset", JSON.Int64 offset;
-                           "size", JSON.Int64 size;
+                           "offset", JSON.Int offset;
+                           "size", JSON.Int size;
                            "file", JSON.Dict [
                                        "driver", JSON.String
"file";
                                        "filename", JSON.String
tar_path]
diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml
index 12fb975d6..30649b33b 100644
--- a/v2v/input_vmx.ml
+++ b/v2v/input_vmx.ml
@@ -245,7 +245,7 @@ and qemu_uri_of_filename vmx_source filename         match
port_of_uri uri with
        | None -> json_params
        | Some port ->
-          ("file.port", JSON.Int port) :: json_params in
+          ("file.port", JSON.Int (Int64.of_int port)) :: json_params
in
 
      "json:" ^ JSON.string_of_doc json_params, format
 
diff --git a/v2v/output_rhv_upload.ml b/v2v/output_rhv_upload.ml
index 966323cae..f03e1ede3 100644
--- a/v2v/output_rhv_upload.ml
+++ b/v2v/output_rhv_upload.ml
@@ -276,7 +276,7 @@ object
 
         let disk_size = ov.ov_virtual_size in
         let json_params -          ("disk_size", JSON.Int64
disk_size) :: json_params in
+          ("disk_size", JSON.Int disk_size) :: json_params in
 
         (* Ask the plugin to write the disk ID to a special file. *)
         let diskid_file = diskid_file_of_id id in
diff --git a/v2v/parse_libvirt_xml.ml b/v2v/parse_libvirt_xml.ml
index dac99511c..255c935a6 100644
--- a/v2v/parse_libvirt_xml.ml
+++ b/v2v/parse_libvirt_xml.ml
@@ -63,8 +63,8 @@ let create_curl_qemu_uri driver host port path    let
json_params = [
     "file.driver", JSON.String driver;  (* "http" or
"https" *)
     "file.url", JSON.String url;
-    "file.timeout", JSON.Int 2000;
-    "file.readahead", JSON.Int (1024 * 1024);
+    "file.timeout", JSON.Int 2000_L;
+    "file.readahead", JSON.Int (1024_L *^ 1024_L);
     (* "file.sslverify", JSON.String "off"; XXX *)
   ] in
 
diff --git a/v2v/utils.ml b/v2v/utils.ml
index 67e2028f3..63ef91c51 100644
--- a/v2v/utils.ml
+++ b/v2v/utils.ml
@@ -114,8 +114,8 @@ let qemu_img_supports_offset_and_size ()    let json = [
       "file", JSON.Dict [
         "driver", JSON.String "raw";
-        "offset", JSON.Int 512;
-        "size", JSON.Int 512;
+        "offset", JSON.Int 512_L;
+        "size", JSON.Int 512_L;
         "file", JSON.Dict [
           "filename", JSON.String tmp
         ]
diff --git a/v2v/vCenter.ml b/v2v/vCenter.ml
index e97d25ce1..cf124f067 100644
--- a/v2v/vCenter.ml
+++ b/v2v/vCenter.ml
@@ -78,14 +78,14 @@ let rec map_source ?readahead ?password_file dcPath uri
server path        "file.driver", JSON.String "https";
       "file.url", JSON.String https_url;
       (* https://bugzilla.redhat.com/show_bug.cgi?id=1146007#c10 *)
-      "file.timeout", JSON.Int 2000;
+      "file.timeout", JSON.Int 2000_L;
     ] in
 
     let json_params        match readahead with
       | None -> json_params
       | Some readahead ->
-         ("file.readahead", JSON.Int readahead) :: json_params in
+         ("file.readahead", JSON.Int (Int64.of_int readahead)) ::
json_params in
 
     let json_params        if sslverify then json_params
-- 
2.18.0

Richard W.M. Jones

2018-Aug-20 16:02 UTC

head link

[Libguestfs] [PATCH 4/4] mltools: JSON: unify JSON_parser type with JSON.json_t.

---
 builder/simplestreams_parser.ml               |  9 +-
 .../test-virt-builder-list-simplestreams.sh   | 64 ++++++-------
 builder/utils.mli                             |  2 +-
 common/mltools/JSON_parser-c.c                | 96 ++++++++++++-------
 common/mltools/JSON_parser.ml                 | 29 ++----
 common/mltools/JSON_parser.mli                | 25 ++---
 common/mltools/JSON_parser_tests.ml           | 77 +++++++--------
 7 files changed, 156 insertions(+), 146 deletions(-)

diff --git a/builder/simplestreams_parser.ml b/builder/simplestreams_parser.ml
index fa5b887ac..ccbfdff67 100644
--- a/builder/simplestreams_parser.ml
+++ b/builder/simplestreams_parser.ml
@@ -59,7 +59,7 @@ let get_index ~downloader ~sigchecker { Sources.uri; proxy }  
error (f_"%s is not a Simple Streams (index) v1.0 JSON file (format:
%s)")
         uri format;
 
-    let index = Array.to_list (object_get_object "index" tree) in
+    let index = object_get_object "index" tree in
     List.filter_map (
       fun (_, desc) ->
         let format = object_get_string "format" desc in
@@ -78,13 +78,12 @@ let get_index ~downloader ~sigchecker { Sources.uri; proxy }
error (f_"%s is not a Simple Streams (products) v1.0 JSON file (format:
%s)")
         uri format;
 
-    let products_node = object_get_object "products" tree in
+    let products = object_get_object "products" tree in
 
-    let products = Array.to_list products_node in
     List.filter_map (
       fun (prod, prod_desc) ->
         let arch = Index.Arch (object_get_string "arch" prod_desc) in
-        let prods = Array.to_list (object_get_object "versions"
prod_desc) in
+        let prods = object_get_object "versions" prod_desc in
         let prods = List.filter_map (
           fun (rel, rel_desc) ->
             let pubname = objects_get_string "pubname" [rel_desc;
prod_desc] in
@@ -106,7 +105,7 @@ let get_index ~downloader ~sigchecker { Sources.uri; proxy }
* the ones related to checksums, explicitly filter
                    * the supported checksums.
                    *)
-                  | ("sha256"|"sha512" as t,
JSON_parser_string c) ->
+                  | ("sha256"|"sha512" as t, JSON.String c)
->
                     Some (Checksums.of_string t c)
                   | _ -> None
                 ) disk_item in
diff --git a/builder/test-virt-builder-list-simplestreams.sh
b/builder/test-virt-builder-list-simplestreams.sh
index 29fbfacce..3158066b1 100755
--- a/builder/test-virt-builder-list-simplestreams.sh
+++ b/builder/test-virt-builder-list-simplestreams.sh
@@ -26,9 +26,9 @@ export
XDG_CONFIG_DIRS="$abs_builddir/test-simplestreams"
 
 short_list=$($VG virt-builder --no-check-signature --no-cache --list)
 
-if [ "$short_list" != "net.cirros-cloud:standard:0.3:i386 i386  
cirros-0.3.4-i386
+if [ "$short_list" != "net.cirros-cloud:standard:0.3:powerpc
powerpc    cirros-0.3.4-powerpc
 net.cirros-cloud:standard:0.3:x86_64 x86_64     cirros-0.3.4-x86_64
-net.cirros-cloud:standard:0.3:powerpc powerpc    cirros-0.3.4-powerpc" ];
then
+net.cirros-cloud:standard:0.3:i386 i386       cirros-0.3.4-i386" ]; then
     echo "$0: unexpected --list output:"
     echo "$short_list"
     exit 1
@@ -38,11 +38,11 @@ long_list=$(virt-builder --no-check-signature --no-cache
--list --long)
 
 if [ "$long_list" != "Source URI:
file://$abs_builddir/test-simplestreams
 
-os-version:              net.cirros-cloud:standard:0.3:i386
-Full name:               cirros-0.3.4-i386
-Architecture:            i386
-Minimum/default size:    11.9M
-Aliases:                 cirros-0.3.4-i386
+os-version:              net.cirros-cloud:standard:0.3:powerpc
+Full name:               cirros-0.3.4-powerpc
+Architecture:            powerpc
+Minimum/default size:    16.4M
+Aliases:                 cirros-0.3.4-powerpc
 
 os-version:              net.cirros-cloud:standard:0.3:x86_64
 Full name:               cirros-0.3.4-x86_64
@@ -50,11 +50,11 @@ Architecture:            x86_64
 Minimum/default size:    12.7M
 Aliases:                 cirros-0.3.4-x86_64
 
-os-version:              net.cirros-cloud:standard:0.3:powerpc
-Full name:               cirros-0.3.4-powerpc
-Architecture:            powerpc
-Minimum/default size:    16.4M
-Aliases:                 cirros-0.3.4-powerpc" ]; then
+os-version:              net.cirros-cloud:standard:0.3:i386
+Full name:               cirros-0.3.4-i386
+Architecture:            i386
+Minimum/default size:    11.9M
+Aliases:                 cirros-0.3.4-i386" ]; then
     echo "$0: unexpected --list --long output:"
     echo "$long_list"
     exit 1
@@ -70,26 +70,6 @@ if [ "$json_list" != "{
     }
   ],
   \"templates\": [
-    {
-      \"os-version\":
\"net.cirros-cloud:standard:0.3:i386\",
-      \"full-name\": \"cirros-0.3.4-i386\",
-      \"arch\": \"i386\",
-      \"size\": 12506112,
-      \"aliases\": [
-        \"cirros-0.3.4-i386\"
-      ],
-      \"hidden\": false
-    },
-    {
-      \"os-version\":
\"net.cirros-cloud:standard:0.3:x86_64\",
-      \"full-name\": \"cirros-0.3.4-x86_64\",
-      \"arch\": \"x86_64\",
-      \"size\": 13287936,
-      \"aliases\": [
-        \"cirros-0.3.4-x86_64\"
-      ],
-      \"hidden\": false
-    },
     {
       \"os-version\":
\"net.cirros-cloud:standard:0.3:powerpc\",
       \"full-name\": \"cirros-0.3.4-powerpc\",
@@ -99,6 +79,26 @@ if [ "$json_list" != "{
         \"cirros-0.3.4-powerpc\"
       ],
       \"hidden\": false
+    },
+    {
+      \"os-version\":
\"net.cirros-cloud:standard:0.3:x86_64\",
+      \"full-name\": \"cirros-0.3.4-x86_64\",
+      \"arch\": \"x86_64\",
+      \"size\": 13287936,
+      \"aliases\": [
+        \"cirros-0.3.4-x86_64\"
+      ],
+      \"hidden\": false
+    },
+    {
+      \"os-version\":
\"net.cirros-cloud:standard:0.3:i386\",
+      \"full-name\": \"cirros-0.3.4-i386\",
+      \"arch\": \"i386\",
+      \"size\": 12506112,
+      \"aliases\": [
+        \"cirros-0.3.4-i386\"
+      ],
+      \"hidden\": false
     }
   ]
 }" ]; then
diff --git a/builder/utils.mli b/builder/utils.mli
index 5dde43a01..c7631636c 100644
--- a/builder/utils.mli
+++ b/builder/utils.mli
@@ -29,7 +29,7 @@ and revision  val string_of_revision : revision -> string
 (** Convert a {!revision} into a string. *)
 
-val get_image_infos : string -> JSON_parser.json_parser_val
+val get_image_infos : string -> JSON.json_t
 (** [get_image_infos path] Run qemu-img info on the image pointed at
     path as JSON tree. *)
 
diff --git a/common/mltools/JSON_parser-c.c b/common/mltools/JSON_parser-c.c
index 32432dc5b..dce9f6a15 100644
--- a/common/mltools/JSON_parser-c.c
+++ b/common/mltools/JSON_parser-c.c
@@ -28,7 +28,12 @@
 #include <stdio.h>
 #include <string.h>
 
-#define Val_none (Val_int (0))
+#define JSON_STRING_TAG 0
+#define JSON_INT_TAG    1
+#define JSON_FLOAT_TAG  2
+#define JSON_BOOL_TAG   3
+#define JSON_LIST_TAG   4
+#define JSON_DICT_TAG   5
 
 value virt_builder_json_parser_tree_parse (value stringv);
 
@@ -36,60 +41,87 @@ static value
 convert_json_t (json_t *val, int level)
 {
   CAMLparam0 ();
-  CAMLlocal4 (rv, lv, v, sv);
+  CAMLlocal5 (rv, v, tv, sv, consv);
 
   if (level > 20)
     caml_invalid_argument ("too many levels of object/array
nesting");
 
   if (json_is_object (val)) {
-    const size_t len = json_object_size (val);
-    size_t i;
     const char *key;
     json_t *jvalue;
-    rv = caml_alloc (1, 3);
-    lv = caml_alloc_tuple (len);
-    i = 0;
+
+    rv = caml_alloc (1, JSON_DICT_TAG);
+    v = Val_int (0);
+    /* This will create the OCaml list backwards, but JSON
+     * dictionaries are supposed to be unordered so that shouldn't
+     * matter, right?  Well except that for some consumers this does
+     * matter (eg. simplestreams which incorrectly uses a dict when it
+     * really should use an array).
+     */
     json_object_foreach (val, key, jvalue) {
-      v = caml_alloc_tuple (2);
+      tv = caml_alloc_tuple (2);
       sv = caml_copy_string (key);
-      Store_field (v, 0, sv);
+      Store_field (tv, 0, sv);
       sv = convert_json_t (jvalue, level + 1);
-      Store_field (v, 1, sv);
-      Store_field (lv, i, v);
-      ++i;
+      Store_field (tv, 1, sv);
+      consv = caml_alloc (2, 0);
+      Store_field (consv, 1, v);
+      Store_field (consv, 0, tv);
+      v = consv;
     }
-    Store_field (rv, 0, lv);
-  } else if (json_is_array (val)) {
+    Store_field (rv, 0, v);
+  }
+  else if (json_is_array (val)) {
     const size_t len = json_array_size (val);
     size_t i;
     json_t *jvalue;
-    rv = caml_alloc (1, 4);
-    lv = caml_alloc_tuple (len);
-    json_array_foreach (val, i, jvalue) {
-      v = convert_json_t (jvalue, level + 1);
-      Store_field (lv, i, v);
+
+    rv = caml_alloc (1, JSON_LIST_TAG);
+    v = Val_int (0);
+    for (i = 0; i < len; ++i) {
+      /* Note we have to create the OCaml list backwards. */
+      jvalue = json_array_get (val, len-i-1);
+      tv = convert_json_t (jvalue, level + 1);
+      consv = caml_alloc (2, 0);
+      Store_field (consv, 1, v);
+      Store_field (consv, 0, tv);
+      v = consv;
     }
-    Store_field (rv, 0, lv);
-  } else if (json_is_string (val)) {
-    rv = caml_alloc (1, 0);
+    Store_field (rv, 0, v);
+  }
+  else if (json_is_string (val)) {
+    rv = caml_alloc (1, JSON_STRING_TAG);
     v = caml_copy_string (json_string_value (val));
     Store_field (rv, 0, v);
-  } else if (json_is_real (val)) {
-    rv = caml_alloc (1, 2);
+  }
+  else if (json_is_real (val)) {
+    rv = caml_alloc (1, JSON_FLOAT_TAG);
     v = caml_copy_double (json_real_value (val));
     Store_field (rv, 0, v);
-  } else if (json_is_integer (val)) {
-    rv = caml_alloc (1, 1);
+  }
+  else if (json_is_integer (val)) {
+    rv = caml_alloc (1, JSON_INT_TAG);
     v = caml_copy_int64 (json_integer_value (val));
     Store_field (rv, 0, v);
-  } else if (json_is_true (val)) {
-    rv = caml_alloc (1, 5);
+  }
+  else if (json_is_true (val)) {
+    rv = caml_alloc (1, JSON_BOOL_TAG);
     Store_field (rv, 0, Val_true);
-  } else if (json_is_false (val)) {
-    rv = caml_alloc (1, 5);
+  }
+  else if (json_is_false (val)) {
+    rv = caml_alloc (1, JSON_BOOL_TAG);
     Store_field (rv, 0, Val_false);
-  } else
-    rv = Val_none;
+  }
+  else {
+    /* Previously we had a special JSON_parser_null value we could
+     * use here, making the returned type (sort of) an option.
+     * This is a best effort which is better than crashing /
+     * throwing an error.
+     */
+    rv = caml_alloc (1, JSON_STRING_TAG);
+    v = caml_copy_string ("");
+    Store_field (rv, 0, v);
+  }
 
   CAMLreturn (rv);
 }
diff --git a/common/mltools/JSON_parser.ml b/common/mltools/JSON_parser.ml
index a82127454..642e24d65 100644
--- a/common/mltools/JSON_parser.ml
+++ b/common/mltools/JSON_parser.ml
@@ -20,20 +20,11 @@ open Std_utils
 open Tools_utils
 open Common_gettext.Gettext
 
-type json_parser_val -| JSON_parser_null
-| JSON_parser_string of string
-| JSON_parser_number of int64
-| JSON_parser_double of float
-| JSON_parser_object of (string * json_parser_val) array
-| JSON_parser_array of json_parser_val array
-| JSON_parser_bool of bool
-
-external json_parser_tree_parse : string -> json_parser_val =
"virt_builder_json_parser_tree_parse"
+external json_parser_tree_parse : string -> JSON.json_t =
"virt_builder_json_parser_tree_parse"
 
 let object_find_optional key = function
-  | JSON_parser_object o ->
-    (match List.filter (fun (k, _) -> k = key) (Array.to_list o) with
+  | JSON.Dict fields ->
+    (match List.filter (fun (k, _) -> k = key) fields with
     | [(k, v)] -> Some v
     | [] -> None
     | _ -> error (f_"more than value for the key ‘%s’") key)
@@ -46,27 +37,27 @@ let object_find key yv  
 let object_get_string key yv    match object_find key yv with
-  | JSON_parser_string s -> s
+  | JSON.String s -> s
   | _ -> error (f_"the value for the key ‘%s’ is not a string")
key
 
 let object_find_object key yv    match object_find key yv with
-  | JSON_parser_object _ as o -> o
+  | JSON.Dict _ as o -> o
   | _ -> error (f_"the value for the key ‘%s’ is not an object")
key
 
 let object_find_objects fn = function
-  | JSON_parser_object o -> List.filter_map fn (Array.to_list o)
+  | JSON.Dict fields -> List.filter_map fn fields
   | _ -> error (f_"the value is not an object")
 
 let object_get_object key yv    match object_find_object key yv with
-  | JSON_parser_object o -> o
+  | JSON.Dict fields -> fields
   | _ -> assert false (* object_find_object already errors out. *)
 
 let object_get_number key yv    match object_find key yv with
-  | JSON_parser_number n -> n
-  | JSON_parser_double d -> Int64.of_float d
+  | JSON.Int n -> n
+  | JSON.Float f -> Int64.of_float f
   | _ -> error (f_"the value for the key ‘%s’ is not an integer")
key
 
 let objects_get_string key yvs @@ -74,7 +65,7 @@ let objects_get_string key yvs
| [] -> None
     | x :: xs ->
       (match object_find_optional key x with
-      | Some (JSON_parser_string s) -> Some s
+      | Some (JSON.String s) -> Some s
       | Some _ -> error (f_"the value for key ‘%s’ is not a string as
expected") key
       | None -> loop xs
       )
diff --git a/common/mltools/JSON_parser.mli b/common/mltools/JSON_parser.mli
index f505953f2..5ad0ef017 100644
--- a/common/mltools/JSON_parser.mli
+++ b/common/mltools/JSON_parser.mli
@@ -16,43 +16,34 @@
  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  *)
 
-type json_parser_val -| JSON_parser_null
-| JSON_parser_string of string
-| JSON_parser_number of int64
-| JSON_parser_double of float
-| JSON_parser_object of (string * json_parser_val) array
-| JSON_parser_array of json_parser_val array
-| JSON_parser_bool of bool
-
-val json_parser_tree_parse : string -> json_parser_val
+val json_parser_tree_parse : string -> JSON.json_t
 (** Parse the JSON string. *)
 
-val object_get_string : string -> json_parser_val -> string
+val object_get_string : string -> JSON.json_t -> string
 (** [object_get_string key yv] gets the value of the [key] field as a string
     in the [yv] structure *)
 
-val object_find_object : string -> json_parser_val -> json_parser_val
+val object_find_object : string -> JSON.json_t -> JSON.json_t
 (** [object_get_object key yv] gets the value of the [key] field as a JSON
     value in the [yv] structure.
 
     Mind the returned type is different from [object_get_object] *)
 
-val object_get_object : string -> json_parser_val -> (string *
json_parser_val) array
+val object_get_object : string -> JSON.json_t -> (string * JSON.json_t)
list
 (** [object_get_object key yv] gets the value of the [key] field as a JSON
     object in the [yv] structure *)
 
-val object_get_number : string -> json_parser_val -> int64
+val object_get_number : string -> JSON.json_t -> int64
 (** [object_get_number key yv] gets the value of the [key] field as an
     integer in the [yv] structure *)
 
-val objects_get_string : string -> json_parser_val list -> string
+val objects_get_string : string -> JSON.json_t list -> string
 (** [objects_get_string key yvs] gets the value of the [key] field as a string
-    in an [yvs] list of json_parser_val structure.
+    in an [yvs] list of JSON.json_t structure.
 
     The key may not be found at all in the list, in which case an error
     is raised *)
 
-val object_find_objects : ((string * json_parser_val) -> 'a option)
-> json_parser_val -> 'a list
+val object_find_objects : ((string * JSON.json_t) -> 'a option) ->
JSON.json_t -> 'a list
 (** [object_find_objects fn obj] returns all the JSON objects matching the [fn]
     function in [obj] list. *)
diff --git a/common/mltools/JSON_parser_tests.ml
b/common/mltools/JSON_parser_tests.ml
index 42045122d..e7e3112b5 100644
--- a/common/mltools/JSON_parser_tests.ml
+++ b/common/mltools/JSON_parser_tests.ml
@@ -27,16 +27,15 @@ let assert_equal_int = assert_equal ~printer:(fun x ->
string_of_int x)
 let assert_equal_int64 = assert_equal ~printer:(fun x -> Int64.to_string x)
 let assert_equal_bool = assert_equal ~printer:(fun x -> string_of_bool x)
 
-let string_of_json_parser_val_type = function
-  | JSON_parser_null -> "null"
-  | JSON_parser_string _ -> "string"
-  | JSON_parser_number _ -> "number"
-  | JSON_parser_double _ -> "float"
-  | JSON_parser_object _ -> "object"
-  | JSON_parser_array _ -> "array"
-  | JSON_parser_bool _ -> "bool"
+let string_of_json_t = function
+  | JSON.String _ -> "string"
+  | JSON.Int _ -> "int"
+  | JSON.Float _ -> "float"
+  | JSON.Dict _ -> "dict"
+  | JSON.List _ -> "list"
+  | JSON.Bool _ -> "bool"
 let type_mismatch_string exp value -  Printf.sprintf "value is not %s but
%s" exp (string_of_json_parser_val_type value)
+  Printf.sprintf "value is not %s but %s" exp (string_of_json_t
value)
 
 let assert_raises_invalid_argument str    (* Replace the Invalid_argument
string with a fixed one, just to check
@@ -54,28 +53,28 @@ let assert_raises_nested str  let assert_is_object value   
assert_bool
     (type_mismatch_string "object" value)
-    (match value with | JSON_parser_object _ -> true | _ -> false)
+    (match value with | JSON.Dict _ -> true | _ -> false)
 let assert_is_string exp = function
-  | JSON_parser_string s -> assert_equal_string exp s
+  | JSON.String s -> assert_equal_string exp s
   | _ as v -> assert_failure (type_mismatch_string "string" v)
 let assert_is_number exp = function
-  | JSON_parser_number n -> assert_equal_int64 exp n
-  | JSON_parser_double d -> assert_equal_int64 exp (Int64.of_float d)
+  | JSON.Int i -> assert_equal_int64 exp i
+  | JSON.Float f -> assert_equal_int64 exp (Int64.of_float f)
   | _ as v -> assert_failure (type_mismatch_string "number/double"
v)
 let assert_is_array value    assert_bool
-    (type_mismatch_string "array" value)
-    (match value with | JSON_parser_array _ -> true | _ -> false)
+    (type_mismatch_string "list" value)
+    (match value with | JSON.List _ -> true | _ -> false)
 let assert_is_bool exp = function
-  | JSON_parser_bool b -> assert_equal_bool exp b
+  | JSON.Bool b -> assert_equal_bool exp b
   | _ as v -> assert_failure (type_mismatch_string "bool" v)
 
-let get_object_list = function
-  | JSON_parser_object x -> x
-  | _ as v -> assert_failure (type_mismatch_string "object" v)
-let get_array = function
-  | JSON_parser_array x -> x
-  | _ as v -> assert_failure (type_mismatch_string "array" v)
+let get_dict = function
+  | JSON.Dict x -> x
+  | _ as v -> assert_failure (type_mismatch_string "dict" v)
+let get_list = function
+  | JSON.List x -> x
+  | _ as v -> assert_failure (type_mismatch_string "list" v)
 
 
 let test_tree_parse_invalid ctx @@ -101,28 +100,26 @@ let test_tree_parse_basic
ctx
 let test_tree_parse_inspect ctx    let value = json_parser_tree_parse
"{\"foo\":5}" in
-  let l = get_object_list value in
-  assert_equal_int 1 (Array.length l);
-  assert_equal_string "foo" (fst (l.(0)));
-  assert_is_number 5_L (snd (l.(0)));
+  let l = get_dict value in
+  assert_equal_int 1 (List.length l);
+  assert_equal_string "foo" (fst (List.hd l));
+  assert_is_number 5_L (snd (List.hd l));
 
   let value = json_parser_tree_parse "[\"foo\", true]" in
-  let a = get_array value in
-  assert_equal_int 2 (Array.length a);
-  assert_is_string "foo" (a.(0));
-  assert_is_bool true (a.(1));
+  let a = get_list value in
+  assert_equal_int 2 (List.length a);
+  assert_is_string "foo" (List.hd a);
+  assert_is_bool true (List.nth a 1);
 
   let value = json_parser_tree_parse "{\"foo\":[false, {}, 10],
\"second\":2}" in
-  let l = get_object_list value in
-  assert_equal_int 2 (Array.length l);
-  assert_equal_string "foo" (fst (l.(0)));
-  let a = get_array (snd (l.(0))) in
-  assert_equal_int 3 (Array.length a);
-  assert_is_bool false (a.(0));
-  assert_is_object (a.(1));
-  assert_is_number 10_L (a.(2));
-  assert_equal_string "second" (fst (l.(1)));
-  assert_is_number 2_L (snd (l.(1)))
+  let l = get_dict value in
+  assert_equal_int 2 (List.length l);
+  let a = get_list (List.assoc "foo" l) in
+  assert_equal_int 3 (List.length a);
+  assert_is_bool false (List.hd a);
+  assert_is_object (List.nth a 1);
+  assert_is_number 10_L (List.nth a 2);
+  assert_is_number 2_L (List.assoc "second" l)
 
 (* Suites declaration. *)
 let suite -- 
2.18.0

Pino Toscano

2018-Aug-22 11:20 UTC

head link

Re: [Libguestfs] [PATCH 4/4] mltools: JSON: unify JSON_parser type with JSON.json_t.

On Monday, 20 August 2018 18:02:06 CEST Richard W.M. Jones
wrote:> -  } else
> -    rv = Val_none;
> +  }
> +  else {
> +    /* Previously we had a special JSON_parser_null value we could
> +     * use here, making the returned type (sort of) an option.
> +     * This is a best effort which is better than crashing /
> +     * throwing an error.
> +     */
> +    rv = caml_alloc (1, JSON_STRING_TAG);
> +    v = caml_copy_string ("");
> +    Store_field (rv, 0, v);
> +  }
NACK, this is not correct.  null is a proper type of value in JSON, and
thus JSON.json_t must represent it as well.

This is even used in other parts, for example the check of backing
files of disks (see guestfs_impl_disk_has_backing_file).

-- 
Pino Toscano

Pino Toscano

2018-Aug-22 11:21 UTC

head link

Re: [Libguestfs] [PATCH 0/4] mltools: JSON unification

On Monday, 20 August 2018 18:02:02 CEST Richard W.M. Jones
wrote:> An evolution of:
> 
> https://www.redhat.com/archives/libguestfs/2018-August/msg00155.html
Patches #1, #2, and #3 LGTM.

Thanks,
-- 
Pino Toscano

Apparently Analagous Threads

Search for more possibly parallel threads

Libguestfs - Aug 2018 - [PATCH 0/4] mltools: JSON unification

[Libguestfs] [PATCH 0/4] mltools: JSON unification

[Libguestfs] [PATCH 1/4] mltools: JSON: Rename Yajl module as JSON_parser and move to common/mltools.

[Libguestfs] [PATCH 2/4] mltools: JSON: Implement JSON.Float.

[Libguestfs] [PATCH 3/4] mltools: JSON: combine JSON.Int and JSON.Int64 into a single variant.

[Libguestfs] [PATCH 4/4] mltools: JSON: unify JSON_parser type with JSON.json_t.

Re: [Libguestfs] [PATCH 4/4] mltools: JSON: unify JSON_parser type with JSON.json_t.

Re: [Libguestfs] [PATCH 0/4] mltools: JSON unification

Apparently Analagous Threads