Cédric Bosdonnat
2017-Feb-10 15:05 UTC
[Libguestfs] [PATCH v3 00/10] Introducing virt-builder-repository
Hi guys, Here is a v3 of the series, including changes to answer Richard's comments. Cédric Bosdonnat (10): mllib: factorize code to add Checksum.get_checksum function Move xml and xpath_helpers OCAML code to mllib mllib: add Xml.parse_file helper lib/osinfo.c: Extract xml processing into a callback lib: extract osinfo DB traversing API mllib: ocaml wrapper for lib/osinfo builder: rename docs test script builder: add Index.write_entry function builder: add a template parameter to get_index Add a virt-builder-repository tool .gitignore | 3 + builder/Makefile.am | 86 +++- builder/builder.ml | 2 +- builder/index.mli | 3 + builder/index_parser.ml | 72 ++- builder/index_parser.mli | 8 +- builder/repository_main.ml | 487 +++++++++++++++++++++ .../{test-virt-builder-docs.sh => test-docs.sh} | 3 + builder/virt-builder-repository.pod | 183 ++++++++ docs/C_SOURCE_FILES | 2 +- lib/Makefile.am | 2 + lib/osinfo-iso.c | 464 ++++++++++++++++++++ lib/osinfo.c | 479 ++------------------ lib/osinfo.h | 27 ++ mllib/Makefile.am | 18 +- mllib/checksums.ml | 25 +- mllib/checksums.mli | 9 + mllib/osinfo-c.c | 100 +++++ mllib/osinfo.ml | 26 ++ mllib/osinfo.mli | 31 ++ {v2v => mllib}/xml-c.c | 47 +- {v2v => mllib}/xml.ml | 49 ++- {v2v => mllib}/xml.mli | 3 + {v2v => mllib}/xpath_helpers.ml | 0 {v2v => mllib}/xpath_helpers.mli | 0 v2v/Makefile.am | 17 +- v2v/test-harness/Makefile.am | 3 +- v2v/test-harness/dummy.c | 2 + 28 files changed, 1631 insertions(+), 520 deletions(-) create mode 100644 builder/repository_main.ml rename builder/{test-virt-builder-docs.sh => test-docs.sh} (89%) create mode 100644 builder/virt-builder-repository.pod create mode 100644 lib/osinfo-iso.c create mode 100644 lib/osinfo.h create mode 100644 mllib/osinfo-c.c create mode 100644 mllib/osinfo.ml create mode 100644 mllib/osinfo.mli rename {v2v => mllib}/xml-c.c (90%) rename {v2v => mllib}/xml.ml (79%) rename {v2v => mllib}/xml.mli (97%) rename {v2v => mllib}/xpath_helpers.ml (100%) rename {v2v => mllib}/xpath_helpers.mli (100%) create mode 100644 v2v/test-harness/dummy.c -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:05 UTC
[Libguestfs] [PATCH v3 01/10] mllib: factorize code to add Checksum.get_checksum function
Getting checksum involves the same code than verifying them. Create a get_checksum function and use it in verify_checksum. --- mllib/checksums.ml | 25 +++++++++++++++---------- mllib/checksums.mli | 9 +++++++++ 2 files changed, 24 insertions(+), 10 deletions(-) diff --git a/mllib/checksums.ml b/mllib/checksums.ml index 1009e131c..000214703 100644 --- a/mllib/checksums.ml +++ b/mllib/checksums.ml @@ -45,14 +45,14 @@ let of_string csum_type csum_value | "sha512" -> SHA512 csum_value | _ -> invalid_arg csum_type -let verify_checksum csum ?tar filename - let prog, csum_ref - match csum with - | SHA1 c -> "sha1sum", c - | SHA256 c -> "sha256sum", c - | SHA512 c -> "sha512sum", c +let compute_checksum csum_type ?tar filename + let prog + match csum_type with + | "sha1" -> "sha1sum" + | "sha256" -> "sha256sum" + | "sha512" -> "sha512sum" + | _ -> error (f_"unhandled checksum type '%s'") csum_type in - let cmd match tar with | None -> @@ -66,9 +66,14 @@ let verify_checksum csum ?tar filename | [] -> error (f_"%s did not return any output") prog | line :: _ -> - let csum_actual = fst (String.split " " line) in - if csum_ref <> csum_actual then - raise (Mismatched_checksum (csum, csum_actual)) + let csum_str = fst (String.split " " line) in + of_string csum_type csum_str + +let verify_checksum csum ?tar filename + let csum_type = string_of_csum_t csum in + let csum_actual = compute_checksum csum_type ?tar filename in + if csum <> csum_actual then + raise (Mismatched_checksum (csum, (string_of_csum csum_actual))) let verify_checksums checksums filename List.iter (fun c -> verify_checksum c filename) checksums diff --git a/mllib/checksums.mli b/mllib/checksums.mli index 9f7041b00..92336a18b 100644 --- a/mllib/checksums.mli +++ b/mllib/checksums.mli @@ -43,3 +43,12 @@ val string_of_csum_t : csum_t -> string val string_of_csum : csum_t -> string (** Return a string representation of the checksum value. *) + +val compute_checksum : string -> ?tar:string -> string -> csum_t +(** [compute_checksum type filename] Computes the checksum of the file. + + The [type] is one the possible results of the [string_of_csum_t] + function. + + When optional [tar] is used it is path to uncompressed tar archive + and the [filename] is a path in the tar archive. *) -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:05 UTC
[Libguestfs] [PATCH v3 02/10] Move xml and xpath_helpers OCAML code to mllib
To allow other pieces of code to process XML files easily, move the xml.ml* and xpath_helpers.ml* from v2v to mllib. --- docs/C_SOURCE_FILES | 2 +- mllib/Makefile.am | 9 ++++++-- {v2v => mllib}/xml-c.c | 47 ++++++++++++++++++++-------------------- {v2v => mllib}/xml.ml | 45 +++++++++++++++++++------------------- {v2v => mllib}/xml.mli | 0 {v2v => mllib}/xpath_helpers.ml | 0 {v2v => mllib}/xpath_helpers.mli | 0 v2v/Makefile.am | 17 +++------------ v2v/test-harness/Makefile.am | 3 +-- v2v/test-harness/dummy.c | 2 ++ 10 files changed, 61 insertions(+), 64 deletions(-) rename {v2v => mllib}/xml-c.c (90%) rename {v2v => mllib}/xml.ml (80%) rename {v2v => mllib}/xml.mli (100%) rename {v2v => mllib}/xpath_helpers.ml (100%) rename {v2v => mllib}/xpath_helpers.mli (100%) create mode 100644 v2v/test-harness/dummy.c diff --git a/docs/C_SOURCE_FILES b/docs/C_SOURCE_FILES index 26b32da55..57c0b6084 100644 --- a/docs/C_SOURCE_FILES +++ b/docs/C_SOURCE_FILES @@ -335,6 +335,7 @@ mllib/progress-c.c mllib/statvfs-c.c mllib/uri-c.c mllib/visit-c.c +mllib/xml-c.c ocaml/guestfs-c-actions.c ocaml/guestfs-c-errnos.c ocaml/guestfs-c.c @@ -393,4 +394,3 @@ utils/qemu-boot/qemu-boot.c utils/qemu-speed-test/qemu-speed-test.c v2v/domainxml-c.c v2v/utils-c.c -v2v/xml-c.c diff --git a/mllib/Makefile.am b/mllib/Makefile.am index 78fdf0211..aa5472ade 100644 --- a/mllib/Makefile.am +++ b/mllib/Makefile.am @@ -28,6 +28,8 @@ EXTRA_DIST = \ test-getopt.sh SOURCES_MLI = \ + xml.mli \ + xpath_helpers.mli \ checksums.mli \ common_utils.mli \ curl.mli \ @@ -67,7 +69,9 @@ SOURCES_ML = \ JSON.ml \ curl.ml \ exit.ml \ - checksums.ml + checksums.ml \ + xml.ml \ + xpath_helpers.ml SOURCES_C = \ ../common/visit/visit.c \ @@ -85,7 +89,8 @@ SOURCES_C = \ progress-c.c \ statvfs-c.c \ uri-c.c \ - visit-c.c + visit-c.c \ + xml-c.c if HAVE_OCAML diff --git a/v2v/xml-c.c b/mllib/xml-c.c similarity index 90% rename from v2v/xml-c.c rename to mllib/xml-c.c index 0fcdbedec..d3e893076 100644 --- a/v2v/xml-c.c +++ b/mllib/xml-c.c @@ -1,5 +1,6 @@ -/* virt-v2v +/* Bindings for libxml2 * Copyright (C) 2009-2017 Red Hat Inc. + * Copyright (C) 2017 SUSE Inc. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -52,7 +53,7 @@ static struct custom_operations docptr_custom_operations = { }; value -v2v_xml_free_docptr (value docv) +mllib_xml_free_docptr (value docv) { CAMLparam1 (docv); xmlDocPtr doc = docptr_val (docv); @@ -74,7 +75,7 @@ static struct custom_operations xpathctxptr_custom_operations = { }; value -v2v_xml_free_xpathctxptr (value xpathctxv) +mllib_xml_free_xpathctxptr (value xpathctxv) { CAMLparam1 (xpathctxv); xmlXPathContextPtr xpathctx = xpathctxptr_val (xpathctxv); @@ -96,7 +97,7 @@ static struct custom_operations xpathobjptr_custom_operations = { }; value -v2v_xml_free_xpathobjptr (value xpathobjv) +mllib_xml_free_xpathobjptr (value xpathobjv) { CAMLparam1 (xpathobjv); xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv); @@ -106,7 +107,7 @@ v2v_xml_free_xpathobjptr (value xpathobjv) } value -v2v_xml_parse_memory (value xmlv) +mllib_xml_parse_memory (value xmlv) { CAMLparam1 (xmlv); CAMLlocal1 (docv); @@ -128,7 +129,7 @@ v2v_xml_parse_memory (value xmlv) } value -v2v_xml_copy_doc (value docv, value recursivev) +mllib_xml_copy_doc (value docv, value recursivev) { CAMLparam2 (docv, recursivev); CAMLlocal1 (copyv); @@ -147,7 +148,7 @@ v2v_xml_copy_doc (value docv, value recursivev) } value -v2v_xml_to_string (value docv, value formatv) +mllib_xml_to_string (value docv, value formatv) { CAMLparam2 (docv, formatv); CAMLlocal1 (strv); @@ -166,7 +167,7 @@ v2v_xml_to_string (value docv, value formatv) } value -v2v_xml_xpath_new_context (value docv) +mllib_xml_xpath_new_context (value docv) { CAMLparam1 (docv); CAMLlocal1 (xpathctxv); @@ -186,7 +187,7 @@ v2v_xml_xpath_new_context (value docv) } value -v2v_xml_xpathctxptr_register_ns (value xpathctxv, value prefix, value uri) +mllib_xml_xpathctxptr_register_ns (value xpathctxv, value prefix, value uri) { CAMLparam3 (xpathctxv, prefix, uri); xmlXPathContextPtr xpathctx; @@ -203,7 +204,7 @@ v2v_xml_xpathctxptr_register_ns (value xpathctxv, value prefix, value uri) } value -v2v_xml_xpathctxptr_eval_expression (value xpathctxv, value exprv) +mllib_xml_xpathctxptr_eval_expression (value xpathctxv, value exprv) { CAMLparam2 (xpathctxv, exprv); CAMLlocal1 (xpathobjv); @@ -223,7 +224,7 @@ v2v_xml_xpathctxptr_eval_expression (value xpathctxv, value exprv) } value -v2v_xml_xpathobjptr_nr_nodes (value xpathobjv) +mllib_xml_xpathobjptr_nr_nodes (value xpathobjv) { CAMLparam1 (xpathobjv); xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv); @@ -235,7 +236,7 @@ v2v_xml_xpathobjptr_nr_nodes (value xpathobjv) } value -v2v_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv) +mllib_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv) { CAMLparam2 (xpathobjv, iv); xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv); @@ -256,7 +257,7 @@ v2v_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv) } value -v2v_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev) +mllib_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev) { CAMLparam2 (xpathctxv, nodev); xmlXPathContextPtr xpathctx = xpathctxptr_val (xpathctxv); @@ -268,7 +269,7 @@ v2v_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev) } value -v2v_xml_nodeptr_name (value nodev) +mllib_xml_nodeptr_name (value nodev) { CAMLparam1 (nodev); xmlNodePtr node = (xmlNodePtr) nodev; @@ -284,7 +285,7 @@ v2v_xml_nodeptr_name (value nodev) } value -v2v_xml_nodeptr_as_string (value docv, value nodev) +mllib_xml_nodeptr_as_string (value docv, value nodev) { CAMLparam2 (docv, nodev); CAMLlocal1 (strv); @@ -316,7 +317,7 @@ v2v_xml_nodeptr_as_string (value docv, value nodev) } value -v2v_xml_nodeptr_set_content (value nodev, value contentv) +mllib_xml_nodeptr_set_content (value nodev, value contentv) { CAMLparam2 (nodev, contentv); xmlNodePtr node = (xmlNodePtr) nodev; @@ -327,7 +328,7 @@ v2v_xml_nodeptr_set_content (value nodev, value contentv) } value -v2v_xml_nodeptr_new_text_child (value nodev, value namev, value contentv) +mllib_xml_nodeptr_new_text_child (value nodev, value namev, value contentv) { CAMLparam3 (nodev, namev, contentv); xmlNodePtr node = (xmlNodePtr) nodev; @@ -339,14 +340,14 @@ v2v_xml_nodeptr_new_text_child (value nodev, value namev, value contentv) if (new_node == NULL) caml_invalid_argument ("nodeptr_new_text_child: failed to create new node"); - /* See comment in v2v_xml_xpathobjptr_get_nodeptr about returning + /* See comment in mllib_xml_xpathobjptr_get_nodeptr about returning * named xmlNodePtr here. */ CAMLreturn ((value) new_node); } value -v2v_xml_nodeptr_set_prop (value nodev, value namev, value valv) +mllib_xml_nodeptr_set_prop (value nodev, value namev, value valv) { CAMLparam3 (nodev, namev, valv); xmlNodePtr node = (xmlNodePtr) nodev; @@ -360,7 +361,7 @@ v2v_xml_nodeptr_set_prop (value nodev, value namev, value valv) } value -v2v_xml_nodeptr_unset_prop (value nodev, value namev) +mllib_xml_nodeptr_unset_prop (value nodev, value namev) { CAMLparam2 (nodev, namev); xmlNodePtr node = (xmlNodePtr) nodev; @@ -372,7 +373,7 @@ v2v_xml_nodeptr_unset_prop (value nodev, value namev) } value -v2v_xml_nodeptr_unlink_node (value nodev) +mllib_xml_nodeptr_unlink_node (value nodev) { CAMLparam1 (nodev); xmlNodePtr node = (xmlNodePtr) nodev; @@ -384,7 +385,7 @@ v2v_xml_nodeptr_unlink_node (value nodev) } value -v2v_xml_doc_get_root_element (value docv) +mllib_xml_doc_get_root_element (value docv) { CAMLparam1 (docv); CAMLlocal1 (v); @@ -402,7 +403,7 @@ v2v_xml_doc_get_root_element (value docv) } value -v2v_xml_parse_uri (value strv) +mllib_xml_parse_uri (value strv) { CAMLparam1 (strv); CAMLlocal3 (rv, sv, ov); diff --git a/v2v/xml.ml b/mllib/xml.ml similarity index 80% rename from v2v/xml.ml rename to mllib/xml.ml index 7ed21cd30..e67245188 100644 --- a/v2v/xml.ml +++ b/mllib/xml.ml @@ -1,5 +1,6 @@ -(* virt-v2v +(* Bindings for libxml2 * Copyright (C) 2009-2017 Red Hat Inc. + * Copyright (C) 2017 SUSE Inc. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -56,87 +57,87 @@ type node = doc * nodeptr type xpathctx = doc * xpathctxptr type xpathobj = xpathctx * xpathobjptr -external free_docptr : docptr -> unit = "v2v_xml_free_docptr" -external free_xpathctxptr : xpathctxptr -> unit = "v2v_xml_free_xpathctxptr" -external free_xpathobjptr : xpathobjptr -> unit = "v2v_xml_free_xpathobjptr" +external free_docptr : docptr -> unit = "mllib_xml_free_docptr" +external free_xpathctxptr : xpathctxptr -> unit = "mllib_xml_free_xpathctxptr" +external free_xpathobjptr : xpathobjptr -> unit = "mllib_xml_free_xpathobjptr" -external _parse_memory : string -> docptr = "v2v_xml_parse_memory" +external _parse_memory : string -> docptr = "mllib_xml_parse_memory" let parse_memory xml let docptr = _parse_memory xml in Gc.finalise free_docptr docptr; docptr -external _copy_doc : docptr -> recursive:bool -> docptr = "v2v_xml_copy_doc" +external _copy_doc : docptr -> recursive:bool -> docptr = "mllib_xml_copy_doc" let copy_doc docptr ~recursive let copy = _copy_doc docptr ~recursive in Gc.finalise free_docptr copy; copy -external to_string : docptr -> format:bool -> string = "v2v_xml_to_string" +external to_string : docptr -> format:bool -> string = "mllib_xml_to_string" external _xpath_new_context : docptr -> xpathctxptr - = "v2v_xml_xpath_new_context" + = "mllib_xml_xpath_new_context" let xpath_new_context docptr let xpathctxptr = _xpath_new_context docptr in Gc.finalise free_xpathctxptr xpathctxptr; docptr, xpathctxptr external xpathctxptr_register_ns : xpathctxptr -> string -> string -> unit - = "v2v_xml_xpathctxptr_register_ns" + = "mllib_xml_xpathctxptr_register_ns" let xpath_register_ns (_, xpathctxptr) prefix uri xpathctxptr_register_ns xpathctxptr prefix uri external xpathctxptr_eval_expression : xpathctxptr -> string -> xpathobjptr - = "v2v_xml_xpathctxptr_eval_expression" + = "mllib_xml_xpathctxptr_eval_expression" let xpath_eval_expression ((_, xpathctxptr) as xpathctx) expr let xpathobjptr = xpathctxptr_eval_expression xpathctxptr expr in Gc.finalise free_xpathobjptr xpathobjptr; xpathctx, xpathobjptr external xpathobjptr_nr_nodes : xpathobjptr -> int - = "v2v_xml_xpathobjptr_nr_nodes" + = "mllib_xml_xpathobjptr_nr_nodes" let xpathobj_nr_nodes (_, xpathobjptr) xpathobjptr_nr_nodes xpathobjptr external xpathobjptr_get_nodeptr : xpathobjptr -> int -> nodeptr - = "v2v_xml_xpathobjptr_get_nodeptr" + = "mllib_xml_xpathobjptr_get_nodeptr" let xpathobj_node ((docptr, _), xpathobjptr) i docptr, xpathobjptr_get_nodeptr xpathobjptr i external xpathctxptr_set_nodeptr : xpathctxptr -> nodeptr -> unit - = "v2v_xml_xpathctx_set_nodeptr" + = "mllib_xml_xpathctx_set_nodeptr" let xpathctx_set_current_context (_, xpathctxptr) (_, nodeptr) xpathctxptr_set_nodeptr xpathctxptr nodeptr -external nodeptr_name : nodeptr -> string = "v2v_xml_nodeptr_name" +external nodeptr_name : nodeptr -> string = "mllib_xml_nodeptr_name" let node_name (_, nodeptr) = nodeptr_name nodeptr external nodeptr_as_string : docptr -> nodeptr -> string - = "v2v_xml_nodeptr_as_string" + = "mllib_xml_nodeptr_as_string" let node_as_string (docptr, nodeptr) = nodeptr_as_string docptr nodeptr external nodeptr_set_content : nodeptr -> string -> unit - = "v2v_xml_nodeptr_set_content" + = "mllib_xml_nodeptr_set_content" let node_set_content (_, nodeptr) = nodeptr_set_content nodeptr external nodeptr_new_text_child : nodeptr -> string -> string -> nodeptr - = "v2v_xml_nodeptr_new_text_child" + = "mllib_xml_nodeptr_new_text_child" let new_text_child (docptr, nodeptr) name content docptr, nodeptr_new_text_child nodeptr name content external nodeptr_set_prop : nodeptr -> string -> string -> unit - = "v2v_xml_nodeptr_set_prop" + = "mllib_xml_nodeptr_set_prop" let set_prop (_, nodeptr) = nodeptr_set_prop nodeptr external nodeptr_unset_prop : nodeptr -> string -> bool - = "v2v_xml_nodeptr_unset_prop" + = "mllib_xml_nodeptr_unset_prop" let unset_prop (_, nodeptr) = nodeptr_unset_prop nodeptr -external nodeptr_unlink_node : nodeptr -> unit = "v2v_xml_nodeptr_unlink_node" +external nodeptr_unlink_node : nodeptr -> unit = "mllib_xml_nodeptr_unlink_node" let unlink_node (_, nodeptr) = nodeptr_unlink_node nodeptr external _doc_get_root_element : docptr -> nodeptr option - = "v2v_xml_doc_get_root_element" + = "mllib_xml_doc_get_root_element" let doc_get_root_element docptr match _doc_get_root_element docptr with | None -> None @@ -154,4 +155,4 @@ type uri = { uri_query_raw : string option; } -external parse_uri : string -> uri = "v2v_xml_parse_uri" +external parse_uri : string -> uri = "mllib_xml_parse_uri" diff --git a/v2v/xml.mli b/mllib/xml.mli similarity index 100% rename from v2v/xml.mli rename to mllib/xml.mli diff --git a/v2v/xpath_helpers.ml b/mllib/xpath_helpers.ml similarity index 100% rename from v2v/xpath_helpers.ml rename to mllib/xpath_helpers.ml diff --git a/v2v/xpath_helpers.mli b/mllib/xpath_helpers.mli similarity index 100% rename from v2v/xpath_helpers.mli rename to mllib/xpath_helpers.mli diff --git a/v2v/Makefile.am b/v2v/Makefile.am index d5ca4cdb4..ffd514426 100644 --- a/v2v/Makefile.am +++ b/v2v/Makefile.am @@ -58,14 +58,10 @@ SOURCES_MLI = \ utils.mli \ vCenter.mli \ windows.mli \ - windows_virtio.mli \ - xml.mli \ - xpath_helpers.mli + windows_virtio.mli SOURCES_ML = \ types.ml \ - xml.ml \ - xpath_helpers.ml \ uefi.ml \ utils.ml \ name_from_disk.ml \ @@ -103,8 +99,7 @@ SOURCES_ML = \ SOURCES_C = \ domainxml-c.c \ - utils-c.c \ - xml-c.c + utils-c.c if HAVE_OCAML @@ -119,7 +114,6 @@ virt_v2v_CPPFLAGS = \ -I$(top_srcdir)/lib virt_v2v_CFLAGS = \ $(WARN_CFLAGS) $(WERROR_CFLAGS) \ - $(LIBXML2_CFLAGS) \ $(LIBVIRT_CFLAGS) BOBJECTS = \ @@ -168,8 +162,7 @@ virt_v2v_LINK = \ virt_v2v_copy_to_local_SOURCES = \ domainxml-c.c \ - utils-c.c \ - xml-c.c + utils-c.c virt_v2v_copy_to_local_CPPFLAGS = \ -I. \ -I$(top_builddir) \ @@ -178,12 +171,9 @@ virt_v2v_copy_to_local_CPPFLAGS = \ -I$(top_srcdir)/lib virt_v2v_copy_to_local_CFLAGS = \ $(WARN_CFLAGS) $(WERROR_CFLAGS) \ - $(LIBXML2_CFLAGS) \ $(LIBVIRT_CFLAGS) COPY_TO_LOCAL_BOBJECTS = \ - xml.cmo \ - xpath_helpers.cmo \ uefi.cmo \ utils.cmo \ vCenter.cmo \ @@ -403,7 +393,6 @@ endif v2v_unit_tests_BOBJECTS = \ types.cmo \ - xml.cmo \ uefi.cmo \ utils.cmo \ DOM.cmo \ diff --git a/v2v/test-harness/Makefile.am b/v2v/test-harness/Makefile.am index 1395e662c..8ce441222 100644 --- a/v2v/test-harness/Makefile.am +++ b/v2v/test-harness/Makefile.am @@ -48,7 +48,6 @@ OCAMLPACKAGES = \ OCAMLFLAGS = $(OCAML_FLAGS) $(OCAML_WARN_ERROR) BOBJECTS = \ - $(top_builddir)/v2v/xml.cmo \ $(SOURCES_ML:.ml=.cmo) \ $(libv2vth_a_OBJECTS) XOBJECTS = $(BOBJECTS:.cmo=.cmx) @@ -82,7 +81,7 @@ libv2vth_a_CFLAGS = \ -fPIC libv2vth_a_SOURCES = \ - ../xml-c.c + dummy.c v2v_test_harness.cmi: $(top_builddir)/ocaml/guestfs.cmi diff --git a/v2v/test-harness/dummy.c b/v2v/test-harness/dummy.c new file mode 100644 index 000000000..ebab6198c --- /dev/null +++ b/v2v/test-harness/dummy.c @@ -0,0 +1,2 @@ +/* Dummy source, to be used for OCaml-based tools with no C sources. */ +enum { foo = 1 }; -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:05 UTC
[Libguestfs] [PATCH v3 03/10] mllib: add Xml.parse_file helper
Provide a helper function rather than having callers read the file and then parse the string. --- mllib/xml.ml | 4 ++++ mllib/xml.mli | 3 +++ 2 files changed, 7 insertions(+) diff --git a/mllib/xml.ml b/mllib/xml.ml index e67245188..78e75b8f2 100644 --- a/mllib/xml.ml +++ b/mllib/xml.ml @@ -67,6 +67,10 @@ let parse_memory xml Gc.finalise free_docptr docptr; docptr +let parse_file file + let xml = Common_utils.read_whole_file file in + parse_memory xml + external _copy_doc : docptr -> recursive:bool -> docptr = "mllib_xml_copy_doc" let copy_doc docptr ~recursive let copy = _copy_doc docptr ~recursive in diff --git a/mllib/xml.mli b/mllib/xml.mli index 9b2bc5c18..92d4977a2 100644 --- a/mllib/xml.mli +++ b/mllib/xml.mli @@ -33,6 +33,9 @@ val parse_memory : string -> doc For security reasons it actually calls xmlReadMemory with the [XML_PARSE_NONET] option set. *) +val parse_file : string -> doc +(** [parse_file path] parses the file pointed by [path].*) + val copy_doc : doc -> recursive:bool -> doc (** http://xmlsoft.org/html/libxml-tree.html#xmlCopyDoc *) -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:05 UTC
[Libguestfs] [PATCH v3 04/10] lib/osinfo.c: Extract xml processing into a callback
In order to further reuse the osinfo database parsing in OCAML, this commit extracts the XML processing for the distro ISOs and places it into a newly created callback. This will later help other code to traverse the osinfo DB files and let them extract what they need from them. --- lib/osinfo.c | 111 ++++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 71 insertions(+), 40 deletions(-) diff --git a/lib/osinfo.c b/lib/osinfo.c index ea2a7659a..b77ff96b3 100644 --- a/lib/osinfo.c +++ b/lib/osinfo.c @@ -43,6 +43,7 @@ * * XXX Currently the database is not freed when the program exits / * library is unloaded, although we should probably do that. + * */ #include <config.h> @@ -52,6 +53,7 @@ #include <string.h> #include <unistd.h> #include <dirent.h> +#include <errno.h> #include <assert.h> #include <sys/types.h> #include <libintl.h> @@ -71,10 +73,41 @@ gl_lock_define_initialized (static, osinfo_db_lock); static ssize_t osinfo_db_size = 0; /* 0 = unread, -1 = error, >= 1 = #records */ static struct osinfo *osinfo_db = NULL; -static int read_osinfo_db (guestfs_h *g); +#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1) +typedef int (*read_osinfo_db_callback) (guestfs_h *g, const char *path, void *opaque); + +static int +read_osinfo_db (guestfs_h *g, + read_osinfo_db_callback callback, void *opaque); +static int read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *data); static void free_osinfo_db_entry (struct osinfo *); -#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1) +#ifndef GUESTFS_PRIVATE +void guestfs_int_debug (guestfs_h *g, const char *fs, ...) +{ + va_list args; + + va_start (args, fs); + vfprintf (stderr, fs, args); + va_end (args); +} + +void +guestfs_int_perrorf (guestfs_h *g, const char *fs, ...) +{ + va_list args; + CLEANUP_FREE char *msg = NULL; + int err; + + va_start (args, fs); + err = vasprintf (&msg, fs, args); + va_end (args); + + if (err < 0) return; + + perror(msg); +} +#endif /* GUESTFS_PRIVATE */ /* Given one or more fields from the header of a CD/DVD/ISO, look up * the media in the libosinfo database and return our best guess for @@ -87,14 +120,24 @@ static void free_osinfo_db_entry (struct osinfo *); */ int guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo, - const struct osinfo **osinfo_ret) + const struct osinfo **osinfo_ret) { size_t i; /* We only need to lock the database when reading it for the first time. */ gl_lock_lock (osinfo_db_lock); if (osinfo_db_size == 0) { - if (read_osinfo_db (g) == -1) { + if (read_osinfo_db (g, read_osinfo_db_xml, NULL) == -1) { + /* Fatal error: free any database entries which have been read, and + * mark the database as having a permanent error. + */ + if (osinfo_db_size > 0) { + for (i = 0; i < (size_t) osinfo_db_size; ++i) + free_osinfo_db_entry (&osinfo_db[i]); + } + free (osinfo_db); + osinfo_db = NULL; + osinfo_db_size = -1; gl_lock_unlock (osinfo_db_lock); return -1; } @@ -156,19 +199,18 @@ guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo, * Try to use the shared osinfo database layout (and location) first: * https://gitlab.com/libosinfo/libosinfo/blob/master/docs/database-layout.txt */ -static int read_osinfo_db_xml (guestfs_h *g, const char *filename); - -static int read_osinfo_db_flat (guestfs_h *g, const char *directory); -static int read_osinfo_db_three_levels (guestfs_h *g, const char *directory); -static int read_osinfo_db_directory (guestfs_h *g, const char *directory); +static int read_osinfo_db_flat (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque); +static int read_osinfo_db_three_levels (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque); +static int read_osinfo_db_directory (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque); static int -read_osinfo_db (guestfs_h *g) +read_osinfo_db (guestfs_h *g, + read_osinfo_db_callback callback, void *opaque) { int r; - size_t i; - - assert (osinfo_db_size == 0); /* (1) Try the shared osinfo directory, using either the * $OSINFO_SYSTEM_DIR envvar or its default value. @@ -181,59 +223,47 @@ read_osinfo_db (guestfs_h *g) if (path == NULL) path = "/usr/share/osinfo"; os_path = safe_asprintf (g, "%s/os", path); - r = read_osinfo_db_three_levels (g, os_path); + r = read_osinfo_db_three_levels (g, os_path, callback, opaque); } if (r == -1) - goto error; + return -1; else if (r == 1) return 0; /* (2) Try the libosinfo directory, using the newer three-directory * layout ($LIBOSINFO_DB_PATH / "os" / $group-ID / [file.xml]). */ - r = read_osinfo_db_three_levels (g, LIBOSINFO_DB_PATH "/os"); + r = read_osinfo_db_three_levels (g, LIBOSINFO_DB_PATH "/os", callback, opaque); if (r == -1) - goto error; + return -1; else if (r == 1) return 0; /* (3) Try the libosinfo directory, using the old flat directory * layout ($LIBOSINFO_DB_PATH / "oses" / [file.xml]). */ - r = read_osinfo_db_flat (g, LIBOSINFO_DB_PATH "/oses"); + r = read_osinfo_db_flat (g, LIBOSINFO_DB_PATH "/oses", callback, opaque); if (r == -1) - goto error; + return -1; else if (r == 1) return 0; /* Nothing found. */ return 0; - - error: - /* Fatal error: free any database entries which have been read, and - * mark the database as having a permanent error. - */ - if (osinfo_db_size > 0) { - for (i = 0; i < (size_t) osinfo_db_size; ++i) - free_osinfo_db_entry (&osinfo_db[i]); - } - free (osinfo_db); - osinfo_db = NULL; - osinfo_db_size = -1; - - return -1; } static int -read_osinfo_db_flat (guestfs_h *g, const char *directory) +read_osinfo_db_flat (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque) { debug (g, "osinfo: loading flat database from %s", directory); - return read_osinfo_db_directory (g, directory); + return read_osinfo_db_directory (g, directory, callback, opaque); } static int -read_osinfo_db_three_levels (guestfs_h *g, const char *directory) +read_osinfo_db_three_levels (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque) { DIR *dir; int r; @@ -259,7 +289,7 @@ read_osinfo_db_three_levels (guestfs_h *g, const char *directory) /* Iterate only on directories. */ if (stat (pathname, &sb) == 0 && S_ISDIR (sb.st_mode)) { - r = read_osinfo_db_directory (g, pathname); + r = read_osinfo_db_directory (g, pathname, callback, opaque); if (r == -1) goto error; } @@ -289,7 +319,8 @@ read_osinfo_db_three_levels (guestfs_h *g, const char *directory) } static int -read_osinfo_db_directory (guestfs_h *g, const char *directory) +read_osinfo_db_directory (guestfs_h *g, const char *directory, + read_osinfo_db_callback callback, void *opaque) { DIR *dir; int r; @@ -311,7 +342,7 @@ read_osinfo_db_directory (guestfs_h *g, const char *directory) CLEANUP_FREE char *pathname = NULL; pathname = safe_asprintf (g, "%s/%s", directory, d->d_name); - r = read_osinfo_db_xml (g, pathname); + r = callback (g, pathname, opaque); if (r == -1) goto error; } @@ -348,7 +379,7 @@ static int read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr o * Only memory allocation failures are fatal errors here. */ static int -read_osinfo_db_xml (guestfs_h *g, const char *pathname) +read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *opaque) { CLEANUP_XMLFREEDOC xmlDocPtr doc = NULL; CLEANUP_XMLXPATHFREECONTEXT xmlXPathContextPtr xpathCtx = NULL; -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 05/10] lib: extract osinfo DB traversing API
Split lib/osinfo.c to provide an API for other pieces of code (namely mllib) to reuse it. The ISO-related processing is thus moved into a lib/osinfo-iso.c file. --- lib/Makefile.am | 2 + lib/osinfo-iso.c | 464 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/osinfo.c | 424 +------------------------------------------------- lib/osinfo.h | 27 ++++ 4 files changed, 495 insertions(+), 422 deletions(-) create mode 100644 lib/osinfo-iso.c create mode 100644 lib/osinfo.h diff --git a/lib/Makefile.am b/lib/Makefile.am index e1ab1bff9..7a3580a00 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -112,7 +112,9 @@ libguestfs_la_SOURCES = \ lpj.c \ match.c \ mountable.c \ + osinfo.h \ osinfo.c \ + osinfo-iso.c \ private-data.c \ proto.c \ qemu.c \ diff --git a/lib/osinfo-iso.c b/lib/osinfo-iso.c new file mode 100644 index 000000000..0c3c19971 --- /dev/null +++ b/lib/osinfo-iso.c @@ -0,0 +1,464 @@ +/* libguestfs + * Copyright (C) 2012 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* Read libosinfo XML files to parse out just the + * os/media/iso/system-id and os/media/iso/volume-id fields, which we + * can then use to map install media to operating systems. + * + * Note some assumptions here: + * + * (1) We have to do some translation of the distro names and versions + * stored in the libosinfo files and the standard names returned by + * libguestfs. + * + * (2) Media detection is only part of the story. We may still need + * to inspect inside the image. + * + * (3) We only read the XML database files (at most) once per process, + * and keep them cached. They are only read at all if someone tries + * to inspect a CD/DVD/ISO. + * + * XXX Currently the database is not freed when the program exits / + * library is unloaded, although we should probably do that. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <dirent.h> +#include <assert.h> +#include <sys/types.h> +#include <libintl.h> +#include <sys/stat.h> + +#include <libxml/parser.h> +#include <libxml/xpath.h> + +#include "ignore-value.h" +#include "glthread/lock.h" +#include "c-ctype.h" + +#include "guestfs.h" +#include "guestfs-internal.h" + +#include "osinfo.h" + +gl_lock_define_initialized (static, osinfo_db_lock); +static ssize_t osinfo_db_size = 0; /* 0 = unread, -1 = error, >= 1 = #records */ +static struct osinfo *osinfo_db = NULL; + + +static void free_osinfo_db_entry (struct osinfo *); + +#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1) + +static int +read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *data); + +/* Given one or more fields from the header of a CD/DVD/ISO, look up + * the media in the libosinfo database and return our best guess for + * the operating system. + * + * This returns: + * -1 => a fatal error ('error' has been called, caller must not ignore it) + * 0 => could not locate the OS + * 1 => matching OS found, the osinfo_ret pointer has been filled in + */ +int +guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo, + const struct osinfo **osinfo_ret) +{ + size_t i; + + /* We only need to lock the database when reading it for the first time. */ + gl_lock_lock (osinfo_db_lock); + if (osinfo_db_size == 0) { + if (read_osinfo_db (g, read_osinfo_db_xml, NULL) == -1) { + /* Fatal error: free any database entries which have been read, and + * mark the database as having a permanent error. + */ + if (osinfo_db_size > 0) { + for (i = 0; i < (size_t) osinfo_db_size; ++i) + free_osinfo_db_entry (&osinfo_db[i]); + } + free (osinfo_db); + osinfo_db = NULL; + osinfo_db_size = -1; + gl_lock_unlock (osinfo_db_lock); + return -1; + } + } + gl_lock_unlock (osinfo_db_lock); + + if (osinfo_db_size <= 0) + return 0; + + /* Look in the database to see if we can find a match. */ + for (i = 0; i < (size_t) osinfo_db_size; ++i) { + if (osinfo_db[i].re_system_id) { + if (!isoinfo->iso_system_id || + !match (g, isoinfo->iso_system_id, osinfo_db[i].re_system_id)) + continue; + } + + if (osinfo_db[i].re_volume_id) { + if (!isoinfo->iso_volume_id || + !match (g, isoinfo->iso_volume_id, osinfo_db[i].re_volume_id)) + continue; + } + + if (osinfo_db[i].re_publisher_id) { + if (!isoinfo->iso_publisher_id || + !match (g, isoinfo->iso_publisher_id, osinfo_db[i].re_publisher_id)) + continue; + } + + if (osinfo_db[i].re_application_id) { + if (!isoinfo->iso_application_id || + !match (g, isoinfo->iso_application_id, osinfo_db[i].re_application_id)) + continue; + } + + debug (g, "osinfo: mapped disk to database entry %zu", i); + + if (osinfo_ret) + *osinfo_ret = &osinfo_db[i]; + return 1; + } + + debug (g, "osinfo: no mapping found"); + + return 0; +} + +static int read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo); +static int read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr media_node, struct osinfo *osinfo); +static int read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr os_node, struct osinfo *osinfo); + +/* Read a single XML file from pathname (which is a full path). + * Only memory allocation failures are fatal errors here. + */ +static int +read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *opaque) +{ + CLEANUP_XMLFREEDOC xmlDocPtr doc = NULL; + CLEANUP_XMLXPATHFREECONTEXT xmlXPathContextPtr xpathCtx = NULL; + CLEANUP_XMLXPATHFREEOBJECT xmlXPathObjectPtr xpathObj = NULL; + xmlNodeSetPtr nodes; + xmlNodePtr iso_node, media_node, os_node; + struct osinfo *osinfo; + size_t i; + + doc = xmlReadFile (pathname, NULL, XML_PARSE_NONET); + if (doc == NULL) { + debug (g, "osinfo: unable to parse XML file %s", pathname); + return 0; + } + + xpathCtx = xmlXPathNewContext (doc); + if (xpathCtx == NULL) { + error (g, _("osinfo: unable to create new XPath context")); + return -1; + } + + /* Get all <iso> nodes at any depth, then use the parent pointers in + * order to work back up the tree. + */ + xpathObj = xmlXPathEvalExpression (BAD_CAST "/libosinfo/os/media/iso", + xpathCtx); + if (xpathObj == NULL) { + error (g, _("osinfo: %s: unable to evaluate XPath expression"), + pathname); + return -1; + } + + nodes = xpathObj->nodesetval; + + if (nodes != NULL) { + for (i = 0; i < (size_t) nodes->nodeNr; ++i) { + iso_node = nodes->nodeTab[i]; + assert (iso_node != NULL); + assert (STREQ ((const char *) iso_node->name, "iso")); + assert (iso_node->type == XML_ELEMENT_NODE); + + media_node = iso_node->parent; + assert (media_node != NULL); + assert (STREQ ((const char *) media_node->name, "media")); + assert (media_node->type == XML_ELEMENT_NODE); + + os_node = media_node->parent; + assert (os_node != NULL); + assert (STREQ ((const char *) os_node->name, "os")); + assert (os_node->type == XML_ELEMENT_NODE); + + /* Allocate an osinfo record. */ + osinfo_db_size++; + osinfo_db = safe_realloc (g, osinfo_db, + sizeof (struct osinfo) * osinfo_db_size); + osinfo = &osinfo_db[osinfo_db_size-1]; + memset (osinfo, 0, sizeof *osinfo); + + /* Read XML fields into the new osinfo record. */ + if (read_iso_node (g, iso_node, osinfo) == -1 || + read_media_node (g, xpathCtx, media_node, osinfo) == -1 || + read_os_node (g, xpathCtx, os_node, osinfo) == -1) { + free_osinfo_db_entry (osinfo); + osinfo_db_size--; + return -1; + } + +#if 0 + debug (g, "osinfo: %s: %s%s%s%s=> arch %s live %s installer %s product %s type %d distro %d version %d.%d", + pathname, + osinfo->re_system_id ? "<system-id/> " : "", + osinfo->re_volume_id ? "<volume-id/> " : "", + osinfo->re_publisher_id ? "<publisher-id/> " : "", + osinfo->re_application_id ? "<application-id/> " : "", + osinfo->arch ? osinfo->arch : "(none)", + osinfo->is_live_disk ? "true" : "false", + osinfo->is_installer ? "true" : "false", + osinfo->product_name ? osinfo->product_name : "(none)", + (int) osinfo->type, (int) osinfo->distro, + osinfo->major_version, osinfo->minor_version); +#endif + } + } + + return 0; +} + +static int compile_re (guestfs_h *g, xmlNodePtr child, pcre **re); + +/* Read the regular expressions under the <iso> node. libosinfo + * itself uses the glib function 'g_regex_match_simple'. That appears + * to implement PCRE, however I have not checked in detail. + */ +static int +read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo) +{ + xmlNodePtr child; + + for (child = iso_node->children; child; child = child->next) { + if (STREQ ((const char *) child->name, "system-id")) { + if (compile_re (g, child, &osinfo->re_system_id) == -1) + return -1; + } + else if (STREQ ((const char *) child->name, "volume-id")) { + if (compile_re (g, child, &osinfo->re_volume_id) == -1) + return -1; + } + else if (STREQ ((const char *) child->name, "publisher-id")) { + if (compile_re (g, child, &osinfo->re_publisher_id) == -1) + return -1; + } + else if (STREQ ((const char *) child->name, "application-id")) { + if (compile_re (g, child, &osinfo->re_application_id) == -1) + return -1; + } + } + + return 0; +} + +static int +compile_re (guestfs_h *g, xmlNodePtr node, pcre **re) +{ + const char *err; + int offset; + CLEANUP_FREE char *content = (char *) xmlNodeGetContent (node); + + if (content) { + *re = pcre_compile (content, 0, &err, &offset, NULL); + if (*re == NULL) + debug (g, "osinfo: could not parse regular expression '%s': %s (ignored)", + content, err); + } + + return 0; +} + +/* Read the attributes of the <media/> node. */ +static int +read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, + xmlNodePtr media_node, struct osinfo *osinfo) +{ + osinfo->arch = (char *) xmlGetProp (media_node, BAD_CAST "arch"); + + osinfo->is_live_disk = 0; /* If no 'live' attr, defaults to false. */ + { + CLEANUP_XMLFREE xmlChar *content = NULL; + content = xmlGetProp (media_node, BAD_CAST "live"); + if (content) + osinfo->is_live_disk = XMLSTREQ (content, BAD_CAST "true"); + } + + osinfo->is_installer = true; /* If no 'installer' attr, defaults to true. */ + { + CLEANUP_XMLFREE xmlChar *content = NULL; + content = xmlGetProp (media_node, BAD_CAST "installer"); + if (content) + osinfo->is_installer = XMLSTREQ (content, BAD_CAST "true"); + } + + return 0; +} + +static int parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); +static int parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); +static int parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); + +/* Read some fields under the <os/> node. */ +static int +read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, + xmlNodePtr os_node, struct osinfo *osinfo) +{ + xmlNodePtr child; + + for (child = os_node->children; child; child = child->next) { + if (STREQ ((const char *) child->name, "name")) + osinfo->product_name = (char *) xmlNodeGetContent (child); + else if (STREQ ((const char *) child->name, "version")) { + if (parse_version (g, child, osinfo) == -1) + return -1; + } + else if (STREQ ((const char *) child->name, "family")) { + if (parse_family (g, child, osinfo) == -1) + return -1; + } + else if (STREQ ((const char *) child->name, "distro")) { + if (parse_distro (g, child, osinfo) == -1) + return -1; + } + } + + return 0; +} + +static int +parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) +{ + CLEANUP_FREE char *content = NULL; + + content = (char *) xmlNodeGetContent (node); + /* We parse either "X.Y" or "X" as version strings, so try to parse + * only if the first character is a digit. + */ + if (content && c_isdigit (content[0])) { + struct version version; + const int res = guestfs_int_version_from_x_y_or_x (g, &version, content); + if (res < 0) + return -1; + else if (res > 0) { + osinfo->major_version = version.v_major; + osinfo->minor_version = version.v_minor; + } + } + + return 0; +} + +static int +parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) +{ + CLEANUP_FREE char *content = NULL; + + osinfo->type = OS_TYPE_UNKNOWN; + + content = (char *) xmlNodeGetContent (node); + if (content) { + if (STREQ (content, "linux")) + osinfo->type = OS_TYPE_LINUX; + else if (STRPREFIX (content, "win")) + osinfo->type = OS_TYPE_WINDOWS; + else if (STREQ (content, "freebsd")) + osinfo->type = OS_TYPE_FREEBSD; + else if (STREQ (content, "netbsd")) + osinfo->type = OS_TYPE_NETBSD; + else if (STREQ (content, "msdos")) + osinfo->type = OS_TYPE_DOS; + else if (STREQ (content, "openbsd")) + osinfo->type = OS_TYPE_OPENBSD; + else + debug (g, "osinfo: warning: unknown <family> '%s'", content); + } + + return 0; +} + +static int +parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) +{ + CLEANUP_FREE char *content = NULL; + + osinfo->distro = OS_DISTRO_UNKNOWN; + + content = (char *) xmlNodeGetContent (node); + if (content) { + if (STREQ (content, "altlinux")) + osinfo->distro = OS_DISTRO_ALTLINUX; + else if (STREQ (content, "centos")) + osinfo->distro = OS_DISTRO_CENTOS; + else if (STREQ (content, "debian")) + osinfo->distro = OS_DISTRO_DEBIAN; + else if (STREQ (content, "fedora")) + osinfo->distro = OS_DISTRO_FEDORA; + else if (STREQ (content, "freebsd")) + osinfo->distro = OS_DISTRO_FREEBSD; + else if (STREQ (content, "mageia")) + osinfo->distro = OS_DISTRO_MAGEIA; + else if (STREQ (content, "mandriva")) + osinfo->distro = OS_DISTRO_MANDRIVA; + else if (STREQ (content, "netbsd")) + osinfo->distro = OS_DISTRO_NETBSD; + else if (STREQ (content, "openbsd")) + osinfo->distro = OS_DISTRO_OPENBSD; + else if (STREQ (content, "opensuse")) + osinfo->distro = OS_DISTRO_OPENSUSE; + else if (STREQ (content, "rhel")) + osinfo->distro = OS_DISTRO_RHEL; + else if (STREQ (content, "sled") || STREQ (content, "sles")) + osinfo->distro = OS_DISTRO_SLES; + else if (STREQ (content, "ubuntu")) + osinfo->distro = OS_DISTRO_UBUNTU; + else if (STRPREFIX (content, "win")) + osinfo->distro = OS_DISTRO_WINDOWS; + else + debug (g, "osinfo: warning: unknown <distro> '%s'", content); + } + + return 0; +} + +static void +free_osinfo_db_entry (struct osinfo *osinfo) +{ + free (osinfo->product_name); + free (osinfo->arch); + + if (osinfo->re_system_id) + pcre_free (osinfo->re_system_id); + if (osinfo->re_volume_id) + pcre_free (osinfo->re_volume_id); + if (osinfo->re_publisher_id) + pcre_free (osinfo->re_publisher_id); + if (osinfo->re_application_id) + pcre_free (osinfo->re_application_id); +} diff --git a/lib/osinfo.c b/lib/osinfo.c index b77ff96b3..131616129 100644 --- a/lib/osinfo.c +++ b/lib/osinfo.c @@ -29,21 +29,6 @@ * safe(-ish) since the media identifiers always change for every * release of an OS. We can easily add support for this if it becomes * necessary. - * - * (3) We have to do some translation of the distro names and versions - * stored in the libosinfo files and the standard names returned by - * libguestfs. - * - * (4) Media detection is only part of the story. We may still need - * to inspect inside the image. - * - * (5) We only read the XML database files (at most) once per process, - * and keep them cached. They are only read at all if someone tries - * to inspect a CD/DVD/ISO. - * - * XXX Currently the database is not freed when the program exits / - * library is unloaded, although we should probably do that. - * */ #include <config.h> @@ -59,28 +44,13 @@ #include <libintl.h> #include <sys/stat.h> -#include <libxml/parser.h> -#include <libxml/xpath.h> - #include "ignore-value.h" -#include "glthread/lock.h" #include "c-ctype.h" #include "guestfs.h" #include "guestfs-internal.h" -gl_lock_define_initialized (static, osinfo_db_lock); -static ssize_t osinfo_db_size = 0; /* 0 = unread, -1 = error, >= 1 = #records */ -static struct osinfo *osinfo_db = NULL; - -#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1) -typedef int (*read_osinfo_db_callback) (guestfs_h *g, const char *path, void *opaque); - -static int -read_osinfo_db (guestfs_h *g, - read_osinfo_db_callback callback, void *opaque); -static int read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *data); -static void free_osinfo_db_entry (struct osinfo *); +#include "osinfo.h" #ifndef GUESTFS_PRIVATE void guestfs_int_debug (guestfs_h *g, const char *fs, ...) @@ -109,82 +79,6 @@ guestfs_int_perrorf (guestfs_h *g, const char *fs, ...) } #endif /* GUESTFS_PRIVATE */ -/* Given one or more fields from the header of a CD/DVD/ISO, look up - * the media in the libosinfo database and return our best guess for - * the operating system. - * - * This returns: - * -1 => a fatal error ('error' has been called, caller must not ignore it) - * 0 => could not locate the OS - * 1 => matching OS found, the osinfo_ret pointer has been filled in - */ -int -guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo, - const struct osinfo **osinfo_ret) -{ - size_t i; - - /* We only need to lock the database when reading it for the first time. */ - gl_lock_lock (osinfo_db_lock); - if (osinfo_db_size == 0) { - if (read_osinfo_db (g, read_osinfo_db_xml, NULL) == -1) { - /* Fatal error: free any database entries which have been read, and - * mark the database as having a permanent error. - */ - if (osinfo_db_size > 0) { - for (i = 0; i < (size_t) osinfo_db_size; ++i) - free_osinfo_db_entry (&osinfo_db[i]); - } - free (osinfo_db); - osinfo_db = NULL; - osinfo_db_size = -1; - gl_lock_unlock (osinfo_db_lock); - return -1; - } - } - gl_lock_unlock (osinfo_db_lock); - - if (osinfo_db_size <= 0) - return 0; - - /* Look in the database to see if we can find a match. */ - for (i = 0; i < (size_t) osinfo_db_size; ++i) { - if (osinfo_db[i].re_system_id) { - if (!isoinfo->iso_system_id || - !match (g, isoinfo->iso_system_id, osinfo_db[i].re_system_id)) - continue; - } - - if (osinfo_db[i].re_volume_id) { - if (!isoinfo->iso_volume_id || - !match (g, isoinfo->iso_volume_id, osinfo_db[i].re_volume_id)) - continue; - } - - if (osinfo_db[i].re_publisher_id) { - if (!isoinfo->iso_publisher_id || - !match (g, isoinfo->iso_publisher_id, osinfo_db[i].re_publisher_id)) - continue; - } - - if (osinfo_db[i].re_application_id) { - if (!isoinfo->iso_application_id || - !match (g, isoinfo->iso_application_id, osinfo_db[i].re_application_id)) - continue; - } - - debug (g, "osinfo: mapped disk to database entry %zu", i); - - if (osinfo_ret) - *osinfo_ret = &osinfo_db[i]; - return 1; - } - - debug (g, "osinfo: no mapping found"); - - return 0; -} - /* Read the libosinfo XML database files. The lock is held while * this is called. * @@ -206,7 +100,7 @@ static int read_osinfo_db_three_levels (guestfs_h *g, const char *directory, static int read_osinfo_db_directory (guestfs_h *g, const char *directory, read_osinfo_db_callback callback, void *opaque); -static int +int read_osinfo_db (guestfs_h *g, read_osinfo_db_callback callback, void *opaque) { @@ -370,317 +264,3 @@ read_osinfo_db_directory (guestfs_h *g, const char *directory, return -1; } - -static int read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo); -static int read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr media_node, struct osinfo *osinfo); -static int read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr os_node, struct osinfo *osinfo); - -/* Read a single XML file from pathname (which is a full path). - * Only memory allocation failures are fatal errors here. - */ -static int -read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *opaque) -{ - CLEANUP_XMLFREEDOC xmlDocPtr doc = NULL; - CLEANUP_XMLXPATHFREECONTEXT xmlXPathContextPtr xpathCtx = NULL; - CLEANUP_XMLXPATHFREEOBJECT xmlXPathObjectPtr xpathObj = NULL; - xmlNodeSetPtr nodes; - xmlNodePtr iso_node, media_node, os_node; - struct osinfo *osinfo; - size_t i; - - doc = xmlReadFile (pathname, NULL, XML_PARSE_NONET); - if (doc == NULL) { - debug (g, "osinfo: unable to parse XML file %s", pathname); - return 0; - } - - xpathCtx = xmlXPathNewContext (doc); - if (xpathCtx == NULL) { - error (g, _("osinfo: unable to create new XPath context")); - return -1; - } - - /* Get all <iso> nodes at any depth, then use the parent pointers in - * order to work back up the tree. - */ - xpathObj = xmlXPathEvalExpression (BAD_CAST "/libosinfo/os/media/iso", - xpathCtx); - if (xpathObj == NULL) { - error (g, _("osinfo: %s: unable to evaluate XPath expression"), - pathname); - return -1; - } - - nodes = xpathObj->nodesetval; - - if (nodes != NULL) { - for (i = 0; i < (size_t) nodes->nodeNr; ++i) { - iso_node = nodes->nodeTab[i]; - assert (iso_node != NULL); - assert (STREQ ((const char *) iso_node->name, "iso")); - assert (iso_node->type == XML_ELEMENT_NODE); - - media_node = iso_node->parent; - assert (media_node != NULL); - assert (STREQ ((const char *) media_node->name, "media")); - assert (media_node->type == XML_ELEMENT_NODE); - - os_node = media_node->parent; - assert (os_node != NULL); - assert (STREQ ((const char *) os_node->name, "os")); - assert (os_node->type == XML_ELEMENT_NODE); - - /* Allocate an osinfo record. */ - osinfo_db_size++; - osinfo_db = safe_realloc (g, osinfo_db, - sizeof (struct osinfo) * osinfo_db_size); - osinfo = &osinfo_db[osinfo_db_size-1]; - memset (osinfo, 0, sizeof *osinfo); - - /* Read XML fields into the new osinfo record. */ - if (read_iso_node (g, iso_node, osinfo) == -1 || - read_media_node (g, xpathCtx, media_node, osinfo) == -1 || - read_os_node (g, xpathCtx, os_node, osinfo) == -1) { - free_osinfo_db_entry (osinfo); - osinfo_db_size--; - return -1; - } - -#if 0 - debug (g, "osinfo: %s: %s%s%s%s=> arch %s live %s installer %s product %s type %d distro %d version %d.%d", - pathname, - osinfo->re_system_id ? "<system-id/> " : "", - osinfo->re_volume_id ? "<volume-id/> " : "", - osinfo->re_publisher_id ? "<publisher-id/> " : "", - osinfo->re_application_id ? "<application-id/> " : "", - osinfo->arch ? osinfo->arch : "(none)", - osinfo->is_live_disk ? "true" : "false", - osinfo->is_installer ? "true" : "false", - osinfo->product_name ? osinfo->product_name : "(none)", - (int) osinfo->type, (int) osinfo->distro, - osinfo->major_version, osinfo->minor_version); -#endif - } - } - - return 0; -} - -static int compile_re (guestfs_h *g, xmlNodePtr child, pcre **re); - -/* Read the regular expressions under the <iso> node. libosinfo - * itself uses the glib function 'g_regex_match_simple'. That appears - * to implement PCRE, however I have not checked in detail. - */ -static int -read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo) -{ - xmlNodePtr child; - - for (child = iso_node->children; child; child = child->next) { - if (STREQ ((const char *) child->name, "system-id")) { - if (compile_re (g, child, &osinfo->re_system_id) == -1) - return -1; - } - else if (STREQ ((const char *) child->name, "volume-id")) { - if (compile_re (g, child, &osinfo->re_volume_id) == -1) - return -1; - } - else if (STREQ ((const char *) child->name, "publisher-id")) { - if (compile_re (g, child, &osinfo->re_publisher_id) == -1) - return -1; - } - else if (STREQ ((const char *) child->name, "application-id")) { - if (compile_re (g, child, &osinfo->re_application_id) == -1) - return -1; - } - } - - return 0; -} - -static int -compile_re (guestfs_h *g, xmlNodePtr node, pcre **re) -{ - const char *err; - int offset; - CLEANUP_FREE char *content = (char *) xmlNodeGetContent (node); - - if (content) { - *re = pcre_compile (content, 0, &err, &offset, NULL); - if (*re == NULL) - debug (g, "osinfo: could not parse regular expression '%s': %s (ignored)", - content, err); - } - - return 0; -} - -/* Read the attributes of the <media/> node. */ -static int -read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, - xmlNodePtr media_node, struct osinfo *osinfo) -{ - osinfo->arch = (char *) xmlGetProp (media_node, BAD_CAST "arch"); - - osinfo->is_live_disk = 0; /* If no 'live' attr, defaults to false. */ - { - CLEANUP_XMLFREE xmlChar *content = NULL; - content = xmlGetProp (media_node, BAD_CAST "live"); - if (content) - osinfo->is_live_disk = XMLSTREQ (content, BAD_CAST "true"); - } - - osinfo->is_installer = true; /* If no 'installer' attr, defaults to true. */ - { - CLEANUP_XMLFREE xmlChar *content = NULL; - content = xmlGetProp (media_node, BAD_CAST "installer"); - if (content) - osinfo->is_installer = XMLSTREQ (content, BAD_CAST "true"); - } - - return 0; -} - -static int parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); -static int parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); -static int parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo); - -/* Read some fields under the <os/> node. */ -static int -read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, - xmlNodePtr os_node, struct osinfo *osinfo) -{ - xmlNodePtr child; - - for (child = os_node->children; child; child = child->next) { - if (STREQ ((const char *) child->name, "name")) - osinfo->product_name = (char *) xmlNodeGetContent (child); - else if (STREQ ((const char *) child->name, "version")) { - if (parse_version (g, child, osinfo) == -1) - return -1; - } - else if (STREQ ((const char *) child->name, "family")) { - if (parse_family (g, child, osinfo) == -1) - return -1; - } - else if (STREQ ((const char *) child->name, "distro")) { - if (parse_distro (g, child, osinfo) == -1) - return -1; - } - } - - return 0; -} - -static int -parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) -{ - CLEANUP_FREE char *content = NULL; - - content = (char *) xmlNodeGetContent (node); - /* We parse either "X.Y" or "X" as version strings, so try to parse - * only if the first character is a digit. - */ - if (content && c_isdigit (content[0])) { - struct version version; - const int res = guestfs_int_version_from_x_y_or_x (g, &version, content); - if (res < 0) - return -1; - else if (res > 0) { - osinfo->major_version = version.v_major; - osinfo->minor_version = version.v_minor; - } - } - - return 0; -} - -static int -parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) -{ - CLEANUP_FREE char *content = NULL; - - osinfo->type = OS_TYPE_UNKNOWN; - - content = (char *) xmlNodeGetContent (node); - if (content) { - if (STREQ (content, "linux")) - osinfo->type = OS_TYPE_LINUX; - else if (STRPREFIX (content, "win")) - osinfo->type = OS_TYPE_WINDOWS; - else if (STREQ (content, "freebsd")) - osinfo->type = OS_TYPE_FREEBSD; - else if (STREQ (content, "netbsd")) - osinfo->type = OS_TYPE_NETBSD; - else if (STREQ (content, "msdos")) - osinfo->type = OS_TYPE_DOS; - else if (STREQ (content, "openbsd")) - osinfo->type = OS_TYPE_OPENBSD; - else - debug (g, "osinfo: warning: unknown <family> '%s'", content); - } - - return 0; -} - -static int -parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo) -{ - CLEANUP_FREE char *content = NULL; - - osinfo->distro = OS_DISTRO_UNKNOWN; - - content = (char *) xmlNodeGetContent (node); - if (content) { - if (STREQ (content, "altlinux")) - osinfo->distro = OS_DISTRO_ALTLINUX; - else if (STREQ (content, "centos")) - osinfo->distro = OS_DISTRO_CENTOS; - else if (STREQ (content, "debian")) - osinfo->distro = OS_DISTRO_DEBIAN; - else if (STREQ (content, "fedora")) - osinfo->distro = OS_DISTRO_FEDORA; - else if (STREQ (content, "freebsd")) - osinfo->distro = OS_DISTRO_FREEBSD; - else if (STREQ (content, "mageia")) - osinfo->distro = OS_DISTRO_MAGEIA; - else if (STREQ (content, "mandriva")) - osinfo->distro = OS_DISTRO_MANDRIVA; - else if (STREQ (content, "netbsd")) - osinfo->distro = OS_DISTRO_NETBSD; - else if (STREQ (content, "openbsd")) - osinfo->distro = OS_DISTRO_OPENBSD; - else if (STREQ (content, "opensuse")) - osinfo->distro = OS_DISTRO_OPENSUSE; - else if (STREQ (content, "rhel")) - osinfo->distro = OS_DISTRO_RHEL; - else if (STREQ (content, "sled") || STREQ (content, "sles")) - osinfo->distro = OS_DISTRO_SLES; - else if (STREQ (content, "ubuntu")) - osinfo->distro = OS_DISTRO_UBUNTU; - else if (STRPREFIX (content, "win")) - osinfo->distro = OS_DISTRO_WINDOWS; - else - debug (g, "osinfo: warning: unknown <distro> '%s'", content); - } - - return 0; -} - -static void -free_osinfo_db_entry (struct osinfo *osinfo) -{ - free (osinfo->product_name); - free (osinfo->arch); - - if (osinfo->re_system_id) - pcre_free (osinfo->re_system_id); - if (osinfo->re_volume_id) - pcre_free (osinfo->re_volume_id); - if (osinfo->re_publisher_id) - pcre_free (osinfo->re_publisher_id); - if (osinfo->re_application_id) - pcre_free (osinfo->re_application_id); -} diff --git a/lib/osinfo.h b/lib/osinfo.h new file mode 100644 index 000000000..68846747e --- /dev/null +++ b/lib/osinfo.h @@ -0,0 +1,27 @@ +/* libguestfs + * Copyright (C) 2017 Red Hat Inc. + * Copyright (C) 2017 SUSE Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef OSINFO_H +#define OSINFO_H + +typedef int (*read_osinfo_db_callback) (guestfs_h *g, const char *path, void *opaque); + +extern int read_osinfo_db (guestfs_h *g, read_osinfo_db_callback callback, void *opaque); + +#endif /* OSINFO_H */ -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 06/10] mllib: ocaml wrapper for lib/osinfo
Provide osinfo database parsing API in OCAML. --- mllib/Makefile.am | 11 ++++-- mllib/osinfo-c.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ mllib/osinfo.ml | 26 ++++++++++++++ mllib/osinfo.mli | 31 +++++++++++++++++ 4 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 mllib/osinfo-c.c create mode 100644 mllib/osinfo.ml create mode 100644 mllib/osinfo.mli diff --git a/mllib/Makefile.am b/mllib/Makefile.am index aa5472ade..2541ce280 100644 --- a/mllib/Makefile.am +++ b/mllib/Makefile.am @@ -40,6 +40,7 @@ SOURCES_MLI = \ getopt.mli \ JSON.mli \ mkdtemp.mli \ + osinfo.mli \ planner.mli \ progress.mli \ regedit.mli \ @@ -71,7 +72,8 @@ SOURCES_ML = \ exit.ml \ checksums.ml \ xml.ml \ - xpath_helpers.ml + xpath_helpers.ml \ + osinfo.ml SOURCES_C = \ ../common/visit/visit.c \ @@ -79,6 +81,9 @@ SOURCES_C = \ ../common/options/keys.c \ ../common/options/uri.c \ ../common/progress/progress.c \ + ../lib/alloc.c \ + ../lib/osinfo.c \ + ../lib/osinfo.h \ common_utils-c.c \ dev_t-c.c \ exit-c.c \ @@ -86,6 +91,7 @@ SOURCES_C = \ fsync-c.c \ getopt-c.c \ mkdtemp-c.c \ + osinfo-c.c \ progress-c.c \ statvfs-c.c \ uri-c.c \ @@ -119,7 +125,8 @@ libmllib_a_CPPFLAGS = \ -I$(top_srcdir)/lib \ -I$(top_srcdir)/common/visit \ -I$(top_srcdir)/common/options \ - -I$(top_srcdir)/common/progress + -I$(top_srcdir)/common/progress \ + -DLIBOSINFO_DB_PATH='"$(datadir)/libosinfo/db"' libmllib_a_CFLAGS = \ $(WARN_CFLAGS) $(WERROR_CFLAGS) \ $(LIBVIRT_CFLAGS) $(LIBXML2_CFLAGS) \ diff --git a/mllib/osinfo-c.c b/mllib/osinfo-c.c new file mode 100644 index 000000000..9546c5984 --- /dev/null +++ b/mllib/osinfo-c.c @@ -0,0 +1,100 @@ +/* Bindings for osinfo db reading function. + * Copyright (C) 2017 SUSE Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include <assert.h> + +#include <caml/alloc.h> +#include <caml/callback.h> +#include <caml/fail.h> +#include <caml/memory.h> +#include <caml/mlvalues.h> + +#include "guestfs.h" +#include "guestfs-internal.h" +#include "osinfo.h" + +#pragma GCC diagnostic ignored "-Wmissing-prototypes" + +struct callback_wrapper_args { + /* In both case we are pointing to local roots, hence why these are + * value* not value. + */ + value *exnp; /* Safe place to store any exception + raised by callback */ + value *fvp; /* callback. */ +}; + +static int read_osinfo_db_callback_wrapper (guestfs_h *g, const char *path, void *opaque); + +value +guestfs_int_mllib_read_osinfo_db (value gv, value fv) +{ + CAMLparam2 (gv, fv); + guestfs_h *g = (guestfs_h *) Int64_val (gv); + struct callback_wrapper_args args; + + /* This stack address is used to point to the exception, if one is + * raised in the visitor_function. Note that the macro initializes + * this to Val_unit, which is how we know if an exception was set. + */ + CAMLlocal1 (exn); + + args.exnp = &exn; + args.fvp = &fv; + + if (read_osinfo_db (g, read_osinfo_db_callback_wrapper, &args) == -1) { + if (exn != Val_unit) { + /* The failure was caused by the callback raising an + * exception. Re-raise it here. + */ + caml_raise (exn); + } + + caml_failwith ("read_osinfo_db"); +} + + CAMLreturn (Val_unit); +} + +static int +read_osinfo_db_callback_wrapper (guestfs_h *g, const char *path, void *opaque) +{ + CAMLparam0 (); + CAMLlocal2 (pathv, v); + struct callback_wrapper_args *args = opaque; + + assert (path != NULL); + assert (args != NULL); + + pathv = caml_copy_string (path); + + v = caml_callback_exn (*args->fvp, pathv); + + if (Is_exception_result (v)) { + *args->exnp = Extract_exception (v); + CAMLreturnT (int, -1); + } + + /* No error, return normally. */ + CAMLreturnT (int, 0); +} diff --git a/mllib/osinfo.ml b/mllib/osinfo.ml new file mode 100644 index 000000000..f5afbd889 --- /dev/null +++ b/mllib/osinfo.ml @@ -0,0 +1,26 @@ +(* virt-builder + * Copyright (C) 2016 - SUSE Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) +open Common_utils + +type osinfo_db_callback = string -> unit + +external c_read_osinfo_db : int64 -> osinfo_db_callback -> unit + "guestfs_int_mllib_read_osinfo_db" + +let read_osinfo_db g f + c_read_osinfo_db (Guestfs.c_pointer g) f diff --git a/mllib/osinfo.mli b/mllib/osinfo.mli new file mode 100644 index 000000000..306cbce0f --- /dev/null +++ b/mllib/osinfo.mli @@ -0,0 +1,31 @@ +(* virt-builder + * Copyright (C) 2016 - SUSE Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +(** Bindings for the src/osinfo.h libosinfo database reading API. *) + +type osinfo_db_callback = string -> unit +(** The osinfo_db_callback is a callback called for each data file + in the libosinfo database. The argument of the function is + the absolute path of the data file. + + The callback may raise an exception, which will cause the whole + database read to fail with an error (raising the same exception). *) + +val read_osinfo_db : Guestfs.t -> osinfo_db_callback -> unit +(** [read_osinfo_db g callback] will find all the libosinfo database + files and call the callback on them. *) -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 07/10] builder: rename docs test script
Rename test-virt-builder-docs.sh into test-docs.sh to include test for another tool's documentation. --- builder/Makefile.am | 4 ++-- builder/{test-virt-builder-docs.sh => test-docs.sh} | 0 2 files changed, 2 insertions(+), 2 deletions(-) rename builder/{test-virt-builder-docs.sh => test-docs.sh} (100%) diff --git a/builder/Makefile.am b/builder/Makefile.am index d56b394b7..218f64b4c 100644 --- a/builder/Makefile.am +++ b/builder/Makefile.am @@ -28,7 +28,7 @@ EXTRA_DIST = \ test-simplestreams/streams/v1/index.json \ test-simplestreams/streams/v1/net.cirros-cloud_released_download.json \ test-virt-builder.sh \ - test-virt-builder-docs.sh \ + test-docs.sh \ test-virt-builder-list.sh \ test-virt-builder-list-simplestreams.sh \ test-virt-builder-planner.sh \ @@ -237,7 +237,7 @@ yajl_tests_LINK = \ $(yajl_tests_THEOBJECTS) -o $@ TESTS = \ - test-virt-builder-docs.sh \ + test-docs.sh \ test-virt-builder-list.sh \ test-virt-index-validate.sh \ $(SLOW_TESTS) diff --git a/builder/test-virt-builder-docs.sh b/builder/test-docs.sh similarity index 100% rename from builder/test-virt-builder-docs.sh rename to builder/test-docs.sh -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 08/10] builder: add Index.write_entry function
Add a function to properly write virt-builder source index entries. Note that this function is very similar to Index.print_entry that is meant for debugging purposes. --- builder/index.mli | 3 +++ builder/index_parser.ml | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ builder/index_parser.mli | 6 ++++++ 3 files changed, 61 insertions(+) diff --git a/builder/index.mli b/builder/index.mli index ff5ec4a35..6202d636e 100644 --- a/builder/index.mli +++ b/builder/index.mli @@ -39,3 +39,6 @@ and entry = { } val print_entry : out_channel -> (string * entry) -> unit +(** Debugging helper function dumping an index entry to a stream. + To write entries for non-debugging purpose, use the + [Index_parser.write_entry] function. *) diff --git a/builder/index_parser.ml b/builder/index_parser.ml index a3cae7d1a..eb72602aa 100644 --- a/builder/index_parser.ml +++ b/builder/index_parser.ml @@ -226,3 +226,55 @@ let get_index ~downloader ~sigchecker in get_index () + +let write_entry chan (name, { Index.printable_name = printable_name; + file_uri = file_uri; + arch = arch; + osinfo = osinfo; + signature_uri = signature_uri; + checksums = checksums; + revision = revision; + format = format; + size = size; + compressed_size = compressed_size; + expand = expand; + lvexpand = lvexpand; + notes = notes; + aliases = aliases; + hidden = hidden }) + let fp fs = fprintf chan fs in + fp "[%s]\n" name; + may (fp "name=%s\n") printable_name; + may (fp "osinfo=%s\n") osinfo; + fp "file=%s\n" file_uri; + fp "arch=%s\n" arch; + may (fp "sig=%s\n") signature_uri; + (match checksums with + | None -> () + | Some checksums -> + List.iter ( + fun c -> + fp "checksum[%s]=%s\n" + (Checksums.string_of_csum_t c) (Checksums.string_of_csum c) + ) checksums + ); + fp "revision=%s\n" (string_of_revision revision); + may (fp "format=%s\n") format; + fp "size=%Ld\n" size; + may (fp "compressed_size=%Ld\n") compressed_size; + may (fp "expand=%s\n") expand; + may (fp "lvexpand=%s\n") lvexpand; + List.iter ( + fun (lang, notes) -> + let format_notes notes + Str.global_replace (Str.regexp "^" ) " " notes in + match lang with + | "" -> fp "notes=%s\n" (format_notes notes) + | lang -> fp "notes[%s]=%s\n" lang (format_notes notes) + ) notes; + (match aliases with + | None -> () + | Some l -> fp "aliases=%s\n" (String.concat " " l) + ); + if hidden then fp "hidden=true\n"; + fp "\n" diff --git a/builder/index_parser.mli b/builder/index_parser.mli index b8d8ddf3d..7c1c423ad 100644 --- a/builder/index_parser.mli +++ b/builder/index_parser.mli @@ -17,3 +17,9 @@ *) val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t -> Sources.source -> Index.index +(** [get_index download sigchecker source] will parse the source index file + into an index entry list. *) + +val write_entry : out_channel -> (string * Index.entry) -> unit +(** [write_entry chan entry] writes the index entry to the chan output + stream.*) -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 09/10] builder: add a template parameter to get_index
get_index now gets a new template parameter. Setting it to true will make the index parsing less picky about missing important data. This can be used to parse a partial index file. --- builder/builder.ml | 2 +- builder/index_parser.ml | 20 ++++++++++++++------ builder/index_parser.mli | 2 +- 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/builder/builder.ml b/builder/builder.ml index 14b42d7ce..809995013 100644 --- a/builder/builder.ml +++ b/builder/builder.ml @@ -208,7 +208,7 @@ let main () ~tmpdir in match source.Sources.format with | Sources.FormatNative -> - Index_parser.get_index ~downloader ~sigchecker source + Index_parser.get_index ~downloader ~sigchecker ~template:false source | Sources.FormatSimpleStreams -> Simplestreams_parser.get_index ~downloader ~sigchecker source ) sources diff --git a/builder/index_parser.ml b/builder/index_parser.ml index eb72602aa..152a3999e 100644 --- a/builder/index_parser.ml +++ b/builder/index_parser.ml @@ -24,7 +24,7 @@ open Utils open Printf open Unix -let get_index ~downloader ~sigchecker +let get_index ~downloader ~sigchecker ~template { Sources.uri = uri; proxy = proxy } let corrupt_file () error (f_"The index file downloaded from '%s' is corrupt.\nYou need to ask the supplier of this file to fix it and upload a fixed version.") uri @@ -112,7 +112,7 @@ let get_index ~downloader ~sigchecker let revision try Rev_int (int_of_string (List.assoc ("revision", None) fields)) with - | Not_found -> Rev_int 1 + | Not_found -> if template then Rev_int 0 else Rev_int 1 | Failure _ -> eprintf (f_"%s: cannot parse 'revision' field for '%s'\n") prog n; corrupt_file () in @@ -122,11 +122,19 @@ let get_index ~downloader ~sigchecker try Int64.of_string (List.assoc ("size", None) fields) with | Not_found -> - eprintf (f_"%s: no 'size' field for '%s'\n") prog n; - corrupt_file () + if template then + Int64.zero + else ( + eprintf (f_"%s: no 'size' field for '%s'\n") prog n; + corrupt_file () + ) | Failure _ -> - eprintf (f_"%s: cannot parse 'size' field for '%s'\n") prog n; - corrupt_file () in + if template then + Int64.zero + else ( + eprintf (f_"%s: cannot parse 'size' field for '%s'\n") prog n; + corrupt_file () + ) in let compressed_size try Some (Int64.of_string (List.assoc ("compressed_size", None) fields)) with diff --git a/builder/index_parser.mli b/builder/index_parser.mli index 7c1c423ad..ae757ad6f 100644 --- a/builder/index_parser.mli +++ b/builder/index_parser.mli @@ -16,7 +16,7 @@ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. *) -val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t -> Sources.source -> Index.index +val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t -> template:bool -> Sources.source -> Index.index (** [get_index download sigchecker source] will parse the source index file into an index entry list. *) -- 2.11.0
Cédric Bosdonnat
2017-Feb-10 15:06 UTC
[Libguestfs] [PATCH v3 10/10] Add a virt-builder-repository tool
virt-builder-repository allows users to easily create or update a virt-builder source repository out of disk images. The tool can be run in either interactive or automated mode. --- .gitignore | 3 + builder/Makefile.am | 82 +++++- builder/repository_main.ml | 487 ++++++++++++++++++++++++++++++++++++ builder/test-docs.sh | 3 + builder/virt-builder-repository.pod | 183 ++++++++++++++ 5 files changed, 757 insertions(+), 1 deletion(-) create mode 100644 builder/repository_main.ml create mode 100644 builder/virt-builder-repository.pod diff --git a/.gitignore b/.gitignore index 0ea250831..bcc9f7753 100644 --- a/.gitignore +++ b/.gitignore @@ -94,13 +94,16 @@ Makefile.in /builder/oUnit-* /builder/*.qcow2 /builder/stamp-virt-builder.pod +/builder/stamp-virt-builder-repository.pod /builder/stamp-virt-index-validate.pod /builder/test-config/virt-builder/repos.d/test-index.conf /builder/test-console-*.sh /builder/test-simplestreams/virt-builder/repos.d/cirros.conf /builder/test-website/virt-builder/repos.d/libguestfs.conf /builder/virt-builder +/builder/virt-builder-repository /builder/virt-builder.1 +/builder/virt-builder-repository.1 /builder/virt-index-validate /builder/virt-index-validate.1 /builder/*.xz diff --git a/builder/Makefile.am b/builder/Makefile.am index 218f64b4c..f5a5138cd 100644 --- a/builder/Makefile.am +++ b/builder/Makefile.am @@ -21,6 +21,8 @@ AM_YFLAGS = -d EXTRA_DIST = \ $(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \ + $(REPOSITORY_SOURCES_ML) \ + $(REPOSITORY_SOURCES_MLI) \ libguestfs.gpg \ opensuse.gpg \ test-console.sh \ @@ -38,6 +40,7 @@ EXTRA_DIST = \ test-virt-index-validate-good-2 \ test-virt-index-validate-good-3 \ virt-builder.pod \ + virt-builder-repository.pod \ virt-index-validate.pod \ yajl_tests.ml @@ -85,13 +88,44 @@ SOURCES_C = \ setlocale-c.c \ yajl-c.c +REPOSITORY_SOURCES_ML = \ + utils.ml \ + index.ml \ + cache.ml \ + downloader.ml \ + sigchecker.ml \ + ini_reader.ml \ + index_parser.ml \ + yajl.ml \ + paths.ml \ + sources.ml \ + repository_main.ml + +REPOSITORY_SOURCES_MLI = \ + cache.mli \ + downloader.mli \ + index.mli \ + index_parser.mli \ + ini_reader.mli \ + sigchecker.mli \ + sources.mli \ + yajl.mli + +REPOSITORY_SOURCES_C = \ + index-scan.c \ + index-struct.c \ + index-parse.c \ + index-parser-c.c \ + yajl-c.c + + man_MANS noinst_DATA bin_PROGRAMS if HAVE_OCAML -bin_PROGRAMS += virt-builder +bin_PROGRAMS += virt-builder virt-builder-repository virt_builder_SOURCES = $(SOURCES_C) virt_builder_CPPFLAGS = \ @@ -115,6 +149,26 @@ virt_builder_CFLAGS = \ BOBJECTS = $(SOURCES_ML:.ml=.cmo) XOBJECTS = $(BOBJECTS:.cmo=.cmx) +virt_builder_repository_SOURCES = $(REPOSITORY_SOURCES_C) +virt_builder_repository_CPPFLAGS = \ + -I. \ + -I$(top_builddir) \ + -I$(top_srcdir)/gnulib/lib -I$(top_builddir)/gnulib/lib \ + -I$(shell $(OCAMLC) -where) \ + -I$(top_srcdir)/gnulib/lib \ + -I$(top_srcdir)/src \ + -I$(top_srcdir)/fish +virt_builder_repository_CFLAGS = \ + -pthread \ + $(WARN_CFLAGS) $(WERROR_CFLAGS) \ + -Wno-unused-macros \ + $(LIBLZMA_CFLAGS) \ + $(LIBTINFO_CFLAGS) \ + $(LIBXML2_CFLAGS) \ + $(YAJL_CFLAGS) +REPOSITORY_BOBJECTS = $(REPOSITORY_SOURCES_ML:.ml=.cmo) +REPOSITORY_XOBJECTS = $(REPOSITORY_BOBJECTS:.cmo=.cmx) + # -I $(top_builddir)/lib/.libs is a hack which forces corresponding -L # option to be passed to gcc, so we don't try linking against an # installed copy of libguestfs. @@ -149,8 +203,10 @@ OCAMLFLAGS = $(OCAML_FLAGS) $(OCAML_WARN_ERROR) if !HAVE_OCAMLOPT OBJECTS = $(BOBJECTS) +REPOSITORY_OBJECTS = $(REPOSITORY_BOBJECTS) else OBJECTS = $(XOBJECTS) +REPOSITORY_OBJECTS = $(REPOSITORY_XOBJECTS) endif OCAMLLINKFLAGS = mlguestfs.$(MLARCHIVE) mllib.$(MLARCHIVE) customize.$(MLARCHIVE) $(LINK_CUSTOM_OCAMLC_ONLY) @@ -165,6 +221,16 @@ virt_builder_LINK = \ $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \ $(OBJECTS) -o $@ +virt_builder_repository_DEPENDENCIES = \ + $(REPOSITORY_OBJECTS) \ + ../mllib/mllib.$(MLARCHIVE) \ + ../customize/customize.$(MLARCHIVE) \ + $(top_srcdir)/ocaml-link.sh +virt_builder_repository_LINK = \ + $(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \ + $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \ + $(REPOSITORY_OBJECTS) -o $@ + # Manual pages and HTML files for the website. man_MANS += virt-builder.1 @@ -183,6 +249,20 @@ stamp-virt-builder.pod: virt-builder.pod $(top_srcdir)/customize/customize-synop $< touch $@ +man_MANS += virt-builder-repository.1 +noinst_DATA += $(top_builddir)/website/virt-builder-repository.1.html + +virt-builder-repository.1 $(top_builddir)/website/virt-builder-repository.1.html: stamp-virt-builder-repository.pod + +stamp-virt-builder-repository.pod: virt-builder-repository.pod + $(PODWRAPPER) \ + --man virt-builder-repository.1 \ + --html $(top_builddir)/website/virt-builder-repository.1.html \ + --license GPLv2+ \ + --warning safe \ + $< + touch $@ + # Tests. TESTS_ENVIRONMENT = $(top_builddir)/run --test diff --git a/builder/repository_main.ml b/builder/repository_main.ml new file mode 100644 index 000000000..89807f9d5 --- /dev/null +++ b/builder/repository_main.ml @@ -0,0 +1,487 @@ +(* virt-builder + * Copyright (C) 2016 SUSE Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + *) + +open Common_gettext.Gettext +open Common_utils +open Getopt.OptionName +open Utils +open Yajl +open Xpath_helpers + +open Printf + +module StringSet = Set.Make(String) + +type cmdline = { + gpg : string; + gpgkey : string option; + interactive : bool; + keep_unsigned : bool; + repo : string; +} + +type disk_image_infos = { + format : string; + size : int64; +} + +let parse_cmdline () + let gpg = ref "gpg" in + let gpgkey = ref "" in + let interactive = ref false in + let keep_unsigned = ref false in + + let argspec = [ + [ L"gpg" ], Getopt.Set_string ("gpg", gpg), s_"Set GPG binary/command"; + [ S 'K'; L"gpg-key" ], Getopt.Set_string ("gpgkey", gpgkey), + s_"ID of the GPG key to sign the repo with"; + [ S 'i'; L"interactive" ], Getopt.Set interactive, s_"Ask the user about missing data"; + [ L"keep-index" ], Getopt.Set keep_unsigned, s_"Keep unsigned index"; + ] in + + let args = ref [] in + let anon_fun s = push_front s args in + let usage_msg + sprintf (f_"\ +%s: create a repository for virt-builder + + virt-builder-repository REPOSITORY_PATH + +A short summary of the options is given below. For detailed help please +read the man page virt-builder-repository(1). +") + prog in + let opthandle = create_standard_options argspec ~anon_fun usage_msg in + Getopt.parse opthandle; + + (* Dereference options. *) + let args = List.rev !args in + let gpg = !gpg in + let gpgkey = match !gpgkey with "" -> None | s -> Some s in + let interactive = !interactive in + let keep_unsigned = !keep_unsigned in + + (* Check options *) + let repo + (match args with + | [repo] -> repo + | [] -> + error (f_"virt-builder-repository /path/to/repo\nUse '/path/to/repo' to point to the repository folder.") + | _ -> + error (f_"too many parameters, only one path to repository is allowed") + ) in + + { + gpg = gpg; + gpgkey = gpgkey; + interactive = interactive; + keep_unsigned = keep_unsigned; + repo = repo; + } + +let increment_revision revision + match revision with + | Utils.Rev_int n -> Utils.Rev_int (n + 1) + | Utils.Rev_string s -> + if Str.string_match (Str.regexp "^\\(.*[-._]\\)\\([0-9]+\\)$") s 0 then + let prefix = Str.matched_group 1 s in + let suffix = int_of_string (Str.matched_group 2 s) in + Utils.Rev_string (prefix ^ (string_of_int (suffix + 1))) + else + Utils.Rev_string (s ^ ".1") + +let osinfo_ids = ref None + +let osinfo_get_short_ids () + match !osinfo_ids with + | Some ids -> ids + | None -> ( + let ids = ref [] in + let g = open_guestfs () in + + Osinfo.read_osinfo_db g#ocaml_handle ( + fun filepath -> + let doc = Xml.parse_file filepath in + let xpathctx = Xml.xpath_new_context doc in + let id = xpath_string_default xpathctx "/libosinfo/os/short-id" "" in + if id <> "" then + ids := id :: !ids + ); + g#close (); + let ids_set = StringSet.of_list(!ids) in + osinfo_ids := Some ids_set; + ids_set + ) + +(* Move files in tmprepo into the repository *) +let do_mv src dest + let cmd = [ "mv"; src; dest ] in + run_command cmd + +let compress_to file outdir + info "Copying image to temporary folder ...%!"; + let outimg = outdir // (Filename.basename file) in + let cmd = [ "cp" ] @ + (if verbose () then [ "-v" ] else []) @ + [ file; outimg ] in + let r = run_command cmd in + if r <> 0 then + error (f_"cp command failed '%s'") file; + + info "Compressing ...%!"; + let cmd = [ "xz"; "-f"; "--best"; + "--block-size=16777216"; outimg ] in + if run_command cmd <> 0 then + exit 1; + outimg ^ ".xz" + +let get_disk_image_infos filepath + let qemuimg_cmd = "qemu-img info --output json " ^ (quote filepath) in + let lines = external_command qemuimg_cmd in + let line = String.concat "\n" lines in + let infos = yajl_tree_parse line in + { format = object_get_string "format" infos; + size = object_get_number "virtual-size" infos } + +let process_image filename repo tmprepo index interactive sigchecker + info "Preparing %s..." filename; + + let filepath = repo // filename in + let { format = format; size = size } = get_disk_image_infos filepath in + let xz_path = compress_to filepath tmprepo in + let checksum = Checksums.compute_checksum "sha512" xz_path in + let compressed_size = (Unix.stat xz_path).Unix.st_size in + + let ask message = ( + printf message; + let value = read_line () in + match value with + | "" -> None + | s -> Some s + ) in + + let rec ask_id () = ( + printf (f_"Identifier: "); + let id = read_line () in + if not (Str.string_match (Str.regexp "[a-zA-Z0-9-_.]+") id 0) then ( + warning (f_"Allowed characters are letters, digits, - _ and ."); + ask_id (); + ) else + id; + ) in + + let ask_arch () = ( + printf (f_"Architecture. Choose one from the list below:\n"); + let arches = ["x86_64"; "aarch64"; "armv7l"; "i686"; "ppc64"; "ppc64le"; "s390x" ] in + iteri ( + fun i arch -> printf " [%d] %s\n" (i + 1) arch + ) arches; + + let arch = ref None in + while !arch <> None do + let input = read_line () in + if input = "exit" || input = "q" || input = "quit" then + exit 0 + else + try + let i = int_of_string input in + arch := Some (List.nth arches (i - 1)) + with Failure _ -> arch := Some input + done; + match !arch with + | None -> "" (* Should never happen *) + | Some arch -> arch + ) in + + let ask_osinfo () + printf (f_ "osinfo short ID (can be found with osinfo-query os command): "); + let value = read_line () in + match value with + | "" -> None + | osinfo -> + let osinfo_ids = osinfo_get_short_ids () in + if not (StringSet.mem osinfo osinfo_ids) then + warning (f_"'%s' is not a libosinfo OS id") osinfo; + Some osinfo in + + (* Do we have an entry for that file already? *) + let file_entry + try + List.hd ( + List.filter ( + fun (id, { Index.file_uri=file_uri }) -> + (Filename.basename file_uri) = ((Filename.basename filename) ^ ".xz") + ) index + ) + with + | Failure _ -> + let entry = { Index.printable_name = None; + osinfo = None; + file_uri = ""; + arch = ""; + signature_uri = None; + checksums = None; + revision = Utils.Rev_int 0; + format = Some format; + size = size; + compressed_size = Some (Int64.of_int compressed_size); + expand = None; + lvexpand = None; + notes = []; + hidden = false; + aliases = None; + sigchecker = sigchecker; + proxy = Curl.UnsetProxy } in + ("", entry) in + + let id, { Index.printable_name = printable_name; + osinfo = osinfo; + arch = arch; + checksums = checksums; + revision = revision; + expand = expand; + lvexpand = lvexpand; + notes = notes; + hidden = hidden; + aliases = aliases } = file_entry in + + let old_checksum + match checksums with + | Some csums -> ( + try + let csum = List.find ( + fun c -> + match c with + | Checksums.SHA512 _ -> true + | _ -> false + ) csums in + csum + with + | _ -> Checksums.SHA512 "" + ) + | None -> Checksums.SHA512 "" in + + let id + if id = "" && interactive then + ask_id () + else ( + if id = "" then + error (f_"Missing image identifier"); + id + ) in + + let printable_name + if printable_name = None && interactive then + ask (f_"Display name: ") + else + printable_name in + + let arch + if arch = "" then ( + if interactive then ask_arch () + else error (f_"missing architecture for %s") id; + ) else arch in + + let osinfo + if osinfo = None && interactive then + ask_osinfo () + else + osinfo in + + let expand + if expand = None && interactive then + ask (f_"Expandable partition: ") + else + expand in + + let lvexpand + if lvexpand = None && interactive then + ask (f_"Expandable volume: ") + else + lvexpand in + + let revision + if old_checksum <> checksum then + increment_revision revision + else + revision in + + (id, { Index.printable_name = printable_name; + osinfo = osinfo; + file_uri = Filename.basename xz_path; + arch = arch; + signature_uri = None; + checksums = Some [checksum]; + revision = revision; + format = Some format; + size = size; + compressed_size = Some (Int64.of_int compressed_size); + expand = expand; + lvexpand = lvexpand; + notes = notes; + hidden = hidden; + aliases = aliases; + sigchecker = sigchecker; + proxy = Curl.UnsetProxy }) + +let main () + let cmdline = parse_cmdline () in + + (* If debugging, echo the command line arguments. *) + debug "command line: %s\n" (String.concat " " (Array.to_list Sys.argv)); + + (* Check that the paths are existing *) + if not (Sys.file_exists cmdline.repo) then + error (f_"Repository folder '%s' doesn't exist") cmdline.repo; + + (* Create a temporary folder to work in *) + let tmpdir = Mkdtemp.temp_dir ~base_dir:cmdline.repo + "virt-builder-repository." "" in + rmdir_on_exit tmpdir; + + let tmprepo = tmpdir // "repo" in + Unix.mkdir tmprepo 0o700; + + let sigchecker = Sigchecker.create ~gpg:cmdline.gpg + ~check_signature:false + ~gpgkey:No_Key + ~tmpdir in + + let index + try + let index_filename + List.find ( + fun filename -> Sys.file_exists (cmdline.repo // filename) + ) [ "index.asc"; "index" ] in + + let downloader = Downloader.create ~curl:"do-not-use-curl" + ~cache:None ~tmpdir in + + let source = { Sources.name = "input"; + uri = cmdline.repo // index_filename; + gpgkey = No_Key; + proxy = Curl.SystemProxy; + format = Sources.FormatNative } in + + Index_parser.get_index ~downloader ~sigchecker ~template:true source + with Not_found -> [] in + + (* Check for index/interactive consistency *) + if not cmdline.interactive && index == [] then + error (f_"the repository needs to contain an index file when running in automated mode"); + + debug "Searching for images ...\n"; + + let images + let is_supported_format file + let extension = last_part_of file '.' in + match extension with + | Some ext -> + let allowed = StringSet.of_list ["qcow2"; "raw"; "img"] in + StringSet.mem ext allowed + | None -> + file <> "index" in + let files = Array.to_list (Sys.readdir cmdline.repo) in + List.filter is_supported_format files in + + debug " + %s\n" (String.concat "\n + " images); + + let outindex_path = tmprepo // "index" in + let index_channel = open_out outindex_path in + + (* Generate entries for uncompressed images *) + let written_ids + List.map ( + fun filename -> + let id, new_entry = process_image filename cmdline.repo tmprepo + index cmdline.interactive sigchecker in + + Index_parser.write_entry index_channel (id, new_entry); + id + ) images in + + (* Write the unchanged entries *) + List.iter ( + fun (id, entry) -> + let written = List.exists (fun written_id -> written_id = id) written_ids in + if not written then + let { Index.file_uri = file_uri } = entry in + if Sys.file_exists file_uri then ( + let rel_path + try + subdirectory cmdline.repo file_uri + with + | Invalid_argument _ -> + file_uri in + debug "adding unchanged entry: %s" rel_path; + let rel_entry = { entry with Index.file_uri = rel_path } in + Index_parser.write_entry index_channel (id, rel_entry); + ); + ) index; + + close_out index_channel; + + (* GPG sign the generated index *) + (match cmdline.gpgkey with + | None -> + debug "Skip index signing" + | Some gpgkey -> + debug "Signing index with GPG key %s" gpgkey; + let cmd = [ cmdline.gpg; "--armor"; + "--output"; (tmprepo // "index.gpg"); + "--export"; gpgkey ] in + if run_command cmd <> 0 then + error (f_"Failed to export GPG key %s") gpgkey; + + let cmd = [ cmdline.gpg; "--armor"; + "--default-key"; gpgkey; + "--clearsign"; (tmprepo // "index") ] in + if run_command cmd <> 0 then + error (f_"Failed to sign index"); + + (* Remove the index file since we have the signed version of it *) + if not cmdline.keep_unsigned then + Sys.remove (tmprepo // "index") + ); + + debug "Creating index backup copy"; + + List.iter ( + fun filename -> + let filepath = cmdline.repo // filename in + if Sys.file_exists filepath then + if do_mv filepath (filepath ^ ".bak") <> 0 then + error (f_"Failed to create %s backup copy") filename + ) ["index"; "index.asc"]; + + debug "Moving files to final destination"; + + Array.iter ( + fun filename -> + if do_mv (tmprepo // filename) cmdline.repo <> 0 then + error (f_"Failed to move %s in repository") (tmprepo // filename) + ) (Sys.readdir tmprepo); + + debug "Cleanup"; + + (* Remove the processed image files *) + List.iter ( + fun filename -> Sys.remove (cmdline.repo // filename) + ) images + +let () = run_main_and_handle_errors main diff --git a/builder/test-docs.sh b/builder/test-docs.sh index 83e2f4961..e65fdd3fe 100755 --- a/builder/test-docs.sh +++ b/builder/test-docs.sh @@ -23,3 +23,6 @@ $srcdir/../podcheck.pl virt-builder.pod virt-builder \ --insert $srcdir/../customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ --insert $srcdir/../customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ --ignore=--check-signatures,--no-check-signatures + +$srcdir/../podcheck.pl virt-builder-repository.pod virt-builder-repository \ + --ignore=--check-signatures,--no-check-signatures diff --git a/builder/virt-builder-repository.pod b/builder/virt-builder-repository.pod new file mode 100644 index 000000000..697deddec --- /dev/null +++ b/builder/virt-builder-repository.pod @@ -0,0 +1,183 @@ +=begin html + +<img src="virt-builder.svg" width="250" + style="float: right; clear: right;" /> + +=end html + +=head1 NAME + +virt-builder-repository - Build virt-builder source repository easily + +=head1 SYNOPSIS + + virt-builder-repository /path/to/repository + [-i|--interactive] [--gpg-key KEYID] + +=head1 DESCRIPTION + +Virt-builder is a tool for quickly building new virtual machines. It can +be configured to use template repositories. However creating and +maintaining a repository involves many tasks which can be automated. +virt-builder-repository is a tool helping to manage these repositories. + +Virt-builder-repository loops over the files in the C</path/to/repository> +folder, compresses the files with a name ending by C<qcow2>, C<raw> or +C<img>, extracts data from them and creates or updates the C<index> file. + +Some of the image-related data needed for the index file can't be +computed from the image file. virt-builder-repository first tries to +find them in the existing index file. If data are still missing after +this, they are prompted in interactive mode, otherwise an error will +be triggered. + +If a C<KEYID> is provided, the generated index file will be signed +with this GPG key. + +=head1 EXAMPLES + +=head2 Create the initial repository + +Create a folder and copy the disk image template files in it. Then +run a command like the following one: + + virt-builder-repository --gpg-key "joe@hacker.org" -i /path/to/folder + +Note that this example command runs in interactive mode. To run in +automated mode, a minimal index file needs to be created before running +the command containing sections like this one: + + [template_id] + name=template display name + file=template_filename.qcow.xz + arch=x86_64 + revision=0 + expand=/dev/sda3 + +The file value needs to match the image name extended with the ".xz" +suffix. Other optional data can be prefilled, for more informations, +see L<virt-builder(1)/Creating and signing the index file> man page. + +=head2 Update images in an existing repository + +In this use case, an new image or a new revision of an existing image +needs to be added to the repository. Place the corresponding image +template files in the repository folder. + +To update the revision of an image, the file needs to have the same +name than the existing one (without the C<xz> extension). + +As in the repository creation use case, a minimal fragment can be +added to the index file for the automated mode. This can be done +on the signed index even if it may sound a strange idea: the index +will be resigned by the tool. + +To remove an image from the repository, just remove the corresponding +compressed image file before running virt-builder-repository. + +Then running the following command will complete and update the index +file: + + virt-builder-repository --gpg-key "joe@hacker.org" -i /path/to/folder + +virt-builder-repository works in a temporary folder inside the repository +one. If anything wrong happens when running the tool, the repository is +left untouched. + +=head1 OPTIONS + +=over 4 + +=item B<--help> + +Display help. + +=item B<--gpg> GPG + +Specify an alternate L<gpg(1)> (GNU Privacy Guard) binary. You can +also use this to add gpg parameters, for example to specify an +alternate home directory: + + virt-builder-repository --gpg "gpg --homedir /tmp" [...] + +This can also be used to avoid gpg asking for the key passphrase: + + virt-builder-repository --gpg "gpg --passphrase-file /tmp/pass --batch" [...] + +=item B<-K> KEYID + +=item B<--gpg-key> KEYID + +Specify the GPG key to be used to sign the repository index file. +If not provided, the index will left unsigned. C<KEYID> is used to +identify the GPG key to use. This value is passed to gpg's +C<--default-key> option and can can thus be an email address or a +fingerprint. + +B<NOTE>: by default, virt-builder-repository searches for the key +in the user's GPG key ring. + +=item B<--keep-index> + +When using a GPG key, don't remove the unsigned index. + +=item B<-i> + +=item B<--interactive> + +Prompt for missing data. + +=item B<--colors> + +=item B<--colours> + +Use ANSI colour sequences to colourize messages. This is the default +when the output is a tty. If the output of the program is redirected +to a file, ANSI colour sequences are disabled unless you use this +option. + +=item B<-q> + +=item B<--quiet> + +Don't print ordinary progress messages. + +=item B<-v> + +=item B<--verbose> + +Enable debug messages and/or produce verbose output. + +When reporting bugs, use this option and attach the complete output to +your bug report. + +=item B<-V> + +=item B<--version> + +Display version number and exit. + +=item B<-x> + +Enable tracing of libguestfs API calls. + + +=back + +=head1 EXIT STATUS + +This program returns 0 if successful, or non-zero if there was an +error. + +=head1 SEE ALSO + +L<virt-builder(1)> +L<http://libguestfs.org/>. + +=head1 AUTHOR + +Cédric Bosdonnat L<mailto:cbosdonnat@suse.com> + +=head1 COPYRIGHT + +Copyright (C) 2016 SUSE Inc. -- 2.11.0
Richard W.M. Jones
2017-Feb-14 15:35 UTC
Re: [Libguestfs] [PATCH v3 04/10] lib/osinfo.c: Extract xml processing into a callback
On Fri, Feb 10, 2017 at 04:05:59PM +0100, Cédric Bosdonnat wrote:> In order to further reuse the osinfo database parsing in OCAML, this > commit extracts the XML processing for the distro ISOs and places it > into a newly created callback. > > This will later help other code to traverse the osinfo DB files and > let them extract what they need from them.> diff --git a/lib/osinfo.c b/lib/osinfo.c > index ea2a7659a..b77ff96b3 100644 > --- a/lib/osinfo.c > +++ b/lib/osinfo.c > @@ -43,6 +43,7 @@ > * > * XXX Currently the database is not freed when the program exits / > * library is unloaded, although we should probably do that. > + * > */An extra line has been added to this comment.> +#ifndef GUESTFS_PRIVATE > +void guestfs_int_debug (guestfs_h *g, const char *fs, ...) > +{ > + va_list args; > + > + va_start (args, fs); > + vfprintf (stderr, fs, args); > + va_end (args); > +} > + > +void > +guestfs_int_perrorf (guestfs_h *g, const char *fs, ...) > +{ > + va_list args; > + CLEANUP_FREE char *msg = NULL; > + int err; > + > + va_start (args, fs); > + err = vasprintf (&msg, fs, args); > + va_end (args); > + > + if (err < 0) return; > + > + perror(msg); > +} > +#endif /* GUESTFS_PRIVATE */Why have these functions been added in this commit? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Richard W.M. Jones
2017-Feb-14 15:39 UTC
Re: [Libguestfs] [PATCH v3 05/10] lib: extract osinfo DB traversing API
On Fri, Feb 10, 2017 at 04:06:00PM +0100, Cédric Bosdonnat wrote:> +static struct osinfo *osinfo_db = NULL; > + > +Extra blank line here.> +static int > +read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *data);Unsplit this. Only put a function name at the beginning of a line if that is the definition of the function, so you can easily find where a function is defined by grepping for ^function_name. This is a GNU coding standard; it's a shame it's not more widely followed in the open source world. https://www.gnu.org/prep/standards/standards.html#Formatting The rest of this commit seems to be code motion AFAICT so that looks fine. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Richard W.M. Jones
2017-Feb-14 15:42 UTC
Re: [Libguestfs] [PATCH v3 06/10] mllib: ocaml wrapper for lib/osinfo
On Fri, Feb 10, 2017 at 04:06:01PM +0100, Cédric Bosdonnat wrote:> Provide osinfo database parsing API in OCAML.This commit looks OK too. I reviewed up to and including this commit. I pushed commits 1, 2 and 3. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Pino Toscano
2017-Feb-17 14:38 UTC
Re: [Libguestfs] [PATCH v3 08/10] builder: add Index.write_entry function
On Friday, 10 February 2017 16:06:03 CET Cédric Bosdonnat wrote:> Add a function to properly write virt-builder source index entries. > Note that this function is very similar to Index.print_entry that is > meant for debugging purposes. > ---Mostly LGTM, with just one note:> + List.iter ( > + fun (lang, notes) -> > + let format_notes notes > + Str.global_replace (Str.regexp "^" ) " " notes inWon't this kill the newlines in the text of notes? Also, this will add one empty space at the beginning of the string. One (inefficient) approach could be: String.concat "\n " (String.nsplit "\n" notes) Thanks, -- Pino Toscano
Pino Toscano
2017-Feb-17 16:42 UTC
Re: [Libguestfs] [PATCH v3 10/10] Add a virt-builder-repository tool
On Friday, 10 February 2017 16:06:05 CET Cédric Bosdonnat wrote:> virt-builder-repository allows users to easily create or update > a virt-builder source repository out of disk images. The tool can > be run in either interactive or automated mode. > ---Still not a comprehensive review, but here there are few more notes.> diff --git a/builder/repository_main.ml b/builder/repository_main.ml > new file mode 100644 > index 000000000..89807f9d5 > --- /dev/null > +++ b/builder/repository_main.ml > @@ -0,0 +1,487 @@ > +(* virt-builder > + * Copyright (C) 2016 SUSE Inc.2017 too? (ditto for the .pod file)> +type disk_image_infos = { > + format : string; > + size : int64; > +}"disk_image_info", since "information" is singular.> + { > + gpg = gpg; > + gpgkey = gpgkey; > + interactive = interactive; > + keep_unsigned = keep_unsigned; > + repo = repo; > + }Indented too much?> +let osinfo_ids = ref None > + > +let osinfo_get_short_ids () > + match !osinfo_ids with > + | Some ids -> ids > + | None -> ( > + let ids = ref [] in > + let g = open_guestfs () in > + > + Osinfo.read_osinfo_db g#ocaml_handle ( > + fun filepath -> > + let doc = Xml.parse_file filepath in > + let xpathctx = Xml.xpath_new_context doc in > + let id = xpath_string_default xpathctx "/libosinfo/os/short-id" "" in > + if id <> "" then > + ids := id :: !ids > + ); > + g#close (); > + let ids_set = StringSet.of_list(!ids) in > + osinfo_ids := Some ids_set; > + ids_set > + )osinfo_get_short_ids does not need an intermediate list to fill, to convert as Set at the end of the reading, as a Set can be filled directly: let set = ref StringSet.empty in ... if id <> "" then set := StringSet.add id !set ... osinfo_ids := Some (!set)> +(* Move files in tmprepo into the repository *) > +let do_mv src dest > + let cmd = [ "mv"; src; dest ] in > + run_command cmdThis must check the result of run_command.> +let compress_to file outdir > + info "Copying image to temporary folder ...%!"; > + let outimg = outdir // (Filename.basename file) in > + let cmd = [ "cp" ] @ > + (if verbose () then [ "-v" ] else []) @ > + [ file; outimg ] in > + let r = run_command cmd in > + if r <> 0 then > + error (f_"cp command failed '%s'") file;Most probably the error message could be improved, e.g.: "copy of %s to %s" file outdir and added to the do_cp from dib/utils.ml, and the function moved to Common_utils.> +let get_disk_image_infos filepath > + let qemuimg_cmd = "qemu-img info --output json " ^ (quote filepath) in > + let lines = external_command qemuimg_cmd in > + let line = String.concat "\n" lines in > + let infos = yajl_tree_parse line in > + { format = object_get_string "format" infos; > + size = object_get_number "virtual-size" infos }Could this be formatted better, e.g.: { format = ...; size = ... }> +let process_image filename repo tmprepo index interactive sigchecker > + info "Preparing %s..." filename;I guess this should be "progress", have no ellipsis, and marked for translation.> + let ask message = (There's no need for ( ... ) in let foo = ... in> + printf message; > + let value = read_line () in > + match value with > + | "" -> None > + | s -> Some s > + ) in > + > + let rec ask_id () = (Ditto.> + printf (f_"Identifier: "); > + let id = read_line () in > + if not (Str.string_match (Str.regexp "[a-zA-Z0-9-_.]+") id 0) then ( > + warning (f_"Allowed characters are letters, digits, - _ and ."); > + ask_id (); > + ) else > + id; > + ) in > + > + let ask_arch () = (Ditto.> + printf (f_"Architecture. Choose one from the list below:\n"); > + let arches = ["x86_64"; "aarch64"; "armv7l"; "i686"; "ppc64"; "ppc64le"; "s390x" ] in > + iteri ( > + fun i arch -> printf " [%d] %s\n" (i + 1) arch > + ) arches; > + > + let arch = ref None in > + while !arch <> None do > + let input = read_line () in > + if input = "exit" || input = "q" || input = "quit" then > + exit 0 > + else > + try > + let i = int_of_string input in > + arch := Some (List.nth arches (i - 1)) > + with Failure _ -> arch := Some input > + done; > + match !arch with > + | None -> "" (* Should never happen *)assert false, then. Also, the while loop seems not executed at all, since arch is None and the condition checks for <> None? Also, if the condition would be inverted, the loop would be executed just once.> + let ask_osinfo () > + printf (f_ "osinfo short ID (can be found with osinfo-query os command): "); > + let value = read_line () in > + match value with > + | "" -> None > + | osinfo -> > + let osinfo_ids = osinfo_get_short_ids () in > + if not (StringSet.mem osinfo osinfo_ids) then > + warning (f_"'%s' is not a libosinfo OS id") osinfo;"'%s' is not a recognized osinfo OS id; using it anyway"> + (* Do we have an entry for that file already? *) > + let file_entry > + try > + List.hd ( > + List.filter ( > + fun (id, { Index.file_uri=file_uri }) -> > + (Filename.basename file_uri) = ((Filename.basename filename) ^ ".xz")let xzfn = Filename.basename filename ^ ".xz" in and declated outside of the List.hd.> + let id > + if id = "" && interactive then > + ask_id () > + else ( > + if id = "" then > + error (f_"Missing image identifier"); > + id > + ) inThe double id = "" check makes it a bit confusing; IMHO it can be simplified as: let id if id = "" then ( if interactive then ask_id () else error (f_"missing image identifier"); ) else id in> + (* Check that the paths are existing *) > + if not (Sys.file_exists cmdline.repo) then > + error (f_"Repository folder '%s' doesn't exist") cmdline.repo;Messages for "error" should not start with a capital letter (this applies to all the occurrencies in this tool, not just this).> + (* Create a temporary folder to work in *) > + let tmpdir = Mkdtemp.temp_dir ~base_dir:cmdline.repo > + "virt-builder-repository." "" in > + rmdir_on_exit tmpdir; > + > + let tmprepo = tmpdir // "repo" in > + Unix.mkdir tmprepo 0o700;I'd use mkdir_p to ensure all the directories are available; in this case Unix.mkdir works too, since the parent directory is created by Mkdtemp.temp_dir, but having it as safety measure won't hurt.> + let images > + let is_supported_format file > + let extension = last_part_of file '.' in > + match extension with > + | Some ext -> > + let allowed = StringSet.of_list ["qcow2"; "raw"; "img"] in > + StringSet.mem ext allowedA list in this case can be fine, since the number of items is very small -- so: List.mem ext [ "qcow2"; "raw"; "img" ]> + debug " + %s\n" (String.concat "\n + " images);List.iter (debug " + %s\n") images;> + (* Write the unchanged entries *) > + List.iter ( > + fun (id, entry) -> > + let written = List.exists (fun written_id -> written_id = id) written_ids inList.mem?> + | Some gpgkey -> > + debug "Signing index with GPG key %s" gpgkey;This could be a "progress" message.> + debug "Creating index backup copy";Ditto.> + debug "Moving files to final destination";Ditto.> diff --git a/builder/test-docs.sh b/builder/test-docs.sh > index 83e2f4961..e65fdd3fe 100755 > --- a/builder/test-docs.sh > +++ b/builder/test-docs.sh > @@ -23,3 +23,6 @@ $srcdir/../podcheck.pl virt-builder.pod virt-builder \ > --insert $srcdir/../customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \ > --insert $srcdir/../customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \ > --ignore=--check-signatures,--no-check-signatures > + > +$srcdir/../podcheck.pl virt-builder-repository.pod virt-builder-repository \ > + --ignore=--check-signatures,--no-check-signaturesThe --ignore here does not apply, since virt-builder-repository does not have these options.> diff --git a/builder/virt-builder-repository.pod b/builder/virt-builder-repository.pod > new file mode 100644 > index 000000000..697deddec > --- /dev/null > +++ b/builder/virt-builder-repository.pod > @@ -0,0 +1,183 @@ > +=begin html > + > +<img src="virt-builder.svg" width="250" > + style="float: right; clear: right;" /> > + > +=end html > + > +=head1 NAME > + > +virt-builder-repository - Build virt-builder source repository easily > + > +=head1 SYNOPSIS > + > + virt-builder-repository /path/to/repository > + [-i|--interactive] [--gpg-key KEYID] > + > +=head1 DESCRIPTION > + > +Virt-builder is a tool for quickly building new virtual machines. It can > +be configured to use template repositories. However creating and > +maintaining a repository involves many tasks which can be automated. > +virt-builder-repository is a tool helping to manage these repositories. > + > +Virt-builder-repository loops over the files in the C</path/to/repository> > +folder, compresses the files with a name ending by C<qcow2>, C<raw> or > +C<img>, extracts data from them and creates or updates the C<index> file. > + > +Some of the image-related data needed for the index file can't be > +computed from the image file. virt-builder-repository first tries to > +find them in the existing index file. If data are still missing after > +this, they are prompted in interactive mode, otherwise an error will > +be triggered. > + > +If a C<KEYID> is provided, the generated index file will be signed > +with this GPG key. > + > +=head1 EXAMPLES > + > +=head2 Create the initial repository > + > +Create a folder and copy the disk image template files in it. Then > +run a command like the following one: > + > + virt-builder-repository --gpg-key "joe@hacker.org" -i /path/to/folder > + > +Note that this example command runs in interactive mode. To run in > +automated mode, a minimal index file needs to be created before running > +the command containing sections like this one: > + > + [template_id] > + name=template display name > + file=template_filename.qcow.xz > + arch=x86_64 > + revision=0 > + expand=/dev/sda3 > + > +The file value needs to match the image name extended with the ".xz" > +suffix. Other optional data can be prefilled, for more informations, > +see L<virt-builder(1)/Creating and signing the index file> man page."man page" can be removed, since it will be an HTML link for the HTML output. Thanks, -- Pino Toscano
Possibly Parallel Threads
- [PATCH v4 2/9] lib: extract osinfo DB traversing API
- [PATCH v7 2/9] lib: extract osinfo DB traversing API
- [PATCH v6 02/10] lib: extract osinfo DB traversing API
- [PATCH v2 3/7] mllib: expose libosinfo DB reading functions in mllib
- [PATCH] inspection: Deprecate APIs and remove support for inspecting installer CDs.