Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 0/7] Introducing virt-builder-repository
Hi all,
Here is a new version of the virt-builder-repository series taking
care of Pino's comments. It has also been rebased on recent master.
Cédric Bosdonnat (7):
mllib: factorize code to add Checksum.get_checksum function
Move xml and xpath_helpers OCAML code to mllib
mllib: expose libosinfo DB reading functions in mllib
builder: rename docs test script
builder: add Index.write_entry function
builder: add a template parameter to get_index
Add a virt-builder-repository tool
.gitignore | 3 +
builder/Makefile.am | 86 +++-
builder/builder.ml | 2 +-
builder/index.mli | 3 +
builder/index_parser.ml | 72 ++-
builder/index_parser.mli | 8 +-
builder/repository_main.ml | 487 +++++++++++++++++++++
.../{test-virt-builder-docs.sh => test-docs.sh} | 3 +
builder/virt-builder-repository.pod | 183 ++++++++
docs/C_SOURCE_FILES | 2 +-
lib/Makefile.am | 2 +
lib/osinfo-iso.c | 464 ++++++++++++++++++++
lib/osinfo.c | 477 ++------------------
lib/osinfo.h | 27 ++
mllib/Makefile.am | 18 +-
mllib/checksums.ml | 25 +-
mllib/checksums.mli | 9 +
mllib/osinfo-c.c | 100 +++++
mllib/osinfo.ml | 26 ++
mllib/osinfo.mli | 31 ++
{v2v => mllib}/xml-c.c | 47 +-
{v2v => mllib}/xml.ml | 49 ++-
{v2v => mllib}/xml.mli | 3 +
{v2v => mllib}/xpath_helpers.ml | 0
{v2v => mllib}/xpath_helpers.mli | 0
v2v/Makefile.am | 17 +-
26 files changed, 1629 insertions(+), 515 deletions(-)
create mode 100644 builder/repository_main.ml
rename builder/{test-virt-builder-docs.sh => test-docs.sh} (89%)
create mode 100644 builder/virt-builder-repository.pod
create mode 100644 lib/osinfo-iso.c
create mode 100644 lib/osinfo.h
create mode 100644 mllib/osinfo-c.c
create mode 100644 mllib/osinfo.ml
create mode 100644 mllib/osinfo.mli
rename {v2v => mllib}/xml-c.c (90%)
rename {v2v => mllib}/xml.ml (79%)
rename {v2v => mllib}/xml.mli (97%)
rename {v2v => mllib}/xpath_helpers.ml (100%)
rename {v2v => mllib}/xpath_helpers.mli (100%)
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 1/7] mllib: factorize code to add Checksum.get_checksum function
Getting checksum involves the same code than verifying them. Create
a get_checksum function and use it in verify_checksum.
---
mllib/checksums.ml | 25 ++++++++++++++++---------
mllib/checksums.mli | 9 +++++++++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/mllib/checksums.ml b/mllib/checksums.ml
index 1009e131c..bee829085 100644
--- a/mllib/checksums.ml
+++ b/mllib/checksums.ml
@@ -45,14 +45,13 @@ let of_string csum_type csum_value | "sha512"
-> SHA512 csum_value
| _ -> invalid_arg csum_type
-let verify_checksum csum ?tar filename - let prog, csum_ref +let
do_compute_checksum csum ?tar filename + let prog match csum with
- | SHA1 c -> "sha1sum", c
- | SHA256 c -> "sha256sum", c
- | SHA512 c -> "sha512sum", c
+ | SHA1 _ -> "sha1sum"
+ | SHA256 _ -> "sha256sum"
+ | SHA512 _ -> "sha512sum"
in
-
let cmd match tar with
| None ->
@@ -66,9 +65,17 @@ let verify_checksum csum ?tar filename | [] ->
error (f_"%s did not return any output") prog
| line :: _ ->
- let csum_actual = fst (String.split " " line) in
- if csum_ref <> csum_actual then
- raise (Mismatched_checksum (csum, csum_actual))
+ fst (String.split " " line)
+
+let compute_checksum csum_type ?tar filename + do_compute_checksum (of_string
csum_type "") ?tar filename
+
+let verify_checksum csum ?tar filename + let csum_ref = string_of_csum csum in
+ let csum_type = string_of_csum_t csum in
+ let csum_actual = compute_checksum csum_type ?tar filename in
+ if csum_ref <> csum_actual then
+ raise (Mismatched_checksum (csum, csum_actual))
let verify_checksums checksums filename List.iter (fun c ->
verify_checksum c filename) checksums
diff --git a/mllib/checksums.mli b/mllib/checksums.mli
index 9f7041b00..5f470738a 100644
--- a/mllib/checksums.mli
+++ b/mllib/checksums.mli
@@ -43,3 +43,12 @@ val string_of_csum_t : csum_t -> string
val string_of_csum : csum_t -> string
(** Return a string representation of the checksum value. *)
+
+val compute_checksum : string -> ?tar:string -> string -> string
+(** [compute_checksum type filename] Computes the checksum of the file.
+
+ The [type] is one the possible results of the [string_of_csum_t]
+ function.
+
+ When optional [tar] is used it is path to uncompressed tar archive
+ and the [filename] is a path in the tar archive. *)
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 2/7] Move xml and xpath_helpers OCAML code to mllib
To allow other pieces of code to process XML files easily, move the
xml.ml* and xpath_helpers.ml* from v2v to mllib.
---
docs/C_SOURCE_FILES | 2 +-
mllib/Makefile.am | 9 ++++++--
{v2v => mllib}/xml-c.c | 47 +++++++++++++++++++-------------------
{v2v => mllib}/xml.ml | 49
++++++++++++++++++++++------------------
{v2v => mllib}/xml.mli | 3 +++
{v2v => mllib}/xpath_helpers.ml | 0
{v2v => mllib}/xpath_helpers.mli | 0
v2v/Makefile.am | 17 +++-----------
8 files changed, 65 insertions(+), 62 deletions(-)
rename {v2v => mllib}/xml-c.c (90%)
rename {v2v => mllib}/xml.ml (79%)
rename {v2v => mllib}/xml.mli (97%)
rename {v2v => mllib}/xpath_helpers.ml (100%)
rename {v2v => mllib}/xpath_helpers.mli (100%)
diff --git a/docs/C_SOURCE_FILES b/docs/C_SOURCE_FILES
index 286723b73..73039c927 100644
--- a/docs/C_SOURCE_FILES
+++ b/docs/C_SOURCE_FILES
@@ -335,6 +335,7 @@ mllib/progress-c.c
mllib/statvfs-c.c
mllib/uri-c.c
mllib/visit-c.c
+mllib/xml-c.c
ocaml/guestfs-c-actions.c
ocaml/guestfs-c-errnos.c
ocaml/guestfs-c.c
@@ -392,4 +393,3 @@ utils/qemu-boot/qemu-boot.c
utils/qemu-speed-test/qemu-speed-test.c
v2v/domainxml-c.c
v2v/utils-c.c
-v2v/xml-c.c
diff --git a/mllib/Makefile.am b/mllib/Makefile.am
index 78fdf0211..aa5472ade 100644
--- a/mllib/Makefile.am
+++ b/mllib/Makefile.am
@@ -28,6 +28,8 @@ EXTRA_DIST = \
test-getopt.sh
SOURCES_MLI = \
+ xml.mli \
+ xpath_helpers.mli \
checksums.mli \
common_utils.mli \
curl.mli \
@@ -67,7 +69,9 @@ SOURCES_ML = \
JSON.ml \
curl.ml \
exit.ml \
- checksums.ml
+ checksums.ml \
+ xml.ml \
+ xpath_helpers.ml
SOURCES_C = \
../common/visit/visit.c \
@@ -85,7 +89,8 @@ SOURCES_C = \
progress-c.c \
statvfs-c.c \
uri-c.c \
- visit-c.c
+ visit-c.c \
+ xml-c.c
if HAVE_OCAML
diff --git a/v2v/xml-c.c b/mllib/xml-c.c
similarity index 90%
rename from v2v/xml-c.c
rename to mllib/xml-c.c
index 0fcdbedec..d3e893076 100644
--- a/v2v/xml-c.c
+++ b/mllib/xml-c.c
@@ -1,5 +1,6 @@
-/* virt-v2v
+/* Bindings for libxml2
* Copyright (C) 2009-2017 Red Hat Inc.
+ * Copyright (C) 2017 SUSE Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -52,7 +53,7 @@ static struct custom_operations docptr_custom_operations = {
};
value
-v2v_xml_free_docptr (value docv)
+mllib_xml_free_docptr (value docv)
{
CAMLparam1 (docv);
xmlDocPtr doc = docptr_val (docv);
@@ -74,7 +75,7 @@ static struct custom_operations xpathctxptr_custom_operations
= {
};
value
-v2v_xml_free_xpathctxptr (value xpathctxv)
+mllib_xml_free_xpathctxptr (value xpathctxv)
{
CAMLparam1 (xpathctxv);
xmlXPathContextPtr xpathctx = xpathctxptr_val (xpathctxv);
@@ -96,7 +97,7 @@ static struct custom_operations xpathobjptr_custom_operations
= {
};
value
-v2v_xml_free_xpathobjptr (value xpathobjv)
+mllib_xml_free_xpathobjptr (value xpathobjv)
{
CAMLparam1 (xpathobjv);
xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv);
@@ -106,7 +107,7 @@ v2v_xml_free_xpathobjptr (value xpathobjv)
}
value
-v2v_xml_parse_memory (value xmlv)
+mllib_xml_parse_memory (value xmlv)
{
CAMLparam1 (xmlv);
CAMLlocal1 (docv);
@@ -128,7 +129,7 @@ v2v_xml_parse_memory (value xmlv)
}
value
-v2v_xml_copy_doc (value docv, value recursivev)
+mllib_xml_copy_doc (value docv, value recursivev)
{
CAMLparam2 (docv, recursivev);
CAMLlocal1 (copyv);
@@ -147,7 +148,7 @@ v2v_xml_copy_doc (value docv, value recursivev)
}
value
-v2v_xml_to_string (value docv, value formatv)
+mllib_xml_to_string (value docv, value formatv)
{
CAMLparam2 (docv, formatv);
CAMLlocal1 (strv);
@@ -166,7 +167,7 @@ v2v_xml_to_string (value docv, value formatv)
}
value
-v2v_xml_xpath_new_context (value docv)
+mllib_xml_xpath_new_context (value docv)
{
CAMLparam1 (docv);
CAMLlocal1 (xpathctxv);
@@ -186,7 +187,7 @@ v2v_xml_xpath_new_context (value docv)
}
value
-v2v_xml_xpathctxptr_register_ns (value xpathctxv, value prefix, value uri)
+mllib_xml_xpathctxptr_register_ns (value xpathctxv, value prefix, value uri)
{
CAMLparam3 (xpathctxv, prefix, uri);
xmlXPathContextPtr xpathctx;
@@ -203,7 +204,7 @@ v2v_xml_xpathctxptr_register_ns (value xpathctxv, value
prefix, value uri)
}
value
-v2v_xml_xpathctxptr_eval_expression (value xpathctxv, value exprv)
+mllib_xml_xpathctxptr_eval_expression (value xpathctxv, value exprv)
{
CAMLparam2 (xpathctxv, exprv);
CAMLlocal1 (xpathobjv);
@@ -223,7 +224,7 @@ v2v_xml_xpathctxptr_eval_expression (value xpathctxv, value
exprv)
}
value
-v2v_xml_xpathobjptr_nr_nodes (value xpathobjv)
+mllib_xml_xpathobjptr_nr_nodes (value xpathobjv)
{
CAMLparam1 (xpathobjv);
xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv);
@@ -235,7 +236,7 @@ v2v_xml_xpathobjptr_nr_nodes (value xpathobjv)
}
value
-v2v_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv)
+mllib_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv)
{
CAMLparam2 (xpathobjv, iv);
xmlXPathObjectPtr xpathobj = xpathobjptr_val (xpathobjv);
@@ -256,7 +257,7 @@ v2v_xml_xpathobjptr_get_nodeptr (value xpathobjv, value iv)
}
value
-v2v_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev)
+mllib_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev)
{
CAMLparam2 (xpathctxv, nodev);
xmlXPathContextPtr xpathctx = xpathctxptr_val (xpathctxv);
@@ -268,7 +269,7 @@ v2v_xml_xpathctx_set_nodeptr (value xpathctxv, value nodev)
}
value
-v2v_xml_nodeptr_name (value nodev)
+mllib_xml_nodeptr_name (value nodev)
{
CAMLparam1 (nodev);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -284,7 +285,7 @@ v2v_xml_nodeptr_name (value nodev)
}
value
-v2v_xml_nodeptr_as_string (value docv, value nodev)
+mllib_xml_nodeptr_as_string (value docv, value nodev)
{
CAMLparam2 (docv, nodev);
CAMLlocal1 (strv);
@@ -316,7 +317,7 @@ v2v_xml_nodeptr_as_string (value docv, value nodev)
}
value
-v2v_xml_nodeptr_set_content (value nodev, value contentv)
+mllib_xml_nodeptr_set_content (value nodev, value contentv)
{
CAMLparam2 (nodev, contentv);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -327,7 +328,7 @@ v2v_xml_nodeptr_set_content (value nodev, value contentv)
}
value
-v2v_xml_nodeptr_new_text_child (value nodev, value namev, value contentv)
+mllib_xml_nodeptr_new_text_child (value nodev, value namev, value contentv)
{
CAMLparam3 (nodev, namev, contentv);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -339,14 +340,14 @@ v2v_xml_nodeptr_new_text_child (value nodev, value namev,
value contentv)
if (new_node == NULL)
caml_invalid_argument ("nodeptr_new_text_child: failed to create new
node");
- /* See comment in v2v_xml_xpathobjptr_get_nodeptr about returning
+ /* See comment in mllib_xml_xpathobjptr_get_nodeptr about returning
* named xmlNodePtr here.
*/
CAMLreturn ((value) new_node);
}
value
-v2v_xml_nodeptr_set_prop (value nodev, value namev, value valv)
+mllib_xml_nodeptr_set_prop (value nodev, value namev, value valv)
{
CAMLparam3 (nodev, namev, valv);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -360,7 +361,7 @@ v2v_xml_nodeptr_set_prop (value nodev, value namev, value
valv)
}
value
-v2v_xml_nodeptr_unset_prop (value nodev, value namev)
+mllib_xml_nodeptr_unset_prop (value nodev, value namev)
{
CAMLparam2 (nodev, namev);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -372,7 +373,7 @@ v2v_xml_nodeptr_unset_prop (value nodev, value namev)
}
value
-v2v_xml_nodeptr_unlink_node (value nodev)
+mllib_xml_nodeptr_unlink_node (value nodev)
{
CAMLparam1 (nodev);
xmlNodePtr node = (xmlNodePtr) nodev;
@@ -384,7 +385,7 @@ v2v_xml_nodeptr_unlink_node (value nodev)
}
value
-v2v_xml_doc_get_root_element (value docv)
+mllib_xml_doc_get_root_element (value docv)
{
CAMLparam1 (docv);
CAMLlocal1 (v);
@@ -402,7 +403,7 @@ v2v_xml_doc_get_root_element (value docv)
}
value
-v2v_xml_parse_uri (value strv)
+mllib_xml_parse_uri (value strv)
{
CAMLparam1 (strv);
CAMLlocal3 (rv, sv, ov);
diff --git a/v2v/xml.ml b/mllib/xml.ml
similarity index 79%
rename from v2v/xml.ml
rename to mllib/xml.ml
index 7ed21cd30..78e75b8f2 100644
--- a/v2v/xml.ml
+++ b/mllib/xml.ml
@@ -1,5 +1,6 @@
-(* virt-v2v
+(* Bindings for libxml2
* Copyright (C) 2009-2017 Red Hat Inc.
+ * Copyright (C) 2017 SUSE Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -56,87 +57,91 @@ type node = doc * nodeptr
type xpathctx = doc * xpathctxptr
type xpathobj = xpathctx * xpathobjptr
-external free_docptr : docptr -> unit = "v2v_xml_free_docptr"
-external free_xpathctxptr : xpathctxptr -> unit =
"v2v_xml_free_xpathctxptr"
-external free_xpathobjptr : xpathobjptr -> unit =
"v2v_xml_free_xpathobjptr"
+external free_docptr : docptr -> unit = "mllib_xml_free_docptr"
+external free_xpathctxptr : xpathctxptr -> unit =
"mllib_xml_free_xpathctxptr"
+external free_xpathobjptr : xpathobjptr -> unit =
"mllib_xml_free_xpathobjptr"
-external _parse_memory : string -> docptr = "v2v_xml_parse_memory"
+external _parse_memory : string -> docptr =
"mllib_xml_parse_memory"
let parse_memory xml let docptr = _parse_memory xml in
Gc.finalise free_docptr docptr;
docptr
-external _copy_doc : docptr -> recursive:bool -> docptr =
"v2v_xml_copy_doc"
+let parse_file file + let xml = Common_utils.read_whole_file file in
+ parse_memory xml
+
+external _copy_doc : docptr -> recursive:bool -> docptr =
"mllib_xml_copy_doc"
let copy_doc docptr ~recursive let copy = _copy_doc docptr ~recursive in
Gc.finalise free_docptr copy;
copy
-external to_string : docptr -> format:bool -> string =
"v2v_xml_to_string"
+external to_string : docptr -> format:bool -> string =
"mllib_xml_to_string"
external _xpath_new_context : docptr -> xpathctxptr
- = "v2v_xml_xpath_new_context"
+ = "mllib_xml_xpath_new_context"
let xpath_new_context docptr let xpathctxptr = _xpath_new_context docptr in
Gc.finalise free_xpathctxptr xpathctxptr;
docptr, xpathctxptr
external xpathctxptr_register_ns : xpathctxptr -> string -> string ->
unit
- = "v2v_xml_xpathctxptr_register_ns"
+ = "mllib_xml_xpathctxptr_register_ns"
let xpath_register_ns (_, xpathctxptr) prefix uri xpathctxptr_register_ns
xpathctxptr prefix uri
external xpathctxptr_eval_expression : xpathctxptr -> string ->
xpathobjptr
- = "v2v_xml_xpathctxptr_eval_expression"
+ = "mllib_xml_xpathctxptr_eval_expression"
let xpath_eval_expression ((_, xpathctxptr) as xpathctx) expr let
xpathobjptr = xpathctxptr_eval_expression xpathctxptr expr in
Gc.finalise free_xpathobjptr xpathobjptr;
xpathctx, xpathobjptr
external xpathobjptr_nr_nodes : xpathobjptr -> int
- = "v2v_xml_xpathobjptr_nr_nodes"
+ = "mllib_xml_xpathobjptr_nr_nodes"
let xpathobj_nr_nodes (_, xpathobjptr) xpathobjptr_nr_nodes xpathobjptr
external xpathobjptr_get_nodeptr : xpathobjptr -> int -> nodeptr
- = "v2v_xml_xpathobjptr_get_nodeptr"
+ = "mllib_xml_xpathobjptr_get_nodeptr"
let xpathobj_node ((docptr, _), xpathobjptr) i docptr,
xpathobjptr_get_nodeptr xpathobjptr i
external xpathctxptr_set_nodeptr : xpathctxptr -> nodeptr -> unit
- = "v2v_xml_xpathctx_set_nodeptr"
+ = "mllib_xml_xpathctx_set_nodeptr"
let xpathctx_set_current_context (_, xpathctxptr) (_, nodeptr)
xpathctxptr_set_nodeptr xpathctxptr nodeptr
-external nodeptr_name : nodeptr -> string = "v2v_xml_nodeptr_name"
+external nodeptr_name : nodeptr -> string =
"mllib_xml_nodeptr_name"
let node_name (_, nodeptr) = nodeptr_name nodeptr
external nodeptr_as_string : docptr -> nodeptr -> string
- = "v2v_xml_nodeptr_as_string"
+ = "mllib_xml_nodeptr_as_string"
let node_as_string (docptr, nodeptr) = nodeptr_as_string docptr nodeptr
external nodeptr_set_content : nodeptr -> string -> unit
- = "v2v_xml_nodeptr_set_content"
+ = "mllib_xml_nodeptr_set_content"
let node_set_content (_, nodeptr) = nodeptr_set_content nodeptr
external nodeptr_new_text_child : nodeptr -> string -> string ->
nodeptr
- = "v2v_xml_nodeptr_new_text_child"
+ = "mllib_xml_nodeptr_new_text_child"
let new_text_child (docptr, nodeptr) name content docptr,
nodeptr_new_text_child nodeptr name content
external nodeptr_set_prop : nodeptr -> string -> string -> unit
- = "v2v_xml_nodeptr_set_prop"
+ = "mllib_xml_nodeptr_set_prop"
let set_prop (_, nodeptr) = nodeptr_set_prop nodeptr
external nodeptr_unset_prop : nodeptr -> string -> bool
- = "v2v_xml_nodeptr_unset_prop"
+ = "mllib_xml_nodeptr_unset_prop"
let unset_prop (_, nodeptr) = nodeptr_unset_prop nodeptr
-external nodeptr_unlink_node : nodeptr -> unit =
"v2v_xml_nodeptr_unlink_node"
+external nodeptr_unlink_node : nodeptr -> unit =
"mllib_xml_nodeptr_unlink_node"
let unlink_node (_, nodeptr) = nodeptr_unlink_node nodeptr
external _doc_get_root_element : docptr -> nodeptr option
- = "v2v_xml_doc_get_root_element"
+ = "mllib_xml_doc_get_root_element"
let doc_get_root_element docptr match _doc_get_root_element docptr with
| None -> None
@@ -154,4 +159,4 @@ type uri = {
uri_query_raw : string option;
}
-external parse_uri : string -> uri = "v2v_xml_parse_uri"
+external parse_uri : string -> uri = "mllib_xml_parse_uri"
diff --git a/v2v/xml.mli b/mllib/xml.mli
similarity index 97%
rename from v2v/xml.mli
rename to mllib/xml.mli
index 9b2bc5c18..92d4977a2 100644
--- a/v2v/xml.mli
+++ b/mllib/xml.mli
@@ -33,6 +33,9 @@ val parse_memory : string -> doc
For security reasons it actually calls xmlReadMemory with the
[XML_PARSE_NONET] option set. *)
+val parse_file : string -> doc
+(** [parse_file path] parses the file pointed by [path].*)
+
val copy_doc : doc -> recursive:bool -> doc
(** http://xmlsoft.org/html/libxml-tree.html#xmlCopyDoc *)
diff --git a/v2v/xpath_helpers.ml b/mllib/xpath_helpers.ml
similarity index 100%
rename from v2v/xpath_helpers.ml
rename to mllib/xpath_helpers.ml
diff --git a/v2v/xpath_helpers.mli b/mllib/xpath_helpers.mli
similarity index 100%
rename from v2v/xpath_helpers.mli
rename to mllib/xpath_helpers.mli
diff --git a/v2v/Makefile.am b/v2v/Makefile.am
index d62ac477e..7b8d5d1ca 100644
--- a/v2v/Makefile.am
+++ b/v2v/Makefile.am
@@ -58,14 +58,10 @@ SOURCES_MLI = \
utils.mli \
vCenter.mli \
windows.mli \
- windows_virtio.mli \
- xml.mli \
- xpath_helpers.mli
+ windows_virtio.mli
SOURCES_ML = \
types.ml \
- xml.ml \
- xpath_helpers.ml \
uefi.ml \
utils.ml \
name_from_disk.ml \
@@ -103,8 +99,7 @@ SOURCES_ML = \
SOURCES_C = \
domainxml-c.c \
- utils-c.c \
- xml-c.c
+ utils-c.c
if HAVE_OCAML
@@ -119,7 +114,6 @@ virt_v2v_CPPFLAGS = \
-I$(top_srcdir)/lib
virt_v2v_CFLAGS = \
$(WARN_CFLAGS) $(WERROR_CFLAGS) \
- $(LIBXML2_CFLAGS) \
$(LIBVIRT_CFLAGS)
BOBJECTS = \
@@ -168,8 +162,7 @@ virt_v2v_LINK = \
virt_v2v_copy_to_local_SOURCES = \
domainxml-c.c \
- utils-c.c \
- xml-c.c
+ utils-c.c
virt_v2v_copy_to_local_CPPFLAGS = \
-I. \
-I$(top_builddir) \
@@ -178,12 +171,9 @@ virt_v2v_copy_to_local_CPPFLAGS = \
-I$(top_srcdir)/lib
virt_v2v_copy_to_local_CFLAGS = \
$(WARN_CFLAGS) $(WERROR_CFLAGS) \
- $(LIBXML2_CFLAGS) \
$(LIBVIRT_CFLAGS)
COPY_TO_LOCAL_BOBJECTS = \
- xml.cmo \
- xpath_helpers.cmo \
uefi.cmo \
utils.cmo \
vCenter.cmo \
@@ -398,7 +388,6 @@ endif
v2v_unit_tests_BOBJECTS = \
types.cmo \
- xml.cmo \
uefi.cmo \
utils.cmo \
DOM.cmo \
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 3/7] mllib: expose libosinfo DB reading functions in mllib
src/osinfo.c has generic code to read the libosinfo DB. Add a wrapper
around it to expose it in mllib for use in ocaml code.
---
lib/Makefile.am | 2 +
lib/osinfo-iso.c | 464 ++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/osinfo.c | 477 ++++++------------------------------------------------
lib/osinfo.h | 27 ++++
mllib/Makefile.am | 11 +-
mllib/osinfo-c.c | 100 ++++++++++++
mllib/osinfo.ml | 26 +++
mllib/osinfo.mli | 31 ++++
8 files changed, 704 insertions(+), 434 deletions(-)
create mode 100644 lib/osinfo-iso.c
create mode 100644 lib/osinfo.h
create mode 100644 mllib/osinfo-c.c
create mode 100644 mllib/osinfo.ml
create mode 100644 mllib/osinfo.mli
diff --git a/lib/Makefile.am b/lib/Makefile.am
index e1ab1bff9..7a3580a00 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -112,7 +112,9 @@ libguestfs_la_SOURCES = \
lpj.c \
match.c \
mountable.c \
+ osinfo.h \
osinfo.c \
+ osinfo-iso.c \
private-data.c \
proto.c \
qemu.c \
diff --git a/lib/osinfo-iso.c b/lib/osinfo-iso.c
new file mode 100644
index 000000000..0c3c19971
--- /dev/null
+++ b/lib/osinfo-iso.c
@@ -0,0 +1,464 @@
+/* libguestfs
+ * Copyright (C) 2012 Red Hat Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/* Read libosinfo XML files to parse out just the
+ * os/media/iso/system-id and os/media/iso/volume-id fields, which we
+ * can then use to map install media to operating systems.
+ *
+ * Note some assumptions here:
+ *
+ * (1) We have to do some translation of the distro names and versions
+ * stored in the libosinfo files and the standard names returned by
+ * libguestfs.
+ *
+ * (2) Media detection is only part of the story. We may still need
+ * to inspect inside the image.
+ *
+ * (3) We only read the XML database files (at most) once per process,
+ * and keep them cached. They are only read at all if someone tries
+ * to inspect a CD/DVD/ISO.
+ *
+ * XXX Currently the database is not freed when the program exits /
+ * library is unloaded, although we should probably do that.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <dirent.h>
+#include <assert.h>
+#include <sys/types.h>
+#include <libintl.h>
+#include <sys/stat.h>
+
+#include <libxml/parser.h>
+#include <libxml/xpath.h>
+
+#include "ignore-value.h"
+#include "glthread/lock.h"
+#include "c-ctype.h"
+
+#include "guestfs.h"
+#include "guestfs-internal.h"
+
+#include "osinfo.h"
+
+gl_lock_define_initialized (static, osinfo_db_lock);
+static ssize_t osinfo_db_size = 0; /* 0 = unread, -1 = error, >= 1 =
#records */
+static struct osinfo *osinfo_db = NULL;
+
+
+static void free_osinfo_db_entry (struct osinfo *);
+
+#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1)
+
+static int
+read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *data);
+
+/* Given one or more fields from the header of a CD/DVD/ISO, look up
+ * the media in the libosinfo database and return our best guess for
+ * the operating system.
+ *
+ * This returns:
+ * -1 => a fatal error ('error' has been called, caller must not
ignore it)
+ * 0 => could not locate the OS
+ * 1 => matching OS found, the osinfo_ret pointer has been filled in
+ */
+int
+guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo,
+ const struct osinfo **osinfo_ret)
+{
+ size_t i;
+
+ /* We only need to lock the database when reading it for the first time. */
+ gl_lock_lock (osinfo_db_lock);
+ if (osinfo_db_size == 0) {
+ if (read_osinfo_db (g, read_osinfo_db_xml, NULL) == -1) {
+ /* Fatal error: free any database entries which have been read, and
+ * mark the database as having a permanent error.
+ */
+ if (osinfo_db_size > 0) {
+ for (i = 0; i < (size_t) osinfo_db_size; ++i)
+ free_osinfo_db_entry (&osinfo_db[i]);
+ }
+ free (osinfo_db);
+ osinfo_db = NULL;
+ osinfo_db_size = -1;
+ gl_lock_unlock (osinfo_db_lock);
+ return -1;
+ }
+ }
+ gl_lock_unlock (osinfo_db_lock);
+
+ if (osinfo_db_size <= 0)
+ return 0;
+
+ /* Look in the database to see if we can find a match. */
+ for (i = 0; i < (size_t) osinfo_db_size; ++i) {
+ if (osinfo_db[i].re_system_id) {
+ if (!isoinfo->iso_system_id ||
+ !match (g, isoinfo->iso_system_id, osinfo_db[i].re_system_id))
+ continue;
+ }
+
+ if (osinfo_db[i].re_volume_id) {
+ if (!isoinfo->iso_volume_id ||
+ !match (g, isoinfo->iso_volume_id, osinfo_db[i].re_volume_id))
+ continue;
+ }
+
+ if (osinfo_db[i].re_publisher_id) {
+ if (!isoinfo->iso_publisher_id ||
+ !match (g, isoinfo->iso_publisher_id,
osinfo_db[i].re_publisher_id))
+ continue;
+ }
+
+ if (osinfo_db[i].re_application_id) {
+ if (!isoinfo->iso_application_id ||
+ !match (g, isoinfo->iso_application_id,
osinfo_db[i].re_application_id))
+ continue;
+ }
+
+ debug (g, "osinfo: mapped disk to database entry %zu", i);
+
+ if (osinfo_ret)
+ *osinfo_ret = &osinfo_db[i];
+ return 1;
+ }
+
+ debug (g, "osinfo: no mapping found");
+
+ return 0;
+}
+
+static int read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo
*osinfo);
+static int read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
xmlNodePtr media_node, struct osinfo *osinfo);
+static int read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr
os_node, struct osinfo *osinfo);
+
+/* Read a single XML file from pathname (which is a full path).
+ * Only memory allocation failures are fatal errors here.
+ */
+static int
+read_osinfo_db_xml (guestfs_h *g, const char *pathname, void *opaque)
+{
+ CLEANUP_XMLFREEDOC xmlDocPtr doc = NULL;
+ CLEANUP_XMLXPATHFREECONTEXT xmlXPathContextPtr xpathCtx = NULL;
+ CLEANUP_XMLXPATHFREEOBJECT xmlXPathObjectPtr xpathObj = NULL;
+ xmlNodeSetPtr nodes;
+ xmlNodePtr iso_node, media_node, os_node;
+ struct osinfo *osinfo;
+ size_t i;
+
+ doc = xmlReadFile (pathname, NULL, XML_PARSE_NONET);
+ if (doc == NULL) {
+ debug (g, "osinfo: unable to parse XML file %s", pathname);
+ return 0;
+ }
+
+ xpathCtx = xmlXPathNewContext (doc);
+ if (xpathCtx == NULL) {
+ error (g, _("osinfo: unable to create new XPath context"));
+ return -1;
+ }
+
+ /* Get all <iso> nodes at any depth, then use the parent pointers in
+ * order to work back up the tree.
+ */
+ xpathObj = xmlXPathEvalExpression (BAD_CAST
"/libosinfo/os/media/iso",
+ xpathCtx);
+ if (xpathObj == NULL) {
+ error (g, _("osinfo: %s: unable to evaluate XPath expression"),
+ pathname);
+ return -1;
+ }
+
+ nodes = xpathObj->nodesetval;
+
+ if (nodes != NULL) {
+ for (i = 0; i < (size_t) nodes->nodeNr; ++i) {
+ iso_node = nodes->nodeTab[i];
+ assert (iso_node != NULL);
+ assert (STREQ ((const char *) iso_node->name, "iso"));
+ assert (iso_node->type == XML_ELEMENT_NODE);
+
+ media_node = iso_node->parent;
+ assert (media_node != NULL);
+ assert (STREQ ((const char *) media_node->name, "media"));
+ assert (media_node->type == XML_ELEMENT_NODE);
+
+ os_node = media_node->parent;
+ assert (os_node != NULL);
+ assert (STREQ ((const char *) os_node->name, "os"));
+ assert (os_node->type == XML_ELEMENT_NODE);
+
+ /* Allocate an osinfo record. */
+ osinfo_db_size++;
+ osinfo_db = safe_realloc (g, osinfo_db,
+ sizeof (struct osinfo) * osinfo_db_size);
+ osinfo = &osinfo_db[osinfo_db_size-1];
+ memset (osinfo, 0, sizeof *osinfo);
+
+ /* Read XML fields into the new osinfo record. */
+ if (read_iso_node (g, iso_node, osinfo) == -1 ||
+ read_media_node (g, xpathCtx, media_node, osinfo) == -1 ||
+ read_os_node (g, xpathCtx, os_node, osinfo) == -1) {
+ free_osinfo_db_entry (osinfo);
+ osinfo_db_size--;
+ return -1;
+ }
+
+#if 0
+ debug (g, "osinfo: %s: %s%s%s%s=> arch %s live %s installer %s
product %s type %d distro %d version %d.%d",
+ pathname,
+ osinfo->re_system_id ? "<system-id/> " :
"",
+ osinfo->re_volume_id ? "<volume-id/> " :
"",
+ osinfo->re_publisher_id ? "<publisher-id/> " :
"",
+ osinfo->re_application_id ? "<application-id/>
" : "",
+ osinfo->arch ? osinfo->arch : "(none)",
+ osinfo->is_live_disk ? "true" : "false",
+ osinfo->is_installer ? "true" : "false",
+ osinfo->product_name ? osinfo->product_name :
"(none)",
+ (int) osinfo->type, (int) osinfo->distro,
+ osinfo->major_version, osinfo->minor_version);
+#endif
+ }
+ }
+
+ return 0;
+}
+
+static int compile_re (guestfs_h *g, xmlNodePtr child, pcre **re);
+
+/* Read the regular expressions under the <iso> node. libosinfo
+ * itself uses the glib function 'g_regex_match_simple'. That appears
+ * to implement PCRE, however I have not checked in detail.
+ */
+static int
+read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo)
+{
+ xmlNodePtr child;
+
+ for (child = iso_node->children; child; child = child->next) {
+ if (STREQ ((const char *) child->name, "system-id")) {
+ if (compile_re (g, child, &osinfo->re_system_id) == -1)
+ return -1;
+ }
+ else if (STREQ ((const char *) child->name, "volume-id")) {
+ if (compile_re (g, child, &osinfo->re_volume_id) == -1)
+ return -1;
+ }
+ else if (STREQ ((const char *) child->name, "publisher-id")) {
+ if (compile_re (g, child, &osinfo->re_publisher_id) == -1)
+ return -1;
+ }
+ else if (STREQ ((const char *) child->name, "application-id"))
{
+ if (compile_re (g, child, &osinfo->re_application_id) == -1)
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+compile_re (guestfs_h *g, xmlNodePtr node, pcre **re)
+{
+ const char *err;
+ int offset;
+ CLEANUP_FREE char *content = (char *) xmlNodeGetContent (node);
+
+ if (content) {
+ *re = pcre_compile (content, 0, &err, &offset, NULL);
+ if (*re == NULL)
+ debug (g, "osinfo: could not parse regular expression '%s':
%s (ignored)",
+ content, err);
+ }
+
+ return 0;
+}
+
+/* Read the attributes of the <media/> node. */
+static int
+read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
+ xmlNodePtr media_node, struct osinfo *osinfo)
+{
+ osinfo->arch = (char *) xmlGetProp (media_node, BAD_CAST
"arch");
+
+ osinfo->is_live_disk = 0; /* If no 'live' attr, defaults to false.
*/
+ {
+ CLEANUP_XMLFREE xmlChar *content = NULL;
+ content = xmlGetProp (media_node, BAD_CAST "live");
+ if (content)
+ osinfo->is_live_disk = XMLSTREQ (content, BAD_CAST "true");
+ }
+
+ osinfo->is_installer = true; /* If no 'installer' attr, defaults
to true. */
+ {
+ CLEANUP_XMLFREE xmlChar *content = NULL;
+ content = xmlGetProp (media_node, BAD_CAST "installer");
+ if (content)
+ osinfo->is_installer = XMLSTREQ (content, BAD_CAST "true");
+ }
+
+ return 0;
+}
+
+static int parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo
*osinfo);
+static int parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo);
+static int parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo);
+
+/* Read some fields under the <os/> node. */
+static int
+read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
+ xmlNodePtr os_node, struct osinfo *osinfo)
+{
+ xmlNodePtr child;
+
+ for (child = os_node->children; child; child = child->next) {
+ if (STREQ ((const char *) child->name, "name"))
+ osinfo->product_name = (char *) xmlNodeGetContent (child);
+ else if (STREQ ((const char *) child->name, "version")) {
+ if (parse_version (g, child, osinfo) == -1)
+ return -1;
+ }
+ else if (STREQ ((const char *) child->name, "family")) {
+ if (parse_family (g, child, osinfo) == -1)
+ return -1;
+ }
+ else if (STREQ ((const char *) child->name, "distro")) {
+ if (parse_distro (g, child, osinfo) == -1)
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
+{
+ CLEANUP_FREE char *content = NULL;
+
+ content = (char *) xmlNodeGetContent (node);
+ /* We parse either "X.Y" or "X" as version strings, so
try to parse
+ * only if the first character is a digit.
+ */
+ if (content && c_isdigit (content[0])) {
+ struct version version;
+ const int res = guestfs_int_version_from_x_y_or_x (g, &version,
content);
+ if (res < 0)
+ return -1;
+ else if (res > 0) {
+ osinfo->major_version = version.v_major;
+ osinfo->minor_version = version.v_minor;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
+{
+ CLEANUP_FREE char *content = NULL;
+
+ osinfo->type = OS_TYPE_UNKNOWN;
+
+ content = (char *) xmlNodeGetContent (node);
+ if (content) {
+ if (STREQ (content, "linux"))
+ osinfo->type = OS_TYPE_LINUX;
+ else if (STRPREFIX (content, "win"))
+ osinfo->type = OS_TYPE_WINDOWS;
+ else if (STREQ (content, "freebsd"))
+ osinfo->type = OS_TYPE_FREEBSD;
+ else if (STREQ (content, "netbsd"))
+ osinfo->type = OS_TYPE_NETBSD;
+ else if (STREQ (content, "msdos"))
+ osinfo->type = OS_TYPE_DOS;
+ else if (STREQ (content, "openbsd"))
+ osinfo->type = OS_TYPE_OPENBSD;
+ else
+ debug (g, "osinfo: warning: unknown <family>
'%s'", content);
+ }
+
+ return 0;
+}
+
+static int
+parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
+{
+ CLEANUP_FREE char *content = NULL;
+
+ osinfo->distro = OS_DISTRO_UNKNOWN;
+
+ content = (char *) xmlNodeGetContent (node);
+ if (content) {
+ if (STREQ (content, "altlinux"))
+ osinfo->distro = OS_DISTRO_ALTLINUX;
+ else if (STREQ (content, "centos"))
+ osinfo->distro = OS_DISTRO_CENTOS;
+ else if (STREQ (content, "debian"))
+ osinfo->distro = OS_DISTRO_DEBIAN;
+ else if (STREQ (content, "fedora"))
+ osinfo->distro = OS_DISTRO_FEDORA;
+ else if (STREQ (content, "freebsd"))
+ osinfo->distro = OS_DISTRO_FREEBSD;
+ else if (STREQ (content, "mageia"))
+ osinfo->distro = OS_DISTRO_MAGEIA;
+ else if (STREQ (content, "mandriva"))
+ osinfo->distro = OS_DISTRO_MANDRIVA;
+ else if (STREQ (content, "netbsd"))
+ osinfo->distro = OS_DISTRO_NETBSD;
+ else if (STREQ (content, "openbsd"))
+ osinfo->distro = OS_DISTRO_OPENBSD;
+ else if (STREQ (content, "opensuse"))
+ osinfo->distro = OS_DISTRO_OPENSUSE;
+ else if (STREQ (content, "rhel"))
+ osinfo->distro = OS_DISTRO_RHEL;
+ else if (STREQ (content, "sled") || STREQ (content,
"sles"))
+ osinfo->distro = OS_DISTRO_SLES;
+ else if (STREQ (content, "ubuntu"))
+ osinfo->distro = OS_DISTRO_UBUNTU;
+ else if (STRPREFIX (content, "win"))
+ osinfo->distro = OS_DISTRO_WINDOWS;
+ else
+ debug (g, "osinfo: warning: unknown <distro>
'%s'", content);
+ }
+
+ return 0;
+}
+
+static void
+free_osinfo_db_entry (struct osinfo *osinfo)
+{
+ free (osinfo->product_name);
+ free (osinfo->arch);
+
+ if (osinfo->re_system_id)
+ pcre_free (osinfo->re_system_id);
+ if (osinfo->re_volume_id)
+ pcre_free (osinfo->re_volume_id);
+ if (osinfo->re_publisher_id)
+ pcre_free (osinfo->re_publisher_id);
+ if (osinfo->re_application_id)
+ pcre_free (osinfo->re_application_id);
+}
diff --git a/lib/osinfo.c b/lib/osinfo.c
index ea2a7659a..6b5ad6de7 100644
--- a/lib/osinfo.c
+++ b/lib/osinfo.c
@@ -30,19 +30,6 @@
* release of an OS. We can easily add support for this if it becomes
* necessary.
*
- * (3) We have to do some translation of the distro names and versions
- * stored in the libosinfo files and the standard names returned by
- * libguestfs.
- *
- * (4) Media detection is only part of the story. We may still need
- * to inspect inside the image.
- *
- * (5) We only read the XML database files (at most) once per process,
- * and keep them cached. They are only read at all if someone tries
- * to inspect a CD/DVD/ISO.
- *
- * XXX Currently the database is not freed when the program exits /
- * library is unloaded, although we should probably do that.
*/
#include <config.h>
@@ -52,95 +39,47 @@
#include <string.h>
#include <unistd.h>
#include <dirent.h>
+#include <errno.h>
#include <assert.h>
#include <sys/types.h>
#include <libintl.h>
#include <sys/stat.h>
-#include <libxml/parser.h>
-#include <libxml/xpath.h>
-
#include "ignore-value.h"
-#include "glthread/lock.h"
#include "c-ctype.h"
#include "guestfs.h"
#include "guestfs-internal.h"
-gl_lock_define_initialized (static, osinfo_db_lock);
-static ssize_t osinfo_db_size = 0; /* 0 = unread, -1 = error, >= 1 =
#records */
-static struct osinfo *osinfo_db = NULL;
-
-static int read_osinfo_db (guestfs_h *g);
-static void free_osinfo_db_entry (struct osinfo *);
+#include "osinfo.h"
-#define XMLSTREQ(a,b) (xmlStrEqual((a),(b)) == 1)
-/* Given one or more fields from the header of a CD/DVD/ISO, look up
- * the media in the libosinfo database and return our best guess for
- * the operating system.
- *
- * This returns:
- * -1 => a fatal error ('error' has been called, caller must not
ignore it)
- * 0 => could not locate the OS
- * 1 => matching OS found, the osinfo_ret pointer has been filled in
- */
-int
-guestfs_int_osinfo_map (guestfs_h *g, const struct guestfs_isoinfo *isoinfo,
- const struct osinfo **osinfo_ret)
+#ifndef GUESTFS_PRIVATE
+void guestfs_int_debug (guestfs_h *g, const char *fs, ...)
{
- size_t i;
-
- /* We only need to lock the database when reading it for the first time. */
- gl_lock_lock (osinfo_db_lock);
- if (osinfo_db_size == 0) {
- if (read_osinfo_db (g) == -1) {
- gl_lock_unlock (osinfo_db_lock);
- return -1;
- }
- }
- gl_lock_unlock (osinfo_db_lock);
-
- if (osinfo_db_size <= 0)
- return 0;
-
- /* Look in the database to see if we can find a match. */
- for (i = 0; i < (size_t) osinfo_db_size; ++i) {
- if (osinfo_db[i].re_system_id) {
- if (!isoinfo->iso_system_id ||
- !match (g, isoinfo->iso_system_id, osinfo_db[i].re_system_id))
- continue;
- }
-
- if (osinfo_db[i].re_volume_id) {
- if (!isoinfo->iso_volume_id ||
- !match (g, isoinfo->iso_volume_id, osinfo_db[i].re_volume_id))
- continue;
- }
+ va_list args;
- if (osinfo_db[i].re_publisher_id) {
- if (!isoinfo->iso_publisher_id ||
- !match (g, isoinfo->iso_publisher_id,
osinfo_db[i].re_publisher_id))
- continue;
- }
-
- if (osinfo_db[i].re_application_id) {
- if (!isoinfo->iso_application_id ||
- !match (g, isoinfo->iso_application_id,
osinfo_db[i].re_application_id))
- continue;
- }
+ va_start (args, fs);
+ vfprintf (stderr, fs, args);
+ va_end (args);
+}
- debug (g, "osinfo: mapped disk to database entry %zu", i);
+void
+guestfs_int_perrorf (guestfs_h *g, const char *fs, ...)
+{
+ va_list args;
+ CLEANUP_FREE char *msg = NULL;
+ int err;
- if (osinfo_ret)
- *osinfo_ret = &osinfo_db[i];
- return 1;
- }
+ va_start (args, fs);
+ err = vasprintf (&msg, fs, args);
+ va_end (args);
- debug (g, "osinfo: no mapping found");
+ if (err < 0) return;
- return 0;
+ perror(msg);
}
+#endif /* GUESTFS_PRIVATE */
/* Read the libosinfo XML database files. The lock is held while
* this is called.
@@ -156,19 +95,18 @@ guestfs_int_osinfo_map (guestfs_h *g, const struct
guestfs_isoinfo *isoinfo,
* Try to use the shared osinfo database layout (and location) first:
* https://gitlab.com/libosinfo/libosinfo/blob/master/docs/database-layout.txt
*/
-static int read_osinfo_db_xml (guestfs_h *g, const char *filename);
+static int read_osinfo_db_flat (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void
*opaque);
+static int read_osinfo_db_three_levels (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void
*opaque);
+static int read_osinfo_db_directory (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void
*opaque);
-static int read_osinfo_db_flat (guestfs_h *g, const char *directory);
-static int read_osinfo_db_three_levels (guestfs_h *g, const char *directory);
-static int read_osinfo_db_directory (guestfs_h *g, const char *directory);
-
-static int
-read_osinfo_db (guestfs_h *g)
+int
+read_osinfo_db (guestfs_h *g,
+ read_osinfo_db_callback callback, void *opaque)
{
int r;
- size_t i;
-
- assert (osinfo_db_size == 0);
/* (1) Try the shared osinfo directory, using either the
* $OSINFO_SYSTEM_DIR envvar or its default value.
@@ -181,59 +119,47 @@ read_osinfo_db (guestfs_h *g)
if (path == NULL)
path = "/usr/share/osinfo";
os_path = safe_asprintf (g, "%s/os", path);
- r = read_osinfo_db_three_levels (g, os_path);
+ r = read_osinfo_db_three_levels (g, os_path, callback, opaque);
}
if (r == -1)
- goto error;
+ return -1;
else if (r == 1)
return 0;
/* (2) Try the libosinfo directory, using the newer three-directory
* layout ($LIBOSINFO_DB_PATH / "os" / $group-ID / [file.xml]).
*/
- r = read_osinfo_db_three_levels (g, LIBOSINFO_DB_PATH "/os");
+ r = read_osinfo_db_three_levels (g, LIBOSINFO_DB_PATH "/os",
callback, opaque);
if (r == -1)
- goto error;
+ return -1;
else if (r == 1)
return 0;
/* (3) Try the libosinfo directory, using the old flat directory
* layout ($LIBOSINFO_DB_PATH / "oses" / [file.xml]).
*/
- r = read_osinfo_db_flat (g, LIBOSINFO_DB_PATH "/oses");
+ r = read_osinfo_db_flat (g, LIBOSINFO_DB_PATH "/oses", callback,
opaque);
if (r == -1)
- goto error;
+ return -1;
else if (r == 1)
return 0;
/* Nothing found. */
return 0;
-
- error:
- /* Fatal error: free any database entries which have been read, and
- * mark the database as having a permanent error.
- */
- if (osinfo_db_size > 0) {
- for (i = 0; i < (size_t) osinfo_db_size; ++i)
- free_osinfo_db_entry (&osinfo_db[i]);
- }
- free (osinfo_db);
- osinfo_db = NULL;
- osinfo_db_size = -1;
-
- return -1;
}
static int
-read_osinfo_db_flat (guestfs_h *g, const char *directory)
+read_osinfo_db_flat (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void *opaque)
{
debug (g, "osinfo: loading flat database from %s", directory);
- return read_osinfo_db_directory (g, directory);
+ return read_osinfo_db_directory (g, directory, callback, opaque);
}
static int
-read_osinfo_db_three_levels (guestfs_h *g, const char *directory)
+read_osinfo_db_three_levels (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void *opaque)
{
DIR *dir;
int r;
@@ -259,7 +185,7 @@ read_osinfo_db_three_levels (guestfs_h *g, const char
*directory)
/* Iterate only on directories. */
if (stat (pathname, &sb) == 0 && S_ISDIR (sb.st_mode)) {
- r = read_osinfo_db_directory (g, pathname);
+ r = read_osinfo_db_directory (g, pathname, callback, opaque);
if (r == -1)
goto error;
}
@@ -289,7 +215,8 @@ read_osinfo_db_three_levels (guestfs_h *g, const char
*directory)
}
static int
-read_osinfo_db_directory (guestfs_h *g, const char *directory)
+read_osinfo_db_directory (guestfs_h *g, const char *directory,
+ read_osinfo_db_callback callback, void *opaque)
{
DIR *dir;
int r;
@@ -311,7 +238,7 @@ read_osinfo_db_directory (guestfs_h *g, const char
*directory)
CLEANUP_FREE char *pathname = NULL;
pathname = safe_asprintf (g, "%s/%s", directory, d->d_name);
- r = read_osinfo_db_xml (g, pathname);
+ r = callback (g, pathname, opaque);
if (r == -1)
goto error;
}
@@ -339,317 +266,3 @@ read_osinfo_db_directory (guestfs_h *g, const char
*directory)
return -1;
}
-
-static int read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo
*osinfo);
-static int read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
xmlNodePtr media_node, struct osinfo *osinfo);
-static int read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx, xmlNodePtr
os_node, struct osinfo *osinfo);
-
-/* Read a single XML file from pathname (which is a full path).
- * Only memory allocation failures are fatal errors here.
- */
-static int
-read_osinfo_db_xml (guestfs_h *g, const char *pathname)
-{
- CLEANUP_XMLFREEDOC xmlDocPtr doc = NULL;
- CLEANUP_XMLXPATHFREECONTEXT xmlXPathContextPtr xpathCtx = NULL;
- CLEANUP_XMLXPATHFREEOBJECT xmlXPathObjectPtr xpathObj = NULL;
- xmlNodeSetPtr nodes;
- xmlNodePtr iso_node, media_node, os_node;
- struct osinfo *osinfo;
- size_t i;
-
- doc = xmlReadFile (pathname, NULL, XML_PARSE_NONET);
- if (doc == NULL) {
- debug (g, "osinfo: unable to parse XML file %s", pathname);
- return 0;
- }
-
- xpathCtx = xmlXPathNewContext (doc);
- if (xpathCtx == NULL) {
- error (g, _("osinfo: unable to create new XPath context"));
- return -1;
- }
-
- /* Get all <iso> nodes at any depth, then use the parent pointers in
- * order to work back up the tree.
- */
- xpathObj = xmlXPathEvalExpression (BAD_CAST
"/libosinfo/os/media/iso",
- xpathCtx);
- if (xpathObj == NULL) {
- error (g, _("osinfo: %s: unable to evaluate XPath expression"),
- pathname);
- return -1;
- }
-
- nodes = xpathObj->nodesetval;
-
- if (nodes != NULL) {
- for (i = 0; i < (size_t) nodes->nodeNr; ++i) {
- iso_node = nodes->nodeTab[i];
- assert (iso_node != NULL);
- assert (STREQ ((const char *) iso_node->name, "iso"));
- assert (iso_node->type == XML_ELEMENT_NODE);
-
- media_node = iso_node->parent;
- assert (media_node != NULL);
- assert (STREQ ((const char *) media_node->name, "media"));
- assert (media_node->type == XML_ELEMENT_NODE);
-
- os_node = media_node->parent;
- assert (os_node != NULL);
- assert (STREQ ((const char *) os_node->name, "os"));
- assert (os_node->type == XML_ELEMENT_NODE);
-
- /* Allocate an osinfo record. */
- osinfo_db_size++;
- osinfo_db = safe_realloc (g, osinfo_db,
- sizeof (struct osinfo) * osinfo_db_size);
- osinfo = &osinfo_db[osinfo_db_size-1];
- memset (osinfo, 0, sizeof *osinfo);
-
- /* Read XML fields into the new osinfo record. */
- if (read_iso_node (g, iso_node, osinfo) == -1 ||
- read_media_node (g, xpathCtx, media_node, osinfo) == -1 ||
- read_os_node (g, xpathCtx, os_node, osinfo) == -1) {
- free_osinfo_db_entry (osinfo);
- osinfo_db_size--;
- return -1;
- }
-
-#if 0
- debug (g, "osinfo: %s: %s%s%s%s=> arch %s live %s installer %s
product %s type %d distro %d version %d.%d",
- pathname,
- osinfo->re_system_id ? "<system-id/> " :
"",
- osinfo->re_volume_id ? "<volume-id/> " :
"",
- osinfo->re_publisher_id ? "<publisher-id/> " :
"",
- osinfo->re_application_id ? "<application-id/>
" : "",
- osinfo->arch ? osinfo->arch : "(none)",
- osinfo->is_live_disk ? "true" : "false",
- osinfo->is_installer ? "true" : "false",
- osinfo->product_name ? osinfo->product_name :
"(none)",
- (int) osinfo->type, (int) osinfo->distro,
- osinfo->major_version, osinfo->minor_version);
-#endif
- }
- }
-
- return 0;
-}
-
-static int compile_re (guestfs_h *g, xmlNodePtr child, pcre **re);
-
-/* Read the regular expressions under the <iso> node. libosinfo
- * itself uses the glib function 'g_regex_match_simple'. That appears
- * to implement PCRE, however I have not checked in detail.
- */
-static int
-read_iso_node (guestfs_h *g, xmlNodePtr iso_node, struct osinfo *osinfo)
-{
- xmlNodePtr child;
-
- for (child = iso_node->children; child; child = child->next) {
- if (STREQ ((const char *) child->name, "system-id")) {
- if (compile_re (g, child, &osinfo->re_system_id) == -1)
- return -1;
- }
- else if (STREQ ((const char *) child->name, "volume-id")) {
- if (compile_re (g, child, &osinfo->re_volume_id) == -1)
- return -1;
- }
- else if (STREQ ((const char *) child->name, "publisher-id")) {
- if (compile_re (g, child, &osinfo->re_publisher_id) == -1)
- return -1;
- }
- else if (STREQ ((const char *) child->name, "application-id"))
{
- if (compile_re (g, child, &osinfo->re_application_id) == -1)
- return -1;
- }
- }
-
- return 0;
-}
-
-static int
-compile_re (guestfs_h *g, xmlNodePtr node, pcre **re)
-{
- const char *err;
- int offset;
- CLEANUP_FREE char *content = (char *) xmlNodeGetContent (node);
-
- if (content) {
- *re = pcre_compile (content, 0, &err, &offset, NULL);
- if (*re == NULL)
- debug (g, "osinfo: could not parse regular expression '%s':
%s (ignored)",
- content, err);
- }
-
- return 0;
-}
-
-/* Read the attributes of the <media/> node. */
-static int
-read_media_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
- xmlNodePtr media_node, struct osinfo *osinfo)
-{
- osinfo->arch = (char *) xmlGetProp (media_node, BAD_CAST
"arch");
-
- osinfo->is_live_disk = 0; /* If no 'live' attr, defaults to false.
*/
- {
- CLEANUP_XMLFREE xmlChar *content = NULL;
- content = xmlGetProp (media_node, BAD_CAST "live");
- if (content)
- osinfo->is_live_disk = XMLSTREQ (content, BAD_CAST "true");
- }
-
- osinfo->is_installer = true; /* If no 'installer' attr, defaults
to true. */
- {
- CLEANUP_XMLFREE xmlChar *content = NULL;
- content = xmlGetProp (media_node, BAD_CAST "installer");
- if (content)
- osinfo->is_installer = XMLSTREQ (content, BAD_CAST "true");
- }
-
- return 0;
-}
-
-static int parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo
*osinfo);
-static int parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo);
-static int parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo);
-
-/* Read some fields under the <os/> node. */
-static int
-read_os_node (guestfs_h *g, xmlXPathContextPtr xpathCtx,
- xmlNodePtr os_node, struct osinfo *osinfo)
-{
- xmlNodePtr child;
-
- for (child = os_node->children; child; child = child->next) {
- if (STREQ ((const char *) child->name, "name"))
- osinfo->product_name = (char *) xmlNodeGetContent (child);
- else if (STREQ ((const char *) child->name, "version")) {
- if (parse_version (g, child, osinfo) == -1)
- return -1;
- }
- else if (STREQ ((const char *) child->name, "family")) {
- if (parse_family (g, child, osinfo) == -1)
- return -1;
- }
- else if (STREQ ((const char *) child->name, "distro")) {
- if (parse_distro (g, child, osinfo) == -1)
- return -1;
- }
- }
-
- return 0;
-}
-
-static int
-parse_version (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
-{
- CLEANUP_FREE char *content = NULL;
-
- content = (char *) xmlNodeGetContent (node);
- /* We parse either "X.Y" or "X" as version strings, so
try to parse
- * only if the first character is a digit.
- */
- if (content && c_isdigit (content[0])) {
- struct version version;
- const int res = guestfs_int_version_from_x_y_or_x (g, &version,
content);
- if (res < 0)
- return -1;
- else if (res > 0) {
- osinfo->major_version = version.v_major;
- osinfo->minor_version = version.v_minor;
- }
- }
-
- return 0;
-}
-
-static int
-parse_family (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
-{
- CLEANUP_FREE char *content = NULL;
-
- osinfo->type = OS_TYPE_UNKNOWN;
-
- content = (char *) xmlNodeGetContent (node);
- if (content) {
- if (STREQ (content, "linux"))
- osinfo->type = OS_TYPE_LINUX;
- else if (STRPREFIX (content, "win"))
- osinfo->type = OS_TYPE_WINDOWS;
- else if (STREQ (content, "freebsd"))
- osinfo->type = OS_TYPE_FREEBSD;
- else if (STREQ (content, "netbsd"))
- osinfo->type = OS_TYPE_NETBSD;
- else if (STREQ (content, "msdos"))
- osinfo->type = OS_TYPE_DOS;
- else if (STREQ (content, "openbsd"))
- osinfo->type = OS_TYPE_OPENBSD;
- else
- debug (g, "osinfo: warning: unknown <family>
'%s'", content);
- }
-
- return 0;
-}
-
-static int
-parse_distro (guestfs_h *g, xmlNodePtr node, struct osinfo *osinfo)
-{
- CLEANUP_FREE char *content = NULL;
-
- osinfo->distro = OS_DISTRO_UNKNOWN;
-
- content = (char *) xmlNodeGetContent (node);
- if (content) {
- if (STREQ (content, "altlinux"))
- osinfo->distro = OS_DISTRO_ALTLINUX;
- else if (STREQ (content, "centos"))
- osinfo->distro = OS_DISTRO_CENTOS;
- else if (STREQ (content, "debian"))
- osinfo->distro = OS_DISTRO_DEBIAN;
- else if (STREQ (content, "fedora"))
- osinfo->distro = OS_DISTRO_FEDORA;
- else if (STREQ (content, "freebsd"))
- osinfo->distro = OS_DISTRO_FREEBSD;
- else if (STREQ (content, "mageia"))
- osinfo->distro = OS_DISTRO_MAGEIA;
- else if (STREQ (content, "mandriva"))
- osinfo->distro = OS_DISTRO_MANDRIVA;
- else if (STREQ (content, "netbsd"))
- osinfo->distro = OS_DISTRO_NETBSD;
- else if (STREQ (content, "openbsd"))
- osinfo->distro = OS_DISTRO_OPENBSD;
- else if (STREQ (content, "opensuse"))
- osinfo->distro = OS_DISTRO_OPENSUSE;
- else if (STREQ (content, "rhel"))
- osinfo->distro = OS_DISTRO_RHEL;
- else if (STREQ (content, "sled") || STREQ (content,
"sles"))
- osinfo->distro = OS_DISTRO_SLES;
- else if (STREQ (content, "ubuntu"))
- osinfo->distro = OS_DISTRO_UBUNTU;
- else if (STRPREFIX (content, "win"))
- osinfo->distro = OS_DISTRO_WINDOWS;
- else
- debug (g, "osinfo: warning: unknown <distro>
'%s'", content);
- }
-
- return 0;
-}
-
-static void
-free_osinfo_db_entry (struct osinfo *osinfo)
-{
- free (osinfo->product_name);
- free (osinfo->arch);
-
- if (osinfo->re_system_id)
- pcre_free (osinfo->re_system_id);
- if (osinfo->re_volume_id)
- pcre_free (osinfo->re_volume_id);
- if (osinfo->re_publisher_id)
- pcre_free (osinfo->re_publisher_id);
- if (osinfo->re_application_id)
- pcre_free (osinfo->re_application_id);
-}
diff --git a/lib/osinfo.h b/lib/osinfo.h
new file mode 100644
index 000000000..68846747e
--- /dev/null
+++ b/lib/osinfo.h
@@ -0,0 +1,27 @@
+/* libguestfs
+ * Copyright (C) 2017 Red Hat Inc.
+ * Copyright (C) 2017 SUSE Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef OSINFO_H
+#define OSINFO_H
+
+typedef int (*read_osinfo_db_callback) (guestfs_h *g, const char *path, void
*opaque);
+
+extern int read_osinfo_db (guestfs_h *g, read_osinfo_db_callback callback, void
*opaque);
+
+#endif /* OSINFO_H */
diff --git a/mllib/Makefile.am b/mllib/Makefile.am
index aa5472ade..2541ce280 100644
--- a/mllib/Makefile.am
+++ b/mllib/Makefile.am
@@ -40,6 +40,7 @@ SOURCES_MLI = \
getopt.mli \
JSON.mli \
mkdtemp.mli \
+ osinfo.mli \
planner.mli \
progress.mli \
regedit.mli \
@@ -71,7 +72,8 @@ SOURCES_ML = \
exit.ml \
checksums.ml \
xml.ml \
- xpath_helpers.ml
+ xpath_helpers.ml \
+ osinfo.ml
SOURCES_C = \
../common/visit/visit.c \
@@ -79,6 +81,9 @@ SOURCES_C = \
../common/options/keys.c \
../common/options/uri.c \
../common/progress/progress.c \
+ ../lib/alloc.c \
+ ../lib/osinfo.c \
+ ../lib/osinfo.h \
common_utils-c.c \
dev_t-c.c \
exit-c.c \
@@ -86,6 +91,7 @@ SOURCES_C = \
fsync-c.c \
getopt-c.c \
mkdtemp-c.c \
+ osinfo-c.c \
progress-c.c \
statvfs-c.c \
uri-c.c \
@@ -119,7 +125,8 @@ libmllib_a_CPPFLAGS = \
-I$(top_srcdir)/lib \
-I$(top_srcdir)/common/visit \
-I$(top_srcdir)/common/options \
- -I$(top_srcdir)/common/progress
+ -I$(top_srcdir)/common/progress \
+ -DLIBOSINFO_DB_PATH='"$(datadir)/libosinfo/db"'
libmllib_a_CFLAGS = \
$(WARN_CFLAGS) $(WERROR_CFLAGS) \
$(LIBVIRT_CFLAGS) $(LIBXML2_CFLAGS) \
diff --git a/mllib/osinfo-c.c b/mllib/osinfo-c.c
new file mode 100644
index 000000000..2f03322d3
--- /dev/null
+++ b/mllib/osinfo-c.c
@@ -0,0 +1,100 @@
+/* Bindings for osinfo db reading function.
+ * Copyright (C) 2017 SUSE Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
USA.
+ */
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <assert.h>
+
+#include <caml/alloc.h>
+#include <caml/callback.h>
+#include <caml/fail.h>
+#include <caml/memory.h>
+#include <caml/mlvalues.h>
+
+#include "guestfs.h"
+#include "guestfs-internal.h"
+#include "osinfo.h"
+
+#pragma GCC diagnostic ignored "-Wmissing-prototypes"
+
+struct callback_wrapper_args {
+ /* In both case we are pointing to local roots, hence why these are
+ * value* not value.
+ */
+ value *exnp; /* Safe place to store any exception
+ raised by callback */
+ value *fvp; /* callback. */
+};
+
+static int read_osinfo_db_callback_wrapper (guestfs_h *g, const char *path,
void *opaque);
+
+value
+guestfs_int_mllib_read_osinfo_db (value gv, value fv)
+{
+ CAMLparam2 (gv, fv);
+ guestfs_h *g = (guestfs_h *) Int64_val (gv);
+ struct callback_wrapper_args args;
+
+ /* This stack address is used to point to the exception, if one is
+ * raised in the visitor_function. Note that the macro initializes
+ * this to Val_unit, which is how we know if an exception was set.
+ */
+ CAMLlocal1 (exn);
+
+ args.exnp = &exn;
+ args.fvp = &fv;
+
+ if (read_osinfo_db (g, read_osinfo_db_callback_wrapper, &args) == -1) {
+ if (exn != Val_unit) {
+ /* The failure was caused by the callback raising an
+ * exception. Re-raise it here.
+ */
+ caml_raise (exn);
+ }
+
+ caml_failwith ("read_osinfo_db");
+}
+
+ CAMLreturn (Val_unit);
+}
+
+static int
+read_osinfo_db_callback_wrapper (guestfs_h *g, const char *path, void *opaque)
+{
+ CAMLparam0 ();
+ CAMLlocal2 (pathv, v);
+ struct callback_wrapper_args *args = opaque;
+
+ assert (path != NULL);
+ assert (args != NULL);
+
+ pathv = caml_copy_string (path);
+
+ v = caml_callback_exn (*args->fvp, pathv);
+
+ if (Is_exception_result (v)) {
+ *args->exnp = Extract_exception (v);
+ return -1;
+ }
+
+ /* No error, return normally. */
+ CAMLreturnT (int, 0);
+}
diff --git a/mllib/osinfo.ml b/mllib/osinfo.ml
new file mode 100644
index 000000000..f5afbd889
--- /dev/null
+++ b/mllib/osinfo.ml
@@ -0,0 +1,26 @@
+(* virt-builder
+ * Copyright (C) 2016 - SUSE Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+open Common_utils
+
+type osinfo_db_callback = string -> unit
+
+external c_read_osinfo_db : int64 -> osinfo_db_callback -> unit +
"guestfs_int_mllib_read_osinfo_db"
+
+let read_osinfo_db g f + c_read_osinfo_db (Guestfs.c_pointer g) f
diff --git a/mllib/osinfo.mli b/mllib/osinfo.mli
new file mode 100644
index 000000000..d106edae2
--- /dev/null
+++ b/mllib/osinfo.mli
@@ -0,0 +1,31 @@
+(* virt-builder
+ * Copyright (C) 2016 - SUSE Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+
+(** Bindings for the src/osinfo.h libosinfo database reading API. *)
+
+type osinfo_db_callback = string -> unit
+(** The osinfo_db_callback is a callback called for each data file
+ in the libosinfo database. The argument of the function is
+ the absolute path of the data file.
+
+ The callback may raise an exception, which will cause the whole
+ database read to fail with an error (raising the same exception). *)
+
+val read_osinfo_db : Guestfs.t -> osinfo_db_callback -> unit
+(** [read_osinfo_db g callback] will find all the libosinfo database
+ files and call the callback on them. *)
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 4/7] builder: rename docs test script
Rename test-virt-builder-docs.sh into test-docs.sh to include test
for another tool's documentation.
---
builder/Makefile.am | 4 ++--
builder/{test-virt-builder-docs.sh => test-docs.sh} | 0
2 files changed, 2 insertions(+), 2 deletions(-)
rename builder/{test-virt-builder-docs.sh => test-docs.sh} (100%)
diff --git a/builder/Makefile.am b/builder/Makefile.am
index d56b394b7..218f64b4c 100644
--- a/builder/Makefile.am
+++ b/builder/Makefile.am
@@ -28,7 +28,7 @@ EXTRA_DIST = \
test-simplestreams/streams/v1/index.json \
test-simplestreams/streams/v1/net.cirros-cloud_released_download.json \
test-virt-builder.sh \
- test-virt-builder-docs.sh \
+ test-docs.sh \
test-virt-builder-list.sh \
test-virt-builder-list-simplestreams.sh \
test-virt-builder-planner.sh \
@@ -237,7 +237,7 @@ yajl_tests_LINK = \
$(yajl_tests_THEOBJECTS) -o $@
TESTS = \
- test-virt-builder-docs.sh \
+ test-docs.sh \
test-virt-builder-list.sh \
test-virt-index-validate.sh \
$(SLOW_TESTS)
diff --git a/builder/test-virt-builder-docs.sh b/builder/test-docs.sh
similarity index 100%
rename from builder/test-virt-builder-docs.sh
rename to builder/test-docs.sh
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 5/7] builder: add Index.write_entry function
Add a function to properly write virt-builder source index entries.
Note that this function is very similar to Index.print_entry that is
meant for debugging purposes.
---
builder/index.mli | 3 +++
builder/index_parser.ml | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
builder/index_parser.mli | 6 ++++++
3 files changed, 61 insertions(+)
diff --git a/builder/index.mli b/builder/index.mli
index ff5ec4a35..6202d636e 100644
--- a/builder/index.mli
+++ b/builder/index.mli
@@ -39,3 +39,6 @@ and entry = {
}
val print_entry : out_channel -> (string * entry) -> unit
+(** Debugging helper function dumping an index entry to a stream.
+ To write entries for non-debugging purpose, use the
+ [Index_parser.write_entry] function. *)
diff --git a/builder/index_parser.ml b/builder/index_parser.ml
index a3cae7d1a..eb72602aa 100644
--- a/builder/index_parser.ml
+++ b/builder/index_parser.ml
@@ -226,3 +226,55 @@ let get_index ~downloader ~sigchecker
in
get_index ()
+
+let write_entry chan (name, { Index.printable_name = printable_name;
+ file_uri = file_uri;
+ arch = arch;
+ osinfo = osinfo;
+ signature_uri = signature_uri;
+ checksums = checksums;
+ revision = revision;
+ format = format;
+ size = size;
+ compressed_size = compressed_size;
+ expand = expand;
+ lvexpand = lvexpand;
+ notes = notes;
+ aliases = aliases;
+ hidden = hidden }) + let fp fs = fprintf chan fs
in
+ fp "[%s]\n" name;
+ may (fp "name=%s\n") printable_name;
+ may (fp "osinfo=%s\n") osinfo;
+ fp "file=%s\n" file_uri;
+ fp "arch=%s\n" arch;
+ may (fp "sig=%s\n") signature_uri;
+ (match checksums with
+ | None -> ()
+ | Some checksums ->
+ List.iter (
+ fun c ->
+ fp "checksum[%s]=%s\n"
+ (Checksums.string_of_csum_t c) (Checksums.string_of_csum c)
+ ) checksums
+ );
+ fp "revision=%s\n" (string_of_revision revision);
+ may (fp "format=%s\n") format;
+ fp "size=%Ld\n" size;
+ may (fp "compressed_size=%Ld\n") compressed_size;
+ may (fp "expand=%s\n") expand;
+ may (fp "lvexpand=%s\n") lvexpand;
+ List.iter (
+ fun (lang, notes) ->
+ let format_notes notes + Str.global_replace (Str.regexp
"^" ) " " notes in
+ match lang with
+ | "" -> fp "notes=%s\n" (format_notes notes)
+ | lang -> fp "notes[%s]=%s\n" lang (format_notes notes)
+ ) notes;
+ (match aliases with
+ | None -> ()
+ | Some l -> fp "aliases=%s\n" (String.concat " " l)
+ );
+ if hidden then fp "hidden=true\n";
+ fp "\n"
diff --git a/builder/index_parser.mli b/builder/index_parser.mli
index b8d8ddf3d..7c1c423ad 100644
--- a/builder/index_parser.mli
+++ b/builder/index_parser.mli
@@ -17,3 +17,9 @@
*)
val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t ->
Sources.source -> Index.index
+(** [get_index download sigchecker source] will parse the source index file
+ into an index entry list. *)
+
+val write_entry : out_channel -> (string * Index.entry) -> unit
+(** [write_entry chan entry] writes the index entry to the chan output
+ stream.*)
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 6/7] builder: add a template parameter to get_index
get_index now gets a new template parameter. Setting it to true will
make the index parsing less picky about missing important data. This
can be used to parse a partial index file.
---
builder/builder.ml | 2 +-
builder/index_parser.ml | 20 ++++++++++++++------
builder/index_parser.mli | 2 +-
3 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/builder/builder.ml b/builder/builder.ml
index 14b42d7ce..809995013 100644
--- a/builder/builder.ml
+++ b/builder/builder.ml
@@ -208,7 +208,7 @@ let main () ~tmpdir in
match source.Sources.format with
| Sources.FormatNative ->
- Index_parser.get_index ~downloader ~sigchecker source
+ Index_parser.get_index ~downloader ~sigchecker ~template:false
source
| Sources.FormatSimpleStreams ->
Simplestreams_parser.get_index ~downloader ~sigchecker source
) sources
diff --git a/builder/index_parser.ml b/builder/index_parser.ml
index eb72602aa..152a3999e 100644
--- a/builder/index_parser.ml
+++ b/builder/index_parser.ml
@@ -24,7 +24,7 @@ open Utils
open Printf
open Unix
-let get_index ~downloader ~sigchecker
+let get_index ~downloader ~sigchecker ~template
{ Sources.uri = uri; proxy = proxy } let corrupt_file () error
(f_"The index file downloaded from '%s' is corrupt.\nYou need to
ask the supplier of this file to fix it and upload a fixed version.") uri
@@ -112,7 +112,7 @@ let get_index ~downloader ~sigchecker
let revision try Rev_int (int_of_string (List.assoc
("revision", None) fields))
with
- | Not_found -> Rev_int 1
+ | Not_found -> if template then Rev_int 0 else Rev_int 1
| Failure _ ->
eprintf (f_"%s: cannot parse 'revision' field for
'%s'\n") prog n;
corrupt_file () in
@@ -122,11 +122,19 @@ let get_index ~downloader ~sigchecker
try Int64.of_string (List.assoc ("size", None) fields)
with
| Not_found ->
- eprintf (f_"%s: no 'size' field for
'%s'\n") prog n;
- corrupt_file ()
+ if template then
+ Int64.zero
+ else (
+ eprintf (f_"%s: no 'size' field for
'%s'\n") prog n;
+ corrupt_file ()
+ )
| Failure _ ->
- eprintf (f_"%s: cannot parse 'size' field for
'%s'\n") prog n;
- corrupt_file () in
+ if template then
+ Int64.zero
+ else (
+ eprintf (f_"%s: cannot parse 'size' field for
'%s'\n") prog n;
+ corrupt_file ()
+ ) in
let compressed_size try Some (Int64.of_string
(List.assoc ("compressed_size", None) fields))
with
diff --git a/builder/index_parser.mli b/builder/index_parser.mli
index 7c1c423ad..ae757ad6f 100644
--- a/builder/index_parser.mli
+++ b/builder/index_parser.mli
@@ -16,7 +16,7 @@
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*)
-val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t ->
Sources.source -> Index.index
+val get_index : downloader:Downloader.t -> sigchecker:Sigchecker.t ->
template:bool -> Sources.source -> Index.index
(** [get_index download sigchecker source] will parse the source index file
into an index entry list. *)
--
2.11.0
Cédric Bosdonnat
2017-Feb-07 15:14 UTC
[Libguestfs] [PATCH v2 7/7] Add a virt-builder-repository tool
virt-builder-repository allows users to easily create or update
a virt-builder source repository out of disk images. The tool can
be run in either interactive or automated mode.
---
.gitignore | 3 +
builder/Makefile.am | 82 +++++-
builder/repository_main.ml | 487 ++++++++++++++++++++++++++++++++++++
builder/test-docs.sh | 3 +
builder/virt-builder-repository.pod | 183 ++++++++++++++
5 files changed, 757 insertions(+), 1 deletion(-)
create mode 100644 builder/repository_main.ml
create mode 100644 builder/virt-builder-repository.pod
diff --git a/.gitignore b/.gitignore
index dde7a529b..152c7e8ff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -93,13 +93,16 @@ Makefile.in
/builder/oUnit-*
/builder/*.qcow2
/builder/stamp-virt-builder.pod
+/builder/stamp-virt-builder-repository.pod
/builder/stamp-virt-index-validate.pod
/builder/test-config/virt-builder/repos.d/test-index.conf
/builder/test-console-*.sh
/builder/test-simplestreams/virt-builder/repos.d/cirros.conf
/builder/test-website/virt-builder/repos.d/libguestfs.conf
/builder/virt-builder
+/builder/virt-builder-repository
/builder/virt-builder.1
+/builder/virt-builder-repository.1
/builder/virt-index-validate
/builder/virt-index-validate.1
/builder/*.xz
diff --git a/builder/Makefile.am b/builder/Makefile.am
index 218f64b4c..f5a5138cd 100644
--- a/builder/Makefile.am
+++ b/builder/Makefile.am
@@ -21,6 +21,8 @@ AM_YFLAGS = -d
EXTRA_DIST = \
$(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \
+ $(REPOSITORY_SOURCES_ML) \
+ $(REPOSITORY_SOURCES_MLI) \
libguestfs.gpg \
opensuse.gpg \
test-console.sh \
@@ -38,6 +40,7 @@ EXTRA_DIST = \
test-virt-index-validate-good-2 \
test-virt-index-validate-good-3 \
virt-builder.pod \
+ virt-builder-repository.pod \
virt-index-validate.pod \
yajl_tests.ml
@@ -85,13 +88,44 @@ SOURCES_C = \
setlocale-c.c \
yajl-c.c
+REPOSITORY_SOURCES_ML = \
+ utils.ml \
+ index.ml \
+ cache.ml \
+ downloader.ml \
+ sigchecker.ml \
+ ini_reader.ml \
+ index_parser.ml \
+ yajl.ml \
+ paths.ml \
+ sources.ml \
+ repository_main.ml
+
+REPOSITORY_SOURCES_MLI = \
+ cache.mli \
+ downloader.mli \
+ index.mli \
+ index_parser.mli \
+ ini_reader.mli \
+ sigchecker.mli \
+ sources.mli \
+ yajl.mli
+
+REPOSITORY_SOURCES_C = \
+ index-scan.c \
+ index-struct.c \
+ index-parse.c \
+ index-parser-c.c \
+ yajl-c.c
+
+
man_MANS noinst_DATA bin_PROGRAMS
if HAVE_OCAML
-bin_PROGRAMS += virt-builder
+bin_PROGRAMS += virt-builder virt-builder-repository
virt_builder_SOURCES = $(SOURCES_C)
virt_builder_CPPFLAGS = \
@@ -115,6 +149,26 @@ virt_builder_CFLAGS = \
BOBJECTS = $(SOURCES_ML:.ml=.cmo)
XOBJECTS = $(BOBJECTS:.cmo=.cmx)
+virt_builder_repository_SOURCES = $(REPOSITORY_SOURCES_C)
+virt_builder_repository_CPPFLAGS = \
+ -I. \
+ -I$(top_builddir) \
+ -I$(top_srcdir)/gnulib/lib -I$(top_builddir)/gnulib/lib \
+ -I$(shell $(OCAMLC) -where) \
+ -I$(top_srcdir)/gnulib/lib \
+ -I$(top_srcdir)/src \
+ -I$(top_srcdir)/fish
+virt_builder_repository_CFLAGS = \
+ -pthread \
+ $(WARN_CFLAGS) $(WERROR_CFLAGS) \
+ -Wno-unused-macros \
+ $(LIBLZMA_CFLAGS) \
+ $(LIBTINFO_CFLAGS) \
+ $(LIBXML2_CFLAGS) \
+ $(YAJL_CFLAGS)
+REPOSITORY_BOBJECTS = $(REPOSITORY_SOURCES_ML:.ml=.cmo)
+REPOSITORY_XOBJECTS = $(REPOSITORY_BOBJECTS:.cmo=.cmx)
+
# -I $(top_builddir)/lib/.libs is a hack which forces corresponding -L
# option to be passed to gcc, so we don't try linking against an
# installed copy of libguestfs.
@@ -149,8 +203,10 @@ OCAMLFLAGS = $(OCAML_FLAGS) $(OCAML_WARN_ERROR)
if !HAVE_OCAMLOPT
OBJECTS = $(BOBJECTS)
+REPOSITORY_OBJECTS = $(REPOSITORY_BOBJECTS)
else
OBJECTS = $(XOBJECTS)
+REPOSITORY_OBJECTS = $(REPOSITORY_XOBJECTS)
endif
OCAMLLINKFLAGS = mlguestfs.$(MLARCHIVE) mllib.$(MLARCHIVE)
customize.$(MLARCHIVE) $(LINK_CUSTOM_OCAMLC_ONLY)
@@ -165,6 +221,16 @@ virt_builder_LINK = \
$(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \
$(OBJECTS) -o $@
+virt_builder_repository_DEPENDENCIES = \
+ $(REPOSITORY_OBJECTS) \
+ ../mllib/mllib.$(MLARCHIVE) \
+ ../customize/customize.$(MLARCHIVE) \
+ $(top_srcdir)/ocaml-link.sh
+virt_builder_repository_LINK = \
+ $(top_srcdir)/ocaml-link.sh -cclib '$(OCAMLCLIBS)' -- \
+ $(OCAMLFIND) $(BEST) $(OCAMLFLAGS) $(OCAMLPACKAGES) $(OCAMLLINKFLAGS) \
+ $(REPOSITORY_OBJECTS) -o $@
+
# Manual pages and HTML files for the website.
man_MANS += virt-builder.1
@@ -183,6 +249,20 @@ stamp-virt-builder.pod: virt-builder.pod
$(top_srcdir)/customize/customize-synop
$<
touch $@
+man_MANS += virt-builder-repository.1
+noinst_DATA += $(top_builddir)/website/virt-builder-repository.1.html
+
+virt-builder-repository.1
$(top_builddir)/website/virt-builder-repository.1.html:
stamp-virt-builder-repository.pod
+
+stamp-virt-builder-repository.pod: virt-builder-repository.pod
+ $(PODWRAPPER) \
+ --man virt-builder-repository.1 \
+ --html $(top_builddir)/website/virt-builder-repository.1.html \
+ --license GPLv2+ \
+ --warning safe \
+ $<
+ touch $@
+
# Tests.
TESTS_ENVIRONMENT = $(top_builddir)/run --test
diff --git a/builder/repository_main.ml b/builder/repository_main.ml
new file mode 100644
index 000000000..1af54cc1b
--- /dev/null
+++ b/builder/repository_main.ml
@@ -0,0 +1,487 @@
+(* virt-builder
+ * Copyright (C) 2016 SUSE Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ *)
+
+open Common_gettext.Gettext
+open Common_utils
+open Getopt.OptionName
+open Utils
+open Yajl
+open Xpath_helpers
+
+open Printf
+
+module StringSet = Set.Make(String)
+
+type cmdline = {
+ gpg : string;
+ gpgkey : string option;
+ interactive : bool;
+ keep_unsigned : bool;
+ repo : string;
+}
+
+type disk_image_infos = {
+ format : string;
+ size : int64;
+}
+
+let parse_cmdline () + let gpg = ref "gpg" in
+ let gpgkey = ref "" in
+ let interactive = ref false in
+ let keep_unsigned = ref false in
+
+ let argspec = [
+ [ L"gpg" ], Getopt.Set_string ("gpg", gpg), s_"Set
GPG binary/command";
+ [ S 'K'; L"gpg-key" ], Getopt.Set_string
("gpgkey", gpgkey),
+ s_"ID of the GPG key to sign the repo with";
+ [ S 'i'; L"interactive" ], Getopt.Set interactive,
s_"Ask the user about missing data";
+ [ L"keep-index" ], Getopt.Set keep_unsigned, s_"Keep
unsigned index";
+ ] in
+
+ let args = ref [] in
+ let anon_fun s = push_front s args in
+ let usage_msg + sprintf (f_"\
+%s: create a repository for virt-builder
+
+ virt-builder-repository REPOSITORY_PATH
+
+A short summary of the options is given below. For detailed help please
+read the man page virt-builder-repository(1).
+")
+ prog in
+ let opthandle = create_standard_options argspec ~anon_fun usage_msg in
+ Getopt.parse opthandle;
+
+ (* Dereference options. *)
+ let args = List.rev !args in
+ let gpg = !gpg in
+ let gpgkey = match !gpgkey with "" -> None | s -> Some s in
+ let interactive = !interactive in
+ let keep_unsigned = !keep_unsigned in
+
+ (* Check options *)
+ let repo + (match args with
+ | [repo] -> repo
+ | [] ->
+ error (f_"virt-builder-repository /path/to/repo\nUse
'/path/to/repo' to point to the repository folder.")
+ | _ ->
+ error (f_"too many parameters, only one path to repository is
allowed")
+ ) in
+
+ {
+ gpg = gpg;
+ gpgkey = gpgkey;
+ interactive = interactive;
+ keep_unsigned = keep_unsigned;
+ repo = repo;
+ }
+
+let increment_revision revision + match revision with
+ | Utils.Rev_int n -> Utils.Rev_int (n + 1)
+ | Utils.Rev_string s ->
+ if Str.string_match (Str.regexp "^\\(.*[-._]\\)\\([0-9]+\\)$") s
0 then
+ let prefix = Str.matched_group 1 s in
+ let suffix = int_of_string (Str.matched_group 2 s) in
+ Utils.Rev_string (prefix ^ (string_of_int (suffix + 1)))
+ else
+ Utils.Rev_string (s ^ ".1")
+
+let osinfo_ids = ref None
+
+let osinfo_get_short_ids () + match !osinfo_ids with
+ | Some ids -> ids
+ | None -> (
+ let ids = ref [] in
+ let g = open_guestfs () in
+
+ Osinfo.read_osinfo_db g#ocaml_handle (
+ fun filepath ->
+ let doc = Xml.parse_file filepath in
+ let xpathctx = Xml.xpath_new_context doc in
+ let id = xpath_string_default xpathctx
"/libosinfo/os/short-id" "" in
+ if id <> "" then
+ ids := id :: !ids
+ );
+ g#close ();
+ let ids_set = StringSet.of_list(!ids) in
+ osinfo_ids := Some ids_set;
+ ids_set
+ )
+
+(* Move files in tmprepo into the repository *)
+let do_mv src dest + let cmd = [ "mv"; src; dest ] in
+ run_command cmd
+
+let compress_to file outdir + info "Copying image to temporary folder
...%!";
+ let outimg = outdir // (Filename.basename file) in
+ let cmd = [ "cp" ] @
+ (if verbose () then [ "-v" ] else []) @
+ [ file; outimg ] in
+ let r = run_command cmd in
+ if r <> 0 then
+ error (f_"cp command failed '%s'") file;
+
+ info "Compressing ...%!";
+ let cmd = [ "xz"; "-f"; "--best";
+ "--block-size=16777216"; outimg ] in
+ if run_command cmd <> 0 then
+ exit 1;
+ outimg ^ ".xz"
+
+let get_disk_image_infos filepath + let qemuimg_cmd = "qemu-img info
--output json " ^ (quote filepath) in
+ let lines = external_command qemuimg_cmd in
+ let line = String.concat "\n" lines in
+ let infos = yajl_tree_parse line in
+ { format = object_get_string "format" infos;
+ size = object_get_number "virtual-size" infos }
+
+let process_image filename repo tmprepo index interactive sigchecker + info
"Preparing %s..." filename;
+
+ let filepath = repo // filename in
+ let { format = format; size = size } = get_disk_image_infos filepath in
+ let xz_path = compress_to filepath tmprepo in
+ let checksum = Checksums.compute_checksum "sha512" xz_path in
+ let compressed_size = (Unix.stat xz_path).Unix.st_size in
+
+ let ask message = (
+ printf message;
+ let value = read_line () in
+ match value with
+ | "" -> None
+ | s -> Some s
+ ) in
+
+ let rec ask_id () = (
+ printf (f_"Identifier: ");
+ let id = read_line () in
+ if not (Str.string_match (Str.regexp "[a-zA-Z0-9-_.]+") id 0)
then (
+ warning (f_"Allowed characters are letters, digits, - _ and
.");
+ ask_id ();
+ ) else
+ id;
+ ) in
+
+ let ask_arch () = (
+ printf (f_"Architecture. Choose one from the list below:\n");
+ let arches = ["x86_64"; "aarch64"; "armv7l";
"i686"; "ppc64"; "ppc64le"; "s390x" ] in
+ iteri (
+ fun i arch -> printf " [%d] %s\n" (i + 1) arch
+ ) arches;
+
+ let arch = ref None in
+ while !arch <> None do
+ let input = read_line () in
+ if input = "exit" || input = "q" || input =
"quit" then
+ exit 0
+ else
+ try
+ let i = int_of_string input in
+ arch := Some (List.nth arches (i - 1))
+ with Failure _ -> arch := Some input
+ done;
+ match !arch with
+ | None -> "" (* Should never happen *)
+ | Some arch -> arch
+ ) in
+
+ let ask_osinfo () + printf (f_ "osinfo short ID (can be found with
osinfo-query os command): ");
+ let value = read_line () in
+ match value with
+ | "" -> None
+ | osinfo ->
+ let osinfo_ids = osinfo_get_short_ids () in
+ if not (StringSet.mem osinfo osinfo_ids) then
+ warning (f_"'%s' is not a libosinfo OS id") osinfo;
+ Some osinfo in
+
+ (* Do we have an entry for that file already? *)
+ let file_entry + try
+ List.hd (
+ List.filter (
+ fun (id, { Index.file_uri=file_uri }) ->
+ (Filename.basename file_uri) = ((Filename.basename filename) ^
".xz")
+ ) index
+ )
+ with
+ | Failure _ ->
+ let entry = { Index.printable_name = None;
+ osinfo = None;
+ file_uri = "";
+ arch = "";
+ signature_uri = None;
+ checksums = None;
+ revision = Utils.Rev_int 0;
+ format = Some format;
+ size = size;
+ compressed_size = Some (Int64.of_int compressed_size);
+ expand = None;
+ lvexpand = None;
+ notes = [];
+ hidden = false;
+ aliases = None;
+ sigchecker = sigchecker;
+ proxy = Curl.UnsetProxy } in
+ ("", entry) in
+
+ let id, { Index.printable_name = printable_name;
+ osinfo = osinfo;
+ arch = arch;
+ checksums = checksums;
+ revision = revision;
+ expand = expand;
+ lvexpand = lvexpand;
+ notes = notes;
+ hidden = hidden;
+ aliases = aliases } = file_entry in
+
+ let old_checksum + match checksums with
+ | Some csums -> (
+ try
+ let csum = List.find (
+ fun c ->
+ match c with
+ | Checksums.SHA512 _ -> true
+ | _ -> false
+ ) csums in
+ Checksums.string_of_csum csum
+ with
+ | _ -> ""
+ )
+ | None -> "" in
+
+ let id + if id = "" && interactive then
+ ask_id ()
+ else (
+ if id = "" then
+ error (f_"Missing image identifier");
+ id
+ ) in
+
+ let printable_name + if printable_name = None && interactive
then
+ ask (f_"Display name: ")
+ else
+ printable_name in
+
+ let arch + if arch = "" then (
+ if interactive then ask_arch ()
+ else error (f_"missing architecture for %s") id;
+ ) else arch in
+
+ let osinfo + if osinfo = None && interactive then
+ ask_osinfo ()
+ else
+ osinfo in
+
+ let expand + if expand = None && interactive then
+ ask (f_"Expandable partition: ")
+ else
+ expand in
+
+ let lvexpand + if lvexpand = None && interactive then
+ ask (f_"Expandable volume: ")
+ else
+ lvexpand in
+
+ let revision + if old_checksum <> checksum then
+ increment_revision revision
+ else
+ revision in
+
+ (id, { Index.printable_name = printable_name;
+ osinfo = osinfo;
+ file_uri = Filename.basename xz_path;
+ arch = arch;
+ signature_uri = None;
+ checksums = Some [(Checksums.SHA512 checksum)];
+ revision = revision;
+ format = Some format;
+ size = size;
+ compressed_size = Some (Int64.of_int compressed_size);
+ expand = expand;
+ lvexpand = lvexpand;
+ notes = notes;
+ hidden = hidden;
+ aliases = aliases;
+ sigchecker = sigchecker;
+ proxy = Curl.UnsetProxy })
+
+let main () + let cmdline = parse_cmdline () in
+
+ (* If debugging, echo the command line arguments. *)
+ debug "command line: %s\n" (String.concat " "
(Array.to_list Sys.argv));
+
+ (* Check that the paths are existing *)
+ if not (Sys.file_exists cmdline.repo) then
+ error (f_"Repository folder '%s' doesn't exist")
cmdline.repo;
+
+ (* Create a temporary folder to work in *)
+ let tmpdir = Mkdtemp.temp_dir ~base_dir:cmdline.repo
+ "virt-builder-repository."
"" in
+ rmdir_on_exit tmpdir;
+
+ let tmprepo = tmpdir // "repo" in
+ Unix.mkdir tmprepo 0o700;
+
+ let sigchecker = Sigchecker.create ~gpg:cmdline.gpg
+ ~check_signature:false
+ ~gpgkey:No_Key
+ ~tmpdir in
+
+ let index + try
+ let index_filename + List.find (
+ fun filename -> Sys.file_exists (cmdline.repo // filename)
+ ) [ "index.asc"; "index" ] in
+
+ let downloader = Downloader.create ~curl:"do-not-use-curl"
+ ~cache:None ~tmpdir in
+
+ let source = { Sources.name = "input";
+ uri = cmdline.repo // index_filename;
+ gpgkey = No_Key;
+ proxy = Curl.SystemProxy;
+ format = Sources.FormatNative } in
+
+ Index_parser.get_index ~downloader ~sigchecker ~template:true source
+ with Not_found -> [] in
+
+ (* Check for index/interactive consistency *)
+ if not cmdline.interactive && index == [] then
+ error (f_"the repository needs to contain an index file when running
in automated mode");
+
+ debug "Searching for images ...\n";
+
+ let images + let is_supported_format file + let extension =
last_part_of file '.' in
+ match extension with
+ | Some ext ->
+ let allowed = StringSet.of_list ["qcow2"; "raw";
"img"] in
+ StringSet.mem ext allowed
+ | None ->
+ file <> "index" in
+ let files = Array.to_list (Sys.readdir cmdline.repo) in
+ List.filter is_supported_format files in
+
+ debug " + %s\n" (String.concat "\n + " images);
+
+ let outindex_path = tmprepo // "index" in
+ let index_channel = open_out outindex_path in
+
+ (* Generate entries for uncompressed images *)
+ let written_ids + List.map (
+ fun filename ->
+ let id, new_entry = process_image filename cmdline.repo tmprepo
+ index cmdline.interactive sigchecker in
+
+ Index_parser.write_entry index_channel (id, new_entry);
+ id
+ ) images in
+
+ (* Write the unchanged entries *)
+ List.iter (
+ fun (id, entry) ->
+ let written = List.exists (fun written_id -> written_id = id)
written_ids in
+ if not written then
+ let { Index.file_uri = file_uri } = entry in
+ if Sys.file_exists file_uri then (
+ let rel_path + try
+ subdirectory cmdline.repo file_uri
+ with
+ | Invalid_argument _ ->
+ file_uri in
+ debug "adding unchanged entry: %s" rel_path;
+ let rel_entry = { entry with Index.file_uri = rel_path } in
+ Index_parser.write_entry index_channel (id, rel_entry);
+ );
+ ) index;
+
+ close_out index_channel;
+
+ (* GPG sign the generated index *)
+ (match cmdline.gpgkey with
+ | None ->
+ debug "Skip index signing"
+ | Some gpgkey ->
+ debug "Signing index with GPG key %s" gpgkey;
+ let cmd = [ cmdline.gpg; "--armor";
+ "--output"; (tmprepo // "index.gpg");
+ "--export"; gpgkey ] in
+ if run_command cmd <> 0 then
+ error (f_"Failed to export GPG key %s") gpgkey;
+
+ let cmd = [ cmdline.gpg; "--armor";
+ "--default-key"; gpgkey;
+ "--clearsign"; (tmprepo // "index") ] in
+ if run_command cmd <> 0 then
+ error (f_"Failed to sign index");
+
+ (* Remove the index file since we have the signed version of it *)
+ if not cmdline.keep_unsigned then
+ Sys.remove (tmprepo // "index")
+ );
+
+ debug "Creating index backup copy";
+
+ List.iter (
+ fun filename ->
+ let filepath = cmdline.repo // filename in
+ if Sys.file_exists filepath then
+ if do_mv filepath (filepath ^ ".bak") <> 0 then
+ error (f_"Failed to create %s backup copy") filename
+ ) ["index"; "index.asc"];
+
+ debug "Moving files to final destination";
+
+ Array.iter (
+ fun filename ->
+ if do_mv (tmprepo // filename) cmdline.repo <> 0 then
+ error (f_"Failed to move %s in repository") (tmprepo //
filename)
+ ) (Sys.readdir tmprepo);
+
+ debug "Cleanup";
+
+ (* Remove the processed image files *)
+ List.iter (
+ fun filename -> Sys.remove (cmdline.repo // filename)
+ ) images
+
+let () = run_main_and_handle_errors main
diff --git a/builder/test-docs.sh b/builder/test-docs.sh
index 83e2f4961..e65fdd3fe 100755
--- a/builder/test-docs.sh
+++ b/builder/test-docs.sh
@@ -23,3 +23,6 @@ $srcdir/../podcheck.pl virt-builder.pod virt-builder \
--insert $srcdir/../customize/customize-synopsis.pod:__CUSTOMIZE_SYNOPSIS__ \
--insert $srcdir/../customize/customize-options.pod:__CUSTOMIZE_OPTIONS__ \
--ignore=--check-signatures,--no-check-signatures
+
+$srcdir/../podcheck.pl virt-builder-repository.pod virt-builder-repository \
+ --ignore=--check-signatures,--no-check-signatures
diff --git a/builder/virt-builder-repository.pod
b/builder/virt-builder-repository.pod
new file mode 100644
index 000000000..697deddec
--- /dev/null
+++ b/builder/virt-builder-repository.pod
@@ -0,0 +1,183 @@
+=begin html
+
+<img src="virt-builder.svg" width="250"
+ style="float: right; clear: right;" />
+
+=end html
+
+=head1 NAME
+
+virt-builder-repository - Build virt-builder source repository easily
+
+=head1 SYNOPSIS
+
+ virt-builder-repository /path/to/repository
+ [-i|--interactive] [--gpg-key KEYID]
+
+=head1 DESCRIPTION
+
+Virt-builder is a tool for quickly building new virtual machines. It can
+be configured to use template repositories. However creating and
+maintaining a repository involves many tasks which can be automated.
+virt-builder-repository is a tool helping to manage these repositories.
+
+Virt-builder-repository loops over the files in the
C</path/to/repository>
+folder, compresses the files with a name ending by C<qcow2>, C<raw>
or
+C<img>, extracts data from them and creates or updates the C<index>
file.
+
+Some of the image-related data needed for the index file can't be
+computed from the image file. virt-builder-repository first tries to
+find them in the existing index file. If data are still missing after
+this, they are prompted in interactive mode, otherwise an error will
+be triggered.
+
+If a C<KEYID> is provided, the generated index file will be signed
+with this GPG key.
+
+=head1 EXAMPLES
+
+=head2 Create the initial repository
+
+Create a folder and copy the disk image template files in it. Then
+run a command like the following one:
+
+ virt-builder-repository --gpg-key "joe@hacker.org" -i
/path/to/folder
+
+Note that this example command runs in interactive mode. To run in
+automated mode, a minimal index file needs to be created before running
+the command containing sections like this one:
+
+ [template_id]
+ name=template display name
+ file=template_filename.qcow.xz
+ arch=x86_64
+ revision=0
+ expand=/dev/sda3
+
+The file value needs to match the image name extended with the ".xz"
+suffix. Other optional data can be prefilled, for more informations,
+see L<virt-builder(1)/Creating and signing the index file> man page.
+
+=head2 Update images in an existing repository
+
+In this use case, an new image or a new revision of an existing image
+needs to be added to the repository. Place the corresponding image
+template files in the repository folder.
+
+To update the revision of an image, the file needs to have the same
+name than the existing one (without the C<xz> extension).
+
+As in the repository creation use case, a minimal fragment can be
+added to the index file for the automated mode. This can be done
+on the signed index even if it may sound a strange idea: the index
+will be resigned by the tool.
+
+To remove an image from the repository, just remove the corresponding
+compressed image file before running virt-builder-repository.
+
+Then running the following command will complete and update the index
+file:
+
+ virt-builder-repository --gpg-key "joe@hacker.org" -i
/path/to/folder
+
+virt-builder-repository works in a temporary folder inside the repository
+one. If anything wrong happens when running the tool, the repository is
+left untouched.
+
+=head1 OPTIONS
+
+=over 4
+
+=item B<--help>
+
+Display help.
+
+=item B<--gpg> GPG
+
+Specify an alternate L<gpg(1)> (GNU Privacy Guard) binary. You can
+also use this to add gpg parameters, for example to specify an
+alternate home directory:
+
+ virt-builder-repository --gpg "gpg --homedir /tmp" [...]
+
+This can also be used to avoid gpg asking for the key passphrase:
+
+ virt-builder-repository --gpg "gpg --passphrase-file /tmp/pass
--batch" [...]
+
+=item B<-K> KEYID
+
+=item B<--gpg-key> KEYID
+
+Specify the GPG key to be used to sign the repository index file.
+If not provided, the index will left unsigned. C<KEYID> is used to
+identify the GPG key to use. This value is passed to gpg's
+C<--default-key> option and can can thus be an email address or a
+fingerprint.
+
+B<NOTE>: by default, virt-builder-repository searches for the key
+in the user's GPG key ring.
+
+=item B<--keep-index>
+
+When using a GPG key, don't remove the unsigned index.
+
+=item B<-i>
+
+=item B<--interactive>
+
+Prompt for missing data.
+
+=item B<--colors>
+
+=item B<--colours>
+
+Use ANSI colour sequences to colourize messages. This is the default
+when the output is a tty. If the output of the program is redirected
+to a file, ANSI colour sequences are disabled unless you use this
+option.
+
+=item B<-q>
+
+=item B<--quiet>
+
+Don't print ordinary progress messages.
+
+=item B<-v>
+
+=item B<--verbose>
+
+Enable debug messages and/or produce verbose output.
+
+When reporting bugs, use this option and attach the complete output to
+your bug report.
+
+=item B<-V>
+
+=item B<--version>
+
+Display version number and exit.
+
+=item B<-x>
+
+Enable tracing of libguestfs API calls.
+
+
+=back
+
+=head1 EXIT STATUS
+
+This program returns 0 if successful, or non-zero if there was an
+error.
+
+=head1 SEE ALSO
+
+L<virt-builder(1)>
+L<http://libguestfs.org/>.
+
+=head1 AUTHOR
+
+Cédric Bosdonnat L<mailto:cbosdonnat@suse.com>
+
+=head1 COPYRIGHT
+
+Copyright (C) 2016 SUSE Inc.
--
2.11.0
Richard W.M. Jones
2017-Feb-07 16:13 UTC
Re: [Libguestfs] [PATCH v2 1/7] mllib: factorize code to add Checksum.get_checksum function
On Tue, Feb 07, 2017 at 04:14:16PM +0100, Cédric Bosdonnat wrote:> Getting checksum involves the same code than verifying them. Create > a get_checksum function and use it in verify_checksum. > --- > mllib/checksums.ml | 25 ++++++++++++++++--------- > mllib/checksums.mli | 9 +++++++++ > 2 files changed, 25 insertions(+), 9 deletions(-) > > diff --git a/mllib/checksums.ml b/mllib/checksums.ml > index 1009e131c..bee829085 100644 > --- a/mllib/checksums.ml > +++ b/mllib/checksums.ml > @@ -45,14 +45,13 @@ let of_string csum_type csum_value > | "sha512" -> SHA512 csum_value > | _ -> invalid_arg csum_type > > -let verify_checksum csum ?tar filename > - let prog, csum_ref > +let do_compute_checksum csum ?tar filename > + let prog > match csum with > - | SHA1 c -> "sha1sum", c > - | SHA256 c -> "sha256sum", c > - | SHA512 c -> "sha512sum", c > + | SHA1 _ -> "sha1sum" > + | SHA256 _ -> "sha256sum" > + | SHA512 _ -> "sha512sum" > in > - > let cmd > match tar with > | None -> > @@ -66,9 +65,17 @@ let verify_checksum csum ?tar filename > | [] -> > error (f_"%s did not return any output") prog > | line :: _ -> > - let csum_actual = fst (String.split " " line) in > - if csum_ref <> csum_actual then > - raise (Mismatched_checksum (csum, csum_actual)) > + fst (String.split " " line) > + > +let compute_checksum csum_type ?tar filename > + do_compute_checksum (of_string csum_type "") ?tar filenameWhy is there a separate do_compute_checksum function here? It seems like it is only called from here, not from anywhere else.> +let verify_checksum csum ?tar filename > + let csum_ref = string_of_csum csum in > + let csum_type = string_of_csum_t csum in > + let csum_actual = compute_checksum csum_type ?tar filename in > + if csum_ref <> csum_actual then > + raise (Mismatched_checksum (csum, csum_actual))This is technically correct because all the checksum types happen to be different lengths, but really wrong. You should be comparing the structured csum_t values, not just the strings. Unfortunately the original verify_checksum is also broken in the same way so this isn't a bug you've added, but it's a bug that needs to be fixed. I think the real problem is that compute_checksum should return a structured csum_t, not a string, and then everything else follows elegantly from that. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Richard W.M. Jones
2017-Feb-07 16:23 UTC
Re: [Libguestfs] [PATCH v2 3/7] mllib: expose libosinfo DB reading functions in mllib
On Tue, Feb 07, 2017 at 04:14:18PM +0100, Cédric Bosdonnat wrote:> src/osinfo.c has generic code to read the libosinfo DB. Add a wrapper > around it to expose it in mllib for use in ocaml code.There are at least 3 changes in this code that I can spot: - A code refactoring that moves some functions from osinfo.c -> osinfo-iso.c. - Something to do with read_osinfo_db_callback which I didn't fully understand. - OCaml bindings. These need to be separate commits, I'm not smart enough to review all three mixed together.> +static int > +read_osinfo_db_callback_wrapper (guestfs_h *g, const char *path, void *opaque) > +{ > + CAMLparam0 (); > + CAMLlocal2 (pathv, v); > + struct callback_wrapper_args *args = opaque; > + > + assert (path != NULL); > + assert (args != NULL); > + > + pathv = caml_copy_string (path); > + > + v = caml_callback_exn (*args->fvp, pathv); > + > + if (Is_exception_result (v)) { > + *args->exnp = Extract_exception (v); > + return -1;You copied buggy code from the OCaml wrapper for visit.c. This code will crash whenever an exception is thrown. Check the latest version of that wrapper for how to fix. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Richard W.M. Jones
2017-Feb-07 16:24 UTC
Re: [Libguestfs] [PATCH v2 4/7] builder: rename docs test script
On Tue, Feb 07, 2017 at 04:14:19PM +0100, Cédric Bosdonnat wrote:> Rename test-virt-builder-docs.sh into test-docs.sh to include test > for another tool's documentation.Is there a follow-up commit related to this which is missing? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Richard W.M. Jones
2017-Feb-07 16:56 UTC
Re: [Libguestfs] [PATCH v2 2/7] Move xml and xpath_helpers OCAML code to mllib
On Tue, Feb 07, 2017 at 04:14:17PM +0100, Cédric Bosdonnat wrote:> To allow other pieces of code to process XML files easily, move the > xml.ml* and xpath_helpers.ml* from v2v to mllib.Strictly speaking this is two commits - the code motion and the addition of Xml.parse_file. However it's a simple enough change. The real problem is this breaks v2v/test-harness. In theory this should fix it: ---------------------------------------------------------------------- --- a/v2v/test-harness/Makefile.am +++ b/v2v/test-harness/Makefile.am @@ -82,7 +82,7 @@ libv2vth_a_CFLAGS = \ -fPIC libv2vth_a_SOURCES = \ - ../xml-c.c + ../../mllib/xml-c.c v2v_test_harness.cmi: $(top_builddir)/ocaml/guestfs.cmi ---------------------------------------------------------------------- but that breaks with some weird automake error: Makefile:1747: ../../mllib/.deps/libv2vth_a-xml-c.Po: No such file or directory which I've no idea how to fix after trying for about an hour. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Possibly Parallel Threads
- [PATCH v2 1/7] mllib: factorize code to add Checksum.get_checksum function
- [PATCH v3 01/10] mllib: factorize code to add Checksum.get_checksum function
- [PATCH 2/5] mllib: factorize code to add Checksum.get_checksum function
- [PATCH 0/4] Consolidate Checksums as common code
- [PATCH v2 0/7] Introducing virt-builder-repository