Pino Toscano
2014-Sep-17 11:58 UTC
[Libguestfs] [PATCH 0/2] supermin: use librpm for rpm support
Hi, this work makes supermin use the rpm library instead of invoking rpm directly. This, together with a needed refactoring of the dependency resolution, should help in make supermin faster on rpm-based systems. Surely the patches will still need polishing, especially for behaviours of newly added stuff, but at least it's a good starting point. Noting that you need rpm-devel on most of rpm systems, feedback of any kind is more than welcome. Pino Toscano (2): package handlers: add possibility for final teardown rpm: use the rpm library instead of invoking rpm configure.ac | 4 + src/Makefile.am | 7 +- src/dpkg.ml | 1 + src/librpm-c.c | 463 ++++++++++++++++++++++++++++++++++++++++++++++++ src/librpm.ml | 51 ++++++ src/librpm.mli | 48 +++++ src/package_handler.ml | 10 ++ src/package_handler.mli | 10 ++ src/pacman.ml | 1 + src/rpm.ml | 218 +++++++++++------------ src/supermin-link.sh.in | 2 +- src/supermin.ml | 4 +- 12 files changed, 702 insertions(+), 117 deletions(-) create mode 100644 src/librpm-c.c create mode 100644 src/librpm.ml create mode 100644 src/librpm.mli -- 1.9.3
Pino Toscano
2014-Sep-17 11:58 UTC
[Libguestfs] [PATCH 1/2] package handlers: add possibility for final teardown
Add a ph_fini callback to package handlers, so they can do teardown operations, if needed, at the very end of the supermin run. Currently all of the current package handlers do nothing. --- src/dpkg.ml | 1 + src/package_handler.ml | 10 ++++++++++ src/package_handler.mli | 10 ++++++++++ src/pacman.ml | 1 + src/rpm.ml | 1 + src/supermin.ml | 4 +++- 6 files changed, 26 insertions(+), 1 deletion(-) diff --git a/src/dpkg.ml b/src/dpkg.ml index 1bb3f7f..ed899cf 100644 --- a/src/dpkg.ml +++ b/src/dpkg.ml @@ -191,6 +191,7 @@ let () let ph = { ph_detect = dpkg_detect; ph_init = dpkg_init; + ph_fini = PHNoFini; ph_package_of_string = dpkg_package_of_string; ph_package_to_string = dpkg_package_to_string; ph_package_name = dpkg_package_name; diff --git a/src/package_handler.ml b/src/package_handler.ml index b1dffc0..74d68f4 100644 --- a/src/package_handler.ml +++ b/src/package_handler.ml @@ -59,6 +59,7 @@ let file_source file type package_handler = { ph_detect : unit -> bool; ph_init : settings -> unit; + ph_fini : ph_fini; ph_package_of_string : string -> package option; ph_package_to_string : package -> string; ph_package_name : package -> string; @@ -76,6 +77,9 @@ and ph_get_files and ph_download_package | PHDownloadPackage of (package -> string -> unit) | PHDownloadAllPackages of (PackageSet.t -> string -> unit) +and ph_fini +| PHNoFini +| PHFini of (unit -> unit) (* Suggested memoization functions. *) let get_memo_functions () @@ -140,6 +144,12 @@ let rec get_package_handler_name () | Some (system, packager, _) -> sprintf "%s/%s" system packager | None -> assert false +let package_handler_shutdown () + let ph = get_package_handler () in + match ph.ph_fini with + | PHNoFini -> () + | PHFini f -> f () + let get_all_requires pkgs let ph = get_package_handler () in match ph.ph_get_requires with diff --git a/src/package_handler.mli b/src/package_handler.mli index 7e17981..7bdf0e8 100644 --- a/src/package_handler.mli +++ b/src/package_handler.mli @@ -99,6 +99,11 @@ type package_handler = { initializes. The [settings] parameter is a struct of general settings and configuration. *) + ph_fini : ph_fini; + (** This is called at the end of the supermin processing. It can + be used to do teardown operations for the package handler, + when no more package-related operations are going to be done. *) + ph_package_of_string : string -> package option; (** Convert a string (from user input) into a package object. If the package is not installed or the string is otherwise @@ -159,6 +164,9 @@ and ph_get_files and ph_download_package | PHDownloadPackage of (package -> string -> unit) | PHDownloadAllPackages of (PackageSet.t -> string -> unit) +and ph_fini +| PHNoFini +| PHFini of (unit -> unit) (** Package handlers could use these memoization functions to convert from the {!package} type to an internal struct and back again, or @@ -172,6 +180,8 @@ val list_package_handlers : unit -> unit val check_system : settings -> unit +val package_handler_shutdown : unit -> unit + val get_package_handler : unit -> package_handler val get_package_handler_name : unit -> string diff --git a/src/pacman.ml b/src/pacman.ml index 8b11ba8..2395796 100644 --- a/src/pacman.ml +++ b/src/pacman.ml @@ -227,6 +227,7 @@ let () let ph = { ph_detect = pacman_detect; ph_init = pacman_init; + ph_fini = PHNoFini; ph_package_of_string = pacman_package_of_string; ph_package_to_string = pacman_package_to_string; ph_package_name = pacman_package_name; diff --git a/src/rpm.ml b/src/rpm.ml index 1195948..e0cdb39 100644 --- a/src/rpm.ml +++ b/src/rpm.ml @@ -394,6 +394,7 @@ let () let fedora = { ph_detect = fedora_detect; ph_init = rpm_init; + ph_fini = PHNoFini; ph_package_of_string = rpm_package_of_string; ph_package_to_string = rpm_package_to_string; ph_package_name = rpm_package_name; diff --git a/src/supermin.ml b/src/supermin.ml index 0153977..2ee61a9 100644 --- a/src/supermin.ml +++ b/src/supermin.ml @@ -261,7 +261,9 @@ let main () *) sprintf "( chmod -R +w %s ; rm -rf %s ) 2>/dev/null &" (quote old_outputdir) (quote old_outputdir) in - ignore (Sys.command cmd) + ignore (Sys.command cmd); + + package_handler_shutdown () let () try main () -- 1.9.3
Pino Toscano
2014-Sep-17 11:58 UTC
[Libguestfs] [PATCH 2/2] rpm: use the rpm library instead of invoking rpm
Look for the rpm library, and use it to query for the information needed, such as: - the list of installed packages - the list of requires for a specified package - the providers of a specified capability - the list of files of a package Also, rework the dependency resolution, using a queue to iterate on the packages not resolved yet (thus resolving each package just once), and caching the provider of each capability. --- configure.ac | 4 + src/Makefile.am | 7 +- src/librpm-c.c | 463 ++++++++++++++++++++++++++++++++++++++++++++++++ src/librpm.ml | 51 ++++++ src/librpm.mli | 48 +++++ src/rpm.ml | 219 +++++++++++------------ src/supermin-link.sh.in | 2 +- 7 files changed, 677 insertions(+), 117 deletions(-) create mode 100644 src/librpm-c.c create mode 100644 src/librpm.ml create mode 100644 src/librpm.mli diff --git a/configure.ac b/configure.ac index 65dab78..e604ea2 100644 --- a/configure.ac +++ b/configure.ac @@ -95,6 +95,10 @@ dnl For yum-rpm handler. AC_PATH_PROG(RPM,[rpm],[no]) AC_PATH_PROG(RPM2CPIO,[rpm2cpio],[no]) AC_PATH_PROG(YUMDOWNLOADER,[yumdownloader],[no]) +PKG_CHECK_MODULES([LIBRPM], [rpm], [librpm=yes], [:]) +if test "x$librpm" = "xyes"; then + AC_DEFINE([HAVE_LIBRPM], [1], [Define if you have librpm]) +fi dnl For Zypper handler. AC_PATH_PROG(ZYPPER,[zypper],[no]) diff --git a/src/Makefile.am b/src/Makefile.am index 90aa773..6261c86 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -42,6 +42,9 @@ SOURCES = \ realpath-c.c \ realpath.ml \ realpath.mli \ + librpm-c.c \ + librpm.ml \ + librpm.mli \ config.ml \ utils.ml \ utils.mli \ @@ -66,6 +69,7 @@ SOURCES_ML = \ fnmatch.ml \ glob.ml \ realpath.ml \ + librpm.ml \ config.ml \ utils.ml \ types.ml \ @@ -86,6 +90,7 @@ SOURCES_C = \ ext2init-c.c \ fnmatch-c.c \ glob-c.c \ + librpm-c.c \ realpath-c.c CLEANFILES = *~ *.cmi *.cmo *.cmx *.o supermin @@ -98,7 +103,7 @@ bin_PROGRAMS = supermin supermin_SOURCES = $(SOURCES_C) supermin_CFLAGS = \ -I$(shell $(OCAMLC) -where) \ - $(EXT2FS_CFLAGS) $(COM_ERR_CFLAGS) \ + $(EXT2FS_CFLAGS) $(COM_ERR_CFLAGS) $(LIBRPM_CFLAGS) \ -Wall $(WERROR_CFLAGS) \ -I$(top_srcdir)/lib -I../lib diff --git a/src/librpm-c.c b/src/librpm-c.c new file mode 100644 index 0000000..c3ec3cb --- /dev/null +++ b/src/librpm-c.c @@ -0,0 +1,463 @@ +/* supermin 5 + * Copyright (C) 2014 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <config.h> + +#include <stdlib.h> +#include <string.h> +#include <glob.h> +#include <assert.h> + +#include <caml/alloc.h> +#include <caml/callback.h> +#include <caml/custom.h> +#include <caml/fail.h> +#include <caml/memory.h> +#include <caml/mlvalues.h> +#include <caml/unixsupport.h> + +#ifdef HAVE_LIBRPM + +#include <rpm/header.h> +#include <rpm/rpmdb.h> +#include <rpm/rpmlib.h> +#include <rpm/rpmlog.h> +#include <rpm/rpmts.h> + +static rpmlogCallback old_log_callback; + +static int +supermin_rpm_log_callback (rpmlogRec rec, rpmlogCallbackData data) +{ + fprintf (stderr, "supermin: rpm: lib: %s%s", + rpmlogLevelPrefix (rpmlogRecPriority (rec)), + rpmlogRecMessage (rec)); + return 0; +} + +struct librpm_data +{ + rpmts ts; + int debug; +}; + +static void librpm_handle_closed (void) __attribute__((noreturn)); + +static void +librpm_handle_closed (void) +{ + caml_failwith ("librpm: function called on a closed handle"); +} + +static void +librpm_raise_multiple_matches (int occurrences) +{ + caml_raise_with_arg (*caml_named_value ("librpm_multiple_matches"), + Val_int (occurrences)); +} + +#define Librpm_val(v) (*((struct librpm_data *)Data_custom_val(v))) +#define Val_none Val_int(0) +#define Some_val(v) Field(v,0) + +static void +librpm_finalize (value rpmv) +{ + struct librpm_data data = Librpm_val (rpmv); + + if (data.ts) { + rpmtsFree (data.ts); + + rpmlogSetCallback (old_log_callback, NULL); + } +} + +static struct custom_operations librpm_custom_operations = { + (char *) "librpm_custom_operations", + librpm_finalize, + custom_compare_default, + custom_hash_default, + custom_serialize_default, + custom_deserialize_default +}; + +static value +Val_librpm (struct librpm_data *data) +{ + CAMLparam0 (); + CAMLlocal1 (rpmv); + + rpmv = caml_alloc_custom (&librpm_custom_operations, + sizeof (struct librpm_data), 0, 1); + Librpm_val (rpmv) = *data; + CAMLreturn (rpmv); +} + +value +supermin_rpm_is_available (value unit) +{ + return Val_true; +} + +value +supermin_rpm_version (value unit) +{ + return caml_copy_string (RPMVERSION); +} + +value +supermin_rpm_open (value debugv) +{ + CAMLparam1 (debugv); + CAMLlocal1 (rpmv); + struct librpm_data data; + int res; + rpmlogLvl lvl; + + data.debug = debugv == Val_none ? 0 : Int_val (Some_val (debugv)); + + switch (data.debug) { + case 3: + lvl = RPMLOG_INFO; + break; + case 2: + lvl = RPMLOG_NOTICE; + break; + case 1: + lvl = RPMLOG_WARNING; + break; + case 0: + default: + lvl = RPMLOG_ERR; + break; + } + + rpmSetVerbosity (lvl); + old_log_callback = rpmlogSetCallback (supermin_rpm_log_callback, NULL); + + res = rpmReadConfigFiles (NULL, NULL); + if (res == -1) + caml_failwith ("rpm_open: rpmReadConfigFiles failed"); + + data.ts = rpmtsCreate (); + if (data.ts == NULL) + caml_failwith ("rpm_open: rpmtsCreate failed"); + + rpmv = Val_librpm (&data); + CAMLreturn (rpmv); +} + +value +supermin_rpm_close (value rpmv) +{ + CAMLparam1 (rpmv); + + librpm_finalize (rpmv); + + /* So we don't double-free in the finalizer. */ + Librpm_val (rpmv).ts = NULL; + + CAMLreturn (Val_unit); +} + +value +supermin_rpm_installed (value rpmv, value pkgv) +{ + CAMLparam2 (rpmv, pkgv); + CAMLlocal2 (rv, v); + struct librpm_data data; + rpmdbMatchIterator iter; + int count, i; + Header h; + + data = Librpm_val (rpmv); + if (data.ts == NULL) + librpm_handle_closed (); + + iter = rpmtsInitIterator (data.ts, RPMTAG_NAME, String_val (pkgv), 0); + if (iter == NULL) + caml_raise_not_found (); + + count = rpmdbGetIteratorCount (iter); + if (data.debug >= 2) + printf ("supermin: rpm: installed: %d occurrences for '%s'\n", count, String_val (pkgv)); + + rv = caml_alloc (count, 0); + i = 0; + + while ((h = rpmdbNextIterator (iter)) != NULL) { + HeaderIterator hi; + rpmtd td; + uint32_t *val; + + v = caml_alloc (5, 0); + hi = headerInitIterator (h); + td = rpmtdNew (); + while (headerNext (hi, td) == 1) { + switch (rpmtdTag (td)) { + case RPMTAG_NAME: + Store_field (v, 0, caml_copy_string (rpmtdGetString (td))); + break; + case RPMTAG_EPOCH: + val = rpmtdGetUint32 (td); + Store_field (v, 1, Val_int ((int) *val)); + break; + case RPMTAG_VERSION: + Store_field (v, 2, caml_copy_string (rpmtdGetString (td))); + break; + case RPMTAG_RELEASE: + Store_field (v, 3, caml_copy_string (rpmtdGetString (td))); + break; + case RPMTAG_ARCH: + Store_field (v, 4, caml_copy_string (rpmtdGetString (td))); + break; + } + rpmtdFreeData (td); + } + Store_field (rv, i, v); + + rpmtdFree (td); + headerFreeIterator (hi); + ++i; + } + + rpmdbFreeIterator (iter); + + CAMLreturn (rv); +} + +value +supermin_rpm_pkg_requires (value rpmv, value pkgv) +{ + CAMLparam2 (rpmv, pkgv); + CAMLlocal1 (rv); + struct librpm_data data; + rpmdbMatchIterator iter; + int count, i; + Header h; + rpmtd td; + + data = Librpm_val (rpmv); + if (data.ts == NULL) + librpm_handle_closed (); + + iter = rpmtsInitIterator (data.ts, RPMDBI_LABEL, String_val (pkgv), 0); + if (iter == NULL) + caml_raise_not_found (); + + count = rpmdbGetIteratorCount (iter); + if (data.debug >= 2) + printf ("supermin: rpm: pkg_requires: %d occurrences for '%s'\n", count, String_val (pkgv)); + if (count != 1) + librpm_raise_multiple_matches (count); + + h = rpmdbNextIterator (iter); + assert (h != NULL); + + td = rpmtdNew (); + i = headerGet (h, RPMTAG_REQUIRENAME, td, HEADERGET_MINMEM); + if (i != 1) + caml_failwith ("rpm_pkg_requires: headerGet failed"); + + rv = caml_alloc (rpmtdCount (td), 0); + for (i = 0; i < rpmtdCount (td); ++i) + Store_field (rv, i, caml_copy_string (rpmtdNextString (td))); + + rpmtdFreeData (td); + rpmtdFree (td); + + rpmdbFreeIterator (iter); + + CAMLreturn (rv); +} + +static rpmdbMatchIterator +createProvidesIterator (rpmts ts, const char *what) +{ + rpmdbMatchIterator mi = NULL; + + if (what[0] != '/') { + mi = rpmtsInitIterator(ts, RPMDBI_PROVIDENAME, what, 0); + if (mi != NULL) + return mi; + } + mi = rpmtsInitIterator(ts, RPMDBI_INSTFILENAMES, what, 0); + if (mi != NULL) + return mi; + + mi = rpmtsInitIterator(ts, RPMDBI_PROVIDENAME, what, 0); + + return mi; +} + +value +supermin_rpm_pkg_whatprovides (value rpmv, value pkgv) +{ + CAMLparam2 (rpmv, pkgv); + CAMLlocal1 (rv); + struct librpm_data data; + rpmdbMatchIterator iter; + int count, i; + Header h; + + data = Librpm_val (rpmv); + if (data.ts == NULL) + librpm_handle_closed (); + + iter = createProvidesIterator (data.ts, String_val (pkgv)); + if (iter == NULL) + caml_raise_not_found (); + + count = rpmdbGetIteratorCount (iter); + if (data.debug >= 2) + printf ("supermin: rpm: pkg_whatprovides: %d occurrences for '%s'\n", count, String_val (pkgv)); + + rv = caml_alloc (count, 0); + i = 0; + + while ((h = rpmdbNextIterator (iter)) != NULL) { + rpmtd td; + int ret; + + td = rpmtdNew (); + ret = headerGet (h, RPMTAG_NAME, td, HEADERGET_MINMEM); + if (ret != 1) + caml_failwith ("rpm_pkg_whatprovides: headerGet failed"); + + Store_field (rv, i, caml_copy_string (rpmtdGetString (td))); + + rpmtdFreeData (td); + rpmtdFree (td); + ++i; + } + + rpmdbFreeIterator (iter); + + CAMLreturn (rv); +} + +value +supermin_rpm_pkg_filelist (value rpmv, value pkgv) +{ + CAMLparam2 (rpmv, pkgv); + CAMLlocal2 (rv, v); + struct librpm_data data; + rpmdbMatchIterator iter; + int count, i; + Header h; + rpmfi fi; + const rpmfiFlags fiflags = RPMFI_NOHEADER | RPMFI_FLAGS_QUERY | RPMFI_NOFILEDIGESTS; + + data = Librpm_val (rpmv); + if (data.ts == NULL) + librpm_handle_closed (); + + iter = rpmtsInitIterator (data.ts, RPMDBI_LABEL, String_val (pkgv), 0); + if (iter == NULL) + caml_raise_not_found (); + + count = rpmdbGetIteratorCount (iter); + if (data.debug >= 2) + printf ("supermin: rpm: pkg_filelist: %d occurrences for '%s'\n", count, String_val (pkgv)); + if (count != 1) + librpm_raise_multiple_matches (count); + + h = rpmdbNextIterator (iter); + assert (h != NULL); + + fi = rpmfiNew (data.ts, h, RPMTAG_BASENAMES, fiflags); + + count = rpmfiFC (fi); + if (count < 0) + count = 0; + + rv = caml_alloc (count, 0); + i = 0; + + fi = rpmfiInit (fi, 0); + while (rpmfiNext (fi) >= 0) { + const char *fn; + + v = caml_alloc (2, 0); + fn = rpmfiFN(fi); + Store_field (v, 0, caml_copy_string (fn)); + if (rpmfiFFlags (fi) & RPMFILE_CONFIG) + Store_field (v, 1, Val_long (1)); /* FileConfig */ + else + Store_field (v, 1, Val_long (0)); /* FileNormal */ + Store_field (rv, i, v); + ++i; + } + rpmfiFree(fi); + + rpmdbFreeIterator (iter); + + CAMLreturn (rv); +} + +#else + +value +supermin_rpm_is_available (value unit) +{ + return Val_false; +} + +value +supermin_rpm_version (value unit) +{ + abort (); +} + +value +supermin_rpm_open (value debugv) +{ + abort (); +} + +value +supermin_rpm_close (value rpmv) +{ + abort (); +} + +value +supermin_rpm_installed (value rpmv, value pkgv) +{ + abort (); +} + +value +supermin_rpm_pkg_required (value rpmv, value pkgv) +{ + abort (); +} + +value +supermin_rpm_pkg_whatprovides (value rpmv, value pkgv) +{ + abort (); +} + +value +supermin_rpm_pkg_filelist (value rpmv, value pkgv) +{ + abort (); +} + +#endif diff --git a/src/librpm.ml b/src/librpm.ml new file mode 100644 index 0000000..aa8d367 --- /dev/null +++ b/src/librpm.ml @@ -0,0 +1,51 @@ +(* supermin 5 + * Copyright (C) 2014 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +external rpm_is_available : unit -> bool = "supermin_rpm_is_available" + +external rpm_version : unit -> string = "supermin_rpm_version" + +type t + +exception Multiple_matches of int + +external rpm_open : ?debug:int -> t = "supermin_rpm_open" +external rpm_close : t -> unit = "supermin_rpm_close" + +type rpm_t = { + name : string; + epoch : int; + version : string; + release : string; + arch : string; +} + +type rpmfile_t = { + filepath : string; + filetype : rpmfiletype_t; +} and rpmfiletype_t + | FileNormal + | FileConfig + +external rpm_installed : t -> string -> rpm_t array = "supermin_rpm_installed" +external rpm_pkg_requires : t -> string -> string array = "supermin_rpm_pkg_requires" +external rpm_pkg_whatprovides : t -> string -> string array = "supermin_rpm_pkg_whatprovides" +external rpm_pkg_filelist : t -> string -> rpmfile_t array = "supermin_rpm_pkg_filelist" + +let () + Callback.register_exception "librpm_multiple_matches" (Multiple_matches 0) diff --git a/src/librpm.mli b/src/librpm.mli new file mode 100644 index 0000000..880a038 --- /dev/null +++ b/src/librpm.mli @@ -0,0 +1,48 @@ +(* supermin 5 + * Copyright (C) 2014 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + *) + +val rpm_is_available : unit -> bool + +val rpm_version : unit -> string + +type t + +exception Multiple_matches of int + +val rpm_open : ?debug:int -> t +val rpm_close : t -> unit + +type rpm_t = { + name : string; + epoch : int; + version : string; + release : string; + arch : string; +} + +type rpmfile_t = { + filepath : string; + filetype : rpmfiletype_t; +} and rpmfiletype_t + | FileNormal + | FileConfig + +val rpm_installed : t -> string -> rpm_t array +val rpm_pkg_requires : t -> string -> string array +val rpm_pkg_whatprovides : t -> string -> string array +val rpm_pkg_filelist : t -> string -> rpmfile_t array diff --git a/src/rpm.ml b/src/rpm.ml index e0cdb39..3bac45f 100644 --- a/src/rpm.ml +++ b/src/rpm.ml @@ -21,9 +21,15 @@ open Printf open Utils open Package_handler +open Librpm + +module StringSet = Set.Make (String) + +let stringset_of_list pkgs + List.fold_left (fun set elem -> StringSet.add elem set) StringSet.empty pkgs let fedora_detect () - Config.rpm <> "no" && Config.rpm2cpio <> "no" && + Config.rpm <> "no" && Config.rpm2cpio <> "no" && (rpm_is_available ()) && Config.yumdownloader <> "no" && try (stat "/etc/redhat-release").st_kind = S_REG || @@ -31,12 +37,12 @@ let fedora_detect () with Unix_error _ -> false let opensuse_detect () - Config.rpm <> "no" && Config.rpm2cpio <> "no" && + Config.rpm <> "no" && Config.rpm2cpio <> "no" && (rpm_is_available ()) && Config.zypper <> "no" && try (stat "/etc/SuSE-release").st_kind = S_REG with Unix_error _ -> false let mageia_detect () - Config.rpm <> "no" && Config.rpm2cpio <> "no" && + Config.rpm <> "no" && Config.rpm2cpio <> "no" && (rpm_is_available ()) && Config.urpmi <> "no" && Config.fakeroot <> "no" && try (stat "/etc/mageia-release").st_kind = S_REG with Unix_error _ -> false @@ -44,6 +50,14 @@ let mageia_detect () let settings = ref no_settings let rpm_major, rpm_minor = ref 0, ref 0 let zypper_major, zypper_minor, zypper_patch = ref 0, ref 0, ref 0 +let t = ref None + +let get_rpm () + match !t with + | None -> + eprintf "supermin: rpm: get_rpm called too early"; + exit 1 + | Some t -> t let rec rpm_init s settings := s; @@ -51,31 +65,26 @@ let rec rpm_init s (* Get RPM version. We have to adjust some RPM commands based on * the version. *) - let cmd = sprintf "%s --version | awk '{print $3}'" Config.rpm in - let lines = run_command_get_lines cmd in + let version = rpm_version () in let major, minor - match lines with + match string_split "." version with | [] -> - eprintf "supermin: rpm --version command had no output\n"; + eprintf "supermin: unable to parse empty rpm version string\n"; exit 1 - | line :: _ -> - let line = string_split "." line in - match line with - | [] -> - eprintf "supermin: unable to parse empty output of rpm --version\n"; - exit 1 - | [x] -> - eprintf "supermin: unable to parse output of rpm --version: %s\n" x; - exit 1 - | major :: minor :: _ -> - try int_of_string major, int_of_string minor - with Failure "int_of_string" -> - eprintf "supermin: unable to parse output of rpm --version: non-numeric\n"; - exit 1 in + | [x] -> + eprintf "supermin: unable to parse rpm version string: %s\n" x; + exit 1 + | major :: minor :: _ -> + try int_of_string major, int_of_string minor + with Failure "int_of_string" -> + eprintf "supermin: unable to parse rpm version string: non-numeric, %s\n" version; + exit 1 in rpm_major := major; rpm_minor := minor; if !settings.debug >= 1 then - printf "supermin: rpm: detected RPM version %d.%d\n" major minor + printf "supermin: rpm: detected RPM version %d.%d\n" major minor; + + t := Some (rpm_open ~debug:!settings.debug) and opensuse_init s rpm_init s; @@ -115,13 +124,10 @@ and opensuse_init s if !settings.debug >= 1 then printf "supermin: rpm: detected zypper version %d.%d.%d\n" major minor patch -type rpm_t = { - name : string; - epoch : int32; - version : string; - release : string; - arch : string; -} +let rpm_fini () + match !t with + | None -> () + | Some t -> rpm_close t (* Memo from package type to internal rpm_t. *) let rpm_of_pkg, pkg_of_rpm = get_memo_functions () @@ -130,43 +136,8 @@ let rpm_of_pkg, pkg_of_rpm = get_memo_functions () let rpmh = Hashtbl.create 13 let rpm_package_of_string str - (* Parse an RPM name into the fields like name and version. Since - * the package is installed (see check below), it's easier to use RPM - * itself to do this parsing rather than haphazardly parsing it - * ourselves. *) - let parse_rpm str - let cmd - sprintf "%s --nosignature --nodigest -q --qf '%%{name} %%{epoch} %%{version} %%{release} %%{arch}\\n' %s" - Config.rpm - (quote str) in - let lines = run_command_get_lines cmd in - let lines = List.map (string_split " ") lines in - let rpms = filter_map ( - function - | [ name; ("0"|"(none)"); version; release; arch ] -> - Some { name = name; - epoch = 0_l; - version = version; release = release; arch = arch } - | [ name; epoch; version; release; arch ] -> - Some { name = name; - epoch = Int32.of_string epoch; - version = version; release = release; arch = arch } - | xs -> - (* grrr, RPM doesn't send errors to stderr *) - None - ) lines in - - if rpms = [] then ( - eprintf "supermin: no output from rpm command could be parsed when searching for '%s'\nThe command was:\n %s\n" - str cmd; - exit 1 - ); - - (* RPM will return multiple hits when either multiple versions or - * multiple arches are installed at the same time. We are only - * interested in the highest version with the best - * architecture. - *) + let query rpm + let rpms = Array.to_list (rpm_installed (get_rpm ()) str) in let cmp { version = v1; arch = a1 } { version = v2; arch = a2 } let i = compare_version v2 v1 in if i <> 0 then i @@ -174,12 +145,6 @@ let rpm_package_of_string str in let rpms = List.sort cmp rpms in List.hd rpms - - (* Check if an RPM is installed. *) - and check_rpm_installed name - let cmd = sprintf "%s --nosignature --nodigest -q %s >/dev/null" - Config.rpm (quote name) in - 0 = Sys.command cmd in try @@ -187,11 +152,8 @@ let rpm_package_of_string str with Not_found -> let r - if check_rpm_installed str then ( - let rpm = parse_rpm str in - Some (pkg_of_rpm rpm) - ) - else None in + try Some (pkg_of_rpm (query str)) + with Not_found -> None in Hashtbl.add rpmh str r; r @@ -212,10 +174,10 @@ let rpm_package_to_string pkg !rpm_major < 4 || (!rpm_major = 4 && !rpm_minor < 11) in let rpm = rpm_of_pkg pkg in - if is_rpm_lt_4_11 || rpm.epoch = 0_l then + if is_rpm_lt_4_11 || rpm.epoch = 0 then sprintf "%s-%s-%s.%s" rpm.name rpm.version rpm.release rpm.arch else - sprintf "%s-%ld:%s-%s.%s" + sprintf "%s-%d:%s-%s.%s" rpm.name rpm.epoch rpm.version rpm.release rpm.arch let rpm_package_name pkg @@ -225,47 +187,74 @@ let rpm_package_name pkg let rpm_get_package_database_mtime () (lstat "/var/lib/rpm/Packages").st_mtime +(* Memo of resolved provides. *) +let rpm_providers = Hashtbl.create 13 + let rpm_get_all_requires pkgs - let get pkgs - let cmd = sprintf "\ - %s --nosignature --nodigest -qR %s | - awk '{print $1}' | - xargs rpm --nosignature --nodigest -q --qf '%%{name}\\n' --whatprovides | - grep -v 'no package provides' | - sort -u" - Config.rpm - (quoted_list (List.map rpm_package_to_string - (PackageSet.elements pkgs))) in - let lines = run_command_get_lines cmd in - let lines = filter_map rpm_package_of_string lines in - PackageSet.union pkgs (package_set_of_list lines) - in - (* The command above only gets one level of dependencies. We need - * to keep iterating until we reach a fixpoint. - *) - let rec loop pkgs - let pkgs' = get pkgs in - if PackageSet.equal pkgs pkgs' then pkgs - else loop pkgs' + let get pkg + let reqs + try + rpm_pkg_requires (get_rpm ()) pkg + with + Multiple_matches _ as ex -> + match rpm_package_of_string pkg with + | None -> raise ex + | Some pkg -> rpm_pkg_requires (get_rpm ()) (rpm_package_to_string pkg) in + let pkgs' = Array.fold_left ( + fun set x -> + try + let provides + try Hashtbl.find rpm_providers x + with Not_found -> rpm_pkg_whatprovides (get_rpm ()) x in + let newset = Array.fold_left ( + fun newset p -> + match rpm_package_of_string p with + | None -> newset + | Some x -> StringSet.add p newset + ) StringSet.empty provides in + StringSet.union set newset + with Not_found -> set + ) StringSet.empty reqs in + pkgs' in - loop pkgs + let queue = Queue.create () in + let final = ref (stringset_of_list + (List.map rpm_package_name + (PackageSet.elements pkgs))) in + StringSet.iter (fun x -> Queue.push x queue) !final; + let resolved = ref StringSet.empty in + while not (Queue.is_empty queue) do + let current = Queue.pop queue in + if not (StringSet.mem current !resolved) then ( + try + let expanded = get current in + let diff = StringSet.diff expanded !final in + if not (StringSet.is_empty diff) then ( + final := StringSet.union !final diff; + StringSet.iter (fun x -> Queue.push x queue) diff; + ) + with Not_found -> (); + resolved := StringSet.add current !resolved + ) + done; + let pkgs' = filter_map rpm_package_of_string (StringSet.elements !final) in + package_set_of_list pkgs' let rpm_get_all_files pkgs - let cmd = sprintf "\ - %s --nosignature --nodigest -q --qf '[%%{FILENAMES}\\t%%{FILEFLAGS:fflags}\\n]' %s | - grep '^/' | - sort -u" - Config.rpm - (quoted_list (List.map rpm_package_to_string (PackageSet.elements pkgs))) in - let lines = run_command_get_lines cmd in - let lines = List.map (string_split "\t") lines in + let files_compare { filepath = a } { filepath = b } + compare a b in + let files = List.map rpm_package_to_string (PackageSet.elements pkgs) in + let files = List.fold_right ( + fun pkg xs -> + let files = Array.to_list (rpm_pkg_filelist (get_rpm ()) pkg) in + files @ xs + ) files [] in + let files = sort_uniq ~cmp:files_compare files in List.map ( - function - | [ path; flags ] -> - let config = String.contains flags 'c' in + fun { filepath = path; filetype = flags } -> + let config = flags = FileConfig in { ft_path = path; ft_source_path = path; ft_config = config } - | _ -> assert false - ) lines + ) files let rec fedora_download_all_packages pkgs dir let tdir = !settings.tmpdir // string_random8 () in @@ -394,7 +383,7 @@ let () let fedora = { ph_detect = fedora_detect; ph_init = rpm_init; - ph_fini = PHNoFini; + ph_fini = PHFini rpm_fini; ph_package_of_string = rpm_package_of_string; ph_package_to_string = rpm_package_to_string; ph_package_name = rpm_package_name; diff --git a/src/supermin-link.sh.in b/src/supermin-link.sh.in index b2d71d9..29b84a1 100644 --- a/src/supermin-link.sh.in +++ b/src/supermin-link.sh.in @@ -21,4 +21,4 @@ # Hack automake to link 'supermin' binary properly. There is no other # way to add the -cclib parameter to the end of the command line. -exec "$@" -linkpkg -cclib '@EXT2FS_LIBS@ @COM_ERR_LIBS@' +exec "$@" -linkpkg -cclib '@EXT2FS_LIBS@ @COM_ERR_LIBS@ @LIBRPM_LIBS@' -- 1.9.3
Richard W.M. Jones
2014-Sep-17 12:17 UTC
Re: [Libguestfs] [PATCH 1/2] package handlers: add possibility for final teardown
On Wed, Sep 17, 2014 at 01:58:24PM +0200, Pino Toscano wrote:> Add a ph_fini callback to package handlers, so they can do teardown > operations, if needed, at the very end of the supermin run. > > Currently all of the current package handlers do nothing. > --- > src/dpkg.ml | 1 + > src/package_handler.ml | 10 ++++++++++ > src/package_handler.mli | 10 ++++++++++ > src/pacman.ml | 1 + > src/rpm.ml | 1 + > src/supermin.ml | 4 +++- > 6 files changed, 26 insertions(+), 1 deletion(-) > > diff --git a/src/dpkg.ml b/src/dpkg.ml > index 1bb3f7f..ed899cf 100644 > --- a/src/dpkg.ml > +++ b/src/dpkg.ml > @@ -191,6 +191,7 @@ let () > let ph = { > ph_detect = dpkg_detect; > ph_init = dpkg_init; > + ph_fini = PHNoFini; > ph_package_of_string = dpkg_package_of_string; > ph_package_to_string = dpkg_package_to_string; > ph_package_name = dpkg_package_name; > diff --git a/src/package_handler.ml b/src/package_handler.ml > index b1dffc0..74d68f4 100644 > --- a/src/package_handler.ml > +++ b/src/package_handler.ml > @@ -59,6 +59,7 @@ let file_source file > type package_handler = { > ph_detect : unit -> bool; > ph_init : settings -> unit; > + ph_fini : ph_fini; > ph_package_of_string : string -> package option; > ph_package_to_string : package -> string; > ph_package_name : package -> string; > @@ -76,6 +77,9 @@ and ph_get_files > and ph_download_package > | PHDownloadPackage of (package -> string -> unit) > | PHDownloadAllPackages of (PackageSet.t -> string -> unit) > +and ph_fini > +| PHNoFini > +| PHFini of (unit -> unit)This is unnecessary, since you can write the default (no handler) as: ph_fini = fun () -> () and thus avoid all the other special cases below. ACK to the patch, but please remove the ph_fini type before committing. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
Richard W.M. Jones
2014-Sep-17 12:24 UTC
Re: [Libguestfs] [PATCH 2/2] rpm: use the rpm library instead of invoking rpm
On Wed, Sep 17, 2014 at 01:58:25PM +0200, Pino Toscano wrote:> - Config.rpm <> "no" && Config.rpm2cpio <> "no" && > + Config.rpm <> "no" && Config.rpm2cpio <> "no" && (rpm_is_available ()) &&Unnecessary parens: function application binds tightest in functional languages. There are a couple more of the same thing below. Seems reasonable after this change, so ACK. Once you've pushed this I'm going to add this to Rawhide to get more testing. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org
Maybe Matching Threads
- [PATCH 0/4] rpm: Choose providers better (RHBZ#1266918).
- [supermin PATCH 1/2] rpm: extend the Multiple_matches exception
- [supermin PATCH 0/5] rpm: fix package selection w/ multilib
- [PATCH 0/3] supermin: miscellaneous cleanups
- [PATCH] rpm: use librpm's rpmvercmp