Libguestfs - Nov 2018 - [PATCH 0/3] inspection: tweak limits of package manager files

I got a recent report of virt-v2v (via virt-p2v) failing to migrate a
Fedora guest.  The issue was that its /var/lib/rpm/Packages was bigger
than our current limit (~410M vs 300M), hence the inspection failed.

I took the liberty to refactor the limits of the these files, bumping
the problematic one, and reducing the ones of the others (as they are
supposed to be way smaller than the limit).

Pino Toscano (3):
  lib: move MAX_PKG_DB_SIZE to inspect-apps
  lib: inspect: split limits of package manager files
  lib: inspect: tweak limits of package manager files

 lib/guestfs-internal.h |  9 ---------
 lib/inspect-apps.c     | 23 +++++++++++++++++++----
 2 files changed, 19 insertions(+), 13 deletions(-)

-- 
2.17.2

It is used only there, so avoid exposing it to all the users of
guestfs-internal.h.

This is just code motion.
---
 lib/guestfs-internal.h | 9 ---------
 lib/inspect-apps.c     | 9 +++++++++
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h
index 0ebb639e7..3274dcdf0 100644
--- a/lib/guestfs-internal.h
+++ b/lib/guestfs-internal.h
@@ -122,15 +122,6 @@
  */
 #define APPLIANCE_TIMEOUT (20*60) /* 20 mins */
 
-/* Some limits on what the inspection code will read, for safety. */
-
-/* Maximum RPM or dpkg database we will download to /tmp.  RPM
- * 'Packages' database can get very large: 70 MB is roughly the
- * standard size for a new Fedora install, and after lots of package
- * installation/removal I have seen well over 100 MB databases.
- */
-#define MAX_PKG_DB_SIZE       (300 * 1000 * 1000)
-
 /* Maximum size of Windows explorer.exe.  2.6MB on Windows 7. */
 #define MAX_WINDOWS_EXPLORER_SIZE (4 * 1000 * 1000)
 
diff --git a/lib/inspect-apps.c b/lib/inspect-apps.c
index f0cf16b38..d0bb340da 100644
--- a/lib/inspect-apps.c
+++ b/lib/inspect-apps.c
@@ -45,6 +45,15 @@
 #include "guestfs-internal-actions.h"
 #include "structs-cleanups.h"
 
+/* Some limits on what the inspection code will read, for safety. */
+
+/* Maximum RPM or dpkg database we will download to /tmp.  RPM
+ * 'Packages' database can get very large: 70 MB is roughly the
+ * standard size for a new Fedora install, and after lots of package
+ * installation/removal I have seen well over 100 MB databases.
+ */
+#define MAX_PKG_DB_SIZE       (300 * 1000 * 1000)
+
 #ifdef DB_DUMP
 static struct guestfs_application2_list *list_applications_rpm (guestfs_h *g,
const char *root);
 #endif
-- 
2.17.2

Use different defines for the limits of the package manager files
downloaded during the inspection of applications.  This way it will be
possible to tweak each of them without affecting the others.

This is a minor refactoring, with no behaviour change.
---
 lib/inspect-apps.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/lib/inspect-apps.c b/lib/inspect-apps.c
index d0bb340da..3441b4b94 100644
--- a/lib/inspect-apps.c
+++ b/lib/inspect-apps.c
@@ -47,12 +47,18 @@
 
 /* Some limits on what the inspection code will read, for safety. */
 
-/* Maximum RPM or dpkg database we will download to /tmp.  RPM
- * 'Packages' database can get very large: 70 MB is roughly the
- * standard size for a new Fedora install, and after lots of package
- * installation/removal I have seen well over 100 MB databases.
+/* Maximum RPM 'Packages' file we will download to /tmp.  This file
+ * can get very large: 70 MB is roughly the standard size for a new
+ * Fedora install, and after lots of package installation/removal
+ * I have seen well over 100 MB databases.
  */
-#define MAX_PKG_DB_SIZE       (300 * 1000 * 1000)
+#define MAX_RPM_PACKAGES_SIZE          (300 * 1000 * 1000)
+/* Maximum RPM 'Name' file we will download to /tmp. */
+#define MAX_RPM_NAME_SIZE              (300 * 1000 * 1000)
+/* Maximum dpkg 'status' file we will download to /tmp. */
+#define MAX_DPKG_STATUS_SIZE           (300 * 1000 * 1000)
+/* Maximum APK 'installed' file we will download to /tmp. */
+#define MAX_APK_INSTALLED_SIZE         (300 * 1000 * 1000)
 
 #ifdef DB_DUMP
 static struct guestfs_application2_list *list_applications_rpm (guestfs_h *g,
const char *root);
@@ -380,12 +386,12 @@ list_applications_rpm (guestfs_h *g, const char *root)
   struct read_package_data data;
 
   Name = guestfs_int_download_to_tmp (g, "/var/lib/rpm/Name", NULL,
-				      MAX_PKG_DB_SIZE);
+				      MAX_RPM_NAME_SIZE);
   if (Name == NULL)
     goto error;
 
   Packages = guestfs_int_download_to_tmp (g, "/var/lib/rpm/Packages",
NULL,
-					  MAX_PKG_DB_SIZE);
+					  MAX_RPM_PACKAGES_SIZE);
   if (Packages == NULL)
     goto error;
 
@@ -437,7 +443,7 @@ list_applications_deb (guestfs_h *g, const char *root)
   size_t continuation_field_len = 0;
 
   status = guestfs_int_download_to_tmp (g, "/var/lib/dpkg/status",
NULL,
-					MAX_PKG_DB_SIZE);
+					MAX_DPKG_STATUS_SIZE);
   if (status == NULL)
     return NULL;
 
@@ -721,7 +727,7 @@ list_applications_apk (guestfs_h *g, const char *root)
     *url = NULL, *description = NULL;
 
   installed = guestfs_int_download_to_tmp (g,
"/lib/apk/db/installed",
-                                           NULL, MAX_PKG_DB_SIZE);
+                                           NULL, MAX_APK_INSTALLED_SIZE);
   if (installed == NULL)
     return NULL;
 
-- 
2.17.2

Tweak the limits of the files of package managers downloaded to list the
installed applications:
- bump RPM 'Packages' up to 500M: Fedora installations upgraded to each
  new Fedora release have an ever-growing file, reported to be even
  slightly bigger than 400M
- reduce the other files down to 50M: they are usually small, not bigger
  than 1M-5M, so 50M is a good enough high threshold
---
 lib/inspect-apps.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/inspect-apps.c b/lib/inspect-apps.c
index 3441b4b94..dbc9d968c 100644
--- a/lib/inspect-apps.c
+++ b/lib/inspect-apps.c
@@ -50,15 +50,15 @@
 /* Maximum RPM 'Packages' file we will download to /tmp.  This file
  * can get very large: 70 MB is roughly the standard size for a new
  * Fedora install, and after lots of package installation/removal
- * I have seen well over 100 MB databases.
+ * I have seen well over 400 MB databases.
  */
-#define MAX_RPM_PACKAGES_SIZE          (300 * 1000 * 1000)
+#define MAX_RPM_PACKAGES_SIZE          (500 * 1000 * 1000)
 /* Maximum RPM 'Name' file we will download to /tmp. */
-#define MAX_RPM_NAME_SIZE              (300 * 1000 * 1000)
+#define MAX_RPM_NAME_SIZE              (50 * 1000 * 1000)
 /* Maximum dpkg 'status' file we will download to /tmp. */
-#define MAX_DPKG_STATUS_SIZE           (300 * 1000 * 1000)
+#define MAX_DPKG_STATUS_SIZE           (50 * 1000 * 1000)
 /* Maximum APK 'installed' file we will download to /tmp. */
-#define MAX_APK_INSTALLED_SIZE         (300 * 1000 * 1000)
+#define MAX_APK_INSTALLED_SIZE         (50 * 1000 * 1000)
 
 #ifdef DB_DUMP
 static struct guestfs_application2_list *list_applications_rpm (guestfs_h *g,
const char *root);
-- 
2.17.2

ACK series.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org