Libguestfs - Oct 2015 - Re: [PATCHv4 0/2] Introduce vfs_minimum_size API to get minimum filesystem size.

Tried to make it in accordance with your comments.

Difference to v1:
Added reply_with_error where necessary.
Changed name get_min_size -> vfs_min_size.

Difference to v2:
Changed name to vfs_minimum_size.
Changed parsing to xstrtol + STR* macros where possible.

Difference to v3:
Decapitalize error messages.

Maxim Perevedentsev (2):
  New API: vfs_minimum_size
  Include resize2fs_P into vfs_minimum_size.

 daemon/Makefile.am   |  1 +
 daemon/daemon.h      |  2 ++
 daemon/ext2.c        | 62 ++++++++++++++++++++++++++++++++-----
 daemon/fs-min-size.c | 49 +++++++++++++++++++++++++++++
 daemon/ntfs.c        | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 generator/actions.ml | 19 ++++++++----
 po/POTFILES          |  1 +
 7 files changed, 207 insertions(+), 14 deletions(-)
 create mode 100644 daemon/fs-min-size.c

--
1.8.3.1

This call provides the way to get minimum size of filesystem.
This is needed for shrinking. The return units are bytes.

---
 daemon/Makefile.am   |  1 +
 daemon/daemon.h      |  1 +
 daemon/fs-min-size.c | 46 +++++++++++++++++++++++++++
 daemon/ntfs.c        | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 generator/actions.ml | 19 ++++++++++++
 po/POTFILES          |  1 +
 src/MAX_PROC_NR      |  2 +-
 7 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 daemon/fs-min-size.c

diff --git a/daemon/Makefile.am b/daemon/Makefile.am
index 4ea3c88..0a01a24 100644
--- a/daemon/Makefile.am
+++ b/daemon/Makefile.am
@@ -116,6 +116,7 @@ guestfsd_SOURCES = \
 	findfs.c \
 	fill.c \
 	find.c \
+	fs-min-size.c \
 	fsck.c \
 	fstrim.c \
 	glob.c \
diff --git a/daemon/daemon.h b/daemon/daemon.h
index 508691a..3b0266d 100644
--- a/daemon/daemon.h
+++ b/daemon/daemon.h
@@ -283,6 +283,7 @@ extern int btrfs_set_uuid_random (const char *device);
 /*-- in ntfs.c --*/
 extern char *ntfs_get_label (const char *device);
 extern int ntfs_set_label (const char *device, const char *label);
+extern int64_t ntfs_minimum_size (const char *device);

 /*-- in swap.c --*/
 extern int swap_set_uuid (const char *device, const char *uuid);
diff --git a/daemon/fs-min-size.c b/daemon/fs-min-size.c
new file mode 100644
index 0000000..652eb0e
--- /dev/null
+++ b/daemon/fs-min-size.c
@@ -0,0 +1,46 @@
+/* libguestfs - the guestfsd daemon
+ * Copyright (C) 2015 Maxim Perevedentsev mperevedentsev@virtuozzo.com
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
USA.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "daemon.h"
+#include "actions.h"
+
+int64_t
+do_vfs_minimum_size (const mountable_t *mountable)
+{
+  int64_t r;
+
+  /* How we set the label depends on the filesystem type. */
+  CLEANUP_FREE char *vfs_type = do_vfs_type (mountable);
+  if (vfs_type == NULL)
+    return -1;
+
+  else if (STREQ (vfs_type, "ntfs"))
+    r = ntfs_minimum_size (mountable->device);
+
+  else
+    NOT_SUPPORTED (-1, "don't know how to get minimum size of
'%s' filesystems",
+                   vfs_type);
+
+  return r;
+}
diff --git a/daemon/ntfs.c b/daemon/ntfs.c
index 1ead159..7a74c9b 100644
--- a/daemon/ntfs.c
+++ b/daemon/ntfs.c
@@ -27,6 +27,7 @@
 #include "daemon.h"
 #include "actions.h"
 #include "optgroups.h"
+#include "xstrtol.h"

 #define MAX_ARGS 64

@@ -153,6 +154,92 @@ do_ntfsresize_size (const char *device, int64_t size)
   return do_ntfsresize (device, size, 0);
 }

+int64_t
+ntfs_minimum_size (const char *device)
+{
+  CLEANUP_FREE char *err = NULL, *out = NULL;
+  CLEANUP_FREE_STRING_LIST char **lines = NULL;
+  int r;
+  size_t i;
+  int64_t volume_size = 0;
+  const char *size_pattern = "You might resize at ",
+             *full_pattern = "Volume is full",
+             *cluster_size_pattern = "Cluster size",
+             *volume_size_pattern = "Current volume size:";
+  int is_full = 0;
+  int32_t cluster_size = 0;
+
+  /* FS may be marked for check, so force ntfsresize */
+  r = command (&out, &err, str_ntfsresize, "--info",
"-ff", device, NULL);
+
+  lines = split_lines (out);
+  if (lines == NULL)
+    return -1;
+
+  if (verbose) {
+    for (i = 0; lines[i] != NULL; ++i)
+      fprintf (stderr, "ntfs_minimum_size: lines[%zu] =
\"%s\"\n", i, lines[i]);
+  }
+
+#if __WORDSIZE == 64
+#define XSTRTOD64 xstrtol
+#else
+#define XSTRTOD64 xstrtoll
+#endif
+
+  if (r == -1) {
+    /* If volume is full, ntfsresize returns error. */
+    for (i = 0; lines[i] != NULL; ++i) {
+      if (strstr (lines[i], full_pattern))
+        is_full = 1;
+      else if (STRPREFIX (lines[i], cluster_size_pattern)) {
+        if (sscanf (lines[i] + strlen (cluster_size_pattern),
+                    "%*[ ]:%" SCNd32, &cluster_size) != 1) {
+          reply_with_error ("cannot parse cluster size");
+          return -1;
+        }
+      }
+      else if (STRPREFIX (lines[i], volume_size_pattern)) {
+        if (XSTRTOD64 (lines[i] + strlen (volume_size_pattern),
+                       NULL, 20, &volume_size, NULL) != LONGINT_OK) {
+          reply_with_error ("cannot parse volume size");
+          return -1;
+        }
+      }
+    }
+    if (is_full) {
+      if (cluster_size == 0) {
+        reply_with_error ("bad cluster size");
+        return -1;
+      }
+      /* In case of a full filesystem, we estimate minimum size
+       * as volume size rounded up to cluster size.
+       */
+      return (volume_size + cluster_size - 1) / cluster_size * cluster_size;
+    }
+
+    reply_with_error ("%s", err);
+    return -1;
+  }
+
+  for (i = 0; lines[i] != NULL; ++i) {
+    if (STRPREFIX (lines[i], size_pattern)) {
+      int64_t ret;
+      if (XSTRTOD64 (lines[i] + strlen (size_pattern),
+                     NULL, 20, &ret, NULL) != LONGINT_OK) {
+        reply_with_error ("cannot parse minimum size");
+        return -1;
+      }
+      return ret;
+    }
+  }
+
+#undef XSTRTOD64
+
+  reply_with_error ("minimum size not found. Check output
format:\n%s", out);
+  return -1;
+}
+
 /* Takes optional arguments, consult optargs_bitmask. */
 int
 do_ntfsfix (const char *device, int clearbadsectors)
diff --git a/generator/actions.ml b/generator/actions.ml
index 274ef3f..e0459c0 100644
--- a/generator/actions.ml
+++ b/generator/actions.ml
@@ -12765,6 +12765,25 @@ Get the estimated minimum filesystem size of an
ext2/3/4 filesystem in blocks.

 See also L<resize2fs(8)>." };

+  { defaults with
+    name = "vfs_minimum_size"; added = (1, 31, 18);
+    style = RInt64 "sizeinbytes", [Mountable "mountable"],
[];
+    proc_nr = Some 458;
+    tests = [
+      InitPartition, IfAvailable "ntfsprogs", TestRun(
+        [["mkfs"; "ntfs"; "/dev/sda1";
""; "NOARG"; ""; ""; "NOARG"];
+         ["vfs_minimum_size"; "/dev/sda1"]]), [];
+    ];
+    shortdesc = "get minimum filesystem size";
+    longdesc = "\
+Get the minimum size of filesystem in bytes.
+This is the minimum possible size for filesystem shrinking.
+
+If getting minimum size of specified filesystem is not supported,
+this will fail and set errno as ENOTSUP.
+
+See also L<ntfsresize(8)>." };
+
 ]

 (* Non-API meta-commands available only in guestfish.
diff --git a/po/POTFILES b/po/POTFILES
index cd2c437..d90772a 100644
--- a/po/POTFILES
+++ b/po/POTFILES
@@ -49,6 +49,7 @@ daemon/file.c
 daemon/fill.c
 daemon/find.c
 daemon/findfs.c
+daemon/fs-min-size.c
 daemon/fsck.c
 daemon/fstrim.c
 daemon/glob.c
diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR
index de2a00c..c92ddb6 100644
--- a/src/MAX_PROC_NR
+++ b/src/MAX_PROC_NR
@@ -1 +1 @@
-457
+458
--
1.8.3.1

---
 daemon/daemon.h      |  1 +
 daemon/ext2.c        | 62 +++++++++++++++++++++++++++++++++++++++++++++-------
 daemon/fs-min-size.c |  3 +++
 generator/actions.ml | 20 ++++-------------
 src/MAX_PROC_NR      |  2 +-
 5 files changed, 63 insertions(+), 25 deletions(-)

diff --git a/daemon/daemon.h b/daemon/daemon.h
index 3b0266d..8bcc9bd 100644
--- a/daemon/daemon.h
+++ b/daemon/daemon.h
@@ -224,6 +224,7 @@ extern int sync_disks (void);
 #define EXT2_LABEL_MAX 16
 extern int fstype_is_extfs (const char *fstype);
 extern int ext_set_uuid_random (const char *device);
+extern int64_t ext_minimum_size (const char *device);

 /*-- in blkid.c --*/
 extern char *get_blkid_tag (const char *device, const char *tag);
diff --git a/daemon/ext2.c b/daemon/ext2.c
index 0cd6a66..01f934b 100644
--- a/daemon/ext2.c
+++ b/daemon/ext2.c
@@ -29,6 +29,7 @@
 #include "daemon.h"
 #include "c-ctype.h"
 #include "actions.h"
+#include "xstrtol.h"

 #define MAX_ARGS 128

@@ -279,15 +280,41 @@ do_resize2fs_M (const char *device)
   return 0;
 }

+static long
+get_block_size (const char *device)
+{
+  CLEANUP_FREE_STRING_LIST char **params = NULL;
+  const char *block_pattern = "Block size";
+  size_t i;
+  long block_size;
+
+  params = do_tune2fs_l (device);
+  if (params == NULL)
+    return -1;
+
+  for (i = 0; params[i] != NULL; i += 2) {
+    if (STREQ (params[i], block_pattern)) {
+      if (xstrtol (params[i + 1], NULL, 10, &block_size, NULL) !=
LONGINT_OK) {
+        reply_with_error ("cannot parse block size");
+        return -1;
+      }
+      return block_size;
+    }
+  }
+
+  reply_with_error ("missing 'Block size' in tune2fs_l
output");
+  return -1;
+}
+
 int64_t
-do_resize2fs_P (const char *device)
+ext_minimum_size (const char *device)
 {
   CLEANUP_FREE char *err = NULL, *out = NULL;
   CLEANUP_FREE_STRING_LIST char **lines = NULL;
   int r;
   size_t i;
-  char *p;
   int64_t ret;
+  long block_size;
   const char *pattern = "Estimated minimum size of the filesystem: ";

   r = command (&out, &err, str_resize2fs, "-P", device,
NULL);
@@ -300,17 +327,36 @@ do_resize2fs_P (const char *device)
   if (lines == NULL)
     return -1;

-  for (i = 0; lines[i] != NULL; ++i) {
-    if (verbose)
-      fprintf (stderr, "resize2fs_P: lines[%zu] =
\"%s\"\n", i, lines[i]);
+#if __WORDSIZE == 64
+#define XSTRTOD64 xstrtol
+#else
+#define XSTRTOD64 xstrtoll
+#endif

-    if ((p = strstr (lines[i], pattern))) {
-      if (sscanf (p + strlen(pattern), "%" SCNd64, &ret) != 1)
+  for (i = 0; lines[i] != NULL; ++i) {
+    if (STRPREFIX (lines[i], pattern)) {
+      if (XSTRTOD64 (lines[i] + strlen (pattern),
+                     NULL, 20, &ret, NULL) != LONGINT_OK) {
+        reply_with_error ("cannot parse minimum size");
+        return -1;
+      }
+      if ((block_size = get_block_size (device)) == -1)
+        return -1;
+      if (verbose) {
+        fprintf (stderr, "Minimum size in blocks: %" SCNd64 \
+                         "\nBlock count: %ld\n", ret, block_size);
+      }
+      if (INT64_MAX / block_size < ret) {
+        reply_with_error ("filesystem size too big: overflow");
         return -1;
-      return ret;
+      }
+      return ret * block_size;
     }
   }

+#undef XSTRTOD64
+
+  reply_with_error ("minimum size not found. Check output
format:\n%s", out);
   return -1;
 }

diff --git a/daemon/fs-min-size.c b/daemon/fs-min-size.c
index 652eb0e..4f93f8c 100644
--- a/daemon/fs-min-size.c
+++ b/daemon/fs-min-size.c
@@ -35,6 +35,9 @@ do_vfs_minimum_size (const mountable_t *mountable)
   if (vfs_type == NULL)
     return -1;

+  else if (fstype_is_extfs (vfs_type))
+    r = ext_minimum_size (mountable->device);
+
   else if (STREQ (vfs_type, "ntfs"))
     r = ntfs_minimum_size (mountable->device);

diff --git a/generator/actions.ml b/generator/actions.ml
index e0459c0..10acb7c 100644
--- a/generator/actions.ml
+++ b/generator/actions.ml
@@ -12752,24 +12752,12 @@ Only some filesystem types support setting UUIDs.
 To read the UUID on a filesystem, call C<guestfs_vfs_uuid>." };

   { defaults with
-    name = "resize2fs_P"; added = (1, 31, 17);
-    style = RInt64 "sizeinblocks", [Device "device"], [];
-    proc_nr = Some 457;
-    tests = [
-      InitBasicFS, Always, TestRun (
-        [["resize2fs_P"; "/dev/sda1"]]), [];
-    ];
-    shortdesc = "get minimum filesystem size in blocks";
-    longdesc = "\
-Get the estimated minimum filesystem size of an ext2/3/4 filesystem in blocks.
-
-See also L<resize2fs(8)>." };
-
-  { defaults with
     name = "vfs_minimum_size"; added = (1, 31, 18);
     style = RInt64 "sizeinbytes", [Mountable "mountable"],
[];
-    proc_nr = Some 458;
+    proc_nr = Some 457;
     tests = [
+      InitBasicFS, Always, TestRun (
+        [["vfs_minimum_size"; "/dev/sda1"]]), [];
       InitPartition, IfAvailable "ntfsprogs", TestRun(
         [["mkfs"; "ntfs"; "/dev/sda1";
""; "NOARG"; ""; ""; "NOARG"];
          ["vfs_minimum_size"; "/dev/sda1"]]), [];
@@ -12782,7 +12770,7 @@ This is the minimum possible size for filesystem
shrinking.
 If getting minimum size of specified filesystem is not supported,
 this will fail and set errno as ENOTSUP.

-See also L<ntfsresize(8)>." };
+See also L<ntfsresize(8)>, L<resize2fs(8)>." };

 ]

diff --git a/src/MAX_PROC_NR b/src/MAX_PROC_NR
index c92ddb6..de2a00c 100644
--- a/src/MAX_PROC_NR
+++ b/src/MAX_PROC_NR
@@ -1 +1 @@
-458
+457
--
1.8.3.1

On Tue, Oct 20, 2015 at 06:45:54PM +0300, Maxim Perevedentsev
wrote:
> Tried to make it in accordance with your comments.
> 
> Difference to v1:
> Added reply_with_error where necessary.
> Changed name get_min_size -> vfs_min_size.
> 
> Difference to v2:
> Changed name to vfs_minimum_size.
> Changed parsing to xstrtol + STR* macros where possible.
> 
> Difference to v3:
> Decapitalize error messages.
> 
> Maxim Perevedentsev (2):
>   New API: vfs_minimum_size
>   Include resize2fs_P into vfs_minimum_size.
> 
>  daemon/Makefile.am   |  1 +
>  daemon/daemon.h      |  2 ++
>  daemon/ext2.c        | 62 ++++++++++++++++++++++++++++++++-----
>  daemon/fs-min-size.c | 49 +++++++++++++++++++++++++++++
>  daemon/ntfs.c        | 87
++++++++++++++++++++++++++++++++++++++++++++++++++++
>  generator/actions.ml | 19 ++++++++----
>  po/POTFILES          |  1 +
>  7 files changed, 207 insertions(+), 14 deletions(-)
>  create mode 100644 daemon/fs-min-size.c
Looks good to me.  I'm going to run some tests on it now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

On Tuesday 20 October 2015 18:45:55 Maxim Perevedentsev
wrote:
> +int64_t
> +do_vfs_minimum_size (const mountable_t *mountable)
> +{
> +  int64_t r;
> +
> +  /* How we set the label depends on the filesystem type. */
Just noticed now: this comment should be really fitting more the new
function... :)

-- 
Pino Toscano

On 10/20/2015 07:35 PM, Richard W.M. Jones wrote:
> Looks good to me. I'm going to run some tests on it now. Rich. 
Thanks. I'm looking forward to hear from you!

-- 
Your sincerely,
Maxim Perevedentsev

I have an issue with btrfs:

There's an API
 > btrfs inspect-internal min-dev-size /sysroot/

But this API is available only in btrfs-progs >= 4.2.
Before this, there's no reliable way to get minimum size (I could parse 
"btrfs filesystem show", but it is veeery inaccurate).

Can we require btrfs-progs v4.2? Or how should I work-around this issue?

-- 
Your sincerely,
Maxim Perevedentsev

On Tue, Oct 20, 2015 at 08:56:25PM +0300, Maxim Perevedentsev
wrote:
> I have an issue with btrfs:
> 
> There's an API
> > btrfs inspect-internal min-dev-size /sysroot/
> 
> But this API is available only in btrfs-progs >= 4.2.
> Before this, there's no reliable way to get minimum size (I could
> parse "btrfs filesystem show", but it is veeery inaccurate).
> 
> Can we require btrfs-progs v4.2? Or how should I work-around this issue?
Well it's a new API, so we can just tell people they need new
btrfs-progs if they want to use the new API.

It would be a problem if we were changing an old API, but that's not
the case here.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On Tuesday 20 October 2015 20:56:25 Maxim Perevedentsev
wrote:
> I have an issue with btrfs:
> 
> There's an API
>  > btrfs inspect-internal min-dev-size /sysroot/
> 
> But this API is available only in btrfs-progs >= 4.2.
> Before this, there's no reliable way to get minimum size (I could parse
> "btrfs filesystem show", but it is veeery inaccurate).
> 
> Can we require btrfs-progs v4.2? Or how should I work-around this issue?
The usual way in cases like this is to detect whether the needed tools
are available, or the needed versions of them, and if not report an
error (or just ENOTSUP).  See for example the btrfs code for setting the
UUID in a btrfs filesystem (daemon/btrfs.c:btrfs_set_uuid).

-- 
Pino Toscano