Libguestfs - May 2016 - [PATCH supermin] Add support for a DAX root filesystem.

DAX is explained in detail here:
https://lwn.net/Articles/610174/

This patch adds support to supermin for using a DAX root filesystem.
The corresponding libguestfs patches will be posted shortly once I've
tested them a bit more.

This requires qemu >= 2.6.

Unfortunately it's not really a win for a few reasons:

 - Requires enabling ACPI, which slows everything down by hundreds of
   milliseconds.  I need to work out exactly what's going on here.
   Since we never required ACPI before now, I never had the impetus to
   find out if this is easy to fix.

 - The extra modules in the supermin mini-initrd greatly increase its
   size, which also slows things down.  qemu fw_cfg DMA should help
   here, if we ever manage to get it upstream.

 - vNVDIMM and DAX are not very widely tested or used yet.

 - Not supported at all by libvirt.

Rich.

With these changes you can provide an ext4 root filesystem on a
virtual NVDIMM.  The filesystem will be mounted using DAX, so files
are directly read, mmapped etc from the underlying host file, saving
guest memory both directly and in the guest page cache.
---
 init/init.c        | 19 ++++++++++++++++---
 src/ext2_initrd.ml |  3 +++
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/init/init.c b/init/init.c
index 3014de2..106be02 100644
--- a/init/init.c
+++ b/init/init.c
@@ -154,11 +154,14 @@ main ()
    */
   char *root, *path;
   size_t len;
+  int dax = 0;
   root = strstr (cmdline, "root=");
   if (root) {
     root += 5;
     if (strncmp (root, "/dev/", 5) == 0)
       root += 5;
+    if (strncmp (root, "pmem", 4) == 0)
+      dax = 1;
     len = strcspn (root, " ");
     root[len] = '\0';
 
@@ -243,10 +246,20 @@ main ()
     exit (EXIT_FAILURE);
   }
 
+  /* Construct the filesystem mount options. */
+  const char *mount_options = "";
+  if (dax)
+    mount_options = "dax";
+
   /* Mount new root and chroot to it. */
-  if (!quiet)
-    fprintf (stderr, "supermin: mounting new root on /root\n");
-  if (mount ("/dev/root", "/root", "ext2",
MS_NOATIME, "") == -1) {
+  if (!quiet) {
+    fprintf (stderr, "supermin: mounting new root on /root");
+    if (mount_options[0] != '\0')
+      fprintf (stderr, " (%s)", mount_options);
+    fprintf (stderr, "\n");
+  }
+  if (mount ("/dev/root", "/root", "ext2",
MS_NOATIME,
+             mount_options) == -1) {
     perror ("mount: /root");
     exit (EXIT_FAILURE);
   }
diff --git a/src/ext2_initrd.ml b/src/ext2_initrd.ml
index 55a38d0..dd7d5a3 100644
--- a/src/ext2_initrd.ml
+++ b/src/ext2_initrd.ml
@@ -45,6 +45,9 @@ let kmods = [
   "crc*.ko*";
   "libcrc*.ko*";
   "ibmvscsic.ko*";
+  "nd_pmem.ko*";
+  "nd_btt.ko*";
+  "nfit.ko*";
 ]
 
 (* A blacklist of kmods which match the above patterns, but which we
-- 
2.7.4

On Mon, May 16, 2016 at 07:09:44PM +0100, Richard W.M. Jones
wrote:
> DAX is explained in detail here:
> https://lwn.net/Articles/610174/
> 
> This patch adds support to supermin for using a DAX root filesystem.
> The corresponding libguestfs patches will be posted shortly once I've
> tested them a bit more.
> 
> This requires qemu >= 2.6.
> 
> Unfortunately it's not really a win for a few reasons:
> 
>  - Requires enabling ACPI, which slows everything down by hundreds of
>    milliseconds.  I need to work out exactly what's going on here.
>    Since we never required ACPI before now, I never had the impetus to
>    find out if this is easy to fix.
Good news is I have fixed the ACPI issue.  The slowdown is now
very minor (< 20ms).

Rich.
>  - The extra modules in the supermin mini-initrd greatly increase its
>    size, which also slows things down.  qemu fw_cfg DMA should help
>    here, if we ever manage to get it upstream.
> 
>  - vNVDIMM and DAX are not very widely tested or used yet.
> 
>  - Not supported at all by libvirt.
> 
> Rich.
> 
> _______________________________________________
> Libguestfs mailing list
> Libguestfs@redhat.com
> https://www.redhat.com/mailman/listinfo/libguestfs
-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

On Mon, May 16, 2016 at 07:09:44PM +0100, Richard W.M. Jones
wrote:
>  - The extra modules in the supermin mini-initrd greatly increase its
>    size, which also slows things down.  qemu fw_cfg DMA should help
>    here, if we ever manage to get it upstream.
Actually this one is fine too.  It turns out the overhead of the extra
modules is modest.  I was looking at the init binary which I had
accidentally linked to glibc instead of dietlibc.

So I think this supermin patch is fine, it can go upstream as it is.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/