Richard W.M. Jones
2019-Feb-19 07:49 UTC
[Libguestfs] [PATCH nbdkit 0/4] New plugin: Add linuxdisk plugin.
Turns out Japanese trains are good for coding! In supermin we have a bunch of code to create the libguestfs appliance. It creates it directly using libext2fs (part of e2fsprogs). We can use the same technique to create ext2 virtual disks in nbdkit, which is what this new plugin does. Why a new plugin instead of modifying the floppy plugin? See the 4/4 commit message for an explanation. The first 3 patches are pretty uncontroversial refactoring. The 4th patch contains the plugin and tests. It's probably best to start with the commit message and the included man page. This is quick even for relatively large filesystems containing large numbers of small files. Here's an example for the nbdkit source code (including the .git directory): $ time ./nbdkit --filter=partition -U - linuxdisk . partition=1 --run 'qemu-img convert $nbd /var/tmp/ext2fs.img' real 0m1.314s user 0m0.424s sys 0m0.889s $ ls -lh /var/tmp/ext2fs.img -rw-r--r--. 1 rjones rjones 351M Feb 19 07:44 /var/tmp/ext2fs.img The tests + valgrind pass on Fedora 29 & RHEL 7. Rich.
Richard W.M. Jones
2019-Feb-19 07:49 UTC
[Libguestfs] [PATCH nbdkit 1/4] common: Move some GPT functionality to a common directory.
From: "Richard W.M. Jones" <rjones@redhat.com>
Headers and code related to GUID Partition Tables are moved to a
common directory to allow us to share that code across multiple
plugins and filters in future. This change is pure refactoring.
---
Makefile.am | 1 +
common/gpt/Makefile.am | 42 +++++++++++
common/gpt/efi-crc32.c | 140 +++++++++++++++++++++++++++++++++++
common/gpt/efi-crc32.h | 42 +++++++++++
common/gpt/gpt.h | 80 ++++++++++++++++++++
configure.ac | 1 +
plugins/partitioning/Makefile.am | 4 +-
plugins/partitioning/efi-crc32.c | 140 -----------------------------------
plugins/partitioning/efi-crc32.h | 42 -----------
plugins/partitioning/partition-gpt.c | 31 ++------
plugins/partitioning/virtual-disk.h | 13 +---
11 files changed, 314 insertions(+), 222 deletions(-)
create mode 100644 common/gpt/Makefile.am
create mode 100644 common/gpt/efi-crc32.c
create mode 100644 common/gpt/efi-crc32.h
create mode 100644 common/gpt/gpt.h
delete mode 100644 plugins/partitioning/efi-crc32.c
delete mode 100644 plugins/partitioning/efi-crc32.h
diff --git a/Makefile.am b/Makefile.am
index 273accb..653956f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -71,6 +71,7 @@ SUBDIRS = \
if HAVE_PLUGINS
SUBDIRS += \
common/bitmap \
+ common/gpt \
common/regions \
common/sparse \
plugins \
diff --git a/common/gpt/Makefile.am b/common/gpt/Makefile.am
new file mode 100644
index 0000000..20bf62f
--- /dev/null
+++ b/common/gpt/Makefile.am
@@ -0,0 +1,42 @@
+# nbdkit
+# Copyright (C) 2019 Red Hat Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+include $(top_srcdir)/common-rules.mk
+
+noinst_LTLIBRARIES = libgpt.la
+
+libgpt_la_SOURCES = \
+ efi-crc32.c \
+ efi-crc32.h \
+ gpt.h
+libgpt_la_CFLAGS = \
+ $(WARNINGS_CFLAGS)
diff --git a/common/gpt/efi-crc32.c b/common/gpt/efi-crc32.c
new file mode 100644
index 0000000..2b080d3
--- /dev/null
+++ b/common/gpt/efi-crc32.c
@@ -0,0 +1,140 @@
+/* This code was taken from parted and indirectly from other sources
+ * as you can see from the messages below. The license is compatible
+ * with the permissive license used in nbdkit. - RWMJ 2018-09-16
+ */
+
+/*
+ * Dec 5, 2000 Matt Domsch <Matt_Domsch@dell.com>
+ * - Copied crc32.c from the linux/drivers/net/cipe directory.
+ * - Now pass seed as an arg
+ * - changed unsigned long to uint32_t, added #include<stdint.h>
+ * - changed len to be an unsigned long
+ * - changed crc32val to be a register
+ * - License remains unchanged! It's still GPL-compatable!
+ */
+
+ /* ============================================================= */
+ /* COPYRIGHT (C) 1986 Gary S. Brown. You may use this program, or */
+ /* code or tables extracted from it, as desired without restriction. */
+ /* */
+ /* First, the polynomial itself and its table of feedback terms. The */
+ /* polynomial is */
+ /* X^32+X^26+X^23+X^22+X^16+X^12+X^11+X^10+X^8+X^7+X^5+X^4+X^2+X^1+X^0 */
+ /* */
+ /* Note that we take it "backwards" and put the highest-order term
in */
+ /* the lowest-order bit. The X^32 term is "implied"; the LSB is
the */
+ /* X^31 term, etc. The X^0 term (usually shown as "+1") results
in */
+ /* the MSB being 1. */
+ /* */
+ /* Note that the usual hardware shift register implementation, which */
+ /* is what we're using (we're merely optimizing it by doing
eight-bit */
+ /* chunks at a time) shifts bits into the lowest-order term. In our */
+ /* implementation, that means shifting towards the right. Why do we */
+ /* do it this way? Because the calculated CRC must be transmitted in */
+ /* order from highest-order term to lowest-order term. UARTs transmit */
+ /* characters in order from LSB to MSB. By storing the CRC this way, */
+ /* we hand it to the UART in the order low-byte to high-byte; the UART */
+ /* sends each low-bit to hight-bit; and the result is transmission bit */
+ /* by bit from highest- to lowest-order term without requiring any bit */
+ /* shuffling on our part. Reception works similarly. */
+ /* */
+ /* The feedback terms table consists of 256, 32-bit entries. Notes: */
+ /* */
+ /* The table can be generated at runtime if desired; code to do so */
+ /* is shown later. It might not be obvious, but the feedback */
+ /* terms simply represent the results of eight shift/xor opera- */
+ /* tions for all combinations of data and CRC register values. */
+ /* */
+ /* The values must be right-shifted by eight bits by the
"updcrc" */
+ /* logic; the shift must be unsigned (bring in zeroes). On some */
+ /* hardware you could probably optimize the shift in assembler by */
+ /* using byte-swap instructions. */
+ /* polynomial $edb88320 */
+ /* */
+ /* -------------------------------------------------------------------- */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdint.h>
+
+#include "efi-crc32.h"
+
+static const uint32_t crc32_tab[] = {
+ 0x00000000L, 0x77073096L, 0xee0e612cL, 0x990951baL, 0x076dc419L,
+ 0x706af48fL, 0xe963a535L, 0x9e6495a3L, 0x0edb8832L, 0x79dcb8a4L,
+ 0xe0d5e91eL, 0x97d2d988L, 0x09b64c2bL, 0x7eb17cbdL, 0xe7b82d07L,
+ 0x90bf1d91L, 0x1db71064L, 0x6ab020f2L, 0xf3b97148L, 0x84be41deL,
+ 0x1adad47dL, 0x6ddde4ebL, 0xf4d4b551L, 0x83d385c7L, 0x136c9856L,
+ 0x646ba8c0L, 0xfd62f97aL, 0x8a65c9ecL, 0x14015c4fL, 0x63066cd9L,
+ 0xfa0f3d63L, 0x8d080df5L, 0x3b6e20c8L, 0x4c69105eL, 0xd56041e4L,
+ 0xa2677172L, 0x3c03e4d1L, 0x4b04d447L, 0xd20d85fdL, 0xa50ab56bL,
+ 0x35b5a8faL, 0x42b2986cL, 0xdbbbc9d6L, 0xacbcf940L, 0x32d86ce3L,
+ 0x45df5c75L, 0xdcd60dcfL, 0xabd13d59L, 0x26d930acL, 0x51de003aL,
+ 0xc8d75180L, 0xbfd06116L, 0x21b4f4b5L, 0x56b3c423L, 0xcfba9599L,
+ 0xb8bda50fL, 0x2802b89eL, 0x5f058808L, 0xc60cd9b2L, 0xb10be924L,
+ 0x2f6f7c87L, 0x58684c11L, 0xc1611dabL, 0xb6662d3dL, 0x76dc4190L,
+ 0x01db7106L, 0x98d220bcL, 0xefd5102aL, 0x71b18589L, 0x06b6b51fL,
+ 0x9fbfe4a5L, 0xe8b8d433L, 0x7807c9a2L, 0x0f00f934L, 0x9609a88eL,
+ 0xe10e9818L, 0x7f6a0dbbL, 0x086d3d2dL, 0x91646c97L, 0xe6635c01L,
+ 0x6b6b51f4L, 0x1c6c6162L, 0x856530d8L, 0xf262004eL, 0x6c0695edL,
+ 0x1b01a57bL, 0x8208f4c1L, 0xf50fc457L, 0x65b0d9c6L, 0x12b7e950L,
+ 0x8bbeb8eaL, 0xfcb9887cL, 0x62dd1ddfL, 0x15da2d49L, 0x8cd37cf3L,
+ 0xfbd44c65L, 0x4db26158L, 0x3ab551ceL, 0xa3bc0074L, 0xd4bb30e2L,
+ 0x4adfa541L, 0x3dd895d7L, 0xa4d1c46dL, 0xd3d6f4fbL, 0x4369e96aL,
+ 0x346ed9fcL, 0xad678846L, 0xda60b8d0L, 0x44042d73L, 0x33031de5L,
+ 0xaa0a4c5fL, 0xdd0d7cc9L, 0x5005713cL, 0x270241aaL, 0xbe0b1010L,
+ 0xc90c2086L, 0x5768b525L, 0x206f85b3L, 0xb966d409L, 0xce61e49fL,
+ 0x5edef90eL, 0x29d9c998L, 0xb0d09822L, 0xc7d7a8b4L, 0x59b33d17L,
+ 0x2eb40d81L, 0xb7bd5c3bL, 0xc0ba6cadL, 0xedb88320L, 0x9abfb3b6L,
+ 0x03b6e20cL, 0x74b1d29aL, 0xead54739L, 0x9dd277afL, 0x04db2615L,
+ 0x73dc1683L, 0xe3630b12L, 0x94643b84L, 0x0d6d6a3eL, 0x7a6a5aa8L,
+ 0xe40ecf0bL, 0x9309ff9dL, 0x0a00ae27L, 0x7d079eb1L, 0xf00f9344L,
+ 0x8708a3d2L, 0x1e01f268L, 0x6906c2feL, 0xf762575dL, 0x806567cbL,
+ 0x196c3671L, 0x6e6b06e7L, 0xfed41b76L, 0x89d32be0L, 0x10da7a5aL,
+ 0x67dd4accL, 0xf9b9df6fL, 0x8ebeeff9L, 0x17b7be43L, 0x60b08ed5L,
+ 0xd6d6a3e8L, 0xa1d1937eL, 0x38d8c2c4L, 0x4fdff252L, 0xd1bb67f1L,
+ 0xa6bc5767L, 0x3fb506ddL, 0x48b2364bL, 0xd80d2bdaL, 0xaf0a1b4cL,
+ 0x36034af6L, 0x41047a60L, 0xdf60efc3L, 0xa867df55L, 0x316e8eefL,
+ 0x4669be79L, 0xcb61b38cL, 0xbc66831aL, 0x256fd2a0L, 0x5268e236L,
+ 0xcc0c7795L, 0xbb0b4703L, 0x220216b9L, 0x5505262fL, 0xc5ba3bbeL,
+ 0xb2bd0b28L, 0x2bb45a92L, 0x5cb36a04L, 0xc2d7ffa7L, 0xb5d0cf31L,
+ 0x2cd99e8bL, 0x5bdeae1dL, 0x9b64c2b0L, 0xec63f226L, 0x756aa39cL,
+ 0x026d930aL, 0x9c0906a9L, 0xeb0e363fL, 0x72076785L, 0x05005713L,
+ 0x95bf4a82L, 0xe2b87a14L, 0x7bb12baeL, 0x0cb61b38L, 0x92d28e9bL,
+ 0xe5d5be0dL, 0x7cdcefb7L, 0x0bdbdf21L, 0x86d3d2d4L, 0xf1d4e242L,
+ 0x68ddb3f8L, 0x1fda836eL, 0x81be16cdL, 0xf6b9265bL, 0x6fb077e1L,
+ 0x18b74777L, 0x88085ae6L, 0xff0f6a70L, 0x66063bcaL, 0x11010b5cL,
+ 0x8f659effL, 0xf862ae69L, 0x616bffd3L, 0x166ccf45L, 0xa00ae278L,
+ 0xd70dd2eeL, 0x4e048354L, 0x3903b3c2L, 0xa7672661L, 0xd06016f7L,
+ 0x4969474dL, 0x3e6e77dbL, 0xaed16a4aL, 0xd9d65adcL, 0x40df0b66L,
+ 0x37d83bf0L, 0xa9bcae53L, 0xdebb9ec5L, 0x47b2cf7fL, 0x30b5ffe9L,
+ 0xbdbdf21cL, 0xcabac28aL, 0x53b39330L, 0x24b4a3a6L, 0xbad03605L,
+ 0xcdd70693L, 0x54de5729L, 0x23d967bfL, 0xb3667a2eL, 0xc4614ab8L,
+ 0x5d681b02L, 0x2a6f2b94L, 0xb40bbe37L, 0xc30c8ea1L, 0x5a05df1bL,
+ 0x2d02ef8dL
+ };
+
+/* Return a 32-bit CRC of the contents of the buffer. */
+
+static uint32_t
+_efi_crc32 (const void *buf, size_t len, uint32_t seed)
+{
+ size_t i;
+ uint32_t crc32val;
+ const unsigned char *s = buf;
+
+ crc32val = seed;
+ for (i = 0; i < len; i++) {
+ crc32val + crc32_tab[(crc32val ^ s[i]) & 0xff] ^
+ (crc32val >> 8);
+ }
+ return crc32val;
+}
+
+uint32_t
+efi_crc32 (const void *buf, size_t len)
+{
+ return _efi_crc32 (buf, len, ~0L) ^ ~0L;
+}
diff --git a/common/gpt/efi-crc32.h b/common/gpt/efi-crc32.h
new file mode 100644
index 0000000..21e28d7
--- /dev/null
+++ b/common/gpt/efi-crc32.h
@@ -0,0 +1,42 @@
+/* nbdkit
+ * Copyright (C) 2018 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#ifndef NBDKIT_EFI_CRC32_H
+#define NBDKIT_EFI_CRC32_H
+
+#include <stdint.h>
+
+extern uint32_t efi_crc32 (const void *buf, size_t len)
+ __attribute__((__nonnull__ (1)));
+
+#endif /* NBDKIT_EFI_CRC32_H */
diff --git a/common/gpt/gpt.h b/common/gpt/gpt.h
new file mode 100644
index 0000000..7e2a2ec
--- /dev/null
+++ b/common/gpt/gpt.h
@@ -0,0 +1,80 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#ifndef NBDKIT_GPT_H
+#define NBDKIT_GPT_H
+
+struct gpt_header {
+ char signature[8];
+ char revision[4];
+ uint32_t header_size;
+ uint32_t crc;
+ uint32_t reserved;
+ uint64_t current_lba;
+ uint64_t backup_lba;
+ uint64_t first_usable_lba;
+ uint64_t last_usable_lba;
+ char guid[16];
+ uint64_t partition_entries_lba;
+ uint32_t nr_partition_entries;
+ uint32_t size_partition_entry;
+ uint32_t crc_partitions;
+};
+
+#define GPT_SIGNATURE "EFI PART"
+#define GPT_REVISION "\0\0\1\0" /* revision 1.0 */
+
+struct gpt_entry {
+ char partition_type_guid[16];
+ char unique_guid[16];
+ uint64_t first_lba;
+ uint64_t last_lba;
+ uint64_t attributes;
+ char name[72]; /* UTF-16LE */
+};
+
+/* GPT_MIN_PARTITIONS is the minimum number of partitions and is
+ * defined by the UEFI standard (assuming 512 byte sector size).
+ *
+ * In plugins such as the partitioning plugin, if we are requested to
+ * allocate more than GPT_MIN_PARTITIONS then we increase the
+ * partition table in chunks of this size. Note that clients may not
+ * support > GPT_MIN_PARTITIONS.
+ *
+ * GPT_PT_ENTRY_SIZE is the minimum specified by the UEFI spec, but
+ * increasing it is not useful.
+ */
+#define GPT_MIN_PARTITIONS 128
+#define GPT_PT_ENTRY_SIZE 128
+
+#endif /* NBDKIT_GPT_H */
diff --git a/configure.ac b/configure.ac
index a0f45e8..bca6134 100644
--- a/configure.ac
+++ b/configure.ac
@@ -833,6 +833,7 @@ AC_CONFIG_FILES([podwrapper.pl],
AC_CONFIG_FILES([Makefile
bash/Makefile
common/bitmap/Makefile
+ common/gpt/Makefile
common/include/Makefile
common/regions/Makefile
common/sparse/Makefile
diff --git a/plugins/partitioning/Makefile.am b/plugins/partitioning/Makefile.am
index 42e0c45..fa0266e 100644
--- a/plugins/partitioning/Makefile.am
+++ b/plugins/partitioning/Makefile.am
@@ -37,8 +37,6 @@ EXTRA_DIST = nbdkit-partitioning-plugin.pod
plugin_LTLIBRARIES = nbdkit-partitioning-plugin.la
nbdkit_partitioning_plugin_la_SOURCES = \
- efi-crc32.c \
- efi-crc32.h \
partitioning.c \
partition-gpt.c \
partition-mbr.c \
@@ -48,6 +46,7 @@ nbdkit_partitioning_plugin_la_SOURCES = \
nbdkit_partitioning_plugin_la_CPPFLAGS = \
-I$(top_srcdir)/include \
+ -I$(top_srcdir)/common/gpt \
-I$(top_srcdir)/common/include \
-I$(top_srcdir)/common/regions \
-I.
@@ -57,6 +56,7 @@ nbdkit_partitioning_plugin_la_LDFLAGS = \
-module -avoid-version -shared \
-Wl,--version-script=$(top_srcdir)/plugins/plugins.syms
nbdkit_partitioning_plugin_la_LIBADD = \
+ $(top_builddir)/common/gpt/libgpt.la \
$(top_builddir)/common/regions/libregions.la
if HAVE_POD
diff --git a/plugins/partitioning/efi-crc32.c b/plugins/partitioning/efi-crc32.c
deleted file mode 100644
index 2b080d3..0000000
--- a/plugins/partitioning/efi-crc32.c
+++ /dev/null
@@ -1,140 +0,0 @@
-/* This code was taken from parted and indirectly from other sources
- * as you can see from the messages below. The license is compatible
- * with the permissive license used in nbdkit. - RWMJ 2018-09-16
- */
-
-/*
- * Dec 5, 2000 Matt Domsch <Matt_Domsch@dell.com>
- * - Copied crc32.c from the linux/drivers/net/cipe directory.
- * - Now pass seed as an arg
- * - changed unsigned long to uint32_t, added #include<stdint.h>
- * - changed len to be an unsigned long
- * - changed crc32val to be a register
- * - License remains unchanged! It's still GPL-compatable!
- */
-
- /* ============================================================= */
- /* COPYRIGHT (C) 1986 Gary S. Brown. You may use this program, or */
- /* code or tables extracted from it, as desired without restriction. */
- /* */
- /* First, the polynomial itself and its table of feedback terms. The */
- /* polynomial is */
- /* X^32+X^26+X^23+X^22+X^16+X^12+X^11+X^10+X^8+X^7+X^5+X^4+X^2+X^1+X^0 */
- /* */
- /* Note that we take it "backwards" and put the highest-order term
in */
- /* the lowest-order bit. The X^32 term is "implied"; the LSB is
the */
- /* X^31 term, etc. The X^0 term (usually shown as "+1") results
in */
- /* the MSB being 1. */
- /* */
- /* Note that the usual hardware shift register implementation, which */
- /* is what we're using (we're merely optimizing it by doing
eight-bit */
- /* chunks at a time) shifts bits into the lowest-order term. In our */
- /* implementation, that means shifting towards the right. Why do we */
- /* do it this way? Because the calculated CRC must be transmitted in */
- /* order from highest-order term to lowest-order term. UARTs transmit */
- /* characters in order from LSB to MSB. By storing the CRC this way, */
- /* we hand it to the UART in the order low-byte to high-byte; the UART */
- /* sends each low-bit to hight-bit; and the result is transmission bit */
- /* by bit from highest- to lowest-order term without requiring any bit */
- /* shuffling on our part. Reception works similarly. */
- /* */
- /* The feedback terms table consists of 256, 32-bit entries. Notes: */
- /* */
- /* The table can be generated at runtime if desired; code to do so */
- /* is shown later. It might not be obvious, but the feedback */
- /* terms simply represent the results of eight shift/xor opera- */
- /* tions for all combinations of data and CRC register values. */
- /* */
- /* The values must be right-shifted by eight bits by the
"updcrc" */
- /* logic; the shift must be unsigned (bring in zeroes). On some */
- /* hardware you could probably optimize the shift in assembler by */
- /* using byte-swap instructions. */
- /* polynomial $edb88320 */
- /* */
- /* -------------------------------------------------------------------- */
-
-#include <config.h>
-
-#include <stdio.h>
-#include <stdint.h>
-
-#include "efi-crc32.h"
-
-static const uint32_t crc32_tab[] = {
- 0x00000000L, 0x77073096L, 0xee0e612cL, 0x990951baL, 0x076dc419L,
- 0x706af48fL, 0xe963a535L, 0x9e6495a3L, 0x0edb8832L, 0x79dcb8a4L,
- 0xe0d5e91eL, 0x97d2d988L, 0x09b64c2bL, 0x7eb17cbdL, 0xe7b82d07L,
- 0x90bf1d91L, 0x1db71064L, 0x6ab020f2L, 0xf3b97148L, 0x84be41deL,
- 0x1adad47dL, 0x6ddde4ebL, 0xf4d4b551L, 0x83d385c7L, 0x136c9856L,
- 0x646ba8c0L, 0xfd62f97aL, 0x8a65c9ecL, 0x14015c4fL, 0x63066cd9L,
- 0xfa0f3d63L, 0x8d080df5L, 0x3b6e20c8L, 0x4c69105eL, 0xd56041e4L,
- 0xa2677172L, 0x3c03e4d1L, 0x4b04d447L, 0xd20d85fdL, 0xa50ab56bL,
- 0x35b5a8faL, 0x42b2986cL, 0xdbbbc9d6L, 0xacbcf940L, 0x32d86ce3L,
- 0x45df5c75L, 0xdcd60dcfL, 0xabd13d59L, 0x26d930acL, 0x51de003aL,
- 0xc8d75180L, 0xbfd06116L, 0x21b4f4b5L, 0x56b3c423L, 0xcfba9599L,
- 0xb8bda50fL, 0x2802b89eL, 0x5f058808L, 0xc60cd9b2L, 0xb10be924L,
- 0x2f6f7c87L, 0x58684c11L, 0xc1611dabL, 0xb6662d3dL, 0x76dc4190L,
- 0x01db7106L, 0x98d220bcL, 0xefd5102aL, 0x71b18589L, 0x06b6b51fL,
- 0x9fbfe4a5L, 0xe8b8d433L, 0x7807c9a2L, 0x0f00f934L, 0x9609a88eL,
- 0xe10e9818L, 0x7f6a0dbbL, 0x086d3d2dL, 0x91646c97L, 0xe6635c01L,
- 0x6b6b51f4L, 0x1c6c6162L, 0x856530d8L, 0xf262004eL, 0x6c0695edL,
- 0x1b01a57bL, 0x8208f4c1L, 0xf50fc457L, 0x65b0d9c6L, 0x12b7e950L,
- 0x8bbeb8eaL, 0xfcb9887cL, 0x62dd1ddfL, 0x15da2d49L, 0x8cd37cf3L,
- 0xfbd44c65L, 0x4db26158L, 0x3ab551ceL, 0xa3bc0074L, 0xd4bb30e2L,
- 0x4adfa541L, 0x3dd895d7L, 0xa4d1c46dL, 0xd3d6f4fbL, 0x4369e96aL,
- 0x346ed9fcL, 0xad678846L, 0xda60b8d0L, 0x44042d73L, 0x33031de5L,
- 0xaa0a4c5fL, 0xdd0d7cc9L, 0x5005713cL, 0x270241aaL, 0xbe0b1010L,
- 0xc90c2086L, 0x5768b525L, 0x206f85b3L, 0xb966d409L, 0xce61e49fL,
- 0x5edef90eL, 0x29d9c998L, 0xb0d09822L, 0xc7d7a8b4L, 0x59b33d17L,
- 0x2eb40d81L, 0xb7bd5c3bL, 0xc0ba6cadL, 0xedb88320L, 0x9abfb3b6L,
- 0x03b6e20cL, 0x74b1d29aL, 0xead54739L, 0x9dd277afL, 0x04db2615L,
- 0x73dc1683L, 0xe3630b12L, 0x94643b84L, 0x0d6d6a3eL, 0x7a6a5aa8L,
- 0xe40ecf0bL, 0x9309ff9dL, 0x0a00ae27L, 0x7d079eb1L, 0xf00f9344L,
- 0x8708a3d2L, 0x1e01f268L, 0x6906c2feL, 0xf762575dL, 0x806567cbL,
- 0x196c3671L, 0x6e6b06e7L, 0xfed41b76L, 0x89d32be0L, 0x10da7a5aL,
- 0x67dd4accL, 0xf9b9df6fL, 0x8ebeeff9L, 0x17b7be43L, 0x60b08ed5L,
- 0xd6d6a3e8L, 0xa1d1937eL, 0x38d8c2c4L, 0x4fdff252L, 0xd1bb67f1L,
- 0xa6bc5767L, 0x3fb506ddL, 0x48b2364bL, 0xd80d2bdaL, 0xaf0a1b4cL,
- 0x36034af6L, 0x41047a60L, 0xdf60efc3L, 0xa867df55L, 0x316e8eefL,
- 0x4669be79L, 0xcb61b38cL, 0xbc66831aL, 0x256fd2a0L, 0x5268e236L,
- 0xcc0c7795L, 0xbb0b4703L, 0x220216b9L, 0x5505262fL, 0xc5ba3bbeL,
- 0xb2bd0b28L, 0x2bb45a92L, 0x5cb36a04L, 0xc2d7ffa7L, 0xb5d0cf31L,
- 0x2cd99e8bL, 0x5bdeae1dL, 0x9b64c2b0L, 0xec63f226L, 0x756aa39cL,
- 0x026d930aL, 0x9c0906a9L, 0xeb0e363fL, 0x72076785L, 0x05005713L,
- 0x95bf4a82L, 0xe2b87a14L, 0x7bb12baeL, 0x0cb61b38L, 0x92d28e9bL,
- 0xe5d5be0dL, 0x7cdcefb7L, 0x0bdbdf21L, 0x86d3d2d4L, 0xf1d4e242L,
- 0x68ddb3f8L, 0x1fda836eL, 0x81be16cdL, 0xf6b9265bL, 0x6fb077e1L,
- 0x18b74777L, 0x88085ae6L, 0xff0f6a70L, 0x66063bcaL, 0x11010b5cL,
- 0x8f659effL, 0xf862ae69L, 0x616bffd3L, 0x166ccf45L, 0xa00ae278L,
- 0xd70dd2eeL, 0x4e048354L, 0x3903b3c2L, 0xa7672661L, 0xd06016f7L,
- 0x4969474dL, 0x3e6e77dbL, 0xaed16a4aL, 0xd9d65adcL, 0x40df0b66L,
- 0x37d83bf0L, 0xa9bcae53L, 0xdebb9ec5L, 0x47b2cf7fL, 0x30b5ffe9L,
- 0xbdbdf21cL, 0xcabac28aL, 0x53b39330L, 0x24b4a3a6L, 0xbad03605L,
- 0xcdd70693L, 0x54de5729L, 0x23d967bfL, 0xb3667a2eL, 0xc4614ab8L,
- 0x5d681b02L, 0x2a6f2b94L, 0xb40bbe37L, 0xc30c8ea1L, 0x5a05df1bL,
- 0x2d02ef8dL
- };
-
-/* Return a 32-bit CRC of the contents of the buffer. */
-
-static uint32_t
-_efi_crc32 (const void *buf, size_t len, uint32_t seed)
-{
- size_t i;
- uint32_t crc32val;
- const unsigned char *s = buf;
-
- crc32val = seed;
- for (i = 0; i < len; i++) {
- crc32val - crc32_tab[(crc32val ^ s[i]) & 0xff] ^
- (crc32val >> 8);
- }
- return crc32val;
-}
-
-uint32_t
-efi_crc32 (const void *buf, size_t len)
-{
- return _efi_crc32 (buf, len, ~0L) ^ ~0L;
-}
diff --git a/plugins/partitioning/efi-crc32.h b/plugins/partitioning/efi-crc32.h
deleted file mode 100644
index 21e28d7..0000000
--- a/plugins/partitioning/efi-crc32.h
+++ /dev/null
@@ -1,42 +0,0 @@
-/* nbdkit
- * Copyright (C) 2018 Red Hat Inc.
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- *
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * * Neither the name of Red Hat nor the names of its contributors may be
- * used to endorse or promote products derived from this software without
- * specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
- * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
- * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
- * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
- * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
- * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#ifndef NBDKIT_EFI_CRC32_H
-#define NBDKIT_EFI_CRC32_H
-
-#include <stdint.h>
-
-extern uint32_t efi_crc32 (const void *buf, size_t len)
- __attribute__((__nonnull__ (1)));
-
-#endif /* NBDKIT_EFI_CRC32_H */
diff --git a/plugins/partitioning/partition-gpt.c
b/plugins/partitioning/partition-gpt.c
index c3d1d03..7a9bd80 100644
--- a/plugins/partitioning/partition-gpt.c
+++ b/plugins/partitioning/partition-gpt.c
@@ -46,6 +46,7 @@
#include "byte-swapping.h"
#include "efi-crc32.h"
+#include "gpt.h"
#include "regions.h"
#include "virtual-disk.h"
@@ -86,28 +87,13 @@ create_gpt_partition_header (const void *pt, bool
is_primary,
unsigned char *out)
{
uint64_t nr_lbas;
- struct gpt_header {
- char signature[8];
- char revision[4];
- uint32_t header_size;
- uint32_t crc;
- uint32_t reserved;
- uint64_t current_lba;
- uint64_t backup_lba;
- uint64_t first_usable_lba;
- uint64_t last_usable_lba;
- char guid[16];
- uint64_t partition_entries_lba;
- uint32_t nr_partition_entries;
- uint32_t size_partition_entry;
- uint32_t crc_partitions;
- } *header = (struct gpt_header *) out;
+ struct gpt_header *header = (struct gpt_header *) out;
nr_lbas = virtual_size (®ions) / SECTOR_SIZE;
memset (header, 0, sizeof *header);
- memcpy (header->signature, "EFI PART", 8);
- memcpy (header->revision, "\0\0\1\0", 4); /* revision 1.0 */
+ memcpy (header->signature, GPT_SIGNATURE, sizeof (header->signature));
+ memcpy (header->revision, GPT_REVISION, sizeof (header->revision));
header->header_size = htole32 (sizeof *header);
if (is_primary) {
header->current_lba = htole64 (1);
@@ -159,14 +145,7 @@ create_gpt_partition_table_entry (const struct region
*region,
{
size_t i, len;
const char *filename;
- struct gpt_entry {
- char partition_type_guid[16];
- char unique_guid[16];
- uint64_t first_lba;
- uint64_t last_lba;
- uint64_t attributes;
- char name[72]; /* UTF-16LE */
- } *entry = (struct gpt_entry *) out;
+ struct gpt_entry *entry = (struct gpt_entry *) out;
assert (sizeof (struct gpt_entry) == GPT_PT_ENTRY_SIZE);
diff --git a/plugins/partitioning/virtual-disk.h
b/plugins/partitioning/virtual-disk.h
index f59df70..49e125d 100644
--- a/plugins/partitioning/virtual-disk.h
+++ b/plugins/partitioning/virtual-disk.h
@@ -40,6 +40,7 @@
#include "rounding.h"
+#include "gpt.h"
#include "regions.h"
#define SECTOR_SIZE UINT64_C(512)
@@ -50,18 +51,6 @@
*/
#define MAX_MBR_DISK_SIZE (UINT32_MAX * SECTOR_SIZE - 5 * MAX_ALIGNMENT)
-/* GPT_MIN_PARTITIONS is the minimum number of partitions and is
- * defined by the UEFI standard (assuming 512 byte sector size). If
- * we are requested to allocate more than GPT_MIN_PARTITIONS then we
- * increase the partition table in chunks of this size. Note that
- * clients may not support > GPT_MIN_PARTITIONS.
- *
- * GPT_PT_ENTRY_SIZE is the minimum specified by the UEFI spec, but
- * increasing it is not useful.
- */
-#define GPT_MIN_PARTITIONS 128
-#define GPT_PT_ENTRY_SIZE 128
-
/* For GPT, the number of entries in the partition table array (PTA),
* and the number of LBAs which the PTA occupies. The latter will be
* 32 if the number of files is <= GPT_MIN_PARTITIONS, which is the
--
1.8.3.1
Richard W.M. Jones
2019-Feb-19 07:49 UTC
[Libguestfs] [PATCH nbdkit 2/4] partition: Update to use new common/gpt/gpt.h header file.
From: "Richard W.M. Jones" <rjones@redhat.com>
---
filters/partition/Makefile.am | 1 +
filters/partition/partition-gpt.c | 62 ++++++++++++++++++---------------------
2 files changed, 29 insertions(+), 34 deletions(-)
diff --git a/filters/partition/Makefile.am b/filters/partition/Makefile.am
index 0932793..7814f09 100644
--- a/filters/partition/Makefile.am
+++ b/filters/partition/Makefile.am
@@ -45,6 +45,7 @@ nbdkit_partition_filter_la_SOURCES = \
nbdkit_partition_filter_la_CPPFLAGS = \
-I$(top_srcdir)/include \
+ -I$(top_srcdir)/common/gpt \
-I$(top_srcdir)/common/include
nbdkit_partition_filter_la_CFLAGS = \
$(WARNINGS_CFLAGS)
diff --git a/filters/partition/partition-gpt.c
b/filters/partition/partition-gpt.c
index ddfaa75..007799a 100644
--- a/filters/partition/partition-gpt.c
+++ b/filters/partition/partition-gpt.c
@@ -41,37 +41,29 @@
#include <nbdkit-filter.h>
#include "byte-swapping.h"
+#include "gpt.h"
#include "partition.h"
-struct gpt_header {
- uint32_t nr_partitions;
- uint32_t partition_entry_size;
-};
-
static void
-get_gpt_header (uint8_t *sector, struct gpt_header *header)
+get_gpt_header (uint8_t *sector,
+ uint32_t *nr_partition_entries,
+ uint32_t *size_partition_entry)
{
- memcpy (&header->nr_partitions, §or[0x50], 4);
- header->nr_partitions = le32toh (header->nr_partitions);
- memcpy (&header->partition_entry_size, §or[0x54], 4);
- header->partition_entry_size = le32toh (header->partition_entry_size);
+ struct gpt_header *header = (struct gpt_header *) sector;
+ *nr_partition_entries = le32toh (header->nr_partition_entries);
+ *size_partition_entry = le32toh (header->size_partition_entry);
}
-struct gpt_partition {
- uint8_t partition_type_guid[16];
- uint64_t first_lba;
- uint64_t last_lba;
-};
-
static void
-get_gpt_partition (uint8_t *bytes, struct gpt_partition *part)
+get_gpt_partition (uint8_t *bytes,
+ uint8_t *partition_type_guid,
+ uint64_t *first_lba, uint64_t *last_lba)
{
- memcpy (&part->partition_type_guid, &bytes[0], 16);
- memcpy (&part->first_lba, &bytes[0x20], 8);
- part->first_lba = le64toh (part->first_lba);
- memcpy (&part->last_lba, &bytes[0x28], 8);
- part->last_lba = le64toh (part->last_lba);
+ struct gpt_entry *entry = (struct gpt_entry *) bytes;
+ memcpy (partition_type_guid, entry->partition_type_guid, 16);
+ *first_lba = le64toh (entry->first_lba);
+ *last_lba = le64toh (entry->last_lba);
}
int
@@ -80,18 +72,19 @@ find_gpt_partition (struct nbdkit_next_ops *next_ops, void
*nxdata,
int64_t *offset_r, int64_t *range_r)
{
uint8_t partition_bytes[128];
- struct gpt_header header;
- struct gpt_partition partition;
+ uint32_t nr_partition_entries, size_partition_entry;
+ uint8_t partition_type_guid[16];
+ uint64_t first_lba, last_lba;
int i;
int err;
- get_gpt_header (header_bytes, &header);
- if (partnum > header.nr_partitions) {
+ get_gpt_header (header_bytes, &nr_partition_entries,
&size_partition_entry);
+ if (partnum > nr_partition_entries) {
nbdkit_error ("GPT partition number out of range");
return -1;
}
- if (header.partition_entry_size < 128) {
+ if (size_partition_entry < 128) {
nbdkit_error ("GPT partition entry size is < 128 bytes");
return -1;
}
@@ -101,23 +94,24 @@ find_gpt_partition (struct nbdkit_next_ops *next_ops, void
*nxdata,
* that the GPT header is bogus.
*/
if (size < INT64_C(3)*SECTOR_SIZE +
- INT64_C(2) * header.nr_partitions * header.partition_entry_size) {
+ INT64_C(2) * nr_partition_entries * size_partition_entry) {
nbdkit_error ("GPT partition table is too large for this disk");
return -1;
}
- for (i = 0; i < header.nr_partitions; ++i) {
+ for (i = 0; i < nr_partition_entries; ++i) {
/* We already checked these are within bounds above. */
if (next_ops->pread (nxdata, partition_bytes, sizeof partition_bytes,
- 2*SECTOR_SIZE + i*header.partition_entry_size, 0,
+ 2*SECTOR_SIZE + i*size_partition_entry, 0,
&err) == -1)
return -1;
- get_gpt_partition (partition_bytes, &partition);
- if (memcmp (partition.partition_type_guid,
+ get_gpt_partition (partition_bytes,
+ partition_type_guid, &first_lba, &last_lba);
+ if (memcmp (partition_type_guid,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) != 0
&&
partnum == i+1) {
- *offset_r = partition.first_lba * SECTOR_SIZE;
- *range_r = (1 + partition.last_lba - partition.first_lba) * SECTOR_SIZE;
+ *offset_r = first_lba * SECTOR_SIZE;
+ *range_r = (1 + last_lba - first_lba) * SECTOR_SIZE;
return 0;
}
}
--
1.8.3.1
Richard W.M. Jones
2019-Feb-19 07:49 UTC
[Libguestfs] [PATCH nbdkit 3/4] common: Move a utility function to a common directory.
From: "Richard W.M. Jones" <rjones@redhat.com>
The shell_quote function is moved to a new common/utils directory.
Eventually more utility functions can be created here.
This change is pure refactoring.
---
Makefile.am | 1 +
common/utils/Makefile.am | 41 ++++++++++++++++++++++++++++
common/utils/utils.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++
common/utils/utils.h | 39 +++++++++++++++++++++++++++
configure.ac | 1 +
plugins/iso/Makefile.am | 3 +++
plugins/iso/iso.c | 36 ++-----------------------
7 files changed, 157 insertions(+), 34 deletions(-)
create mode 100644 common/utils/Makefile.am
create mode 100644 common/utils/utils.c
create mode 100644 common/utils/utils.h
diff --git a/Makefile.am b/Makefile.am
index 653956f..99f0852 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -74,6 +74,7 @@ SUBDIRS += \
common/gpt \
common/regions \
common/sparse \
+ common/utils \
plugins \
filters
endif
diff --git a/common/utils/Makefile.am b/common/utils/Makefile.am
new file mode 100644
index 0000000..e002586
--- /dev/null
+++ b/common/utils/Makefile.am
@@ -0,0 +1,41 @@
+# nbdkit
+# Copyright (C) 2019 Red Hat Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+include $(top_srcdir)/common-rules.mk
+
+noinst_LTLIBRARIES = libutils.la
+
+libutils_la_SOURCES = \
+ utils.c \
+ utils.h
+libutils_la_CFLAGS = \
+ $(WARNINGS_CFLAGS)
diff --git a/common/utils/utils.c b/common/utils/utils.c
new file mode 100644
index 0000000..458f489
--- /dev/null
+++ b/common/utils/utils.c
@@ -0,0 +1,70 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+/* Print str to fp, shell quoting if necessary. This comes from
+ * libguestfs, but was written by me so I'm relicensing it to a BSD
+ * license for nbdkit.
+ */
+void
+shell_quote (const char *str, FILE *fp)
+{
+ const char *safe_chars +
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_=,:/";
+ size_t i, len;
+
+ /* If the string consists only of safe characters, output it as-is. */
+ len = strlen (str);
+ if (len == strspn (str, safe_chars)) {
+ fputs (str, fp);
+ return;
+ }
+
+ /* Double-quote the string. */
+ fputc ('"', fp);
+ for (i = 0; i < len; ++i) {
+ switch (str[i]) {
+ case '$': case '`': case '\\': case
'"':
+ fputc ('\\', fp);
+ /*FALLTHROUGH*/
+ default:
+ fputc (str[i], fp);
+ }
+ }
+ fputc ('"', fp);
+}
diff --git a/common/utils/utils.h b/common/utils/utils.h
new file mode 100644
index 0000000..a012183
--- /dev/null
+++ b/common/utils/utils.h
@@ -0,0 +1,39 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#ifndef NBDKIT_UTILS_H
+#define NBDKIT_UTILS_H
+
+extern void shell_quote (const char *str, FILE *fp);
+
+#endif /* NBDKIT_UTILS_H */
diff --git a/configure.ac b/configure.ac
index bca6134..229e38c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -837,6 +837,7 @@ AC_CONFIG_FILES([Makefile
common/include/Makefile
common/regions/Makefile
common/sparse/Makefile
+ common/utils/Makefile
docs/Makefile
include/Makefile
plugins/Makefile
diff --git a/plugins/iso/Makefile.am b/plugins/iso/Makefile.am
index 061c161..673d1ee 100644
--- a/plugins/iso/Makefile.am
+++ b/plugins/iso/Makefile.am
@@ -43,6 +43,7 @@ nbdkit_iso_plugin_la_SOURCES = \
$(top_srcdir)/include/nbdkit-plugin.h
nbdkit_iso_plugin_la_CPPFLAGS = \
+ -I$(top_srcdir)/common/utils \
-I$(top_srcdir)/include \
-I.
nbdkit_iso_plugin_la_CFLAGS = \
@@ -50,6 +51,8 @@ nbdkit_iso_plugin_la_CFLAGS = \
nbdkit_iso_plugin_la_LDFLAGS = \
-module -avoid-version -shared \
-Wl,--version-script=$(top_srcdir)/plugins/plugins.syms
+nbdkit_iso_plugin_la_LIBADD = \
+ $(top_builddir)/common/utils/libutils.la
if HAVE_POD
diff --git a/plugins/iso/iso.c b/plugins/iso/iso.c
index 7431b48..53fccf0 100644
--- a/plugins/iso/iso.c
+++ b/plugins/iso/iso.c
@@ -43,6 +43,8 @@
#include <nbdkit-plugin.h>
+#include "utils.h"
+
/* List of directories parsed from the command line. */
static char **dirs = NULL;
static size_t nr_dirs = 0;
@@ -59,8 +61,6 @@ static const char *params = NULL;
static int fd = -1;
/* Construct the temporary ISO. */
-static void shell_quote (const char *str, FILE *fp);
-
static int
make_iso (void)
{
@@ -136,38 +136,6 @@ make_iso (void)
return 0;
}
-/* Print str to fp, shell quoting if necessary. This comes from
- * libguestfs, but was written by me so I'm relicensing it to a BSD
- * license for nbdkit.
- */
-static void
-shell_quote (const char *str, FILE *fp)
-{
- const char *safe_chars -
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_=,:/";
- size_t i, len;
-
- /* If the string consists only of safe characters, output it as-is. */
- len = strlen (str);
- if (len == strspn (str, safe_chars)) {
- fputs (str, fp);
- return;
- }
-
- /* Double-quote the string. */
- fputc ('"', fp);
- for (i = 0; i < len; ++i) {
- switch (str[i]) {
- case '$': case '`': case '\\': case
'"':
- fputc ('\\', fp);
- /*FALLTHROUGH*/
- default:
- fputc (str[i], fp);
- }
- }
- fputc ('"', fp);
-}
-
static void
iso_unload (void)
{
--
1.8.3.1
Richard W.M. Jones
2019-Feb-19 07:49 UTC
[Libguestfs] [PATCH nbdkit 4/4] Add linuxdisk plugin.
From: "Richard W.M. Jones" <rjones@redhat.com>
This plugin allows you to create a complete ext2 filesystem in a GPT
partitioned disk image. This can be attached as a disk to a Linux
virtual machine. It is implemented using libext2fs (the same as
supermin).
Although there is some overlap with nbdkit-iso-plugin and
nbdkit-floppy-plugin, the implementations and use cases of all three
plugins are sufficiently different that it seems to make sense to add
another plugin rather than attempting to extend one of the existing
plugins.
Largely to avoid user error this plugin is read-only. This is a major
difference from the floppy plugin: that plugin allows files to be
modified (but not resized or created) and writes those changes through
to the backing filesystem. While this plugin could easily be made
writable, this would cause almost certain disk corruption when someone
connected two clients at the same time. In any case it doesn't make
much sense for it to be writable by default since the expectation that
writes would somehow modify the original directory on the host
filesystem cannot be satisfied by this or any reasonable
implementation. Users can add the cow filter on top if they really
want writes and know what they are doing: instructions plus disclaimer
about this are included in the man page.
As mentioned above, this implementation is based on the same idea as
the appliance creation code in supermin. Eventually we could replace
that supermin code with this plugin, but there are some missing
features that would need to be implemented first.
---
README | 2 +-
TODO | 2 +
configure.ac | 4 +-
plugins/floppy/nbdkit-floppy-plugin.pod | 1 +
plugins/iso/nbdkit-iso-plugin.pod | 6 +-
plugins/linuxdisk/Makefile.am | 78 ++
plugins/linuxdisk/filesystem.c | 935 +++++++++++++++++++++
plugins/linuxdisk/linuxdisk.c | 236 ++++++
plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 197 +++++
plugins/linuxdisk/partition-gpt.c | 211 +++++
plugins/linuxdisk/virtual-disk.c | 163 ++++
plugins/linuxdisk/virtual-disk.h | 96 +++
.../partitioning/nbdkit-partitioning-plugin.pod | 5 +-
tests/Makefile.am | 11 +
tests/test-linuxdisk-copy-out.sh | 76 ++
tests/test-linuxdisk.sh | 94 +++
16 files changed, 2111 insertions(+), 6 deletions(-)
create mode 100644 plugins/linuxdisk/Makefile.am
create mode 100644 plugins/linuxdisk/filesystem.c
create mode 100644 plugins/linuxdisk/linuxdisk.c
create mode 100644 plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod
create mode 100644 plugins/linuxdisk/partition-gpt.c
create mode 100644 plugins/linuxdisk/virtual-disk.c
create mode 100644 plugins/linuxdisk/virtual-disk.h
create mode 100755 tests/test-linuxdisk-copy-out.sh
create mode 100755 tests/test-linuxdisk.sh
diff --git a/README b/README
index e394a1f..4ba9d2a 100644
--- a/README
+++ b/README
@@ -96,7 +96,7 @@ For the libguestfs plugin, and to run parts of the test suite:
- guestfish (from libguestfs)
-For the ext2 plugin:
+For the ext2 and linuxdisk plugins:
- ext2fs
diff --git a/TODO b/TODO
index 72e03b6..96066b2 100644
--- a/TODO
+++ b/TODO
@@ -71,6 +71,8 @@ General ideas for improvements
and also look at the implementation of the -swap option in
nbd-client.
+* Implement extended attributes in the linuxdisk plugin.
+
Suggestions for plugins
-----------------------
diff --git a/configure.ac b/configure.ac
index 229e38c..b8400d4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -735,7 +735,7 @@ dnl Check for guestfish (only needed for some of the tests).
AC_CHECK_PROG([GUESTFISH], [guestfish], [guestfish], [no])
AM_CONDITIONAL([HAVE_GUESTFISH], [test "x$GUESTFISH" !=
"xno"])
-dnl Check for ext2fs and com_err, for the ext2 plugin.
+dnl Check for ext2fs and com_err, for the ext2 and linuxdisk plugins.
AC_ARG_WITH([ext2],
[AS_HELP_STRING([--without-ext2],
[disable ext2 plugin @<:@default=check@:>@])],
@@ -793,6 +793,7 @@ non_lang_plugins="\
gzip \
iso \
libvirt \
+ linuxdisk \
memory \
nbd \
null \
@@ -855,6 +856,7 @@ AC_CONFIG_FILES([Makefile
plugins/gzip/Makefile
plugins/iso/Makefile
plugins/libvirt/Makefile
+ plugins/linuxdisk/Makefile
plugins/lua/Makefile
plugins/memory/Makefile
plugins/nbd/Makefile
diff --git a/plugins/floppy/nbdkit-floppy-plugin.pod
b/plugins/floppy/nbdkit-floppy-plugin.pod
index ae14946..ff84f68 100644
--- a/plugins/floppy/nbdkit-floppy-plugin.pod
+++ b/plugins/floppy/nbdkit-floppy-plugin.pod
@@ -74,6 +74,7 @@ important, and it simplifies the implementation greatly.
L<nbdkit(1)>,
L<nbdkit-plugin(3)>,
L<nbdkit-file-plugin(1)>,
+L<nbdkit-linuxdisk-plugin(1)>,
L<nbdkit-iso-plugin(1)>.
=head1 AUTHORS
diff --git a/plugins/iso/nbdkit-iso-plugin.pod
b/plugins/iso/nbdkit-iso-plugin.pod
index 4d9cf41..90e26f0 100644
--- a/plugins/iso/nbdkit-iso-plugin.pod
+++ b/plugins/iso/nbdkit-iso-plugin.pod
@@ -17,8 +17,9 @@ read-only over the NBD protocol.
This plugin uses L<genisoimage(1)> or L<mkisofs(1)> to create the
ISO
content.
-To create a virtual floppy disk instead of a CD, see
-L<nbdkit-floppy-plugin(1)>.
+To create a FAT-formatted virtual floppy disk instead of a CD, see
+L<nbdkit-floppy-plugin(1)>. To create a Linux compatible virtual
+disk, see L<nbdkit-linuxdisk-plugin(1)>.
=head1 EXAMPLE
@@ -96,6 +97,7 @@ L<nbdkit(1)>,
L<nbdkit-plugin(3)>,
L<nbdkit-file-plugin(1)>,
L<nbdkit-floppy-plugin(1)>,
+L<nbdkit-linuxdisk-plugin(1)>,
L<genisoimage(1)>,
L<mkisofs(1)>.
diff --git a/plugins/linuxdisk/Makefile.am b/plugins/linuxdisk/Makefile.am
new file mode 100644
index 0000000..277efe8
--- /dev/null
+++ b/plugins/linuxdisk/Makefile.am
@@ -0,0 +1,78 @@
+# nbdkit
+# Copyright (C) 2019 Red Hat Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+include $(top_srcdir)/common-rules.mk
+
+EXTRA_DIST = nbdkit-linuxdisk-plugin.pod
+
+if HAVE_EXT2
+
+plugin_LTLIBRARIES = nbdkit-linuxdisk-plugin.la
+
+nbdkit_linuxdisk_plugin_la_SOURCES = \
+ filesystem.c \
+ linuxdisk.c \
+ partition-gpt.c \
+ virtual-disk.c \
+ virtual-disk.h \
+ $(top_srcdir)/include/nbdkit-plugin.h
+
+nbdkit_linuxdisk_plugin_la_CPPFLAGS = \
+ -I$(top_srcdir)/common/gpt \
+ -I$(top_srcdir)/common/include \
+ -I$(top_srcdir)/common/regions \
+ -I$(top_srcdir)/common/utils \
+ -I$(top_srcdir)/include
+nbdkit_linuxdisk_plugin_la_CFLAGS = \
+ $(WARNINGS_CFLAGS) \
+ $(EXT2FS_CFLAGS) $(COM_ERR_CFLAGS)
+nbdkit_linuxdisk_plugin_la_LIBADD = \
+ $(top_builddir)/common/gpt/libgpt.la \
+ $(top_builddir)/common/regions/libregions.la \
+ $(top_builddir)/common/utils/libutils.la \
+ $(EXT2FS_LIBS) $(COM_ERR_LIBS)
+nbdkit_linuxdisk_plugin_la_LDFLAGS = \
+ -module -avoid-version -shared \
+ -Wl,--version-script=$(top_srcdir)/plugins/plugins.syms
+
+if HAVE_POD
+
+man_MANS = nbdkit-linuxdisk-plugin.1
+CLEANFILES += $(man_MANS)
+
+nbdkit-linuxdisk-plugin.1: nbdkit-linuxdisk-plugin.pod
+ $(PODWRAPPER) --section=1 --man $@ \
+ --html $(top_builddir)/html/$@.html \
+ $<
+
+endif HAVE_POD
+endif
diff --git a/plugins/linuxdisk/filesystem.c b/plugins/linuxdisk/filesystem.c
new file mode 100644
index 0000000..f1d3a45
--- /dev/null
+++ b/plugins/linuxdisk/filesystem.c
@@ -0,0 +1,935 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <dirent.h>
+#include <errno.h>
+#include <assert.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/sysmacros.h>
+
+/* Inlining is broken in the ext2fs header file. Disable it by
+ * defining the following:
+ */
+#define NO_INLINE_FUNCS
+#include <ext2fs.h>
+
+#include <nbdkit-plugin.h>
+
+#include "get-current-dir-name.h"
+#include "minmax.h"
+#include "rounding.h"
+#include "utils.h"
+
+#include "virtual-disk.h"
+
+int linuxdisk_debug_filesystem; /* -D linuxdisk.filesystem=1 */
+
+/* Per-file information collected in the first pass. */
+struct file {
+ char *pathname; /* Full host path to the file. */
+ char *name; /* Base name of the file. */
+ ext2_ino_t dir_ino; /* Containing directory inode in target fs. */
+ struct stat statbuf; /* File information. */
+
+ /* These fields are only populated once the file has been created
+ * in the target filesystem.
+ */
+ ext2_ino_t ino; /* Inode. */
+ int ino_flags; /* Inode flags. */
+ const char *type; /* File type (used only for debugging). */
+};
+
+static int64_t estimate_size (void);
+static int mke2fs (const char *filename);
+static int visit (const char *dir, ext2_filsys fs, ext2_ino_t dir_ino,
+ struct file **files, size_t *nr_files);
+static int compare_inodes (const void *f1, const void *f2);
+static int e2mkdir (ext2_filsys fs, ext2_ino_t dir_ino, const char *name,
+ const struct stat *statbuf, ext2_ino_t *ino);
+static int e2emptyinode (ext2_filsys fs, ext2_ino_t dir_ino,
+ const char *name, const struct stat *statbuf,
+ int ino_flags, ext2_ino_t *ino);
+static int e2copyfile (ext2_filsys fs, struct file *file);
+static int e2hardlink (ext2_filsys fs, struct file *to, struct file *file);
+static int e2link (ext2_filsys fs, ext2_ino_t dir_ino, const char *name,
+ ext2_ino_t ino, int flags);
+static int e2copyfiledata (ext2_filsys fs, struct file *file);
+
+void
+load_filesystem (void)
+{
+ initialize_ext2_error_table ();
+}
+
+int
+create_filesystem (struct virtual_disk *disk)
+{
+ const char *tmpdir;
+ char *filename;
+ errcode_t err;
+ ext2_filsys fs;
+ size_t i;
+ struct file *files = NULL;
+ size_t nr_files = 0;
+ struct file *last_file;
+ bool hardlinked;
+
+ /* Estimate the filesystem size and compute the final virtual size
+ * of the disk. We only need to do this if the user didn't specify
+ * the exact size on the command line.
+ */
+ if (size == 0 || size_add_estimate) {
+ int64_t estimate;
+
+ estimate = estimate_size ();
+ if (estimate == -1)
+ return -1;
+
+ nbdkit_debug ("filesystem size estimate: %" PRIi64, estimate);
+
+ /* Add 20% to the estimate to account for the overhead of
+ * filesystem metadata. Also set a minimum size. Note we are
+ * only wasting virtual space (since this will be stored sparsely
+ * under $TMDIR) so we can be generous here.
+ *
+ * If we decide to create ext3/4 filesystems in future we will
+ * need to account for the journal here.
+ */
+ estimate = estimate * 6 / 5;
+ estimate = MAX (estimate, 1024*1024);
+
+ if (size_add_estimate)
+ size += estimate;
+ else
+ size = estimate;
+ }
+
+ /* The minimum filesystem that libext2fs will let us create is 64M.
+ * Why?! Anyway it works if I do this, but not otherwise. XXX
+ */
+ size = MAX (size, 64*1024*1024);
+
+ /* Round the final size up to a whole number of sectors. */
+ size = ROUND_UP (size, SECTOR_SIZE);
+ disk->filesystem_size = size;
+
+ nbdkit_debug ("filesystem virtual size: %" PRIi64, size);
+
+ /* Create the filesystem file. */
+ tmpdir = getenv ("TMPDIR");
+ if (tmpdir == NULL)
+ tmpdir = LARGE_TMPDIR;
+ if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) {
+ nbdkit_error ("asprintf: %m");
+ return -1;
+ }
+
+ disk->fd = mkstemp (filename);
+ if (disk->fd == -1) {
+ nbdkit_error ("mkstemp: %s: %m", filename);
+ free (filename);
+ return -1;
+ }
+ if (ftruncate (disk->fd, size) == -1) {
+ nbdkit_error ("ftruncate: %s: %m", filename);
+ free (filename);
+ return -1;
+ }
+
+ /* Create the filesystem. */
+ if (mke2fs (filename) == -1) {
+ unlink (filename);
+ free (filename);
+ return -1;
+ }
+
+ /* Open the filesystem. */
+ err = ext2fs_open (filename, EXT2_FLAG_RW|EXT2_FLAG_64BITS, 0, 0,
+ unix_io_manager, &fs);
+ unlink (filename);
+ free (filename);
+ if (err) {
+ nbdkit_error ("ext2fs_open: %s", error_message (err));
+ return -1;
+ }
+
+ err = ext2fs_read_bitmaps (fs);
+ if (err) {
+ nbdkit_error ("ext2fs_read_bitmaps: %s", error_message (err));
+ ext2fs_close (fs);
+ return -1;
+ }
+
+ /* First pass: This creates subdirectories in the filesystem and
+ * also builds a list of files so we can identify hard links.
+ */
+ for (i = 0; i < nr_dirs; ++i) {
+ if (visit (dirs[i], fs, EXT2_ROOT_INO, &files, &nr_files) == -1)
+ return -1;
+ }
+
+ if (nr_files > 0) {
+ assert (files != NULL);
+
+ /* Sort the files by device and inode number to identify hard links. */
+ qsort (files, nr_files, sizeof (struct file), compare_inodes);
+
+ /* Second pass: Copy/create the files. */
+ last_file = NULL;
+ for (i = 0; i < nr_files; ++i) {
+ hardlinked = last_file && compare_inodes (last_file,
&files[i]) == 0;
+
+ if (!hardlinked) {
+ /* Normal case: creating a new inode. */
+ last_file = &files[i];
+ assert (!S_ISDIR (files[i].statbuf.st_mode));
+
+ if (e2copyfile (fs, &files[i]) == -1)
+ return -1;
+
+ if (linuxdisk_debug_filesystem)
+ nbdkit_debug ("%s: <%" PRIu32 ">/%s ->
<%" PRIu32 ">",
+ files[i].type,
+ files[i].dir_ino, files[i].name, files[i].ino);
+ }
+ else {
+ /* Creating a hard link to an existing inode. */
+ if (e2hardlink (fs, last_file, &files[i]) == -1)
+ return -1;
+
+ if (linuxdisk_debug_filesystem)
+ nbdkit_debug ("hard link: <%" PRIu32 ">/%s ->
<%" PRIu32 ">",
+ files[i].dir_ino, files[i].name, last_file->ino);
+ }
+ }
+
+ for (i = 0; i < nr_files; ++i) {
+ free (files[i].pathname);
+ free (files[i].name);
+ }
+ free (files);
+ }
+
+ /* Close the filesystem. Note we don't bother to sync it because
+ * it's a private temporary file which only we will read.
+ */
+ ext2fs_close2 (fs, EXT2_FLAG_FLUSH_NO_SYNC);
+ return 0;
+}
+
+/* Use ‘du’ to estimate the size of the filesystem quickly.
+ *
+ * Typical output from ‘du -cs dir1 dir2’ is:
+ *
+ * 12345 dir1
+ * 34567 dir2
+ * 46912 total
+ *
+ * We ignore everything except the first number on the last line.
+ */
+static int64_t
+estimate_size (void)
+{
+ char *command = NULL, *line = NULL;
+ size_t len = 0;
+ FILE *fp;
+ size_t i;
+ int64_t ret;
+
+ /* Create the du command. */
+ fp = open_memstream (&command, &len);
+ if (fp == NULL) {
+ nbdkit_error ("open_memstream: %m");
+ return -1;
+ }
+ fprintf (fp, "du -c -k -s");
+ for (i = 0; i < nr_dirs; ++i) {
+ fputc (' ', fp);
+ shell_quote (dirs[i], fp);
+ }
+ if (fclose (fp) == EOF) {
+ nbdkit_error ("memstream failed: %m");
+ return -1;
+ }
+
+ /* Run the command. */
+ nbdkit_debug ("%s", command);
+ fp = popen (command, "r");
+ free (command);
+ if (fp == NULL) {
+ nbdkit_error ("du command failed: %m");
+ return -1;
+ }
+
+ /* Ignore everything up to the last line. */
+ len = 0;
+ while (getline (&line, &len, fp) != -1)
+ /* empty */;
+ if (ferror (fp)) {
+ nbdkit_error ("getline failed: %m");
+ free (line);
+ fclose (fp);
+ return -1;
+ }
+
+ fclose (fp);
+
+ /* Parse the last line. */
+ if (sscanf (line, "%" SCNi64, &ret) != 1 || ret < 0) {
+ nbdkit_error ("could not parse last line of output: %s", line);
+ free (line);
+ return -1;
+ }
+ free (line);
+
+ /* Result is in 1K blocks, convert it to bytes. */
+ ret *= 1024;
+ return ret;
+}
+
+static int
+mke2fs (const char *filename)
+{
+ char *command = NULL;
+ size_t len = 0;
+ FILE *fp;
+ int r;
+
+ /* Create the mke2fs command. */
+ fp = open_memstream (&command, &len);
+ if (fp == NULL) {
+ nbdkit_error ("open_memstream: %m");
+ return -1;
+ }
+
+ fprintf (fp, "mke2fs -t ext2 -F ");
+ if (label) {
+ fprintf (fp, "-L ");
+ shell_quote (label, fp);
+ fputc (' ', fp);
+ }
+ if (!linuxdisk_debug_filesystem)
+ fprintf (fp, "-q "); /* quiet unless extra debugging enabled
*/
+ shell_quote (filename, fp);
+
+ if (fclose (fp) == EOF) {
+ nbdkit_error ("memstream failed: %m");
+ return -1;
+ }
+
+ /* Run the command. */
+ nbdkit_debug ("%s", command);
+ r = system (command);
+ free (command);
+
+ if (WIFEXITED (r) && WEXITSTATUS (r) != 0) {
+ nbdkit_error ("mke2fs command failed with exit code %d",
WEXITSTATUS (r));
+ return -1;
+ }
+ else if (WIFSIGNALED (r)) {
+ nbdkit_error ("mke2fs command was killed by signal %d", WTERMSIG
(r));
+ return -1;
+ }
+ else if (WIFSTOPPED (r)) {
+ nbdkit_error ("mke2fs command was stopped by signal %d", WSTOPSIG
(r));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+visit (const char *dir, ext2_filsys fs, ext2_ino_t dir_ino,
+ struct file **files, size_t *nr_files)
+{
+ char *origdir;
+ DIR *DIR;
+ struct dirent *d;
+ struct stat statbuf;
+ int err;
+ char *subname = NULL;
+
+ /* Because this is called from config_complete, before nbdkit
+ * daemonizes or starts any threads, it's safe to use chdir here and
+ * greatly simplifies the code. However we must chdir back to the
+ * original directory at the end.
+ */
+ origdir = get_current_dir_name ();
+ if (origdir == NULL) {
+ nbdkit_error ("get_current_dir_name: %m");
+ goto error0;
+ }
+ if (chdir (dir) == -1) {
+ nbdkit_error ("chdir: %s: %m", dir);
+ goto error1;
+ }
+
+ DIR = opendir (".");
+ if (DIR == NULL) {
+ nbdkit_error ("opendir: %s: %m", dir);
+ goto error1;
+ }
+
+ while (errno = 0, (d = readdir (DIR)) != NULL) {
+ if (strcmp (d->d_name, ".") == 0 ||
+ strcmp (d->d_name, "..") == 0)
+ continue;
+
+ free (subname);
+ subname = NULL;
+ if (asprintf (&subname, "%s/%s", dir, d->d_name) == -1) {
+ nbdkit_error ("asprintf: %m");
+ goto error2;
+ }
+
+ if (lstat (d->d_name, &statbuf) == -1) {
+ nbdkit_error ("lstat: %s: %m", subname);
+ goto error2;
+ }
+
+ if (S_ISDIR (statbuf.st_mode)) {
+ /* File type: Directory. */
+ ext2_ino_t ino;
+
+ /* Create the directory in the target filesystem. */
+ if (e2mkdir (fs, dir_ino, d->d_name, &statbuf, &ino) == -1)
+ goto error2;
+
+ if (linuxdisk_debug_filesystem)
+ nbdkit_debug ("mkdir: <%" PRIu32 ">/%s ->
<%" PRIu32 ">",
+ dir_ino, d->d_name, ino);
+
+ /* Visit the subdirectory. */
+ if (visit (subname, fs, ino, files, nr_files) == -1)
+ goto error2;
+ }
+ else {
+ /* Any other non-directory is added to files. We will further
+ * process it in the second pass.
+ */
+ struct file *new_files;
+
+ new_files = realloc (*files, (*nr_files+1) * sizeof (struct file));
+ if (new_files == NULL) {
+ nbdkit_error ("realloc: %m");
+ goto error2;
+ }
+ *files = new_files;
+ new_files[*nr_files].name = strdup (d->d_name);
+ if (new_files[*nr_files].name == NULL) {
+ nbdkit_error ("strdup: %m");
+ goto error2;
+ }
+ new_files[*nr_files].pathname = subname;
+ subname = NULL; /* pass ownership to struct */
+ new_files[*nr_files].dir_ino = dir_ino;
+ new_files[*nr_files].statbuf = statbuf;
+ (*nr_files)++;
+ }
+ }
+
+ /* Did readdir fail? */
+ if (errno != 0) {
+ nbdkit_error ("readdir: %s: %m", dir);
+ goto error2;
+ }
+
+ if (closedir (DIR) == -1) {
+ nbdkit_error ("closedir: %s: %m", dir);
+ goto error1;
+ }
+
+ if (chdir (origdir) == -1) {
+ nbdkit_error ("chdir: %s: %m", origdir);
+ goto error1;
+ }
+
+ free (origdir);
+ free (subname);
+
+ return 0;
+
+ error2:
+ closedir (DIR);
+ error1:
+ err = errno;
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wunused-result"
+ chdir (origdir);
+#pragma GCC diagnostic pop
+ errno = err;
+ free (origdir);
+ error0:
+ free (subname);
+ return -1;
+}
+
+/* To identify hard links we sort the files array by device and inode
+ * number (thus hard linked files will be next to each other after the
+ * sort). Hence this odd sorting function.
+ */
+static int
+compare_inodes (const void *vp1, const void *vp2)
+{
+ const struct file *f1 = vp1;
+ const struct file *f2 = vp2;
+
+ if (f1->statbuf.st_dev < f2->statbuf.st_dev)
+ return -1;
+ else if (f1->statbuf.st_dev > f2->statbuf.st_dev)
+ return 1;
+ else {
+ if (f1->statbuf.st_ino < f2->statbuf.st_ino)
+ return -1;
+ else if (f1->statbuf.st_ino > f2->statbuf.st_ino)
+ return 1;
+ else
+ return 0;
+ }
+}
+
+static int
+e2mkdir (ext2_filsys fs, ext2_ino_t dir_ino, const char *name,
+ const struct stat *statbuf, ext2_ino_t *ino)
+{
+ errcode_t err;
+ mode_t mode = LINUX_S_IFDIR | (statbuf->st_mode & 03777);
+ ext2_ino_t temp_ino;
+ struct ext2_inode inode;
+
+ /* It's possible that the directory exists (eg. because the user has
+ * two identically named subdirectories in two directory
+ * parameters). If this happens, skip.
+ */
+ err = ext2fs_namei (fs, EXT2_ROOT_INO, dir_ino, name, &temp_ino);
+ if (err == 0)
+ return 0;
+
+ /* Create the new inode. */
+ err = ext2fs_new_inode (fs, dir_ino, mode, 0, ino);
+ if (err) {
+ nbdkit_error ("ext2fs_new_inode: %s", error_message (err));
+ return -1;
+ }
+
+ again:
+ err = ext2fs_mkdir (fs, dir_ino, *ino, name);
+ if (err) {
+ /* http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=217892 */
+ if (err == EXT2_ET_DIR_NO_SPACE) {
+ err = ext2fs_expand_dir (fs, dir_ino);
+ if (err) {
+ nbdkit_error ("ext2fs_expand_dir: %s", error_message (err));
+ return -1;
+ }
+ goto again;
+ }
+ nbdkit_error ("ext2fs_mkdir: %s", error_message (err));
+ return -1;
+ }
+
+ /* Copy the permissions etc to the inode. */
+ err = ext2fs_read_inode (fs, *ino, &inode);
+ if (err) {
+ nbdkit_error ("ext2fs_read_inode: %s", error_message (err));
+ return -1;
+ }
+ inode.i_mode = mode;
+ inode.i_uid = statbuf->st_uid;
+ inode.i_gid = statbuf->st_gid;
+ inode.i_ctime = statbuf->st_ctime;
+ inode.i_atime = statbuf->st_atime;
+ inode.i_mtime = statbuf->st_mtime;
+ /* XXX nanosecond times? */
+ err = ext2fs_write_inode (fs, *ino, &inode);
+ if (err) {
+ nbdkit_error ("ext2fs_write_inode: %s", error_message (err));
+ return -1;
+ }
+
+ return 0;
+}
+
+/* A wrapper around ext2fs_new_inode. This creates an inode and a
+ * directory entry and does some other bookkeeping. This is
+ * sufficient for specials, but to create a complete regular file or
+ * symlink you have to do some other stuff. For directories, use
+ * e2mkdir instead.
+ */
+static int
+e2emptyinode (ext2_filsys fs, ext2_ino_t dir_ino,
+ const char *name, const struct stat *statbuf,
+ int ino_flags, ext2_ino_t *ino)
+{
+ errcode_t err;
+ struct ext2_inode inode;
+ int major_, minor_;
+
+ err = ext2fs_new_inode (fs, dir_ino, statbuf->st_mode, 0, ino);
+ if (err) {
+ nbdkit_error ("ext2fs_new_inode: %s", error_message (err));
+ return -1;
+ }
+
+ memset (&inode, 0, sizeof inode);
+ inode.i_mode = statbuf->st_mode;
+ inode.i_uid = statbuf->st_uid;
+ inode.i_gid = statbuf->st_gid;
+ inode.i_blocks = 0;
+ inode.i_links_count = 1;
+ /* XXX nanosecond times? */
+ inode.i_ctime = statbuf->st_ctime;
+ inode.i_atime = statbuf->st_atime;
+ inode.i_mtime = statbuf->st_mtime;
+ inode.i_size = 0;
+ if (S_ISBLK (statbuf->st_mode) || S_ISCHR (statbuf->st_mode)) {
+ major_ = major (statbuf->st_rdev);
+ minor_ = minor (statbuf->st_rdev);
+ inode.i_block[0] + (minor_ & 0xff) | (major_ << 8) |
((minor_ & ~0xff) << 12);
+ }
+ else
+ inode.i_block[0] = 0;
+
+ err = ext2fs_write_new_inode (fs, *ino, &inode);
+ if (err) {
+ nbdkit_error ("ext2fs_write_inode: %s", error_message (err));
+ return -1;
+ }
+
+ if (e2link (fs, dir_ino, name, *ino, ino_flags) == -1)
+ return -1;
+
+ ext2fs_inode_alloc_stats2 (fs, *ino, 1, 0);
+
+ return 0;
+}
+
+/* Copy a file (also symlink, special, etc) from the host to the target. */
+static int
+e2copyfile (ext2_filsys fs, struct file *file)
+{
+ errcode_t err;
+ ext2_ino_t temp_ino;
+
+ /* This cannot happen because we created directories in the first pass. */
+ assert (!S_ISDIR (file->statbuf.st_mode));
+
+ /* It's possible that the file already exists, if the user has two
+ * identically named files in two directory parameters.
+ *
+ * Supermin overwrites the existing file in this case, but the code
+ * to do this is complex because it involves deleting the file we
+ * previously created (and perhaps freeing up its blocks). Supermin
+ * doesn't handle extended attrs or free them which would be even
+ * more complex.
+ *
+ * Therefore we make this a user error, forbidding the user from
+ * doing this.
+ */
+ err = ext2fs_namei (fs, EXT2_ROOT_INO, file->dir_ino, file->name,
&temp_ino);
+ if (err == 0) {
+ nbdkit_error ("linuxdisk: two identically named files were found in
different subdirectories, which is forbidden: second filename %s",
+ file->pathname);
+ return -1;
+ }
+
+ /* Regular file. */
+ if (S_ISREG (file->statbuf.st_mode)) {
+ file->ino_flags = EXT2_FT_REG_FILE;
+ file->type = "regular file";
+ if (e2emptyinode (fs, file->dir_ino, file->name,
&file->statbuf,
+ file->ino_flags, &file->ino) == -1)
+ return -1;
+
+ if (file->statbuf.st_size > 0) {
+ if (e2copyfiledata (fs, file) == -1)
+ return -1;
+ }
+ }
+ /* Symbolic link. */
+ else if (S_ISLNK (file->statbuf.st_mode)) {
+ char *buf;
+ ssize_t r;
+
+ file->ino_flags = EXT2_FT_SYMLINK;
+ file->type = "symbolic link";
+ if (e2emptyinode (fs, file->dir_ino, file->name,
&file->statbuf,
+ file->ino_flags, &file->ino) == -1)
+ return -1;
+
+ buf = malloc (file->statbuf.st_size + 1);
+ if (buf == NULL) {
+ nbdkit_error ("malloc: %m");
+ return -1;
+ }
+ r = readlink (file->pathname, buf, file->statbuf.st_size);
+ if (r == -1) {
+ nbdkit_error ("readlink: %s: %m", file->pathname);
+ free (buf);
+ return -1;
+ }
+ if (r > file->statbuf.st_size)
+ r = file->statbuf.st_size;
+ buf[r] = '\0';
+ err = ext2fs_symlink (fs, file->dir_ino, file->ino, NULL, buf);
+ free (buf);
+ if (err) {
+ nbdkit_error ("ext2fs_symlink: %s", error_message (err));
+ return -1;
+ }
+ }
+ /* Specials. */
+ else if (S_ISBLK (file->statbuf.st_mode)) {
+ file->ino_flags = EXT2_FT_BLKDEV;
+ file->type = "block device";
+ goto special;
+ }
+ else if (S_ISCHR (file->statbuf.st_mode)) {
+ file->ino_flags = EXT2_FT_CHRDEV;
+ file->type = "char device";
+ goto special;
+ }
+ else if (S_ISFIFO (file->statbuf.st_mode)) {
+ file->ino_flags = EXT2_FT_FIFO;
+ file->type = "FIFO";
+ goto special;
+ }
+ else if (S_ISSOCK (file->statbuf.st_mode)) {
+ file->ino_flags = EXT2_FT_SOCK;
+ file->type = "socket";
+ special:
+ return e2emptyinode (fs, file->dir_ino, file->name,
+ &file->statbuf, file->ino_flags,
&file->ino);
+ }
+ else {
+ /* Unknown host file type. Skip it but emit a debug message. */
+ file->type = "unknown";
+ nbdkit_debug ("ignoring unknown file type: %s mode %o",
+ file->pathname, (unsigned) file->statbuf.st_mode);
+ }
+
+ return 0;
+}
+
+/* Create a hard link. The new directory entry is ‘file’. The
+ * existing file/inode is ‘to’.
+ */
+static int
+e2hardlink (ext2_filsys fs, struct file *to, struct file *file)
+{
+ errcode_t err;
+ struct ext2_inode inode;
+
+ if (e2link (fs, file->dir_ino, file->name, to->ino,
to->ino_flags) == -1)
+ return -1;
+
+ /* We need to increase the link count in the inode.
+ * See e2fsprogs/misc/create_inode.c:add_link
+ */
+ err = ext2fs_read_inode (fs, to->ino, &inode);
+ if (err) {
+ nbdkit_error ("ext2fs_read_inode: %s", error_message (err));
+ return -1;
+ }
+ inode.i_links_count++;
+ err = ext2fs_write_inode (fs, to->ino, &inode);
+ if (err) {
+ nbdkit_error ("ext2fs_write_inode: %s", error_message (err));
+ return -1;
+ }
+
+ return 0;
+}
+
+/* This is a helper wrapper around ext2fs_link which expands the
+ * parent directory if it fills up.
+ *
+ * The flags field is required even for hard links because the
+ * directory stores some data about the file type (returned by
+ * getdents(2)).
+ */
+static int
+e2link (ext2_filsys fs, ext2_ino_t dir_ino, const char *name,
+ ext2_ino_t ino, int flags)
+{
+ errcode_t err;
+
+ again:
+ err = ext2fs_link (fs, dir_ino, name, ino, flags);
+ if (err) {
+ if (err == EXT2_ET_DIR_NO_SPACE) {
+ err = ext2fs_expand_dir (fs, dir_ino);
+ if (err) {
+ nbdkit_error ("ext2fs_expand_dir: %s", error_message (err));
+ return -1;
+ }
+ goto again;
+ }
+ nbdkit_error ("ext2fs_link: %s", error_message (err));
+ return -1;
+ }
+
+ return 0;
+}
+
+/* For regular files, this copies the file data blocks. The inode has
+ * already been created and linked into the filesystem.
+ */
+static int
+e2copyfiledata (ext2_filsys fs, struct file *file)
+{
+ int fd = -1;
+ errcode_t err;
+ ext2_file_t f;
+ char buf[BUFSIZ];
+ size_t n;
+ ssize_t r;
+ unsigned int written;
+ off_t offset, data_start, data_end;
+
+ fd = open (file->pathname, O_RDONLY);
+ if (fd == -1) {
+ /* Supermin skips unreadable files. Here we disallow that.
+ * However we should probably add options to make this behaviour
+ * controllable. XXX
+ */
+ nbdkit_error ("open: %s: %m", file->pathname);
+ goto error;
+ }
+
+ err = ext2fs_file_open2 (fs, file->ino, NULL, EXT2_FILE_WRITE, &f);
+ if (err) {
+ nbdkit_error ("ext2fs_file_open2: %s", error_message (err));
+ goto error;
+ }
+
+ offset = 0;
+ do {
+ data_start = lseek (fd, offset, SEEK_DATA);
+ if (data_start == -1) {
+ nbdkit_error ("lseek: %s: SEEK_DATA: %m", file->pathname);
+ goto error;
+ }
+ data_end = lseek (fd, data_start, SEEK_HOLE);
+ if (data_end == -1) {
+ nbdkit_error ("lseek: %s: SEEK_HOLE: %m", file->pathname);
+ goto error;
+ }
+
+ if (linuxdisk_debug_filesystem)
+ nbdkit_debug ("%s: data region [%" PRIu64 "..%"
PRIu64 "]",
+ file->pathname,
+ (uint64_t) data_start, (uint64_t) (data_end-1));
+
+ /* Copy data from [data_start..data_end-1]. */
+ n = data_end - data_start;
+
+ if (lseek (fd, data_start, SEEK_SET) == -1) {
+ nbdkit_error ("lseek: %s: %m", file->pathname);
+ goto error;
+ }
+ err = ext2fs_file_llseek (f, data_start, EXT2_SEEK_SET, NULL);
+ if (err) {
+ nbdkit_error ("ext2fs_file_lseek: %s", error_message (err));
+ goto error;
+ }
+
+ while (n > 0) {
+ r = read (fd, buf, MIN (n, sizeof buf));
+ if (r == -1) {
+ nbdkit_error ("read: %s: %m", file->pathname);
+ goto error;
+ }
+ if (r == 0) {
+ nbdkit_error ("read: %s: unexpected end of file",
file->pathname);
+ goto error;
+ }
+
+ err = ext2fs_file_write (f, buf, r, &written);
+ if (err) {
+ nbdkit_error ("ext2fs_file_write: %s", error_message (err));
+ goto error;
+ }
+ if ((ssize_t) written != r) {
+ nbdkit_error ("ext2fs_file_write: "
+ "requested write size != bytes written");
+ goto error;
+ }
+
+ n -= written;
+ }
+
+ offset = data_end;
+ } while (offset < file->statbuf.st_size);
+
+ if (close (fd) == -1) {
+ nbdkit_error ("close: %s: %m", file->pathname);
+ fd = -1;
+ goto error;
+ }
+ fd = -1;
+
+ /* Set the true size of the file in the inode. This should handle
+ * the case of a sparse file with a hole at the end.
+ */
+ err = ext2fs_file_set_size2 (f, file->statbuf.st_size);
+ if (err) {
+ nbdkit_error ("ext2fs_file_set_size2: %s", error_message (err));
+ goto error;
+ }
+
+ err = ext2fs_file_close (f);
+ if (err) {
+ nbdkit_error ("ext2fs_file_close: %s", error_message (err));
+ goto error;
+ }
+
+ return 0;
+
+ error:
+ if (fd >= 0)
+ close (fd);
+ return -1;
+}
diff --git a/plugins/linuxdisk/linuxdisk.c b/plugins/linuxdisk/linuxdisk.c
new file mode 100644
index 0000000..36781db
--- /dev/null
+++ b/plugins/linuxdisk/linuxdisk.c
@@ -0,0 +1,236 @@
+/* nbdkit
+ * Copyright (C) 2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+#include <time.h>
+
+#define NBDKIT_API_VERSION 2
+
+#include <nbdkit-plugin.h>
+
+#include "random.h"
+#include "regions.h"
+
+#include "virtual-disk.h"
+
+/* Directory, label, size parameters. */
+char **dirs;
+size_t nr_dirs;
+const char *label;
+int64_t size;
+bool size_add_estimate; /* if size=+SIZE was used */
+
+/* Virtual disk. */
+static struct virtual_disk disk;
+
+/* Used to create a random GUID for the partition. */
+struct random_state random_state;
+
+static void
+linuxdisk_load (void)
+{
+ load_filesystem ();
+ init_virtual_disk (&disk);
+ xsrandom (time (NULL), &random_state);
+}
+
+static void
+linuxdisk_unload (void)
+{
+ size_t i;
+
+ free_virtual_disk (&disk);
+
+ for (i = 0; i < nr_dirs; ++i)
+ free (dirs[i]);
+ free (dirs);
+}
+
+static int
+linuxdisk_config (const char *key, const char *value)
+{
+ if (strcmp (key, "dir") == 0) {
+ char **new_dirs;
+ char *dir;
+
+ dir = nbdkit_realpath (value);
+ if (dir == NULL)
+ return -1;
+
+ new_dirs = realloc (dirs, (nr_dirs+1) * sizeof (char *));
+ if (new_dirs == NULL) {
+ nbdkit_error ("realloc: %m");
+ free (dir);
+ return -1;
+ }
+ dirs = new_dirs;
+ dirs[nr_dirs] = dir;
+ nr_dirs++;
+ }
+ else if (strcmp (key, "label") == 0) {
+ label = value;
+ }
+ else if (strcmp (key, "size") == 0) {
+ if (value[0] == '+') {
+ size_add_estimate = true;
+ value++;
+ }
+ else
+ size_add_estimate = false;
+ size = nbdkit_parse_size (value);
+ if (size == -1)
+ return -1;
+ }
+ else {
+ nbdkit_error ("unknown parameter '%s'", key);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+linuxdisk_config_complete (void)
+{
+ if (nr_dirs == 0) {
+ nbdkit_error ("you must supply the dir=<DIRECTORY> parameter
"
+ "after the plugin name on the command line");
+ return -1;
+ }
+
+ return create_virtual_disk ((const char **) dirs, nr_dirs, label,
+ size, size_add_estimate,
+ &disk);
+}
+
+#define linuxdisk_config_help \
+ "dir=<DIRECTORY> (required) The directory to serve.\n" \
+ "label=<LABEL> The filesystem label.\n" \
+ "size=[+]<SIZE> The virtual filesystem size."
+
+static void *
+linuxdisk_open (int readonly)
+{
+ return NBDKIT_HANDLE_NOT_NEEDED;
+}
+
+#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL
+
+/* Get the file size. */
+static int64_t
+linuxdisk_get_size (void *handle)
+{
+ return virtual_size (&disk.regions);
+}
+
+/* Serves the same data over multiple connections. */
+static int
+linuxdisk_can_multi_conn (void *handle)
+{
+ return 1;
+}
+
+/* Read data from the virtual disk. */
+static int
+linuxdisk_pread (void *handle, void *buf, uint32_t count, uint64_t offset,
+ uint32_t flags)
+{
+ while (count > 0) {
+ const struct region *region = find_region (&disk.regions, offset);
+ size_t len;
+ ssize_t r;
+
+ /* Length to end of region. */
+ len = region->end - offset + 1;
+ if (len > count)
+ len = count;
+
+ switch (region->type) {
+ case region_file:
+ /* We don't use region->u.i since there is only one backing
+ * file, and we have that open already (in ‘disk.fd’).
+ */
+ r = pread (disk.fd, buf, len, offset - region->start);
+ if (r == -1) {
+ nbdkit_error ("pread: %m");
+ return -1;
+ }
+ if (r == 0) {
+ nbdkit_error ("pread: unexpected end of file");
+ return -1;
+ }
+ len = r;
+ break;
+
+ case region_data:
+ memcpy (buf, ®ion->u.data[offset - region->start], len);
+ break;
+
+ case region_zero:
+ memset (buf, 0, len);
+ break;
+ }
+
+ count -= len;
+ buf += len;
+ offset += len;
+ }
+
+ return 0;
+}
+
+static struct nbdkit_plugin plugin = {
+ .name = "linuxdisk",
+ .longname = "nbdkit Linux virtual disk plugin",
+ .version = PACKAGE_VERSION,
+ .load = linuxdisk_load,
+ .unload = linuxdisk_unload,
+ .config = linuxdisk_config,
+ .config_complete = linuxdisk_config_complete,
+ .config_help = linuxdisk_config_help,
+ .magic_config_key = "dir",
+ .open = linuxdisk_open,
+ .get_size = linuxdisk_get_size,
+ .can_multi_conn = linuxdisk_can_multi_conn,
+ .pread = linuxdisk_pread,
+ .errno_is_preserved = 1,
+};
+
+NBDKIT_REGISTER_PLUGIN(plugin)
diff --git a/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod
b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod
new file mode 100644
index 0000000..5777b82
--- /dev/null
+++ b/plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod
@@ -0,0 +1,197 @@
+=head1 NAME
+
+nbdkit-linuxdisk-plugin - create virtual Linux disk from directory
+
+=head1 SYNOPSIS
+
+ nbdkit linuxdisk [dir=]DIRECTORY [[dir=]DIRECTORY ...]
+ [label=LABEL] [size=[+]SIZE]
+
+=head1 DESCRIPTION
+
+C<nbdkit-linuxdisk-plugin> is a plugin for L<nbdkit(1)> which
creates
+an ext2-formatted disk image from a directory on the fly. The files
+in the specified directory (and subdirectories) appear in the virtual
+disk, which is served read-only over the NBD protocol.
+
+The virtual disk is partitioned with a single GPT partition containing
+the filesystem.
+
+The virtual disk can be used as a Linux root (or other) filesystem.
+Most features of Linux filesystems are supported, such as hard links,
+symbolic links, block special devices etc. Multiple directories can
+be given on the command line and they are merged into the final
+filesystem.
+
+=head1 EXAMPLES
+
+=over 4
+
+=item nbdkit linuxdisk /path/to/directory label=ROOTFS
+
+Create a virtual disk, giving it a filesystem label. Note that
+clients will not be able to modify the filesystem, so it is safe to
+share it with multiple clients.
+
+=item nbdkit --filter=cow linuxdisk /path/to/directory
+
+Add a writable overlay (see L<nbdkit-cow-filter(1)>, allowing the disk
+to be written by the client. B<Multiple clients must not be allowed
+to connect at the same time> (even if they all mount it read-only) as
+this will cause disk corruption.
+
+=item nbdkit --filter=cow linuxdisk /path/to/directory size=+1G
+
+The same but specifying that at least 1G of free space should be
+available in the filesystem (not including the space taken by the
+initial filesystem).
+
+=item nbdkit --filter=partition linuxdisk /path/to/directory partition=1
+
+Instead of serving a partitioned disk image, serve just the "naked"
+filesystem (ie. the first partition, see
+L<nbdkit-partition-filter(1)>).
+
+=item nbdkit -U - linuxdisk /path/to/directory
+--run 'qemu-img convert $nbd ext2fs.img'
+
+This serves nothing. Instead it turns a directory into an ext2
+filesystem image, writing it to F<ext2fs.img> (see
+L<nbdkit-captive(1)>).
+
+The resulting image is a partitioned disk. If you want just the naked
+filesystem then add the partition filter as in the previous example.
+
+=item nbdkit linuxdisk /path/to/dir1 /path/to/dir2
+
+Merge two directories into a single disk image. Files in F<dir1> and
+F<dir2> will appear in the root directory of the merged disk image.
+(About duplicates appearing in both directories, see L</NOTES> below).
+
+=back
+
+=head1 PARAMETERS
+
+=over 4
+
+=item [B<dir=>]DIRECTORY
+
+Specify the directory containing files and subdirectories which will
+be added to the virtual disk. Files inside this directory will appear
+in the root directory of the virtual disk.
+
+This parameter is required at least once. If it is given multiple
+times then the directories are merged into the final filesystem.
+
+C<dir=> is a magic config key and may be omitted in most cases.
+See L<nbdkit(1)/Magic parameters>.
+
+=item B<label=>LABEL
+
+The optional label for the filesystem.
+
+=item B<size=>SIZE
+
+=item B<size=+>SIZE
+
+The total (virtual) size of the filesystem.
+
+If the C<size> parameter is omitted the plugin will try to size the
+filesystem with just enough space to contain the files and directories
+that are initially loaded, and there will not be much extra space.
+
+Using C<size=SIZE> specifies the required virtual size of the whole
+filesystem (including initial files and extra space). If this is set
+too small for the initial filesystem then the plugin will fail to
+start.
+
+Using C<size=+SIZE> specifies the minimum free space required after
+the initial filesystem has been loaded. (The actual free space might
+be slightly larger).
+
+=back
+
+=head1 NOTES
+
+=head2 Hard links
+
+This plugin preserves hard links. That is, if two or more files are
+hard linked in the source directory, they will be hard linked in the
+resulting filesystem.
+
+=head2 Symbolic links
+
+Symbolic links are copied exactly. This means they might not point to
+the same file, or to any file, once copied to the filesystem.
+
+=head2 Sparse files
+
+This plugin preserves sparseness in files to the extent possible. It
+might not always be possible depending on the filesystem block size.
+It does not create sparse files if the source file is not sparse.
+
+=head2 Block/char special devices, FIFOs, sockets, etc.
+
+These are copied into (ie. recreated in) the filesystem exactly as far
+as possible.
+
+=head2 Users, groups, permissions, file times
+
+The original file UID, GID, permissions and file times are recreated
+as far as possible. Note that UIDs/GIDs will likely map to different
+users and groups when read by a virtual machine or other NBD client
+machine.
+
+=head2 Duplicate directories allowed, duplicate files forbidden
+
+When specifying multiple source directories on the command line:
+
+ nbdkit linuxdisk dir1 dir2
+
+it is possible that there will be duplicates in the target filesystem,
+eg. F<dir1/dir> and F<dir2/dir> both requiring F</dir> to be
created
+on the target. For directories, this is permitted. The permissions,
+ownership etc of the first directory will be used.
+
+For files this is not allowed (even if the files are identical). The
+plugin will give an error in this case.
+
+=head1 UNIMPLEMENTED
+
+=head2 Extended attributes
+
+Extended attributes are not yet implemented, which means (amongst
+other things) that SELinux labels will not be copied to the target
+filesystem. This should be fixed in future.
+
+=head1 ENVIRONMENT VARIABLES
+
+=over 4
+
+=item C<TMPDIR>
+
+The filesystem image is stored in a temporary file located in
+F</var/tmp> by default. You can override this location by setting the
+C<TMPDIR> environment variable before starting nbdkit.
+
+=back
+
+=head1 SEE ALSO
+
+L<nbdkit(1)>,
+L<nbdkit-plugin(3)>,
+L<nbdkit-captive(1)>,
+L<nbdkit-cow-filter(1)>,
+L<nbdkit-file-plugin(1)>,
+L<nbdkit-floppy-plugin(1)>,
+L<nbdkit-iso-plugin(1)>,
+L<nbdkit-partition-filter(1)>,
+L<nbdkit-partitioning-plugin(1)>.
+
+=head1 AUTHORS
+
+Richard W.M. Jones
+
+=head1 COPYRIGHT
+
+Copyright (C) 2019 Red Hat Inc.
diff --git a/plugins/linuxdisk/partition-gpt.c
b/plugins/linuxdisk/partition-gpt.c
new file mode 100644
index 0000000..3278229
--- /dev/null
+++ b/plugins/linuxdisk/partition-gpt.c
@@ -0,0 +1,211 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <nbdkit-plugin.h>
+
+#include "efi-crc32.h"
+#include "gpt.h"
+#include "isaligned.h"
+#include "rounding.h"
+#include "regions.h"
+
+#include "virtual-disk.h"
+
+#define PARTITION_TYPE_GUID "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
+
+static void create_gpt_protective_mbr (struct virtual_disk *disk,
+ unsigned char *out);
+static void create_gpt_partition_header (struct virtual_disk *disk,
+ const void *pt, bool is_primary,
+ unsigned char *out);
+static void create_gpt_partition_table (struct virtual_disk *disk,
+ unsigned char *out);
+
+/* Initialize the partition table structures. */
+int
+create_partition_table (struct virtual_disk *disk)
+{
+ create_gpt_protective_mbr (disk, disk->protective_mbr);
+
+ create_gpt_partition_table (disk, disk->pt);
+
+ create_gpt_partition_header (disk, disk->pt, true,
disk->primary_header);
+ create_gpt_partition_header (disk, disk->pt, false,
disk->secondary_header);
+
+ return 0;
+}
+
+static void
+chs_too_large (unsigned char *out)
+{
+ const int c = 1023, h = 254, s = 63;
+
+ out[0] = h;
+ out[1] = (c & 0x300) >> 2 | s;
+ out[2] = c & 0xff;
+}
+
+static void
+create_mbr_partition_table_entry (const struct region *region,
+ bool bootable, int partition_id,
+ unsigned char *out)
+{
+ uint64_t start_sector, nr_sectors;
+ uint32_t u32;
+
+ assert (IS_ALIGNED (region->start, SECTOR_SIZE));
+
+ start_sector = region->start / SECTOR_SIZE;
+ nr_sectors = DIV_ROUND_UP (region->len, SECTOR_SIZE);
+
+ assert (start_sector <= UINT32_MAX);
+ assert (nr_sectors <= UINT32_MAX);
+
+ out[0] = bootable ? 0x80 : 0;
+ chs_too_large (&out[1]);
+ out[4] = partition_id;
+ chs_too_large (&out[5]);
+ u32 = htole32 (start_sector);
+ memcpy (&out[8], &u32, 4);
+ u32 = htole32 (nr_sectors);
+ memcpy (&out[12], &u32, 4);
+}
+
+static void
+create_gpt_protective_mbr (struct virtual_disk *disk, unsigned char *out)
+{
+ struct region region;
+ uint64_t end;
+
+ /* Protective MBR creates an MBR partition with partition ID 0xee
+ * which covers the whole of the disk, or as much of the disk as
+ * expressible with MBR.
+ */
+ region.start = 512;
+ end = virtual_size (&disk->regions) - 1;
+ if (end > UINT32_MAX * SECTOR_SIZE)
+ end = UINT32_MAX * SECTOR_SIZE;
+ region.end = end;
+ region.len = region.end - region.start + 1;
+
+ create_mbr_partition_table_entry (®ion, false, 0xee, &out[0x1be]);
+
+ /* Boot sector signature. */
+ out[0x1fe] = 0x55;
+ out[0x1ff] = 0xaa;
+}
+
+static void
+create_gpt_partition_header (struct virtual_disk *disk,
+ const void *pt, bool is_primary,
+ unsigned char *out)
+{
+ uint64_t nr_lbas;
+ struct gpt_header *header = (struct gpt_header *) out;
+
+ nr_lbas = virtual_size (&disk->regions) / SECTOR_SIZE;
+
+ memset (header, 0, sizeof *header);
+ memcpy (header->signature, GPT_SIGNATURE, sizeof (header->signature));
+ memcpy (header->revision, GPT_REVISION, sizeof (header->revision));
+ header->header_size = htole32 (sizeof *header);
+ if (is_primary) {
+ header->current_lba = htole64 (1);
+ header->backup_lba = htole64 (nr_lbas - 1);
+ }
+ else {
+ header->current_lba = htole64 (nr_lbas - 1);
+ header->backup_lba = htole64 (1);
+ }
+ header->first_usable_lba = htole64 (34);
+ header->last_usable_lba = htole64 (nr_lbas - 34);
+ if (is_primary)
+ header->partition_entries_lba = htole64 (2);
+ else
+ header->partition_entries_lba = htole64 (nr_lbas - 33);
+ header->nr_partition_entries = htole32 (GPT_MIN_PARTITIONS);
+ header->size_partition_entry = htole32 (GPT_PT_ENTRY_SIZE);
+ header->crc_partitions + htole32 (efi_crc32 (pt, GPT_PT_ENTRY_SIZE *
GPT_MIN_PARTITIONS));
+
+ /* Must be computed last. */
+ header->crc = htole32 (efi_crc32 (header, sizeof *header));
+}
+
+static void
+create_gpt_partition_table_entry (const struct region *region,
+ bool bootable,
+ char partition_type_guid[16],
+ char guid[16],
+ unsigned char *out)
+{
+ struct gpt_entry *entry = (struct gpt_entry *) out;
+
+ assert (sizeof (struct gpt_entry) == GPT_PT_ENTRY_SIZE);
+
+ memcpy (entry->partition_type_guid, partition_type_guid, 16);
+ memcpy (entry->unique_guid, guid, 16);
+
+ entry->first_lba = htole64 (region->start / SECTOR_SIZE);
+ entry->last_lba = htole64 (region->end / SECTOR_SIZE);
+ entry->attributes = htole64 (bootable ? 4 : 0);
+}
+
+static void
+create_gpt_partition_table (struct virtual_disk *disk, unsigned char *out)
+{
+ size_t j;
+
+ for (j = 0; j < nr_regions (&disk->regions); ++j) {
+ const struct region *region = get_region (&disk->regions, j);
+
+ /* Find the (only) partition region, which has type region_file. */
+ if (region->type == region_file) {
+ create_gpt_partition_table_entry (region, true,
+ PARTITION_TYPE_GUID,
+ disk->guid,
+ out);
+ out += GPT_PT_ENTRY_SIZE;
+ }
+ }
+}
diff --git a/plugins/linuxdisk/virtual-disk.c b/plugins/linuxdisk/virtual-disk.c
new file mode 100644
index 0000000..915dc72
--- /dev/null
+++ b/plugins/linuxdisk/virtual-disk.c
@@ -0,0 +1,163 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <nbdkit-plugin.h>
+
+#include "random.h"
+#include "regions.h"
+
+#include "virtual-disk.h"
+
+static int create_regions (struct virtual_disk *disk);
+
+void
+init_virtual_disk (struct virtual_disk *disk)
+{
+ memset (disk, 0, sizeof *disk);
+ disk->fd = -1;
+
+ init_regions (&disk->regions);
+}
+
+int
+create_virtual_disk (const char **dirs, size_t nr_dirs, const char *label,
+ int64_t size, bool size_add_estimate,
+ struct virtual_disk *disk)
+{
+ size_t i;
+
+ /* Allocate the partition table structures. We can't fill them in
+ * until we have created the disk layout.
+ */
+ disk->protective_mbr = calloc (1, SECTOR_SIZE);
+ disk->primary_header = calloc (1, SECTOR_SIZE);
+ disk->pt = calloc (1, 32*SECTOR_SIZE);
+ disk->secondary_header = calloc (1, SECTOR_SIZE);
+ if (disk->protective_mbr == NULL ||
+ disk->primary_header == NULL ||
+ disk->pt == NULL ||
+ disk->secondary_header == NULL) {
+ nbdkit_error ("calloc: %m");
+ return -1;
+ }
+
+ /* Create the filesystem. This fills in disk->filesystem_size and
+ * disk->id.
+ */
+ if (create_filesystem (disk) == -1)
+ return -1;
+
+ /* Create a random GUID used as "Unique partition GUID". However
+ * this doesn't follow GUID conventions so in theory could make an
+ * invalid value.
+ */
+ for (i = 0; i < 16; ++i)
+ disk->guid[i] = xrandom (&random_state) & 0xff;
+
+ /* Create the virtual disk regions. */
+ if (create_regions (disk) == -1)
+ return -1;
+
+ /* Initialize partition table structures. This depends on
+ * disk->regions so must be done last.
+ */
+ if (create_partition_table (disk) == -1)
+ return -1;
+
+ return 0;
+}
+
+void
+free_virtual_disk (struct virtual_disk *disk)
+{
+ free_regions (&disk->regions);
+ free (disk->protective_mbr);
+ free (disk->primary_header);
+ free (disk->pt);
+ free (disk->secondary_header);
+ if (disk->fd >= 0)
+ close (disk->fd);
+}
+
+/* Lay out the final disk. */
+static int
+create_regions (struct virtual_disk *disk)
+{
+ /* Protective MBR. */
+ if (append_region_len (&disk->regions, "Protective MBR",
+ SECTOR_SIZE, 0, 0,
+ region_data, (void *) disk->protective_mbr) == -1)
+ return -1;
+
+ /* GPT primary partition table header (LBA 1). */
+ if (append_region_len (&disk->regions, "GPT primary header",
+ SECTOR_SIZE, 0, 0,
+ region_data, (void *) disk->primary_header) == -1)
+ return -1;
+
+ /* GPT primary PT (LBA 2..33). */
+ if (append_region_len (&disk->regions, "GPT primary PT",
+ 32*SECTOR_SIZE, 0, 0,
+ region_data, (void *) disk->pt) == -1)
+ return -1;
+
+ /* Partition containing the filesystem. Align it to 2048 sectors. */
+ if (append_region_len (&disk->regions, "Filesystem",
+ disk->filesystem_size, 2048*SECTOR_SIZE, 0,
+ region_file, 0 /* unused */) == -1)
+ return -1;
+
+ /* GPT secondary PT (LBA -33..-2). */
+ if (append_region_len (&disk->regions, "GPT secondary PT",
+ 32*SECTOR_SIZE, SECTOR_SIZE, 0,
+ region_data, (void *) disk->pt) == -1)
+ return -1;
+
+ /* GPT secondary PT header (LBA -1). */
+ if (append_region_len (&disk->regions, "GPT secondary
header",
+ SECTOR_SIZE, 0, 0,
+ region_data, (void *) disk->secondary_header) ==
-1)
+ return -1;
+
+ return 0;
+}
diff --git a/plugins/linuxdisk/virtual-disk.h b/plugins/linuxdisk/virtual-disk.h
new file mode 100644
index 0000000..ae0a70e
--- /dev/null
+++ b/plugins/linuxdisk/virtual-disk.h
@@ -0,0 +1,96 @@
+/* nbdkit
+ * Copyright (C) 2018-2019 Red Hat Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#ifndef NBDKIT_VIRTUAL_DISK_H
+#define NBDKIT_VIRTUAL_DISK_H
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include "regions.h"
+
+extern char **dirs;
+extern size_t nr_dirs;
+extern const char *label;
+extern int64_t size;
+extern bool size_add_estimate;
+
+extern struct random_state random_state;
+
+#define SECTOR_SIZE 512
+
+struct virtual_disk {
+ /* Virtual disk layout. */
+ struct regions regions;
+
+ /* Disk protective MBR. */
+ uint8_t *protective_mbr;
+
+ /* GPT primary partition table header. */
+ uint8_t *primary_header;
+
+ /* GPT primary and secondary (backup) PTs. These are the same. */
+ uint8_t *pt;
+
+ /* GPT secondary (backup) PT header. */
+ uint8_t *secondary_header;
+
+ /* Size of the filesystem in bytes. */
+ uint64_t filesystem_size;
+
+ /* Unique partition GUID. */
+ char guid[16];
+
+ /* File descriptor of the temporary file containing the filesystem. */
+ int fd;
+};
+
+/* virtual-disk.c */
+extern void init_virtual_disk (struct virtual_disk *disk)
+ __attribute__((__nonnull__ (1)));
+extern int create_virtual_disk (const char **dirs, size_t nr_dirs,
+ const char *label,
+ int64_t size, bool size_add_estimate,
+ struct virtual_disk *disk)
+ __attribute__((__nonnull__ (1, 6)));
+extern void free_virtual_disk (struct virtual_disk *disk)
+ __attribute__((__nonnull__ (1)));
+
+/* partition-gpt.c */
+extern int create_partition_table (struct virtual_disk *disk);
+
+/* filesystem.c */
+extern void load_filesystem (void);
+extern int create_filesystem (struct virtual_disk *disk);
+
+#endif /* NBDKIT_VIRTUAL_DISK_H */
diff --git a/plugins/partitioning/nbdkit-partitioning-plugin.pod
b/plugins/partitioning/nbdkit-partitioning-plugin.pod
index f3d6996..d045e4c 100644
--- a/plugins/partitioning/nbdkit-partitioning-plugin.pod
+++ b/plugins/partitioning/nbdkit-partitioning-plugin.pod
@@ -19,8 +19,8 @@ If you just want to concatenate files together (without adding
a
partition table) use L<nbdkit-split-plugin(1)>. If you want to select
a single partition from an existing disk, use
L<nbdkit-partition-filter(1)>. If you want to create a complete disk
-with a filesystem, look at L<nbdkit-floppy-plugin(1)> or
-L<nbdkit-iso-plugin(1)>.
+with a filesystem, look at L<nbdkit-floppy-plugin(1)>,
+L<nbdkit-iso-plugin(1)> or L<nbdkit-linuxdisk-plugin(1)>.
The plugin supports read/write access. To limit clients to read-only
access use the I<-r> flag.
@@ -172,6 +172,7 @@ L<nbdkit(1)>,
L<nbdkit-file-plugin(1)>,
L<nbdkit-floppy-plugin(1)>,
L<nbdkit-iso-plugin(1)>,
+L<nbdkit-linuxdisk-plugin(1)>,
L<nbdkit-partition-filter(1)>,
L<nbdkit-split-plugin(1)>,
L<nbdkit-plugin(3)>.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index c75a9de..6a49e23 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -71,6 +71,8 @@ EXTRA_DIST = \
test-ip.sh \
test-iso.sh \
test-layers.sh \
+ test-linuxdisk.sh \
+ test-linuxdisk-copy-out.sh \
test-log.sh \
test.lua \
test-memory-largest.sh \
@@ -447,6 +449,15 @@ TESTS += test-iso.sh
endif HAVE_GUESTFISH
endif HAVE_ISO
+# linuxdisk plugin test.
+if HAVE_EXT2
+if HAVE_GUESTFISH
+TESTS += \
+ test-linuxdisk.sh \
+ test-linuxdisk-copy-out.sh
+endif HAVE_GUESTFISH
+endif HAVE_EXT2
+
# memory plugin test.
LIBGUESTFS_TESTS += test-memory
TESTS += test-memory-largest.sh test-memory-largest-for-qemu.sh
diff --git a/tests/test-linuxdisk-copy-out.sh b/tests/test-linuxdisk-copy-out.sh
new file mode 100755
index 0000000..1b1af56
--- /dev/null
+++ b/tests/test-linuxdisk-copy-out.sh
@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# nbdkit
+# Copyright (C) 2018-2019 Red Hat Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+# Test the linuxdisk plugin with captive nbdkit, as described
+# in the man page.
+
+source ./functions.sh
+set -e
+set -x
+
+requires qemu-img --version
+
+files="linuxdisk-copy-out.img
+ linuxdisk-copy-out.test1 linuxdisk-copy-out.test2
+ linuxdisk-copy-out.test3 linuxdisk-copy-out.test4"
+rm -f $files
+cleanup_fn rm -f $files
+
+nbdkit -f -v -D linuxdisk.filesystem=1 -U - \
+ --filter=partition \
+ linuxdisk $srcdir/../plugins partition=1 label=ROOT \
+ --run 'qemu-img convert $nbd linuxdisk-copy-out.img'
+
+# Check the disk content.
+guestfish --ro -a linuxdisk-copy-out.img -m /dev/sda <<EOF
+# Check some known files and directories exist.
+ ll /
+ ll /linuxdisk
+ is-dir /linuxdisk
+ is-file /linuxdisk/Makefile.am
+
+# This reads out all the directory entries and all file contents.
+ tar-out / - | cat >/dev/null
+
+# Download some files and compare to local copies.
+ download /linuxdisk/Makefile linuxdisk-copy-out.test1
+ download /linuxdisk/Makefile.am linuxdisk-copy-out.test2
+ download /linuxdisk/nbdkit-linuxdisk-plugin.pod linuxdisk-copy-out.test3
+ download /linuxdisk/filesystem.c linuxdisk-copy-out.test4
+EOF
+
+# Compare downloaded files to local versions.
+cmp linuxdisk-copy-out.test1 $srcdir/../plugins/linuxdisk/Makefile
+cmp linuxdisk-copy-out.test2 $srcdir/../plugins/linuxdisk/Makefile.am
+cmp linuxdisk-copy-out.test3
$srcdir/../plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod
+cmp linuxdisk-copy-out.test4 $srcdir/../plugins/linuxdisk/filesystem.c
diff --git a/tests/test-linuxdisk.sh b/tests/test-linuxdisk.sh
new file mode 100755
index 0000000..02e0437
--- /dev/null
+++ b/tests/test-linuxdisk.sh
@@ -0,0 +1,94 @@
+#!/usr/bin/env bash
+# nbdkit
+# Copyright (C) 2018-2019 Red Hat Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# * Neither the name of Red Hat nor the names of its contributors may be
+# used to endorse or promote products derived from this software without
+# specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS
IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+# SUCH DAMAGE.
+
+# Test the linuxdisk plugin.
+
+source ./functions.sh
+set -e
+set -x
+
+requires mkfifo --version
+
+d=linuxdisk.d
+rm -rf $d
+cleanup_fn rm -rf $d
+
+# Create a test directory with some regular files, subdirectories and
+# special files.
+mkdir $d
+mkfifo $d/fifo
+mkdir $d/sub
+cp $srcdir/Makefile.am $d/sub/Makefile.am
+ln $d/sub/Makefile.am $d/sub/hardlink
+ln -s $d/sub/Makefile.am $d/sub/symlink
+
+# It would be nice to use the Unix domain socket to test that the
+# socket gets created, but in fact that won't work because this socket
+# isn't created until after the plugin creates the virtual disk.
+start_nbdkit -P $d/linuxdisk.pid \
+ -U $d/linuxdisk.sock \
+ -D linuxdisk.filesystem=1 \
+ linuxdisk $d
+
+# Check the disk content.
+guestfish --ro --format=raw -a "nbd://?socket=$PWD/$d/linuxdisk.sock"
-m /dev/sda1 <<EOF
+ ll /
+ ll /sub
+
+# Check regular files exist.
+ is-file /sub/Makefile.am
+ is-file /sub/hardlink
+# XXX Test sparse files in future.
+
+# Check the specials exist.
+ is-fifo /fifo
+ is-symlink /sub/symlink
+# XXX Test sockets, etc. in future.
+
+# Check hard linked files.
+ lstatns /sub/Makefile.am | cat > $d/nlink.1
+ lstatns /sub/hardlink | cat > $d/nlink.2
+
+# This reads out all the directory entries and all file contents.
+ tar-out / - | cat >/dev/null
+
+# Download file and compare to local copy.
+ download /sub/Makefile.am $d/Makefile.am
+EOF
+
+# Check the two hard linked files have st_nlink == 2.
+grep "st_nlink: 2" $d/nlink.1
+grep "st_nlink: 2" $d/nlink.2
+
+# Compare downloaded file to local version.
+cmp $d/Makefile.am $srcdir/Makefile.am
--
1.8.3.1
Eric Blake
2019-Feb-19 14:06 UTC
Re: [Libguestfs] [PATCH nbdkit 1/4] common: Move some GPT functionality to a common directory.
On 2/19/19 1:49 AM, Richard W.M. Jones wrote:> From: "Richard W.M. Jones" <rjones@redhat.com> > > Headers and code related to GUID Partition Tables are moved to a > common directory to allow us to share that code across multiple > plugins and filters in future. This change is pure refactoring. > ---> +++ b/common/gpt/Makefile.am> + > +libgpt_la_SOURCES = \ > + efi-crc32.c \ > + efi-crc32.h \ > + gpt.hInconsistent TAB usage Otherwise looks good. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Eric Blake
2019-Feb-19 14:15 UTC
Re: [Libguestfs] [PATCH nbdkit 3/4] common: Move a utility function to a common directory.
On 2/19/19 1:49 AM, Richard W.M. Jones wrote:> From: "Richard W.M. Jones" <rjones@redhat.com> > > The shell_quote function is moved to a new common/utils directory. > Eventually more utility functions can be created here. > > This change is pure refactoring.Indeed, so no problem with the patch as-is, but you may have a low-priority latent bug (here and in libguestfs) to address in a separate patch:> +/* Print str to fp, shell quoting if necessary. This comes from > + * libguestfs, but was written by me so I'm relicensing it to a BSD > + * license for nbdkit. > + */ > +void > +shell_quote (const char *str, FILE *fp) > +{ > + const char *safe_chars > + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_=,:/";'=' is not always a safe shell character. Consider: $ echo echo hi > a=b $ chmod +x a=b $ (PATH=$PATH:; 'a=b') hi (I used the subshell to force a PATH lookup in the current directory, since not everyone's PATH has that by default.) That is, adding quotes around a string containing = determines whether that string can operate as a variable assignment or as a program name. Then, there's the fact that: myprog a=b c="d e" looks nicer than myprog "a=b" "c=d e" In general, since you are unlikely to have argv[0] containing '=', the current code treating = as safe is probably okay, but if we DO want to special case = in argv[0], then we may also want to special case quoting for any other arguments that resemble name=value to only quote after the first = if the content before the = resembles a shell name. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Eric Blake
2019-Feb-19 15:21 UTC
Re: [Libguestfs] [PATCH nbdkit 4/4] Add linuxdisk plugin.
On 2/19/19 1:49 AM, Richard W.M. Jones wrote:> From: "Richard W.M. Jones" <rjones@redhat.com> > > This plugin allows you to create a complete ext2 filesystem in a GPT > partitioned disk image. This can be attached as a disk to a Linux > virtual machine. It is implemented using libext2fs (the same as > supermin). > > Although there is some overlap with nbdkit-iso-plugin and > nbdkit-floppy-plugin, the implementations and use cases of all three > plugins are sufficiently different that it seems to make sense to add > another plugin rather than attempting to extend one of the existing > plugins. > > Largely to avoid user error this plugin is read-only. This is a major > difference from the floppy plugin: that plugin allows files to be > modified (but not resized or created) and writes those changes through > to the backing filesystem. While this plugin could easily be made > writable, this would cause almost certain disk corruption when someone > connected two clients at the same time. In any case it doesn't make > much sense for it to be writable by default since the expectation that > writes would somehow modify the original directory on the host > filesystem cannot be satisfied by this or any reasonable > implementation. Users can add the cow filter on top if they really > want writes and know what they are doing: instructions plus disclaimer > about this are included in the man page. > > As mentioned above, this implementation is based on the same idea as > the appliance creation code in supermin. Eventually we could replace > that supermin code with this plugin, but there are some missing > features that would need to be implemented first. > ---> +++ b/plugins/floppy/nbdkit-floppy-plugin.pod > @@ -74,6 +74,7 @@ important, and it simplifies the implementation greatly. > L<nbdkit(1)>, > L<nbdkit-plugin(3)>, > L<nbdkit-file-plugin(1)>, > +L<nbdkit-linuxdisk-plugin(1)>, > L<nbdkit-iso-plugin(1)>. > > =head1 AUTHORS > diff --git a/plugins/iso/nbdkit-iso-plugin.pod b/plugins/iso/nbdkit-iso-plugin.pod > index 4d9cf41..90e26f0 100644Should this have as much text pointing to the other disk plugins...> --- a/plugins/iso/nbdkit-iso-plugin.pod > +++ b/plugins/iso/nbdkit-iso-plugin.pod > @@ -17,8 +17,9 @@ read-only over the NBD protocol. > This plugin uses L<genisoimage(1)> or L<mkisofs(1)> to create the ISO > content. > > -To create a virtual floppy disk instead of a CD, see > -L<nbdkit-floppy-plugin(1)>. > +To create a FAT-formatted virtual floppy disk instead of a CD, see > +L<nbdkit-floppy-plugin(1)>. To create a Linux compatible virtual > +disk, see L<nbdkit-linuxdisk-plugin(1)>....as what you did here?> +++ b/plugins/linuxdisk/filesystem.c> +int > +create_filesystem (struct virtual_disk *disk) > +{> + /* Add 20% to the estimate to account for the overhead of > + * filesystem metadata. Also set a minimum size. Note we are > + * only wasting virtual space (since this will be stored sparsely > + * under $TMDIR) so we can be generous here.TMPDIR> + /* Create the filesystem file. */ > + tmpdir = getenv ("TMPDIR"); > + if (tmpdir == NULL) > + tmpdir = LARGE_TMPDIR; > + if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) { > + nbdkit_error ("asprintf: %m"); > + return -1; > + } > + > + disk->fd = mkstemp (filename); > + if (disk->fd == -1) { > + nbdkit_error ("mkstemp: %s: %m", filename); > + free (filename); > + return -1; > + } > + if (ftruncate (disk->fd, size) == -1) { > + nbdkit_error ("ftruncate: %s: %m", filename); > + free (filename);Missing an unlink(). Also, does the caller properly use close(disk->fd) on early exits like this, or does it not matter because we're just going to exit()?> + return -1; > + } > + > + /* Create the filesystem. */ > + if (mke2fs (filename) == -1) { > + unlink (filename); > + free (filename); > + return -1;Another example of early exit leaving disk->fd open.> + } > + > + /* Open the filesystem. */ > + err = ext2fs_open (filename, EXT2_FLAG_RW|EXT2_FLAG_64BITS, 0, 0, > + unix_io_manager, &fs); > + unlink (filename); > + free (filename); > + if (err) { > + nbdkit_error ("ext2fs_open: %s", error_message (err)); > + return -1; > + } > + > + err = ext2fs_read_bitmaps (fs); > + if (err) { > + nbdkit_error ("ext2fs_read_bitmaps: %s", error_message (err)); > + ext2fs_close (fs); > + return -1; > + } > + > + /* First pass: This creates subdirectories in the filesystem and > + * also builds a list of files so we can identify hard links. > + */ > + for (i = 0; i < nr_dirs; ++i) { > + if (visit (dirs[i], fs, EXT2_ROOT_INO, &files, &nr_files) == -1) > + return -1; > + } > + > + if (nr_files > 0) { > + assert (files != NULL); > + > + /* Sort the files by device and inode number to identify hard links. */ > + qsort (files, nr_files, sizeof (struct file), compare_inodes); > + > + /* Second pass: Copy/create the files. */ > + last_file = NULL; > + for (i = 0; i < nr_files; ++i) { > + hardlinked = last_file && compare_inodes (last_file, &files[i]) == 0; > + > + if (!hardlinked) { > + /* Normal case: creating a new inode. */ > + last_file = &files[i]; > + assert (!S_ISDIR (files[i].statbuf.st_mode)); > + > + if (e2copyfile (fs, &files[i]) == -1) > + return -1; > + > + if (linuxdisk_debug_filesystem) > + nbdkit_debug ("%s: <%" PRIu32 ">/%s -> <%" PRIu32 ">", > + files[i].type, > + files[i].dir_ino, files[i].name, files[i].ino); > + } > + else { > + /* Creating a hard link to an existing inode. */ > + if (e2hardlink (fs, last_file, &files[i]) == -1) > + return -1;More early exits; this time, skipping...> + > + if (linuxdisk_debug_filesystem) > + nbdkit_debug ("hard link: <%" PRIu32 ">/%s -> <%" PRIu32 ">", > + files[i].dir_ino, files[i].name, last_file->ino); > + } > + } > + > + for (i = 0; i < nr_files; ++i) { > + free (files[i].pathname); > + free (files[i].name); > + } > + free (files);...the memory cleanup. Again, I guess that's okay if we know the caller is going to exit() on our failure.> + } > + > + /* Close the filesystem. Note we don't bother to sync it because > + * it's a private temporary file which only we will read. > + */ > + ext2fs_close2 (fs, EXT2_FLAG_FLUSH_NO_SYNC); > + return 0; > +} > + > +/* Use ‘du’ to estimate the size of the filesystem quickly. > + * > + * Typical output from ‘du -cs dir1 dir2’ is: > + * > + * 12345 dir1 > + * 34567 dir2 > + * 46912 total > + * > + * We ignore everything except the first number on the last line. > + */ > +static int64_t > +estimate_size (void) > +{> + fprintf (fp, "du -c -k -s");'du -c' is not specified by POSIX, but is reasonable to expect (we can't build this plugin without ext2 support, which implies we are probably on a Linux system to begin with). If so, should we rely on having GNU coreutils' du, where we can use -B1 to force output in bytes...> + /* Run the command. */ > + nbdkit_debug ("%s", command); > + fp = popen (command, "r"); > + free (command); > + if (fp == NULL) { > + nbdkit_error ("du command failed: %m"); > + return -1; > + } > + > + /* Ignore everything up to the last line. */ > + len = 0; > + while (getline (&line, &len, fp) != -1) > + /* empty */; > + if (ferror (fp)) { > + nbdkit_error ("getline failed: %m"); > + free (line); > + fclose (fp);You should really use pclose(fp) here, to avoid libc leaking internal data associated with the pipe.> + return -1; > + } > + > + fclose (fp);and again here, and maybe even check WIFEXITED() and WEXITSTATUS() of the results.> + > + /* Parse the last line. */ > + if (sscanf (line, "%" SCNi64, &ret) != 1 || ret < 0) { > + nbdkit_error ("could not parse last line of output: %s", line); > + free (line); > + return -1; > + } > + free (line); > + > + /* Result is in 1K blocks, convert it to bytes. */ > + ret *= 1024;...rather than having to scale the output of -k?> + return ret; > +} > + > +static int > +mke2fs (const char *filename) > +{> + /* Run the command. */ > + nbdkit_debug ("%s", command); > + r = system (command); > + free (command); > + > + if (WIFEXITED (r) && WEXITSTATUS (r) != 0) { > + nbdkit_error ("mke2fs command failed with exit code %d", WEXITSTATUS (r)); > + return -1; > + } > + else if (WIFSIGNALED (r)) { > + nbdkit_error ("mke2fs command was killed by signal %d", WTERMSIG (r)); > + return -1; > + } > + else if (WIFSTOPPED (r)) { > + nbdkit_error ("mke2fs command was stopped by signal %d", WSTOPSIG (r)); > + return -1; > + }Is WIFSTOPPED() even a possible result of system()? 'man waitpid' says this status is only possible for a call made with WUNTRACED, which seems contradictory to system() running the command to completion without tracing.> + > + return 0; > +} > + > +static int > +visit (const char *dir, ext2_filsys fs, ext2_ino_t dir_ino, > + struct file **files, size_t *nr_files) > +{> + error2: > + closedir (DIR); > + error1: > + err = errno; > +#pragma GCC diagnostic push > +#pragma GCC diagnostic ignored "-Wunused-result" > + chdir (origdir); > +#pragma GCC diagnostic popFailure to return to the original directory may cause the rest of nbdkit to misbehave, shouldn't we check the result after all and exit on error?> + > +/* To identify hard links we sort the files array by device and inode > + * number (thus hard linked files will be next to each other after the > + * sort). Hence this odd sorting function. > + */ > +static int > +compare_inodes (const void *vp1, const void *vp2) > +{ > + const struct file *f1 = vp1; > + const struct file *f2 = vp2; > + > + if (f1->statbuf.st_dev < f2->statbuf.st_dev) > + return -1; > + else if (f1->statbuf.st_dev > f2->statbuf.st_dev) > + return 1; > + else { > + if (f1->statbuf.st_ino < f2->statbuf.st_ino) > + return -1; > + else if (f1->statbuf.st_ino > f2->statbuf.st_ino) > + return 1; > + else > + return 0;The sort is not a complete order (two files with the same inode have an arbitrary order), but I don't think that matters. And it's not that odd to visit files in inode order - GNU coreutils has discovered that when deleting a large directory, performance is often faster if you visit files in inode order rather than in readdir() order.> +static int > +e2emptyinode (ext2_filsys fs, ext2_ino_t dir_ino, > + const char *name, const struct stat *statbuf, > + int ino_flags, ext2_ino_t *ino) > +{ > + errcode_t err; > + struct ext2_inode inode; > + int major_, minor_; > + > + err = ext2fs_new_inode (fs, dir_ino, statbuf->st_mode, 0, ino); > + if (err) { > + nbdkit_error ("ext2fs_new_inode: %s", error_message (err)); > + return -1; > + } > + > + memset (&inode, 0, sizeof inode); > + inode.i_mode = statbuf->st_mode; > + inode.i_uid = statbuf->st_uid; > + inode.i_gid = statbuf->st_gid; > + inode.i_blocks = 0; > + inode.i_links_count = 1; > + /* XXX nanosecond times? */ > + inode.i_ctime = statbuf->st_ctime; > + inode.i_atime = statbuf->st_atime; > + inode.i_mtime = statbuf->st_mtime; > + inode.i_size = 0;Not for this patch, but now that newer Linux has the statx() call that can expose birthtime, should we be worrying about that (well, ext4 supports birthtime, but if you are targetting just ext2, it probably doesn't matter). Otherwise looks reasonable. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Possibly Parallel Threads
- Re: [PATCH nbdkit 4/4] Add linuxdisk plugin.
- [PATCH 1/3] ext2: create a struct for the OCaml 't' type
- [PATCH] ext2: Don't load whole files into memory when copying to the appliance (RHBZ#1113065).
- [PATCH supermin v2] ext2: Create symlinks properly (RHBZ#1470157).
- [PATCH nbdkit 0/4] New plugin: Add linuxdisk plugin.