search for: large_tmpdir

...se we must uncompress it into a temporary. *) - let temp_dir = (open_guestfs ())#get_cachedir () in - let new_filename = Filename.temp_file ~temp_dir "ova" ".vmdk" in + let new_filename = + Filename.temp_file ~temp_dir:Utils.large_tmpdir "ova" ".vmdk" in unlink_on_exit new_filename; let cmd = sprintf "zcat %s > %s" (quote filename) (quote new_filename) in diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml index f1d143e97..7a7647e53 100644 --- a/v2v/input_vmx.ml ++...

...ms which use the > /tmp-on-tmpdir misfeature. On these systems, /tmp has only a small, > limited amount of space and is backed by memory, and we must use > /var/tmp for large files. I don't know how to fix this. Perhaps a > simple: > > AS_CASE([$host_os], > [haiku*], [LARGE_TMPDIR=/tmp] > [LARGE_TMPDIR=/var/tmp] > ) > AC_DEFINE_UNQUOTED([LARGE_TMPDIR],["$LARGE_TMPDIR"], > [Temporary directory for large files]) > > would be enough for now? That should do. I made a patch here: https://github.com/mmuman/nbdkit/commits/tmpfix...

v1 was here: https://www.redhat.com/archives/libguestfs/2020-April/msg00007.html There's a BZ for this now which I forgot to add to the commit message: https://bugzilla.redhat.com/show_bug.cgi?id=1814611 For v2: - Fix incorrect reference to $TMPDIR in existing manual. - Separate handling for small temporary files and large temporary files. Small temporary files go into $TMPDIR

...#39;t work well on some Linux systems which use the /tmp-on-tmpdir misfeature. On these systems, /tmp has only a small, limited amount of space and is backed by memory, and we must use /var/tmp for large files. I don't know how to fix this. Perhaps a simple: AS_CASE([$host_os], [haiku*], [LARGE_TMPDIR=/tmp] [LARGE_TMPDIR=/var/tmp] ) AC_DEFINE_UNQUOTED([LARGE_TMPDIR],["$LARGE_TMPDIR"], [Temporary directory for large files]) would be enough for now? Finally about the list of failing tests: https://pastebin.com/1PFG85DS Mostly these are because of the old qemu-i...

..._ops, void *nxdata, void *handle, uint32_t flags, int *err); static void cache_load (void) @@ -104,6 +99,8 @@ cache_load (void) size_t len; char *template; + bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); + tmpdir = getenv ("TMPDIR"); if (!tmpdir) tmpdir = LARGE_TMPDIR; @@ -180,28 +177,8 @@ cache_open (nbdkit_next_open *next, void *nxdata, int readonly) static int blk_set_size (uint64_t new_size) { - uint8_t *new_bm; - const size_t old_bm_size = bm_size; - uint64_t new_bm_size_u64 = DIV_ROUND_UP (new_size, BLKSIZE*8/2); - size_t new_bm_size; - - if (new_b...

..._ops, void *nxdata, void *handle, uint32_t flags, int *err); static void cache_load (void) @@ -104,6 +99,8 @@ cache_load (void) size_t len; char *template; + bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); + tmpdir = getenv ("TMPDIR"); if (!tmpdir) tmpdir = LARGE_TMPDIR; @@ -133,6 +130,8 @@ cache_unload (void) { if (fd >= 0) close (fd); + + bitmap_free (&bm); } static int @@ -180,28 +179,8 @@ cache_open (nbdkit_next_open *next, void *nxdata, int readonly) static int blk_set_size (uint64_t new_size) { - uint8_t *new_bm; - const size_t old_...

..._ops, void *nxdata, void *handle, uint32_t flags, int *err); static void cache_load (void) @@ -104,6 +99,8 @@ cache_load (void) size_t len; char *template; + bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); + tmpdir = getenv ("TMPDIR"); if (!tmpdir) tmpdir = LARGE_TMPDIR; @@ -133,6 +130,8 @@ cache_unload (void) { if (fd >= 0) close (fd); + + bitmap_free (&bm); } static int @@ -180,28 +179,8 @@ cache_open (nbdkit_next_open *next, void *nxdata, int readonly) static int blk_set_size (uint64_t new_size) { - uint8_t *new_bm; - const size_t old_...

I have some patches I'm working on to fix the cache filter. However this is a prelude. It should be simply pure refactoring. All tests pass still. Rich.

v2: https://www.redhat.com/archives/libguestfs/2018-December/msg00039.html v2 -> v3: - Fix all the issues raised in Eric's review. - Precompute some numbers to make the calculations easier. - Calculations now use bitshifts and masks in preference to division and modulo. - Clear existing bits before setting (which fixes a bug in the cache filter). Rich.

This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.

...imum size. Note we are > + * only wasting virtual space (since this will be stored sparsely > + * under $TMDIR) so we can be generous here. TMPDIR > + /* Create the filesystem file. */ > + tmpdir = getenv ("TMPDIR"); > + if (tmpdir == NULL) > + tmpdir = LARGE_TMPDIR; > + if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) { > + nbdkit_error ("asprintf: %m"); > + return -1; > + } > + > + disk->fd = mkstemp (filename); > + if (disk->fd == -1) { > + nbdkit_error ("mkstemp: %s: %m&qu...

...e = 128 * 1024; + + assert (size == -1); + + /* Get the size of the underlying plugin. */ + compressed_size = next_ops->get_size (nxdata); + if (compressed_size == -1) + return -1; + + /* Create the temporary file. */ + tmpdir = getenv ("TMPDIR"); + if (!tmpdir) + tmpdir = LARGE_TMPDIR; + + len = strlen (tmpdir) + 8; + template = alloca (len); + snprintf (template, len, "%s/XXXXXX", tmpdir); + +#ifdef HAVE_MKOSTEMP + fd = mkostemp (template, O_CLOEXEC); +#else + /* This is only invoked serially with the lock held, so this is safe. */ + fd = mkstemp (template); +...

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

...+ } + + /* Round the final size up to a whole number of sectors. */ + size = ROUND_UP (size, SECTOR_SIZE); + + nbdkit_debug ("filesystem virtual size: %" PRIi64, size); + + /* Create the filesystem file. */ + tmpdir = getenv ("TMPDIR"); + if (tmpdir == NULL) + tmpdir = LARGE_TMPDIR; + if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) { + nbdkit_error ("asprintf: %m"); + goto error; + } + + fd = mkstemp (filename); + if (fd == -1) { + nbdkit_error ("mkstemp: %s: %m", filename); + goto error; + } + if (ftruncate (f...

Turns out Japanese trains are good for coding! In supermin we have a bunch of code to create the libguestfs appliance. It creates it directly using libext2fs (part of e2fsprogs). We can use the same technique to create ext2 virtual disks in nbdkit, which is what this new plugin does. Why a new plugin instead of modifying the floppy plugin? See the 4/4 commit message for an explanation. The

...to a whole number of sectors. */ + size = ROUND_UP (size, SECTOR_SIZE); + disk->filesystem_size = size; + + nbdkit_debug ("filesystem virtual size: %" PRIi64, size); + + /* Create the filesystem file. */ + tmpdir = getenv ("TMPDIR"); + if (tmpdir == NULL) + tmpdir = LARGE_TMPDIR; + if (asprintf (&filename, "%s/linuxdiskXXXXXX", tmpdir) == -1) { + nbdkit_error ("asprintf: %m"); + return -1; + } + + disk->fd = mkstemp (filename); + if (disk->fd == -1) { + nbdkit_error ("mkstemp: %s: %m", filename); + free (filename); +...

For v3 I reimplemented this using mke2fs -d. This obviously makes the implementation a whole lot simpler, but cannot support multiple directory merging. Patches 1-3 are the same as before. I've also reproduced the notes from v2 below. v2: - Fix inconsistent tab/space. - All 3 plugins now contain a block of text pointing to the other 2 plugins. - TMDIR -> TMPDIR - Unlink the

Another interesting thing you can do with this plugin: https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/ v2: - Fix inconsistent tab/space. - All 3 plugins now contain a block of text pointing to the other 2 plugins. - TMDIR -> TMPDIR - Unlink the temporary file and other cleanups along error paths. - fclose -> pclose, and check the return value for errors. -