search for: v2vovl

...here, and there are no "funny" restrictions, > to be able to create the libguestfs appliance. I proved this by > setting TMPDIR to a temporary directory under / and running > libguestfs-test-tool. > > There appears to be quite a lot of free space here, so in fact the > v2vovl files could easily be stored here too. (They only store the > conversion delta, not the full guest images.) > > * /var/tmp => an NFS mount from a PVC > > This is a large (2T) external NFS mount. I actually started two pods > to see if they got the same NFS mount point, and...

...es which are currently placed in cachedir)? > That said, it is not clear to me why /var/tmp should be shared among > containers. I also don't think /var/tmp should be shared, nor use NFS. However the way that virt-v2v works at the moment means you cannot put the large files (especially v2vovl*.qcow2) in a different place from the libguestfs appliance. This means that if you have only "some" space in /var/tmp -- enough for the appliance, but not enough for the potentially much larger space required by v2vovl*.qcow2 with multiple copies of virt-v2v running -- then you cannot se...

...t; > How about if we confine the change to just large files (ie. ones > > which are currently placed in cachedir)? > > This is already the case, isn't it? > > > However the way that virt-v2v works at the moment means you cannot put > > the large files (especially v2vovl*.qcow2) in a different place from > > the libguestfs appliance. This means that if you have only "some" > > space in /var/tmp -- enough for the appliance, but not enough for the > > potentially much larger space required by v2vovl*.qcow2 with multiple > > copies o...

...ay to protect the source image(s). *) and create_overlays source_disks = @@ -286,7 +284,7 @@ and create_overlays source_disks = List.mapi ( fun i ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in + Filename.temp_file ~temp_dir:tmpdir "v2vovl" ".qcow2" in unlink_on_exit overlay_file; (* There is a specific reason to use the newer qcow2 variant: @@ -822,8 +820,7 @@ and actual_target_size target_file disk_stats = and prese...

...files will linger way more > > How about if we confine the change to just large files (ie. ones > which are currently placed in cachedir)? This is already the case, isn't it? > However the way that virt-v2v works at the moment means you cannot put > the large files (especially v2vovl*.qcow2) in a different place from > the libguestfs appliance. This means that if you have only "some" > space in /var/tmp -- enough for the appliance, but not enough for the > potentially much larger space required by v2vovl*.qcow2 with multiple > copies of virt-v2v running -...

...s. There is sufficient space here, and there are no "funny" restrictions, to be able to create the libguestfs appliance. I proved this by setting TMPDIR to a temporary directory under / and running libguestfs-test-tool. There appears to be quite a lot of free space here, so in fact the v2vovl files could easily be stored here too. (They only store the conversion delta, not the full guest images.) * /var/tmp => an NFS mount from a PVC This is a large (2T) external NFS mount. I actually started two pods to see if they got the same NFS mount point, and they do. Also I wrote files t...

...here, and there are no "funny" restrictions, > to be able to create the libguestfs appliance. I proved this by > setting TMPDIR to a temporary directory under / and running > libguestfs-test-tool. > > There appears to be quite a lot of free space here, so in fact the > v2vovl files could easily be stored here too. (They only store the > conversion delta, not the full guest images.) The thing is that nobody can guarantee that there will be enough space. The underlying filesystem (not the mountpoint) is shared between all the containers running on the host. This is t...

...t the source from being modified"); - let overlay_dir = (new Guestfs.guestfs ())#get_cachedir () in - let overlays = - List.map ( - fun ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> - let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in - unlink_on_exit overlay_file; - - let options = - "compat=1.1" ^ - (match format with None -> "" - | Some fmt -> ",backing_fmt=" ^ fmt) in - let cmd = - sprintf "qemu-i...

...t; > How about if we confine the change to just large files (ie. ones > > which are currently placed in cachedir)? > > This is already the case, isn't it? > > > However the way that virt-v2v works at the moment means you cannot put > > the large files (especially v2vovl*.qcow2) in a different place from > > the libguestfs appliance. This means that if you have only "some" > > space in /var/tmp -- enough for the appliance, but not enough for the > > potentially much larger space required by v2vovl*.qcow2 with multiple > > copies o...

...Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com> --- tmp/.gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/tmp/.gitignore b/tmp/.gitignore index 28ac388f3..912a946b6 100644 --- a/tmp/.gitignore +++ b/tmp/.gitignore @@ -2,4 +2,5 @@ /guestfs.* /libguestfs??????/ /run-* +/v2vovl*.qcow2 /valgrind-*.log -- 2.16.1

...ing overlays to protect the sources from being modified"); + let overlay_dir = (new Guestfs.guestfs ())#get_cachedir () in + List.mapi ( + fun i ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> + let overlay_file = + Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in + unlink_on_exit overlay_file; + + let options = + "compat=1.1" ^ + (match format with None -> "" + | Some fmt -> ",backing_fmt=" ^ fmt) in + let cmd = + sprintf "qemu-img create -q -...

...e the change to just large files (ie. ones > > > which are currently placed in cachedir)? > > > > This is already the case, isn't it? > > > > > However the way that virt-v2v works at the moment means you cannot put > > > the large files (especially v2vovl*.qcow2) in a different place from > > > the libguestfs appliance. This means that if you have only "some" > > > space in /var/tmp -- enough for the appliance, but not enough for the > > > potentially much larger space required by v2vovl*.qcow2 with multiple &gt...

...ay to protect the source image(s). *) and create_overlays source_disks = @@ -286,8 +284,7 @@ and create_overlays source_disks = List.mapi ( fun i ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in - unlink_on_exit overlay_file; + Filename.temp_file ~temp_dir:cachedir "v2vovl" ".qcow2" in (* There is a specific reason to use the newer qcow2 variant: * Because the L2 table can store zero clusters efficiently, and @@ -...

...t the source from being modified"); let overlay_dir = (new Guestfs.guestfs ())#get_cachedir () in - let overlays = - List.map ( - fun ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> - let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in - unlink_on_exit overlay_file; - - let options = - "compat=1.1" ^ - (match format with None -> "" - | Some fmt -> ",backing_fmt=" ^ fmt) in - let cmd = - sprintf "qemu-i...

...unny" restrictions, > > to be able to create the libguestfs appliance. I proved this by > > setting TMPDIR to a temporary directory under / and running > > libguestfs-test-tool. > > > > There appears to be quite a lot of free space here, so in fact the > > v2vovl files could easily be stored here too. (They only store the > > conversion delta, not the full guest images.) > > The thing is that nobody can guarantee that there will be enough space. > The underlying filesystem (not the mountpoint) is shared between all the > containers runni...

...ay to protect the source image(s). *) and create_overlays source_disks = @@ -286,7 +284,7 @@ and create_overlays source_disks = List.mapi ( fun i ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> let overlay_file = - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in + Filename.temp_file ~temp_dir:large_tmpdir "v2vovl" ".qcow2" in unlink_on_exit overlay_file; (* There is a specific reason to use the newer qcow2 variant: @@ -823,7 +821,7 @@ and preserve_overlays overlays src_name = List.it...

...re. Put that into a separate commit, as it's trivial to review on its own. > - let overlays = > - List.map ( > - fun ({ s_qemu_uri = qemu_uri; s_format = format } as source) -> > - let overlay_file = > - Filename.temp_file ~temp_dir:overlay_dir "v2vovl" ".qcow2" in > - unlink_on_exit overlay_file; > - > - let options = > - "compat=1.1" ^ > - (match format with None -> "" > - | Some fmt -> ",backing_fmt=" ^ fmt) in > - let cm...

Since this new temporary directory will contain UNIX sockets for communicating with nbdkit, then its path must not be too long. Use the existing directory that libguestfs exposes for this, i.e. sockdir. --- v2v/nbdkit.ml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/v2v/nbdkit.ml b/v2v/nbdkit.ml index 65317f9b..46b20c9d 100644 --- a/v2v/nbdkit.ml +++ b/v2v/nbdkit.ml

v1 was here: https://www.redhat.com/archives/libguestfs/2020-April/msg00007.html There's a BZ for this now which I forgot to add to the commit message: https://bugzilla.redhat.com/show_bug.cgi?id=1814611 For v2: - Fix incorrect reference to $TMPDIR in existing manual. - Separate handling for small temporary files and large temporary files. Small temporary files go into $TMPDIR

...ize='512'. +check_blocksize_for_device 3 512 + +rm -f "$DEBUG_QEMU_FILE" diff --git a/tmp/.gitignore b/tmp/.gitignore index 912a946b6..ff989cc68 100644 --- a/tmp/.gitignore +++ b/tmp/.gitignore @@ -1,6 +1,7 @@ /.guestfs-* /guestfs.* /libguestfs??????/ +/testdisk?????? /run-* /v2vovl*.qcow2 /valgrind-*.log -- 2.17.2