Richard W.M. Jones
2017-Feb-14 16:33 UTC
Re: [Libguestfs] [PATCH 09/10] New API: mksquashfs
On Tue, Feb 14, 2017 at 09:12:10AM +0100, Pino Toscano wrote:> Introduce a new API to create a new squashfs filesystem out of a path > in the guest. It can be configured to exclude paths based on patterns, > and to select which compression use for the filesystem. > > The advantage of running mksquashfs directly in the appliance is that > ownerships are properly saved, as opposed to tar_out + local untar.I'm fairly sure this was proposed before as an API, but I cannot find that right now. Was rejected because mksquashfs doesn't support streaming output. However I'm fine with this, although perhaps there should be a warning in the longdesc saying that there may be problems with large files. I think this API will break in the region of 3GB. Rich.> diff --git a/generator/actions.ml b/generator/actions.ml > index fd6cc9f..409f399 100644 > --- a/generator/actions.ml > +++ b/generator/actions.ml > @@ -13292,6 +13292,32 @@ is removed." }; > shortdesc = "search the entries associated to the given inode"; > longdesc = "Internal function for find_inode." }; > > + { defaults with > + name = "mksquashfs"; added = (1, 35, 25); > + style = RErr, [Pathname "path"; FileOut "filename"], [OString "compress"; OStringList "excludes"]; > + proc_nr = Some 471; > + optional = Some "squashfs"; > + cancellable = true; > + shortdesc = "create a squashfs filesystem"; > + longdesc = "\ > +Create a squashfs filesystem for the specified C<path>. > + > +The optional C<compress> flag controls compression. If not given, > +then the output compressed using C<gzip>. Otherwise one > +of the following strings may be given to select the compression > +type of the squashfs: C<gzip>, C<lzma>, C<lzo>, C<lz4>, C<xz>. > + > +The other optional arguments are: > + > +=over 4 > + > +=item C<excludes> > + > +A list of wildcards. Files are excluded if they match any of the > +wildcards. > +-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
On Tuesday, 14 February 2017 16:33:57 CET Richard W.M. Jones wrote:> On Tue, Feb 14, 2017 at 09:12:10AM +0100, Pino Toscano wrote: > > Introduce a new API to create a new squashfs filesystem out of a path > > in the guest. It can be configured to exclude paths based on patterns, > > and to select which compression use for the filesystem. > > > > The advantage of running mksquashfs directly in the appliance is that > > ownerships are properly saved, as opposed to tar_out + local untar. > > I'm fairly sure this was proposed before as an API, but I cannot find > that right now. Was rejected because mksquashfs doesn't support > streaming output.Yes, and it does not have any streaming for input either, so things like `cat my.tar | mksquashfs - my.squash` cannot be done :-(> However I'm fine with this, although perhaps there should be a warning > in the longdesc saying that there may be problems with large files. I > think this API will break in the region of 3GB.Also because the temporary squashfs file is stored inside the appliance, which has < 3.5G space available. It makes sense -- what about the following addendum to the longdesc? Please note that this API may fail when used to compress directories with large files, such as the resulting squashfs will be over 3GB big. Thanks, -- Pino Toscano
Richard W.M. Jones
2017-Feb-15 11:47 UTC
Re: [Libguestfs] [PATCH 09/10] New API: mksquashfs
On Wed, Feb 15, 2017 at 11:06:34AM +0100, Pino Toscano wrote:> On Tuesday, 14 February 2017 16:33:57 CET Richard W.M. Jones wrote: > > On Tue, Feb 14, 2017 at 09:12:10AM +0100, Pino Toscano wrote: > > > Introduce a new API to create a new squashfs filesystem out of a path > > > in the guest. It can be configured to exclude paths based on patterns, > > > and to select which compression use for the filesystem. > > > > > > The advantage of running mksquashfs directly in the appliance is that > > > ownerships are properly saved, as opposed to tar_out + local untar. > > > > I'm fairly sure this was proposed before as an API, but I cannot find > > that right now. Was rejected because mksquashfs doesn't support > > streaming output. > > Yes, and it does not have any streaming for input either, so things > like `cat my.tar | mksquashfs - my.squash` cannot be done :-( > > > However I'm fine with this, although perhaps there should be a warning > > in the longdesc saying that there may be problems with large files. I > > think this API will break in the region of 3GB. > > Also because the temporary squashfs file is stored inside the appliance, > which has < 3.5G space available. > > It makes sense -- what about the following addendum to the longdesc? > > Please note that this API may fail when used to compress directories > with large files, such as the resulting squashfs will be over 3GB big.Yup, seems reasonable. (ACK series BTW in case I wasn't clear about that yesterday). Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org