I have written and am currently testing a patch that provides a new ioctl that waits for snapshot deletion to finish before returning; The work is being sponsored, and I can''t distribute it until the sponsorship details are more solid, but I''d like to reserve ioctl #21 for this purpose (the next one from btrfs-unstable) or receive advice on what ioctl number to use instead. Also, if you would like to test this function on a trade-secret basis (until the sponsorship comes through) please contact me off-list. David Nicol -- l''égalité des droits pour les ambidextres -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
this I can release, the user-facing API. diff --git a/man/btrfsctl.8.in b/man/btrfsctl.8.in index c2d4488..94403a4 100644 --- a/man/btrfsctl.8.in +++ b/man/btrfsctl.8.in @@ -35,6 +35,10 @@ Scans all devices present in the system for btrfs filesystem. .TP \fB\-c\fR Forces a filesystem sync. +.TP +\fB\-C\fR \fIdirectory [ms]\fR +Wait until all the space from all deleted snapshots have been recovered. +The optional timeout parameter is in milliseconds. .SH AVAILABILITY .B btrfsctl is part of btrfs-progs. Btrfs is currently under heavy development, -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi David, please don''t do that. "btrfsctl" is obsolete, now the develop should involve the "btrfs" command. I you need help in updating the "btrfs" command source files, don''t hesitate to contact me. Regards G.Baroncelli On Monday, 13 September, 2010, David Nicol wrote:> this I can release, the user-facing API. > > > diff --git a/man/btrfsctl.8.in b/man/btrfsctl.8.in > index c2d4488..94403a4 100644 > --- a/man/btrfsctl.8.in > +++ b/man/btrfsctl.8.in > @@ -35,6 +35,10 @@ Scans all devices present in the system for btrfsfilesystem.> .TP > \fB\-c\fR > Forces a filesystem sync. > +.TP > +\fB\-C\fR \fIdirectory [ms]\fR > +Wait until all the space from all deleted snapshots have been recovered. > +The optional timeout parameter is in milliseconds. > .SH AVAILABILITY > .B btrfsctl > is part of btrfs-progs. Btrfs is currently under heavy development, > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >-- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijack@inwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
https://btrfs.wiki.kernel.org/index.php/Category:Documentation does not mention any btrfs command. the btrfsctl command has the -c sync tool, and the tools for creating and deleting snapshots and subvolumes, so it seems that unless this btrfs program does all of that I have amended the right tool to provide a command line access to a new ioctl that waits for deleted subvolume cleanups to finish. Nothing on https://btrfs.wiki.kernel.org/index.php/Btrfsctl says anything about deprecation or obsolescence. On Mon, Sep 13, 2010 at 12:23 PM, Goffredo Baroncelli <kreijack@gmail.com> wrote:> Hi David, > > please don''t do that. "btrfsctl" is obsolete, now the develop should involve > the "btrfs" command. > I you need help in updating the "btrfs" command source files, don''t hesitate > to contact me. > > Regards > G.Baroncelli-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I see btrfs utility appears here: http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git;a=commit;h=6d2cf042471cc728b5399b2beae54603739bc66a I shall be patching it, too, with the new ioctl which I made this thread to reserve. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Monday, 13 September, 2010, David Nicol wrote:> https://btrfs.wiki.kernel.org/index.php/Category:Documentation does > not mention any btrfs command. the btrfsctl command has the -c sync > tool, and the tools for creating and deleting snapshots and > subvolumes, so it seems that unless this btrfs program does all of > that I have amended the right tool to provide a command line access to > a new ioctl that waits for deleted subvolume cleanups to finish.$ /sbin/btrfs help Usage: btrfs subvolume snapshot <source> [<dest>/]<name> Create a writable snapshot of the subvolume <source> with the name <name> in the <dest> directory. btrfs subvolume delete <subvolume> Delete the subvolume <subvolume>. btrfs subvolume create [<dest>/]<name> Create a subvolume in <dest> (or the current directory if not passed). btrfs subvolume list <path> List the snapshot/subvolume of a filesystem. btrfs subvolume find-new <path> <last_gen> List the recently modified files in a filesystem. btrfs filesystem defragment [-vcf] [-s start] [-l len] [-t size] <file>|<dir> [<file>|<dir>...] Defragment a file or a directory. btrfs subvolume set-default <id> <path> Set the subvolume of the filesystem <path> which will be mounted as default. btrfs filesystem sync <path> Force a sync on the filesystem <path>. btrfs filesystem resize [+/-]<newsize>[gkm]|max <filesystem> Resize the file system. If ''max'' is passed, the filesystem will occupe all available space on the device. btrfs filesystem show [<uuid>|<label>] Show the info of a btrfs filesystem. If no <uuid> or <label> is passed, info of all the btrfs filesystem are shown. btrfs filesystem df <path> Show space usage information for a mount point . btrfs filesystem balance <path> Balance the chunks across the device. btrfs device scan [<device> [<device>..] Scan all device for or the passed device for a btrfs filesystem. btrfs device add <dev> [<dev>..] <path> Add a device to a filesystem. btrfs device delete <dev> [<dev>..] <path> Remove a device from a filesystem. btrfs help|--help|-h Show the help. Btrfs Btrfs v0.19> > Nothing on https://btrfs.wiki.kernel.org/index.php/Btrfsctl says > anything about deprecation or obsolescence. > > On Mon, Sep 13, 2010 at 12:23 PM, Goffredo Baroncelli > <kreijack@gmail.com> wrote: > > Hi David, > > > > please don''t do that. "btrfsctl" is obsolete, now the develop shouldinvolve> > the "btrfs" command. > > I you need help in updating the "btrfs" command source files, don''thesitate> > to contact me. > > > > Regards > > G.Baroncelli > >-- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) <kreijack@inwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Goffredo Baroncelli
2010-Sep-14 19:25 UTC
Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
Hi all, On Monday, 13 September, 2010, David Nicol wrote:> https://btrfs.wiki.kernel.org/index.php/Category:Documentation does > not mention any btrfs command.[...] I updated the wiki, inserting the btrfs command. For now I left the old commands in the examples (some distros have the old version of the btrfs- progs); I put a "notice" saying: "the new command btrfs is replacing the old one btrfsctl." I think to remove the old commands from the wiki when the next generation of linux distros will appear. Thoughts ? Regards G.Baroncelli -- gpg key@ keyserver.linux.it:Goffredo Baroncelli (ghigo) <kreijackATinwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Nicol
2010-Sep-15 14:10 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
Does the manpatch below describe an acceptable API for a hook to a facility for waiting for the completion of deleted snapshot cleanup? diff --git a/man/btrfs.8.in b/man/btrfs.8.in index 26ef982..290d202 100644 --- a/man/btrfs.8.in +++ b/man/btrfs.8.in @@ -19,6 +19,8 @@ btrfs \- control a btrfs filesystem .PP \fBbtrfs\fP \fBfilesystem sync\fP\fI <path> \fP .PP +\fBbtrfs\fP \fBfilesystem GC\fP\fI <path> [ms] \fP +.PP \fBbtrfs\fP \fBfilesystem resize\fP\fI [+/\-]<size>[gkm]|max <filesystem>\fP .PP \fBbtrfs\fP \fBdevice scan\fP\fI [<device> [<device>..]]\fP @@ -115,6 +117,12 @@ all the block devices. Force a sync for the filesystem identified by \fI<path>\fR. .TP +\fBfilesystem GC\fR\fI <path> [ms] \fR +wait for completion of salvage of deleted subvolumes on +the filesystem identified by \fI<path>\fR. The optional +timeout is in milliseconds. +.TP + .\" .\" Some wording are extracted by the resize2fs man page .\" -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Goffredo Baroncelli
2010-Sep-15 17:51 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
Hi David, the btrfs command is smart enough to accept short form of the sub-command if there is no ambiguity. For example "btrfs fi" is a synonymous of "btrfs filesystem". Because that the acronyms are discouraged. Instead a long form is preferred. So instead of CG (which means ? Garbage Collector ? ) I suggest btrfs filesystem wait-subvolume-deletion or better btrfs subvolume wait-deletion which is a better self-explain form and may be shortened as btrfs s w Only my 2¢ regards G.Baroncelli On Wednesday, 15 September, 2010, David Nicol wrote:> Does the manpatch below describe an acceptable API for a hook to a > facility for waiting for the completion of deleted snapshot cleanup? > > diff --git a/man/btrfs.8.in b/man/btrfs.8.in > index 26ef982..290d202 100644 > --- a/man/btrfs.8.in > +++ b/man/btrfs.8.in > @@ -19,6 +19,8 @@ btrfs \- control a btrfs filesystem > .PP > \fBbtrfs\fP \fBfilesystem sync\fP\fI <path> \fP > .PP > +\fBbtrfs\fP \fBfilesystem GC\fP\fI <path> [ms] \fP > +.PP > \fBbtrfs\fP \fBfilesystem resize\fP\fI [+/\-]<size>[gkm]|max<filesystem>\fP> .PP > \fBbtrfs\fP \fBdevice scan\fP\fI [<device> [<device>..]]\fP > @@ -115,6 +117,12 @@ all the block devices. > Force a sync for the filesystem identified by \fI<path>\fR. > .TP > > +\fBfilesystem GC\fR\fI <path> [ms] \fR > +wait for completion of salvage of deleted subvolumes on > +the filesystem identified by \fI<path>\fR. The optional > +timeout is in milliseconds. > +.TP > + > .\" > .\" Some wording are extracted by the resize2fs man page > .\" > >-- gpg key@ keyserver.linux.it:Goffredo Baroncelli (ghigo) <kreijackATinwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Nicol
2010-Sep-15 18:35 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
btrfs s ... is imprecise, as the operation isn''t on a subvolume, but on the fi where the s used to be but isn''t any more. With your approval I''ll go with "filesystem garbage-collection" which can be abbrevd to btrfs fi g /butter 30000 || echo there should be some more space now \ but it would have taken more than 30 sec to clean it all and in some imaginary future might trigger something instead of simply waiting for something. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Josh Berry
2010-Sep-15 18:41 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
On Wed, Sep 15, 2010 at 07:10, David Nicol <davidnicol@gmail.com> wrote:> Does the manpatch below describe an acceptable API for a hook to a > facility for waiting for the completion of deleted snapshot cleanup?...> +\fBfilesystem GC\fR\fI <path> [ms] \fR > +wait for completion of salvage of deleted subvolumes on > +the filesystem identified by \fI<path>\fR. The optional > +timeout is in milliseconds. > +.TPI''d suggest different wording for the manpage: filesystem GC <filesystem> [timeout_in_ms] Wait for btrfs to free any space that was used by deleted subvolumes in <filesystem>. If timeout_in_ms is specified, wait up to that many milliseconds, then exit with status 1 [or whatever code is chosen] if not all space could be freed in that time. Personally, I like "gc" or "clean". "wait-subvolume-deletion", "garbage-collect" or similar is too verbose for my taste. -- Josh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Goffredo Baroncelli
2010-Sep-15 19:29 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
On Wednesday, 15 September, 2010, David Nicol wrote:> btrfs s ... is imprecise, as the operation isn''t on a subvolume, but > on the fi where the s used to be but isn''t any more.ok> > With your approval I''ll go with "filesystem garbage-collection" which > can be abbrevd to > > btrfs fi g /butter 30000 || echo there should be some more space now \ > but it would have taken more than 30 sec to clean it all > > and in some imaginary future might trigger something instead of simply > waiting for something.I don''t like too much "garbage-collector". It seems that you can start a "garbage collector". Instead we wait the finish of the garbage collector activity. I would like also to have the option to wait in "btrfs subvolume delete" # btrfs subvolume delete --wait /path/to/subvolume (instead of "--wait", we can use "--sync"...) In any case could you highlight the behavior of your ioctl when two snapshot deletions are performed. Does the ioctl wait the end of both the snapshot deletion ? Regards G.Baroncelli -- gpg key@ keyserver.linux.it:Goffredo Baroncelli (ghigo) <kreijackATinwind.it> Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Nicol
2010-Sep-15 20:54 UTC
Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]
On Wed, Sep 15, 2010 at 1:41 PM, Josh Berry <des@condordes.net> wrote:> Personally, I like "gc" or "clean". "wait-subvolume-deletion", > "garbage-collect" or similar is too verbose for my taste. > > -- Joshthe question IMO is how much implementation details to saddle the user with. Anything with the word "garbage" in it is just yucky; anything with "wait" is revealing an implementation detail that hopefully will change eventually. adding a --wait or --sync switch to the delete command is certainly possible, you''d want to add it to the last in a series of deletes. I don''t like "clean" for end-user API because it seems to imply erasing data, like it''s going to get overwritten with zeroes or something like that to hide it from forensic analysis. among the listings at http://thesaurus.com/browse/recycle "reclaim" seems best: "salvage" connotes that the thing being salvaged is damaged. Is everyone happy with btrfs filesystem reclaim <path> [ms] then? (I''m pleasantly surprised that I''m not having to defend milliseconds.) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi, > adding a --wait or --sync switch to the delete command is > certainly possible, you''d want to add it to the last in a series > of deletes. I think this (--sync) is a much better idea than the alternatives so far. Thanks, -- Chris Ball <cjb@laptop.org> <http://printf.net/> One Laptop Per Child -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Sep 15, 2010 at 14:07, Chris Ball <cjb@laptop.org> wrote:> Hi, > > > adding a --wait or --sync switch to the delete command is > > certainly possible, you''d want to add it to the last in a series > > of deletes. > > I think this (--sync) is a much better idea than the alternatives so far.Or make --sync the default behavior. This is probably what most people are expecting anyway (similar to how standard filesystem commands like "rm" work). Add an --aysnc option for those that only care about knowing when the subvolume is taken out of the tree. But I agree, an option to "subvolume delete" is probably better. -- Josh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi, > Or make --sync the default behavior. This is probably what most > people are expecting anyway (similar to how standard filesystem > commands like "rm" work). Add an --aysnc option for those that > only care about knowing when the subvolume is taken out of the > tree. Yeah. We''ve also talked about making snapshot _creation_ perform an FS sync first by default, since otherwise you get a snapshot with stale files, or without files that existed (not yet on disk) at creation-time. - Chris. -- Chris Ball <cjb@laptop.org> <http://printf.net/> One Laptop Per Child -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Sep 15, 2010 at 14:52, Chris Ball <cjb@laptop.org> wrote:> Hi, > > > Or make --sync the default behavior. This is probably what most > > people are expecting anyway (similar to how standard filesystem > > commands like "rm" work). Add an --aysnc option for those that > > only care about knowing when the subvolume is taken out of the > > tree. > > Yeah. We''ve also talked about making snapshot _creation_ perform an > FS sync first by default, since otherwise you get a snapshot with stale > files, or without files that existed (not yet on disk) at creation-time.Oh wow, I didn''t know this. Yes, sync-before-create would be nice. *runs off to update his snapshot script* -- Josh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 15 Sep 2010, Chris Ball wrote:> Hi, > > > Or make --sync the default behavior. This is probably what most > > people are expecting anyway (similar to how standard filesystem > > commands like "rm" work). Add an --aysnc option for those that > > only care about knowing when the subvolume is taken out of the > > tree. > > Yeah. We''ve also talked about making snapshot _creation_ perform an > FS sync first by default, since otherwise you get a snapshot with stale > files, or without files that existed (not yet on disk) at creation-time.Actually, that was fixed in 2.6.34 (0bdb1db2). Creating a snapshot syncs all dirty data and metadata to disk, so you get a fully consistent point in time snapshot. No ''sync'' is necessary (and doing a ''sync'' yourself would be racy any). For subvolume removal, racing doesn''t seem like an issue. Why not just do # btrfs subvolume delete /path/to/subvolume1 # btrfs subvolume delete /path/to/subvolume2 # ... # sync ? sage -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 15 Sep 2010, Sage Weil wrote:> For subvolume removal, racing doesn''t seem like an issue. Why not just > do > > # btrfs subvolume delete /path/to/subvolume1 > # btrfs subvolume delete /path/to/subvolume2 > # ... > # sync > > ?Nevermind, read the whole thread this time. I''ll defer to others on the best way to wait for the old space to be freed. sage -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Sage, > Actually, that was fixed in 2.6.34 (0bdb1db2). Creating a > snapshot syncs all dirty data and metadata to disk, so you get a > fully consistent point in time snapshot. No ''sync'' is necessary > (and doing a ''sync'' yourself would be racy any). Oh, thanks! Am happy to see that. > Why not just do: > # btrfs subvolume delete /path/to/subvolume1 > # btrfs subvolume delete /path/to/subvolume2 > # sync Or even better, "btrfs filesystem sync <path>". I''m assuming that the caller wants to wait until the subvol delete has happened, but is unwilling to wait until all other writes to the FS have finished, and is even more unwilling to wait for all writes on the rest of the system to finish. I don''t know whether that''s actually a reasonable/plausible use case, though. - Chris. -- Chris Ball <cjb@laptop.org> <http://printf.net/> One Laptop Per Child -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Sep 15, 2010 at 15:37, Chris Ball <cjb@laptop.org> wrote: ...> > Why not just do: > > # btrfs subvolume delete /path/to/subvolume1 > > # btrfs subvolume delete /path/to/subvolume2 > > # sync > > Or even better, "btrfs filesystem sync <path>". > > I''m assuming that the caller wants to wait until the subvol delete > has happened, but is unwilling to wait until all other writes to the > FS have finished, and is even more unwilling to wait for all writes > on the rest of the system to finish. > > I don''t know whether that''s actually a reasonable/plausible use case, > though.I think so, because I have that use case (sorta). I have a home fileserver with btrfs, and a cron job on that machine runs hourly, taking snapshots. However, once the disk fills up, I want to delete the oldest snapshots: while [[ disk > xx% full ]]; do btrfs subvol delete /the/oldest/snapshot wait for space to free up done If delete does not wait to free space, I''ll delete too many snapshots. But if I force a filesystem-wide sync after every delete, the loop may take longer, which on a system busier than mine could take more than the hour to complete. -- Josh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html