Hubert Kario
2012-May-01 12:40 UTC
[PATCH v2 1/5] btrfs: add command to zero out superblock
Signed-off-by: Hubert Kario <kario@wit.edu.pl> diff --git a/cmds-device.c b/cmds-device.c index db625a6..05a549c 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -246,11 +246,58 @@ static int cmd_scan_dev(int argc, char **argv) return 0; } +static const char * const cmd_zero_dev_usage[] = { + "btrfs device zero-superblock <device> [<device> ...]", + "Remove btrfs filesystem superblock from devices.", + "WARNING! This command will make filesystem residing on the devices", + "completely unmountable!", + NULL +}; + +static int cmd_zero_dev(int argc, char **argv) +{ + int fd; + char *file; + int arg_processed; + int ret = 0; + int n; + u64 device_len; + int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */ + const int ZERO_END = 1; + + if( argc < 2 ) { + usage(cmd_zero_dev_usage); + } + + for(arg_processed = 1; arg_processed < argc; arg_processed++) { + file = argv[arg_processed]; + + fd = open(file, O_RDWR); + if (fd < 0) { + fprintf(stderr, "Unable to open %s\n", file); + ret |= 1; + continue; + } + + n = btrfs_prepare_device(fd, file, ZERO_END, &device_len, + &mixed_mode_needed); + if (n) { + fprintf(stderr, "Error when zeroing out %s\n", file); + ret |= n; + } + + close(fd); + } + + return ret; +} + const struct cmd_group device_cmd_group = { device_cmd_group_usage, NULL, { { "add", cmd_add_dev, cmd_add_dev_usage, NULL, 0 }, { "delete", cmd_rm_dev, cmd_rm_dev_usage, NULL, 0 }, { "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 }, + { "zero-superblock", cmd_zero_dev, cmd_zero_dev_usage, NULL, 0 }, { 0, 0, 0, 0, 0 } } }; diff --git a/man/btrfs.8.in b/man/btrfs.8.in index be478e0..a840f7e 100644 --- a/man/btrfs.8.in +++ b/man/btrfs.8.in @@ -39,6 +39,8 @@ btrfs \- control a btrfs filesystem .PP \fBbtrfs\fP \fBdevice delete\fP\fI <device> [<device>...] <path> \fP .PP +\fBbtrfs\fP \fBdevice zero-superblock\fP\fI <device> [<device>...] \fP +.PP \fBbtrfs\fP \fBscrub start\fP [-Bdqru] {\fI<path>\fP|\fI<device>\fP} .PP \fBbtrfs\fP \fBscrub cancel\fP {\fI<path>\fP|\fI<device>\fP} @@ -230,6 +232,11 @@ Finally, if \fB--all-devices\fP is passed, all the devices under /dev are scanned. .TP +\fBdevice zero-superblock\fR\fI <dev> [<dev>..]\fR +The space on the disk where btrfs metadata can reside is overwritten with +zeros. +.TP + \fBscrub start\fP [-Bdqru] {\fI<path>\fP|\fI<device>\fP} Start a scrub on all devices of the filesystem identified by \fI<path>\fR or on a single \fI<device>\fR. Without options, scrub is started as a background -- 1.7.10 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hubert Kario
2012-May-01 12:40 UTC
[PATCH v2 2/5] handle null pointers in btrfs_prepare_device
When calling the function from `btrfs device zero-super` we don''t need the additional information returned and don''t want the "SMALL VOLUME" warning printed. Signed-off-by: Hubert Kario <kario@wit.edu.pl> diff --git a/utils.c b/utils.c index ee7fa1b..6773be0 100644 --- a/utils.c +++ b/utils.c @@ -557,7 +557,7 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret, } zero_end = 1; - if (block_count < 1024 * 1024 * 1024 && !(*mixed)) { + if (mixed && block_count < 1024 * 1024 * 1024 && !(*mixed)) { printf("SMALL VOLUME: forcing mixed metadata/data groups\n"); *mixed = 1; } @@ -588,7 +588,9 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret, exit(1); } } - *block_count_ret = block_count; + + if (block_count_ret) + *block_count_ret = block_count; return 0; } -- 1.7.10 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hubert Kario
2012-May-01 12:40 UTC
[PATCH v2 3/5] Remove unused option in btrfs_prepare_device
zero_end is set explicitly to 1 inside the fuction so the device end always will be zeroed out Signed-off-by: Hubert Kario <kario@wit.edu.pl> diff --git a/btrfs-vol.c b/btrfs-vol.c index 0efdbc1..c7b9f80 100644 --- a/btrfs-vol.c +++ b/btrfs-vol.c @@ -150,7 +150,7 @@ int main(int ac, char **av) if (cmd == BTRFS_IOC_ADD_DEV) { int mixed = 0; - ret = btrfs_prepare_device(devfd, device, 1, &dev_block_count, &mixed); + ret = btrfs_prepare_device(devfd, device, &dev_block_count, &mixed); if (ret) { fprintf(stderr, "Unable to init %s\n", device); exit(1); diff --git a/cmds-device.c b/cmds-device.c index 05a549c..a28752f 100644 --- a/cmds-device.c +++ b/cmds-device.c @@ -107,7 +107,7 @@ static int cmd_add_dev(int argc, char **argv) continue; } - res = btrfs_prepare_device(devfd, argv[i], 1, &dev_block_count, &mixed); + res = btrfs_prepare_device(devfd, argv[i], &dev_block_count, &mixed); if (res) { fprintf(stderr, "ERROR: Unable to init ''%s''\n", argv[i]); close(devfd); @@ -263,7 +263,6 @@ static int cmd_zero_dev(int argc, char **argv) int n; u64 device_len; int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */ - const int ZERO_END = 1; if( argc < 2 ) { usage(cmd_zero_dev_usage); @@ -279,8 +278,8 @@ static int cmd_zero_dev(int argc, char **argv) continue; } - n = btrfs_prepare_device(fd, file, ZERO_END, &device_len, - &mixed_mode_needed); + n = btrfs_prepare_device(fd, file, &device_len, + &mixed_mode_needed); if (n) { fprintf(stderr, "Error when zeroing out %s\n", file); ret |= n; diff --git a/mkfs.c b/mkfs.c index c531ef2..7d1165f 100644 --- a/mkfs.c +++ b/mkfs.c @@ -1209,7 +1209,6 @@ int main(int ac, char **av) u32 sectorsize = 4096; u32 nodesize = leafsize; u32 stripesize = 4096; - int zero_end = 1; int option_index = 0; int fd; int ret; @@ -1264,7 +1263,6 @@ int main(int ac, char **av) "metadata/data groups\n"); mixed = 1; } - zero_end = 0; break; case ''V'': print_version(); @@ -1311,7 +1309,7 @@ int main(int ac, char **av) exit(1); } first_file = file; - ret = btrfs_prepare_device(fd, file, zero_end, &dev_block_count, &mixed); + ret = btrfs_prepare_device(fd, file, &dev_block_count, &mixed); if (block_count == 0) block_count = dev_block_count; } else { @@ -1376,7 +1374,6 @@ int main(int ac, char **av) btrfs_register_one_device(file); - zero_end = 1; while(ac-- > 0) { int old_mixed = mixed; @@ -1404,8 +1401,7 @@ int main(int ac, char **av) close(fd); continue; } - ret = btrfs_prepare_device(fd, file, zero_end, - &dev_block_count, &mixed); + ret = btrfs_prepare_device(fd, file, &dev_block_count, &mixed); mixed = old_mixed; BUG_ON(ret); diff --git a/utils.c b/utils.c index 6773be0..e2c72ad 100644 --- a/utils.c +++ b/utils.c @@ -536,8 +536,7 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, return 0; } -int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret, - int *mixed) +int btrfs_prepare_device(int fd, char *file, u64 *block_count_ret, int *mixed) { u64 block_count; u64 bytenr; @@ -555,7 +554,6 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret, fprintf(stderr, "unable to find %s size\n", file); exit(1); } - zero_end = 1; if (mixed && block_count < 1024 * 1024 * 1024 && !(*mixed)) { printf("SMALL VOLUME: forcing mixed metadata/data groups\n"); @@ -581,12 +579,10 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret, zero_blocks(fd, bytenr, BTRFS_SUPER_INFO_SIZE); } - if (zero_end) { - ret = zero_dev_end(fd, block_count); - if (ret) { - fprintf(stderr, "failed to zero device end %d\n", ret); - exit(1); - } + ret = zero_dev_end(fd, block_count); + if (ret) { + fprintf(stderr, "failed to zero device end %d\n", ret); + exit(1); } if (block_count_ret) diff --git a/utils.h b/utils.h index c5f55e1..b7ba663 100644 --- a/utils.h +++ b/utils.h @@ -26,8 +26,8 @@ int make_btrfs(int fd, const char *device, const char *label, u32 leafsize, u32 sectorsize, u32 stripesize); int btrfs_make_root_dir(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 objectid); -int btrfs_prepare_device(int fd, char *file, int zero_end, - u64 *block_count_ret, int *mixed); +int btrfs_prepare_device(int fd, char *file, u64 *block_count_ret, + int *mixed); int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, struct btrfs_root *root, int fd, char *path, u64 block_count, u32 io_width, u32 io_align, -- 1.7.10 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Sterba
2012-May-02 14:28 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote:> +static const char * const cmd_zero_dev_usage[] = { > + "btrfs device zero-superblock <device> [<device> ...]",FYI, this step is named ''clear superblock'' in kernel code as done after the device is removed, and I suggest to consider to name the command like ''clear-superblock'' or ''clear-sb''. Also, kernel clears only the "superblock magic" string, ie. the "_BHRfS_M" bytes. This leaves the rest of the superblock intact, for possible recovery when it''s cleared accidentally. I had prototyped a similar utility (in perl, so nothing for progs inclusion for now) and rewrote the magic string with _BHRf$_M ie. the S -> $ for visual similarity with the action performed. This allows to detect cleared superblocks and activate them back eventually. I''m not sure if this is useful and sensible usecase, clearing superblock is a one-time action anyway, so it''s more for the sake of tool flexibility. To your implementation: I think adding a function doing the superblock reset would be enough here. Something like this (in pseudocode): for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) { bytenr = btrfs_sb_offset(i); "break if bytenr > device size" memset(superblock buffer, CLEARPATTERN, sizeof(...)) } write_all_supers(root); david> + "Remove btrfs filesystem superblock from devices.", > + "WARNING! This command will make filesystem residing on the devices", > + "completely unmountable!", > + NULL > +}; > + > +static int cmd_zero_dev(int argc, char **argv) > +{ > + int fd; > + char *file; > + int arg_processed; > + int ret = 0; > + int n; > + u64 device_len; > + int mixed_mode_needed = 1; /* keep btrfs_prepare_device() quiet */ > + const int ZERO_END = 1; > + > + if( argc < 2 ) { > + usage(cmd_zero_dev_usage); > + } > + > + for(arg_processed = 1; arg_processed < argc; arg_processed++) { > + file = argv[arg_processed]; > + > + fd = open(file, O_RDWR); > + if (fd < 0) { > + fprintf(stderr, "Unable to open %s\n", file); > + ret |= 1; > + continue; > + } > + > + n = btrfs_prepare_device(fd, file, ZERO_END, &device_len, > + &mixed_mode_needed); > + if (n) { > + fprintf(stderr, "Error when zeroing out %s\n", file); > + ret |= n; > + } > + > + close(fd); > + } > + > + return ret; > +} > + > const struct cmd_group device_cmd_group = { > device_cmd_group_usage, NULL, { > { "add", cmd_add_dev, cmd_add_dev_usage, NULL, 0 }, > { "delete", cmd_rm_dev, cmd_rm_dev_usage, NULL, 0 }, > { "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 }, > + { "zero-superblock", cmd_zero_dev, cmd_zero_dev_usage, NULL, 0 }, > { 0, 0, 0, 0, 0 } > } > }; > diff --git a/man/btrfs.8.in b/man/btrfs.8.in > index be478e0..a840f7e 100644 > --- a/man/btrfs.8.in > +++ b/man/btrfs.8.in > @@ -39,6 +39,8 @@ btrfs \- control a btrfs filesystem > .PP > \fBbtrfs\fP \fBdevice delete\fP\fI <device> [<device>...] <path> \fP > .PP > +\fBbtrfs\fP \fBdevice zero-superblock\fP\fI <device> [<device>...] \fP > +.PP > \fBbtrfs\fP \fBscrub start\fP [-Bdqru] {\fI<path>\fP|\fI<device>\fP} > .PP > \fBbtrfs\fP \fBscrub cancel\fP {\fI<path>\fP|\fI<device>\fP} > @@ -230,6 +232,11 @@ Finally, if \fB--all-devices\fP is passed, all the devices under /dev are > scanned. > .TP > > +\fBdevice zero-superblock\fR\fI <dev> [<dev>..]\fR > +The space on the disk where btrfs metadata can reside is overwritten with > +zeros. > +.TP > + > \fBscrub start\fP [-Bdqru] {\fI<path>\fP|\fI<device>\fP} > Start a scrub on all devices of the filesystem identified by \fI<path>\fR or on > a single \fI<device>\fR. Without options, scrub is started as a background > -- > 1.7.10 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Sterba
2012-May-02 14:48 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 04:28:43PM +0200, David Sterba wrote:> I had prototyped a similar utility (in perl, so nothing for progs > inclusion for now)attached. david
Hubert Kario
2012-May-02 16:42 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 16:28:43 David Sterba wrote:> On Tue, May 01, 2012 at 02:40:29PM +0200, Hubert Kario wrote: > > +static const char * const cmd_zero_dev_usage[] = { > > + "btrfs device zero-superblock <device> [<device> ...]", > > FYI, this step is named ''clear superblock'' in kernel code as done after the > device is removed, and I suggest to consider to name the command like > ''clear-superblock'' or ''clear-sb''.A similar function in mdadm is called zero superblock so I just re used the name (according to the principle of least surprise). Users, even admins, generally don''t read kernel code...> Also, kernel clears only the "superblock magic" string, ie. the > "_BHRfS_M" bytes. This leaves the rest of the superblock intact, for > possible recovery when it''s cleared accidentally.That''s when a device is removed from the filesystem, not when a filesystem is just not used any more and you want to re-purpose the devices.> I had prototyped a similar utility (in perl, so nothing for progs > inclusion for now) and rewrote the magic string with _BHRf$_M ie. the > S -> $ for visual similarity with the action performed. This allows to > detect cleared superblocks and activate them back eventually. > > I''m not sure if this is useful and sensible usecase, clearing superblock > is a one-time action anyway, so it''s more for the sake of tool > flexibility.Clearing superblock is not a light decision and should generally be performed just before formatting the partition with some other fs or physical volume for LVM. IMHO recoverability of "cleared" superblock is a function hardly anyone would use.> To your implementation: I think adding a function doing the superblock > reset would be enough here. Something like this (in pseudocode): > > for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) { > bytenr = btrfs_sb_offset(i); > "break if bytenr > device size" > memset(superblock buffer, CLEARPATTERN, sizeof(...)) > } > write_all_supers(root);That''s exactly what btrfs_prepare_device does. And it''s a function run by btfs just before btrfs dev add and by mkfs. Duplicating its code would be a bad idea. Regards, -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Sterba
2012-May-02 17:36 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote:> A similar function in mdadm is called zero superblock so I just re used the > name (according to the principle of least surprise). Users, even admins, > generally don''t read kernel code...I intended to point out that the functionality is there under different name, but the ''least surprise'' and tool compatibility plays for ''zero-superblock'' naming.> > I''m not sure if this is useful and sensible usecase, clearing superblock > > is a one-time action anyway, so it''s more for the sake of tool > > flexibility. > > Clearing superblock is not a light decision and should generally be performed > just before formatting the partition with some other fs or physical volume for > LVM. IMHO recoverability of "cleared" superblock is a function hardly anyone > would use.googled, a few users asking about recovering from md zero-superblock, and the solution was to recreate the array, md is said to be smart and recognize traces of previous array and will not destroy it if the parameters are same. Point for md, btrfs does not do this.> > To your implementation: I think adding a function doing the superblock > > reset would be enough here. Something like this (in pseudocode): > > > > for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) { > > bytenr = btrfs_sb_offset(i); > > "break if bytenr > device size" > > memset(superblock buffer, CLEARPATTERN, sizeof(...)) > > } > > write_all_supers(root); > > That''s exactly what btrfs_prepare_device does. And it''s a function run by btfs > just before btrfs dev add and by mkfs. Duplicating its code would be a bad > idea.Not ''exactly'' IMO: * calls TRIM/discard on the device * zeroes first 2 megabytes * zeroes all reachable superblocks * zeroes last 2 megabytes Too many undocumented and unobvious side-efects. Code duplication can be avoided by factoring the ''zero superblock'' into a function and calling it from btrfs_prepare_device() . david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hubert Kario
2012-May-03 13:11 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 02 of May 2012 19:36:29 David Sterba wrote:> On Wed, May 02, 2012 at 06:42:16PM +0200, Hubert Kario wrote: > > > I''m not sure if this is useful and sensible usecase, clearing superblock > > > is a one-time action anyway, so it''s more for the sake of tool > > > flexibility. > > > > Clearing superblock is not a light decision and should generally be > > performed just before formatting the partition with some other fs or > > physical volume for LVM. IMHO recoverability of "cleared" superblock is a > > function hardly anyone would use. > > googled, a few users asking about recovering from md zero-superblock, and > the solution was to recreate the array, md is said to be smart and > recognize traces of previous array and will not destroy it if the > parameters are same. Point for md, btrfs does not do this.nice, didn''t know about this. Such functionality would be nice to have. But then I don''t think that a "recreate the array if the parameters are the same" is actually a good idea, lots of space for error. A pair of functions: btrfs dev zero-superblock btrfs dev restore-superblock would be a better solution IMO> > > To your implementation: I think adding a function doing the superblock > > > reset would be enough here. Something like this (in pseudocode): > > > > > > for (i = 0 ; i < BTRFS_SUPER_MIRROR_MAX; i++) { > > > > > > bytenr = btrfs_sb_offset(i); > > > "break if bytenr > device size" > > > memset(superblock buffer, CLEARPATTERN, sizeof(...)) > > > > > > } > > > write_all_supers(root); > > > > That''s exactly what btrfs_prepare_device does. And it''s a function run by > > btfs just before btrfs dev add and by mkfs. Duplicating its code would be > > a bad idea. > > Not ''exactly'' IMO: > > * calls TRIM/discard on the device > * zeroes first 2 megabytes > * zeroes all reachable superblocks > * zeroes last 2 megabytes > > Too many undocumented and unobvious side-efects.True. But close enough ;)> Code duplication can be avoided by factoring the ''zero superblock'' into > a function and calling it from btrfs_prepare_device().Then there''s also the "actually zero" vs "reversibly destroy" difference but it''s trivial to fix using a single option. Regards -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Sterba
2012-May-09 17:18 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote:> nice, didn''t know about this. Such functionality would be nice to have. > But then I don''t think that a "recreate the array if the parameters are the > same" is actually a good idea, lots of space for error. A pair of functions: > > btrfs dev zero-superblock > btrfs dev restore-superblockAs a user, I''m not sure what can I expect from the restore command. From where does it restore? Eg. a file? As a tester I have use for a temporary clearing of a superblock on a device, then mount it with -o degraded, work work, and then undo clearing. So, my idea is like btrfs device zero-superblock --undo with the obvious sanity checks. A regular user would never need to call this. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hubert Kario
2012-May-09 17:23 UTC
Re: [PATCH v2 1/5] btrfs: add command to zero out superblock
On Wednesday 09 of May 2012 19:18:07 David Sterba wrote:> On Thu, May 03, 2012 at 03:11:45PM +0200, Hubert Kario wrote: > > nice, didn''t know about this. Such functionality would be nice to have. > > But then I don''t think that a "recreate the array if the parameters are > > the > > same" is actually a good idea, lots of space for error. A pair of > > functions: > > > > btrfs dev zero-superblock > > btrfs dev restore-superblock > > As a user, I''m not sure what can I expect from the restore command. From > where does it restore? Eg. a file? > > As a tester I have use for a temporary clearing of a superblock on a > device, then mount it with -o degraded, work work, and then undo > clearing. So, my idea is like > > btrfs device zero-superblock --undo > > with the obvious sanity checks. A regular user would never need to call > this.Yes, that''s a better idea. -- Hubert Kario QBS - Quality Business Software 02-656 Warszawa, ul. Ksawerów 30/85 tel. +48 (22) 646-61-51, 646-74-24 www.qbs.com.pl -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html