I am on Fedora 20-Beta and we just updated to btrfs-progs 0.20.rc1-20131114git9f0c53f Previously, when you did a btrfs fi show, you got a list with one output for each btrfs storage volume whether it was a single device or multi-device volume. Now, I get multiple outputs for each storage volume. For example, I have a system with two SSDs as one btrfs storage volume and two USB round-and-brown disks as a second btrfs storage volume. Each has three subvols defined and mounted. Previously, I got two outputs; one for each volume. Now I get three outputs for the SSD storage volume and SIX outputs for the out volume. Is this a bug or a feature? If it is a feature, what is it telling me? Gene -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Nov 16, 2013, at 7:58 AM, Gene Czarcinski <gene@czarc.net> wrote:> I am on Fedora 20-Beta and we just updated to btrfs-progs 0.20.rc1-20131114git9f0c53f > > Previously, when you did a btrfs fi show, you got a list with one output for each btrfs storage volume whether it was a single device or multi-device volume. > > Now, I get multiple outputs for each storage volume. For example, I have a system with two SSDs as one btrfs storage volume and two USB round-and-brown disks as a second btrfs storage volume. Each has three subvols defined and mounted. > > Previously, I got two outputs; one for each volume. > > Now I get three outputs for the SSD storage volume and SIX outputs for the out volume. > > Is this a bug or a feature? If it is a feature, what is it telling me?I filed a bug on it. https://bugzilla.redhat.com/show_bug.cgi?id=1031299 Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Chris Murphy posted on Sat, 16 Nov 2013 09:04:31 -0700 as excerpted:> On Nov 16, 2013, at 7:58 AM, Gene Czarcinski <gene@czarc.net> wrote: > >> I am on Fedora 20-Beta and we just updated to btrfs-progs >> 0.20.rc1-20131114git9f0c53f >> >> Previously, when you did a btrfs fi show, you got a list with one >> output for each btrfs storage volume whether it was a single device or >> multi-device volume. >> >> Now, I get multiple outputs for each storage volume. For example, I >> have a system with two SSDs as one btrfs storage volume and two USB >> round-and-brown disks as a second btrfs storage volume. Each has three >> subvols defined and mounted. >> >> Previously, I got two outputs; one for each volume. >> >> Now I get three outputs for the SSD storage volume and SIX outputs for >> the out volume. >> >> Is this a bug or a feature? If it is a feature, what is it telling me? > > I filed a bug on it. > > https://bugzilla.redhat.com/show_bug.cgi?id=1031299As I understand things based on the on-list patch-flow and discussion as well as reading the git commit log and current manpages, previously the output was potentially outdated; if a mkfs.btrfs had just been run, a btrfs device add had been issued, or a btrfs device delete had just completed, and a btrfs device scan had not been run since, the output (sometime?) didn''t reflect the most current state, sometimes surprising users. Here''s the relevant git commit from the git whatchanged log: commit dde4b9214162d994f0190e42f53d2e1d6fdcc893 Author: Anand Jain <anand.jain@oracle.com> Date: Tue Oct 8 11:41:40 2013 +0800 btrfs-progs: use BTRFS_SCAN_LBLKID as default scan in filesystem show btrfs progs has to scan for the btrfs disks for two main reasons, one to register them with the btrfs kernel (under btrfs dev scan) 2nd to report btrfs disks to the user (under btrfs fi show) (there few more minor reasons like check_mounted etc..). To facilitate the scan, in total we have the following methods to scan for the btrfs BTRFS_SCAN_PROC which uses the /proc/partitions to look for the disks, when scanning it does it twice first would look for non dm- paths and in the 2nd scan it would pick only dm- paths. BTRFS_SCAN_DEV which scans all the block dev under /dev as they appear during scanning. BTRFS_SCAN_LBLKID this uses the library functions provided by the lblkid to get only disks which contains the btrfs SB. The better method to use would be BTRFS_SCAN_LBLKID for the obvious reasons we don''t have to reinvent that feature with in btrfs-progs. For the btrfs fi show - This patch will.. - make BTRFS_SCAN_LBLKID as the default scan option (BTRFS_SCAN_DEV is accessible under the option --all-devices and BTRFS_SCAN_PROC won''t be used by btrfs fi show any more) Signed-off-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <chris.mason@fusionio.com> cmds-filesystem.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) And the following commit: commit b4c3cd0bbc68ca61b69cd58b1faadc995e2bf77c Author: Anand Jain <anand.jain@oracle.com> Date: Fri Oct 11 19:52:53 2013 +0800 btrfs-progs: device scan use BTRFS_SCAN_LBLKID by default with this patch, BTRFS_SCAN_LBLKID (which leverages lblkid to look for btrfs disks) would be the default scan method to look for the btrfs disks. And thus the output as seen in the latest btrfs fi show and btrfs fi show -m for the mounted disks will have the consistent disks path. (it was inconsistent (across disks) because btrfs dev scan provided a different path from the mount command eg. below) devid 1 size 1.98GiB used 435.00MiB path /dev/mapper/mpatha devid 2 size 2.00GiB used 415.00MiB path /dev/dm-1 Signed-off-by: Anand Jain <anand.jain@oracle.com> Signed-off-by: David Sterba <dsterba@suse.cz> Signed-off-by: Chris Mason <chris.mason@fusionio.com> cmds-device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) From that you can look back on the patches on-list and their discussion, if desired. Meanwhile, behavior from previous /has/ changed, making things a bit confusing for people already using this after all still officially experimental and thus prone to changing UI product, but that''s something people testing still officially unstable product get used to after awhile. At some point the interface and behavior should settle down and be subject to rather less change. So here''s the relevant btrfs manpage snippet (reformatted for posting): filesystem show [--mounted|--all-devices|<uuid>] Show the btrfs filesystem with some additional info. If no option or UUID is passed, btrfs shows information of all the btrfs filesystem both mounted and unmounted. If --mounted is passed, it would probe btrfs kernel to list mounted btrfs filesystem(s); If --all-devices is passed, all the devices under /dev are scanned; otherwise the devices list is extracted from the /proc/partitions file. And the filesystem show --help output: usage: btrfs filesystem show [options|<path>|<uuid>] Show the structure of a filesystem -d|--all-devices show only disks under /dev containing btrfs filesystem -m|--mounted show only mounted btrfs If no argument is given, structure of all present filesystems is shown. Meanwhile, in plain English based on my observation here, the new behavior works like this: 1) If you run a generic btrfs filesystem show (without options or path), you *WILL* likely get the same base filesystem listed multiple times, as the default lookup appears to get the info from at least two different places, one appearing right away that only appears to include currently mounted filesystems, then a pause, then a more complete listing that includes currently unmounted btrfs volumes as well. 2) If you use -m (mounted only), you''ll still get dups as the information apparently still comes from two different kernel lists, but the list won''t include unmounted filesystems and the pause will be gone as the lookup won''t have to scan for the unmounted ones. 3) If you use -d, you''ll get a different list, based on devices in /dev so it includes unmounted filesystems (and thus the pause), but at least here, this list is BOTH comprehensive AND avoids dups! However, I don''t have device-mapper turned on in my kernel (thus no lvm2 or similar volumes), so I''m unsure the practical effect on systems that use it. 4) If you''re only looking for information on a specific filesystem, use the optional path/uuid parameter. This can be either the mountpoint (if the filesystem is mounted) or a component device-path. If you use component device, however, you /might/ get multiple listings for subvolumes too -- I don''t know as I prefer the not having all my data eggs in the same filesystem basket and thus use independent partitions, not subvolumes, here. But ($>> indicating my prompt, being preceded by <LF> so there''s a blank line between commands, other lines being output): $>>btrfs filesystem show / Label: rt0238gcnx+35l0 uuid: 886dcca6-29b4-4e6e-8cdc-8a8ea435d510 Total devices 2 FS bytes used 1.91GiB devid 1 size 8.00GiB used 3.53GiB path /dev/sdc5 devid 2 size 8.00GiB used 3.53GiB path /dev/sda5 Btrfs v0.20-rc1-596-ge9ac73b>>s btrfs filesystem show /dev/sda5Label: rt0238gcnx+35l0 uuid: 886dcca6-29b4-4e6e-8cdc-8a8ea435d510 Total devices 2 FS bytes used 1.91GiB devid 1 size 8.00GiB used 3.53GiB path /dev/sdc5 devid 2 size 8.00GiB used 3.53GiB path /dev/sda5 Btrfs v0.20-rc1-596-ge9ac73b $>> Bottom line, try either btrfs filesystem show -d, or btrfs filesystem show <path>. Both avoid dups here (tho using component device path may produce dups if you use subvolumes, but using mountpoint shouldn''t), while -d is comprehensive and path limits display to just the filesystem of interest. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Nov 16, 2013, at 1:33 PM, Duncan <1i5t5.duncan@cox.net> wrote:> > Meanwhile, in plain English based on my observation here, the new > behavior works like this: > > 1) If you run a generic btrfs filesystem show (without options or path), > you *WILL* likely get the same base filesystem listed multiple times, as > the default lookup appears to get the info from at least two different > places, one appearing right away that only appears to include currently > mounted filesystems, then a pause, then a more complete listing that > includes currently unmounted btrfs volumes as well.Thanks for the explanation. I''m still considering it a bug though. Duplicates of the same UUID should be filtered out. In my case, there''s no pause. Two identical listings are instantly produced for one btrfs volume.> > 3) If you use -d, you''ll get a different list, based on devices in /dev > so it includes unmounted filesystems (and thus the pause), but at least > here, this list is BOTH comprehensive AND avoids dups!This does show me a single instance of the singular volume I have. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Chris Murphy posted on Sat, 16 Nov 2013 14:50:00 -0700 as excerpted:> On Nov 16, 2013, at 1:33 PM, Duncan <1i5t5.duncan@cox.net> wrote:>> 1) If you run a generic btrfs filesystem show (without options or >> path), you *WILL* likely get the same base filesystem listed multiple >> times, as the default lookup appears to get the info from at least two >> different places, one appearing right away that only appears to include >> currently mounted filesystems, then a pause, then a more complete >> listing that includes currently unmounted btrfs volumes as well. > > Thanks for the explanation. I''m still considering it a bug though. > Duplicates of the same UUID should be filtered out.Reasonable point, and I /guess/ (not being a coder) it shouldn''t take much to do that directly in the btrfs filesystem show code itself.> In my case, there''s no pause. Two identical listings are instantly > produced for one btrfs volume.Probably because you just have that one volume, and it''s mounted. I have something like eleven separate btrfs filesystems, mostly two- component-device raid1 mode btrfs, with some of them being primary backups of others and thus not mounted by default, and others being stuff like /boot and my packages tree, that are only mounted when I''m doing updates. The mounted ones show up instantly (with dups in at least some cases), but then it pauses, presumably because it''s scanning and then parsing devices and matching up components for each unmounted filesystem before displaying it. But if you had no such unmounted btrfs and/or if they were only single device (I''m not sure which bit triggers the pause), you''ll probably not see the pause I see here.>> 3) If you use -d, you''ll get a different list, based on devices in /dev >> so it includes unmounted filesystems (and thus the pause), but at least >> here, this list is BOTH comprehensive AND avoids dups! > > This does show me a single instance of the singular volume I have.=:^) If they don''t decide to do the filtering, perhaps the current -d can become the default. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
There is a fix for this (which David has integrated into the branch recently). This also needs the kernel side patch which is listed below. [PATCH v4 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control [PATCH v4 2/3] btrfs-progs: fs show should handle if subvol(s) mounted [PATCH v2] btrfs: add framework to read fs info from btrfs-control Thanks, Anand On 11/16/2013 10:58 PM, Gene Czarcinski wrote:> I am on Fedora 20-Beta and we just updated to btrfs-progs > 0.20.rc1-20131114git9f0c53f > > Previously, when you did a btrfs fi show, you got a list with one output > for each btrfs storage volume whether it was a single device or > multi-device volume. > > Now, I get multiple outputs for each storage volume. For example, I > have a system with two SSDs as one btrfs storage volume and two USB > round-and-brown disks as a second btrfs storage volume. Each has three > subvols defined and mounted. > > Previously, I got two outputs; one for each volume. > > Now I get three outputs for the SSD storage volume and SIX outputs for > the out volume. > > Is this a bug or a feature? If it is a feature, what is it telling me? > > Gene > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoting Anand Jain (2013-11-17 21:17:19)> There is a fix for this (which David has integrated into the branch > recently). This also needs the kernel side patch which is listed below. > > [PATCH v4 1/3] btrfs-progs: mechanism to fetch fsinfo from btrfs-control > [PATCH v4 2/3] btrfs-progs: fs show should handle if subvol(s) mounted > > [PATCH v2] btrfs: add framework to read fs info from btrfs-controlAnand''s updated patches are a better way to solve this, but until we get the kernel feature in, I''m about to commit this patch: From 5aff090a3951e7d787b32bb5c49adfec65091385 Mon Sep 17 00:00:00 2001 From: Chris Mason <chris.mason@fusionio.com> Date: Mon, 18 Nov 2013 14:18:08 -0500 Subject: [PATCH] btrfs filesystem show: skip duplicate fsids If a given filesystem is mounted more than once, btrfs fi show will print dups. This adds a quick and dirty hash table of fsids it has already printed and makes sure we don''t print any fsid more than once. Signed-off-by: Chris Mason <chris.mason@fusionio.com> --- cmds-filesystem.c | 79 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/cmds-filesystem.c b/cmds-filesystem.c index aa361d6..1c1926b 100644 --- a/cmds-filesystem.c +++ b/cmds-filesystem.c @@ -38,6 +38,74 @@ #include "commands.h" #include "list_sort.h" + +/* + * for btrfs fi show, we maintain a hash of fsids we''ve already printed. + * This way we don''t print dups if a given FS is mounted more than once. + */ +#define SEEN_FSID_HASH_SIZE 256 + +struct seen_fsid { + u8 fsid[BTRFS_FSID_SIZE]; + struct seen_fsid *next; +}; + +static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,}; + +static int add_seen_fsid(u8 *fsid) +{ + u8 hash = fsid[0]; + int slot = hash % SEEN_FSID_HASH_SIZE; + struct seen_fsid *seen = seen_fsid_hash[slot]; + struct seen_fsid *alloc; + + if (!seen) + goto insert; + + while (1) { + if (memcmp(seen->fsid, fsid, BTRFS_FSID_SIZE) == 0) + return -EEXIST; + + if (!seen->next) + break; + + seen = seen->next; + } + +insert: + + alloc = malloc(sizeof(*alloc)); + if (!alloc) + return -ENOMEM; + + alloc->next = NULL; + memcpy(alloc->fsid, fsid, BTRFS_FSID_SIZE); + + if (seen) + seen->next = alloc; + else + seen_fsid_hash[slot] = alloc; + + return 0; +} + +static void free_seen_fsid(void) +{ + int slot; + struct seen_fsid *seen; + struct seen_fsid *next; + + for (slot = 0; slot < SEEN_FSID_HASH_SIZE; slot++) { + seen = seen_fsid_hash[slot]; + while (seen) { + next = seen->next; + free(seen); + seen = next; + } + seen_fsid_hash[slot] = NULL; + } +} + static const char * const filesystem_cmd_group_usage[] = { "btrfs filesystem [<group>] <command> [<args>]", NULL @@ -224,6 +292,9 @@ static void print_one_uuid(struct btrfs_fs_devices *fs_devices) u64 devs_found = 0; u64 total; + if (add_seen_fsid(fs_devices->fsid)) + return; + uuid_unparse(fs_devices->fsid, uuidbuf); device = list_entry(fs_devices->devices.next, struct btrfs_device, dev_list); @@ -274,6 +345,13 @@ static int print_one_fs(struct btrfs_ioctl_fs_info_args *fs_info, int i; char uuidbuf[37]; struct btrfs_ioctl_dev_info_args *tmp_dev_info; + int ret; + + ret = add_seen_fsid(fs_info->fsid); + if (ret == -EEXIST) + return 0; + else if (ret) + return ret; uuid_unparse(fs_info->fsid, uuidbuf); printf("Label: %s uuid: %s\n", @@ -478,6 +556,7 @@ devs_only: out: printf("%s\n", BTRFS_BUILD_VERSION); + free_seen_fsid(); return 0; } -- 1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html