-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all,
this is a new attempt to improve the output of the command "btrfs fi
df".
The previous attempt received a good reception. However there was no a
general consensus about the wording.
Moreover I still didn''t understand how btrfs was using the disks.
A my first attempt was to develop a new command which shows how the
disks are divided in chunks. However it still was no clear. Then I
realised that I need to merge the two output.
Below the results. It shows how the disks is used by the different chunks.
The code is pullable from
http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git
branch
info-cmd
I don''t publish the patched because aren''t in a good shape.
However I
really like the output. The example is a filesystem based on three
disks of 3GB.
It is clear that:
- - RAID0 uses all the disks
- - RAID1 uses two different disks
Comments are welcome.
Known bugs:
- - if a filesystem uses a disk but there is any chunk, the disk is not
shown (solvable)
- - this command need root capability (I use the BTRFS_IOC_TREE_SEARCH
to get the chunk info; so that is unavoidable)
ghigo@emulato:~$ sudo ./btrfs fi df /mnt/btrfs1/
[sudo] password for ghigo:
Path: /mnt/btrfs1
Summary:
Disk_size: 9.00GB
Disk_allocated: 1.83GB
Disk_unallocated: 7.17GB
Used: 284.00KB
Free_(Estimated): 6.76GB (Max: 8.54GB, min: 4.96GB)
Data_to_disk_ratio: 75 %
Allocated_area:
Data,RAID0: Size:921.75MB, Used:256.00KB
/dev/vdc 307.25MB
/dev/vdb 307.25MB
/dev/vdd 307.25MB
Data,Single: Size:8.00MB, Used:0.00
/dev/vdb 8.00MB
System,RAID1: Size:8.00MB, Used:4.00KB
/dev/vdd 8.00MB
/dev/vdc 8.00MB
System,Single: Size:4.00MB, Used:0.00
/dev/vdb 4.00MB
Metadata,RAID1: Size:460.94MB, Used:24.00KB
/dev/vdb 460.94MB
/dev/vdd 460.94MB
Metadata,Single: Size:8.00MB, Used:0.00
/dev/vdb 8.00MB
Unused:
/dev/vdb 2.23GB
/dev/vdc 2.69GB
/dev/vdd 2.24GB
- --
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBAgAGBQJQiZFRAAoJEA7amzeLguC1v6QQAIk22PRdfXJJZC3Jq0zugQrg
e+4N2FZXDrLtYaV1+skXpLQXSQg60IMjlv2cCuMKx3LhDEIvLIdyi4YgL9rMc6fF
4cyS8eKYcU4JRrq/SzRhRuqQlu/eUm9MtmniMcu9enI2HM5DjusKcPMWvV3tGMNf
+WPYtuMmLNVbI1JAdLEpmUkhYM6jFOyvrUccl0MC49xZzqGUFNnrzCBUn/2hO1pH
c3CeebLxS1cOxCoOtN/dM86VUA0QyV5t9Na7quSBvTOPnRr2JwZ3YSjyfnXqOqO1
1m9O0xgHP0PG6s0rPZGLAWgAoNcBoFfS8OAl+GtCq+k55ZZz7zNBouFm/LLa0I4X
3nLiID8DoY+Bk5JLVolmN+DFRbHGC2CCkcoS5myLq2pxQXnSA28cMQBk9YhATkkz
QpPgnxwVZ6ooLjbM+DAv1mPE5yrnDBICVgi1oTLUS6uqY/AG4fl+TuTG9Oq2mcDH
krPopOJQRoeXIorEXbzoSg+FAlbp86KU2+v5sA552jxjs1WvB//GGyEdtTgd4fl6
EaztCPZvPdkV/NmnLs3rylfVNzmR4d3tFTaRwmByX7RFw/cPGF7/7IxP7Yjdv9un
CiVun2edm5gwMgjUU3qIYjyiqxFiET/sl+NK6SIEdh3R9XVQ2638dXOlHzE255jM
o7tCMjCmCOHSoAg95ZiD
=TaPn
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
> I don''t publish the patched because aren''t in a good shape. However I > really like the output. The example is a filesystem based on three > disks of 3GB. > > It is clear that: > - - RAID0 uses all the disks > - - RAID1 uses two different disks > > Comments are welcome. > > Known bugs: > - - if a filesystem uses a disk but there is any chunk, the disk is not > shown (solvable) > - - this command need root capability (I use the BTRFS_IOC_TREE_SEARCH > to get the chunk info; so that is unavoidable) > > > ghigo@emulato:~$ sudo ./btrfs fi df /mnt/btrfs1/ > [sudo] password for ghigo: > Path: /mnt/btrfs1 > Summary: > Disk_size: 9.00GB > Disk_allocated: 1.83GB > Disk_unallocated: 7.17GB > Used: 284.00KB > Free_(Estimated): 6.76GB (Max: 8.54GB, min: 4.96GB) > Data_to_disk_ratio: 75 % > > Allocated_area: > Data,RAID0: Size:921.75MB, Used:256.00KB > /dev/vdc 307.25MB > /dev/vdb 307.25MB > /dev/vdd 307.25MB > > Data,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > System,RAID1: Size:8.00MB, Used:4.00KB > /dev/vdd 8.00MB > /dev/vdc 8.00MB > > System,Single: Size:4.00MB, Used:0.00 > /dev/vdb 4.00MB > > Metadata,RAID1: Size:460.94MB, Used:24.00KB > /dev/vdb 460.94MB > /dev/vdd 460.94MB > > Metadata,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > Unused: > /dev/vdb 2.23GB > /dev/vdc 2.69GB > /dev/vdd 2.24GBCouple minor things, in order of personal opinion of severity: * Devices should be listed in a consistent order; device names are just too consistently similar * System chunks shouldn''t be listed between data and metadata; really, they''re just noise 99% of the time anyway * I think it may be more useful to display each disk, with the profiles in use underneath. With a larger number of disks, that would make it _much_ easier to tell at-a-glance what is currently on a disk (that I may want to remove, or which I may suspect to be unreliable). * I''d rename "Unused" to "Unallocated" for consistency with the section title * (and I still detest the_underscores_between_all_the_words; it doesn''t make parsing significantly easier, and it''s an eyesore) * Three coats of blue paint plus a clear-coat is the One True Paint-Job. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-25 21:40, cwillu wrote:>> I don''t publish the patched because aren''t in a good shape. However I >> really like the output. The example is a filesystem based on three >> disks of 3GB. >> >> It is clear that: >> - - RAID0 uses all the disks >> - - RAID1 uses two different disks >> >> Comments are welcome. >> >> Known bugs: >> - - if a filesystem uses a disk but there is any chunk, the disk is not >> shown (solvable) >> - - this command need root capability (I use the BTRFS_IOC_TREE_SEARCH >> to get the chunk info; so that is unavoidable) >> >> >> ghigo@emulato:~$ sudo ./btrfs fi df /mnt/btrfs1/ >> [sudo] password for ghigo: >> Path: /mnt/btrfs1 >> Summary: >> Disk_size: 9.00GB >> Disk_allocated: 1.83GB >> Disk_unallocated: 7.17GB >> Used: 284.00KB >> Free_(Estimated): 6.76GB (Max: 8.54GB, min: 4.96GB) >> Data_to_disk_ratio: 75 % >> >> Allocated_area: >> Data,RAID0: Size:921.75MB, Used:256.00KB >> /dev/vdc 307.25MB >> /dev/vdb 307.25MB >> /dev/vdd 307.25MB >> >> Data,Single: Size:8.00MB, Used:0.00 >> /dev/vdb 8.00MB >> >> System,RAID1: Size:8.00MB, Used:4.00KB >> /dev/vdd 8.00MB >> /dev/vdc 8.00MB >> >> System,Single: Size:4.00MB, Used:0.00 >> /dev/vdb 4.00MB >> >> Metadata,RAID1: Size:460.94MB, Used:24.00KB >> /dev/vdb 460.94MB >> /dev/vdd 460.94MB >> >> Metadata,Single: Size:8.00MB, Used:0.00 >> /dev/vdb 8.00MB >> >> Unused: >> /dev/vdb 2.23GB >> /dev/vdc 2.69GB >> /dev/vdd 2.24GB > > Couple minor things, in order of personal opinion of severity: > > * Devices should be listed in a consistent order; device names are > just too consistently similarCould you elaborate ? I didn''t understood well> > * System chunks shouldn''t be listed between data and metadata; really, > they''re just noise 99% of the time anywayFrankly speaking I don''t know what contains. We can discuss a sorting or an option to show/hide (depending by by the default)> > * I think it may be more useful to display each disk, with the > profiles in use underneath. With a larger number of disks, that would > make it _much_ easier to tell at-a-glance what is currently on a disk > (that I may want to remove, or which I may suspect to be unreliable). > > * I''d rename "Unused" to "Unallocated" for consistency with the section titleYes> * (and I still detest the_underscores_between_all_the_words; it > doesn''t make parsing significantly easier, and it''s an eyesore):-) we can work on that.> > * Three coats of blue paint plus a clear-coat is the One True Paint-Job. > . >-- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 25, 2012, at 1:21 PM, Goffredo Baroncelli <kreijack@inwind.it> wrote:> > Moreover I still didn''t understand how btrfs was using the disks.This comment has less to do with the RFC, and more about user confusion in a specific case of the existing fi df behavior. But since I have the same misunderstanding of how btrfs is using the disks, I decided to reply to this thread. While working with Fedora 18''s new System Storage Manager [1], I came across this problem. For reference the bug report [2] which seems less of a bug with ssm than a peculiarity with btrfs chunk allocation and how fi df report usage. 80GB VDI, Virtual Box VM, containing Fedora 18: installed and yum updated 2-3 times. That''s it, yet for some reason, 76 GB of chunks have been allocated and they''re all full? This doesn''t make sense when there''s just under 4GB of data on this single device. [root@f18v ~]# btrfs fi show Label: ''fedora'' uuid: 780b8553-4097-4136-92a4-c6fd48779b0c Total devices 1 FS bytes used 3.93GB devid 1 size 76.06GB used 76.06GB path /dev/sda1 [root@f18v ~]# btrfs fi df / Data: total=72.03GB, used=3.67GB System, DUP: total=8.00MB, used=16.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=2.00GB, used=257.54MB Metadata: total=8.00MB, used=0.00 I decided to rebalance, and while things become a lot more sensible, I''m still confused: [chris@f18v ~]$ sudo btrfs fi show failed to read /dev/sr0 Label: ''fedora'' uuid: 780b8553-4097-4136-92a4-c6fd48779b0c Total devices 1 FS bytes used 3.91GB devid 1 size 76.06GB used 9.13GB path /dev/sda1 [chris@f18v ~]$ sudo btrfs fi df / Data: total=5.00GB, used=3.66GB System, DUP: total=64.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=2.00GB, used=256.84MB Points of confusion: 1. Why is FS bytes used = 3.91GB, yet devid 1 used is 9.13 GB? 2. Why before a rebalance does ''fi df'' show extra lines, and then after rebalance there are fewer lines? Another case with raid10, ''fi df'' shows six lines of data, but then after rebalance is shows three lines? 3. How does Data: total=72GB before rebalance, but is 5GB after rebalance? This was a brand new file system, file system installed, with maybe 2-3 updates, and a dozen or two reboots. That''s it. No VM''s created on that volume (it''s a VDI itself), and the VDI file itself never grew beyond 9GB. Chris Murphy [1] https://fedoraproject.org/wiki/Features/SystemStorageManager [2] https://bugzilla.redhat.com/show_bug.cgi?id=869489-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
>>> Allocated_area: >>> Data,RAID0: Size:921.75MB, Used:256.00KB >>> /dev/vdc 307.25MB >>> /dev/vdb 307.25MB >>> /dev/vdd 307.25MB >>> >>> Data,Single: Size:8.00MB, Used:0.00 >>> /dev/vdb 8.00MB >>> >>> System,RAID1: Size:8.00MB, Used:4.00KB >>> /dev/vdd 8.00MB >>> /dev/vdc 8.00MB >>> >>> System,Single: Size:4.00MB, Used:0.00 >>> /dev/vdb 4.00MB >>> >>> Metadata,RAID1: Size:460.94MB, Used:24.00KB >>> /dev/vdb 460.94MB >>> /dev/vdd 460.94MB >>> >>> Metadata,Single: Size:8.00MB, Used:0.00 >>> /dev/vdb 8.00MB >>> >>> Unused: >>> /dev/vdb 2.23GB >>> /dev/vdc 2.69GB >>> /dev/vdd 2.24GB >> >> Couple minor things, in order of personal opinion of severity: >> >> * Devices should be listed in a consistent order; device names are >> just too consistently similar > Could you elaborate ? I didn''t understood wellThereby demonstrating the problem :) Data,RAID0: Size:921.75MB, Used:256.00KB /dev/vdc 307.25MB /dev/vdb 307.25MB /dev/vdd 307.25MB Unused: /dev/vdb 2.23GB /dev/vdc 2.69GB /dev/vdd 2.24GB The first goes c, b, d; the second goes b, c, d. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Oct 25, 2012 at 2:03 PM, Chris Murphy <lists@colorremedies.com> wrote:> > On Oct 25, 2012, at 1:21 PM, Goffredo Baroncelli <kreijack@inwind.it> wrote: >> >> Moreover I still didn''t understand how btrfs was using the disks. > > This comment has less to do with the RFC, and more about user confusion in a specific case of the existing fi df behavior. But since I have the same misunderstanding of how btrfs is using the disks, I decided to reply to this thread. > > While working with Fedora 18''s new System Storage Manager [1], I came across this problem. For reference the bug report [2] which seems less of a bug with ssm than a peculiarity with btrfs chunk allocation and how fi df report usage. > > 80GB VDI, Virtual Box VM, containing Fedora 18: installed and yum updated 2-3 times. That''s it, yet for some reason, 76 GB of chunks have been allocated and they''re all full? This doesn''t make sense when there''s just under 4GB of data on this single device. > > [root@f18v ~]# btrfs fi show > Label: ''fedora'' uuid: 780b8553-4097-4136-92a4-c6fd48779b0c > Total devices 1 FS bytes used 3.93GB > devid 1 size 76.06GB used 76.06GB path /dev/sda1 > > [root@f18v ~]# btrfs fi df / > Data: total=72.03GB, used=3.67GB > System, DUP: total=8.00MB, used=16.00KB > System: total=4.00MB, used=0.00 > Metadata, DUP: total=2.00GB, used=257.54MB > Metadata: total=8.00MB, used=0.00 > > I decided to rebalance, and while things become a lot more sensible, I''m still confused: > > [chris@f18v ~]$ sudo btrfs fi show > failed to read /dev/sr0 > Label: ''fedora'' uuid: 780b8553-4097-4136-92a4-c6fd48779b0c > Total devices 1 FS bytes used 3.91GB > devid 1 size 76.06GB used 9.13GB path /dev/sda1 > > [chris@f18v ~]$ sudo btrfs fi df / > Data: total=5.00GB, used=3.66GB > System, DUP: total=64.00MB, used=4.00KB > System: total=4.00MB, used=0.00 > Metadata, DUP: total=2.00GB, used=256.84MB > > Points of confusion: > > 1. Why is FS bytes used = 3.91GB, yet devid 1 used is 9.13 GB?"FS bytes" is what du -sh would show. "devid 1 used" is space allocated to some block group (without that block group itself being entirely used)> 2. Why before a rebalance does ''fi df'' show extra lines, and then after rebalance there are fewer lines? Another case with raid10, ''fi df'' shows six lines of data, but then after rebalance is shows three lines?A bug in mkfs causes some tiny blockgroups with the wrong profile to be created; as they''re unused, they get cleaned up by the balance.> 3. How does Data: total=72GB before rebalance, but is 5GB after rebalance? This was a brand new file system, file system installed, with maybe 2-3 updates, and a dozen or two reboots. That''s it. No VM''s created on that volume (it''s a VDI itself), and the VDI file itself never grew beyond 9GB.Combine the previous two answers: You had 72GB allocated to block groups which are mostly empty. After the balance, the contents of those groups have been shuffled around such that most of them could be freed. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
My suggestion is that by default a summary similar to the existing df command be mimicked, where it makes sense, for btrfs fi df. - I like the Capacity %. If there is a reliable equivalent, it need not be inode based, that would be great. - I care far less about the actual physical device information, more about the btrfs volume(s) as a whole. How big is the volume, how much of that is used, and how much is available? I understand the challenges behind estimating the amount available. So if the value for available/free is precided by a ~ in each case, or as a heading "~Avail" or "~Free" I''d be OK with that disclaimer. I think the examples so far are reporting too much information and it''s difficult to get just what I want. Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-25 22:11, cwillu wrote:>>> 3. How does Data: total=72GB before rebalance, but is 5GB after >>> rebalance? This was a brand new file system, file system >>> installed, with maybe 2-3 updates, and a dozen or two reboots. >>> That''s it. No VM''s created on that volume (it''s a VDI itself), >>> and the VDI file itself never grew beyond 9GB. > Combine the previous two answers: You had 72GB allocated to block > groups which are mostly empty. After the balance, the contents of > those groups have been shuffled around such that most of them could > be freed.Did btrfs rebalance when it needs spaces ? The example reported by Chris suggested that due to the filesystem history, btrfs allocate some chunk that became empty. If the user/admin don''t make a (re)balance, is btrfs capable to move some chunk from the DATA pool to the METADATA pool when the space became low autonomously ? I don''t think, but I want a confirmation. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Oct 25, 2012 at 2:36 PM, Chris Murphy <lists@colorremedies.com> wrote:> My suggestion is that by default a summary similar to the existing df command be mimicked, where it makes sense, for btrfs fi df. > > - I like the Capacity %. If there is a reliable equivalent, it need not be inode based, that would be great. > > - I care far less about the actual physical device information, more about the btrfs volume(s) as a whole. How big is the volume, how much of that is used, and how much is available? > > I understand the challenges behind estimating the amount available. So if the value for available/free is precided by a ~ in each case, or as a heading "~Avail" or "~Free" I''d be OK with that disclaimer. > > I think the examples so far are reporting too much information and it''s difficult to get just what I want.Plain old "/bin/df" is adequate for that though, and in the mean time one _does_ need _all_ of that information to work with the filesystem. However, the detailed breakdown is vital to answer many questions: "Why can''t I write to my filesystem with 80gb free? <btrfs fi df> Oh, because metadata is raid1 and the one disk is 80gb smaller than the other." "How much data is on this disk that started giving SMART errors?" "How many GB of vm image files (or other large files) can I probably fit on this fs?" "How many GB of mail (or other tiny files) can I probably fit on this fs?" "Is there enough space to remove this disk from the fs, and how much free space will I have then?" And the all-important "Could you please run btrfs fi df and pastebin the output so we can tell what the hell is going on?" :) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-25 22:49, cwillu wrote:> And the all-important "Could you please run btrfs fi df and pastebin > the output so we can tell what the hell is going on?" :)I fully agree ! -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 25, 2012, at 2:11 PM, cwillu <cwillu@cwillu.com> wrote:>> >> 1. Why is FS bytes used = 3.91GB, yet devid 1 used is 9.13 GB? > > "FS bytes" is what du -sh would show. "devid 1 used" is space > allocated to some block group (without that block group itself being > entirely used)I agree the word "used" shouldn''t be used twice when it can have two different values like this; or drop one of the values entirely. Which more correctly reflects the amount of data+metadata?>> 2. Why before a rebalance does ''fi df'' show extra lines, and then after rebalance there are fewer lines? Another case with raid10, ''fi df'' shows six lines of data, but then after rebalance is shows three lines? > > A bug in mkfs causes some tiny blockgroups with the wrong profile to > be created; as they''re unused, they get cleaned up by the balance.OK. I''d have to try to reproduce. I''m virtually certain right after mkfs.btrfs and subsequent OS installation, but before reboot from the installer that ''fi df'' did not indicate 72GB of allocated or used space.> Plain old "/bin/df" is adequate for that though, and in the mean time > one _does_ need _all_ of that information to work with the filesystem. > However, the detailed breakdown is vital to answer many questions:If we''re going to borrow an old term like df, which equates to "display free (space)" then it kinda needs to do that, or try to, as it''s point of emphasis. And right now that''s a significant missing piece to ''btrfs fi df'' is the free part. Inferring it from a LOT of other information I''d characterize as quite less than ideal. If> "Why can''t I write to my filesystem with 80gb free? <btrfs fi df> > Oh, because metadata is raid1 and the one disk is 80gb smaller than > the other." > > "How much data is on this disk that started giving SMART errors?" > > "How many GB of vm image files (or other large files) can I probably > fit on this fs?" > > "How many GB of mail (or other tiny files) can I probably fit on this fs?" > > "Is there enough space to remove this disk from the fs, and how much > free space will I have then?" > > And the all-important "Could you please run btrfs fi df and pastebin > the output so we can tell what the hell is going on?" :)So what''s the intended distinction between ''fi df'' and ''fi show''? Because for months using btrfs I''d constantly be confused which command was going to show me what information I wanted, and that tells me there should be some better distinction between the commands. Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Oct 25, 2012 at 8:33 PM, Chris Murphy <lists@colorremedies.com> wrote:> So what''s the intended distinction between ''fi df'' and ''fi show''? Because for months using btrfs I''d constantly be confused which command was going to show me what information I wanted, and that tells me there should be some better distinction between the commands.Or the distinction should be removed, which is what this patch effectively does. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 25, 2012, at 9:36 PM, cwillu <cwillu@cwillu.com> wrote:> On Thu, Oct 25, 2012 at 8:33 PM, Chris Murphy <lists@colorremedies.com> wrote: >> So what''s the intended distinction between ''fi df'' and ''fi show''? Because for months using btrfs I''d constantly be confused which command was going to show me what information I wanted, and that tells me there should be some better distinction between the commands. > > Or the distinction should be removed, which is what this patch effectively does.Ok fine. But I still think it needs a -s (summary) or -v (more detail) switch. The combined amount of information from existing df and show in a single command is too much information at a glance. The idea that only a human being can evaluate that pile of data into a meaningful answer to their questions strikes me as flawed. That it''s a non-trivial boolean problem I''ll accept, but if a human can consistently draw reasonable conclusions from this data dump then something can summarize it into the same - albeit not answering all possible questions. Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 25, 2012, at 9:36 PM, cwillu <cwillu@cwillu.com> wrote:> On Thu, Oct 25, 2012 at 8:33 PM, Chris Murphy <lists@colorremedies.com> wrote: >> So what''s the intended distinction between ''fi df'' and ''fi show''? Because for months using btrfs I''d constantly be confused which command was going to show me what information I wanted, and that tells me there should be some better distinction between the commands. > > Or the distinction should be removed, which is what this patch effectively does.Ok fine. But I still think it needs a -s (summary) or -v (more detail) switch. The combined amount of information from existing df and show in a single command is too much information at a glance. The idea that only a human being can evaluate that pile of data into a meaningful answer to their questions strikes me as flawed. That it''s a non-trivial boolean problem I''ll accept, but if a human can consistently draw reasonable conclusions from this data dump then something can summarize it into the same - albeit not answering all possible questions. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Donnerstag, 25. Oktober 2012 schrieb Goffredo Baroncelli:> Hi all, > > this is a new attempt to improve the output of the command "btrfs fi > df". > > The previous attempt received a good reception. However there was no a > general consensus about the wording. > > Moreover I still didn''t understand how btrfs was using the disks. > A my first attempt was to develop a new command which shows how the > disks are divided in chunks. However it still was no clear. Then I > realised that I need to merge the two output. > > Below the results. It shows how the disks is used by the different > chunks. > > The code is pullable from > http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git > branch > info-cmd > > I don''t publish the patched because aren''t in a good shape. However I > really like the output. The example is a filesystem based on three > disks of 3GB. > > It is clear that: > - RAID0 uses all the disks > - RAID1 uses two different disks > > Comments are welcome. > > Known bugs: > - if a filesystem uses a disk but there is any chunk, the disk is not > shown (solvable) > - this command need root capability (I use the BTRFS_IOC_TREE_SEARCH > to get the chunk info; so that is unavoidable) > > > ghigo@emulato:~$ sudo ./btrfs fi df /mnt/btrfs1/ > [sudo] password for ghigo: > Path: /mnt/btrfs1 > Summary: > Disk_size: 9.00GB > Disk_allocated: 1.83GB > Disk_unallocated: 7.17GB > Used: 284.00KB > Free_(Estimated): 6.76GB (Max: 8.54GB, min: 4.96GB) > Data_to_disk_ratio: 75 % > > Allocated_area: > Data,RAID0: Size:921.75MB, Used:256.00KB > /dev/vdc 307.25MB > /dev/vdb 307.25MB > /dev/vdd 307.25MB > > Data,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > System,RAID1: Size:8.00MB, Used:4.00KB > /dev/vdd 8.00MB > /dev/vdc 8.00MB > > System,Single: Size:4.00MB, Used:0.00 > /dev/vdb 4.00MB > > Metadata,RAID1: Size:460.94MB, Used:24.00KB > /dev/vdb 460.94MB > /dev/vdd 460.94MB > > Metadata,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > Unused: > /dev/vdb 2.23GB > /dev/vdc 2.69GB > /dev/vdd 2.24GBJust a quick feedback: I think this is rather long. And I find it more complicated than the older output. But maybe its more the vertically oriented formatting. How about: ghigo@emulato:~$ sudo ./btrfs fi df /mnt/btrfs1/ === regular output ==[sudo] password for ghigo: Path: /mnt/btrfs1 Summary: Disk_size: 9.00GB Disk_allocated: 1.83GB Disk_unallocated: 7.17GB Used: 284.00KB Free_(Estimated): 6.76GB (Max: 8.54GB, min: 4.96GB) Data_to_disk_ratio: 75 % Allocated_area: Data,RAID0: 256.00KB of 921.75MB used Data,Single: 0 of 8.00MB used System,RAID1: 4 KB of 8.00MB used System,Single: 0 of 4.00MB used Metadata,RAID1: 24KB of 460.94MB used Metadata,Single: 0 of 8.00MB used ^^^ possibly tabular ^^^ === only with -d / --disks or -v? == /dev/vdb: Data, RAID 0 307.25MB Data,Single 8MB System,Single 4MB Metadata,RAID1 460.94MB Metadata,Single 8MB Unused 2,23GB /dev/vdc: Data,RAID 0 307,25MB System,RAID1 8MB Unused 2,69GB /dev/vdd: Data,RAID 0 307.25MB System,RAID1 8MB Metadata,RAID1 460.94MB Unused 2.24GB (one could include the usage in single disk view as well.) Hmmm... but that also looks quite heavy. Possibly this could be done tabular as well, like: vdb vdc vdd Data, RAID 0 307,25MB 307,25MB 307,25MB … System,RAID1 - 8MB 8MB … Unused 2,23GB 2,69GB 2,24GB I like this. But what if the filesystem has 100 disks? Hmmm, how about: vdb vdc vdd Data, RAID 0 307,25MB 307,25MB 307,25MB vde vdf vdg 307,25MB 200MB 200MB … vdb vdc vdd System,RAID1 - 8MB 8MB … vdb vdc vdd Unused 2,23GB 2,69GB 2,24GB vde vdf vdg 2,25GB 2,8GB 2,21GB in such a case? Then each line has to have headings with devices. As long as one line is enough, it only needs to be printed once at the beginning of the table. So in some form a tabular matrix of devices. Other idea might be: Allocated_area: Data,RAID0: Size:921.75MB, Used:256.00KB /dev/vdb: 307.25MB /dev/vdc: 307.25MB /dev/vdd: 307.25MB And if same possibly: Allocated_area: Data,RAID0: Size:921.75MB, Used:256.00KB /dev/vd[bcd]: 307.25MB Hmmm, this is all playing with the same presentation. Possibly one can at least hide empty trees. Or have them not created at all at mkfs.btrfs time. There is no point in having / showing Metadata,Single I think as long as it is not used. Will think about this a bit. Anyway I am fine with starting somewhere. Improvents can be made later on also taking user feedback into account. Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (Martin Steigerwald):> Possibly this could be done tabular as well, like: > > vdb vdc vdd > Data, RAID 0 307,25MB 307,25MB 307,25MB > … > System,RAID1 - 8MB 8MB > … > Unused 2,23GB 2,69GB 2,24GB > > I like this. But what if the filesystem has 100 disks?Maybe I''m just not familiar enough with btrfs yet to punch an immediate hole in the idea, but how about pivoting that table? Columns for data values ("data, raid 0", "system, raid 1", "unused", ...) and rows for the underlying devices? Something like this, copying the numbers from your example. And I''m using colon here rather than comma, because I _believe_ that it better captures the intent. Data: RAID 0 System: RAID 1 Unused /dev/vdb 307.25 MB - 2.23 GB /dev/vdc 307.25 MB 8 MB 2.69 GB /dev/vdd 307.25 MB 8 MB 2.24 GB ============ ============== ===========TOTAL 921.75 MB 16 MB 7.16 GB This feels like it should work quite well as long as only 3-5 columns plus the device specifier are needed (which would appear to be the case), and it gives a quick run-down of the numbers at a glance. If the filesystem consists of a large number of devices, the header could possibly be repeated just before the "total" row (then something like "btrfs fi df | tail -n2" will still work for getting the executive summary). The risk would be if extremely long device names are used, but that should be relatively trivial to solve. Maybe something like this if the terminal width does not allow the data for each device to fit on a single line? Data: RAID 0 System: RAID 1 Unused /dev/disk/by-uuid/f72d74f4-206d-11e2-aa11-cb4348b38f9e 307.25 MB - 2.23 GB /dev/disk/by-uuid/044e48de-206e-11e2-8a63-8366c6174d47 307.25 MB 8 MB 2.69 GB /dev/disk/by-uuid/0b1309a2-206e-11e2-aa21-234780dc3782 307.25 MB 8 MB 2.24 GB ============ ============== =========TOTAL 921.75 MB 16 MB 7.16 GB Both of the above, as you will note, are very similar to how GNU df solves the same problem. Line breaking could also be disabled if standard output is not a TTY, which means that for example passing the output to grep would still work like one would expect. Any reason why such an output format wouldn''t work for a summary view? -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling:> On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (MartinSteigerwald):> > Possibly this could be done tabular as well, like: > > > > vdb vdc vdd > > > > Data, RAID 0 307,25MB 307,25MB 307,25MB > > … > > System,RAID1 - 8MB 8MB > > … > > Unused 2,23GB 2,69GB 2,24GB > > > > > > > > I like this. But what if the filesystem has 100 disks? > > Maybe I''m just not familiar enough with btrfs yet to punch an > immediate hole in the idea, but how about pivoting that table? Columns > for data values ("data, raid 0", "system, raid 1", "unused", ...) and > rows for the underlying devices? Something like this, copying the > numbers from your example. And I''m using colon here rather than comma, > because I believe that it better captures the intent. > > Data: RAID 0 System: RAID 1 Unused > /dev/vdb 307.25 MB - 2.23 GB > /dev/vdc 307.25 MB 8 MB 2.69 GB > /dev/vdd 307.25 MB 8 MB 2.24 GB > ============ ============== ===========> TOTAL 921.75 MB 16 MB 7.16 GBHmmm, good idea. I like it this way around. It would scale better with the number of drives and there is a good way to place the totals. I wonder about how to possibly include the used part of each tree. With mostly 5 columns it might be doable. -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote:> Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: > > On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (Martin > Steigerwald): > > > Possibly this could be done tabular as well, like: > > > > > > vdb vdc vdd > > > > > > Data, RAID 0 307,25MB 307,25MB 307,25MB > > > … > > > System,RAID1 - 8MB 8MB > > > … > > > Unused 2,23GB 2,69GB 2,24GB > > > > > > > > > > > > I like this. But what if the filesystem has 100 disks? > > > > Maybe I''m just not familiar enough with btrfs yet to punch an > > immediate hole in the idea, but how about pivoting that table? Columns > > for data values ("data, raid 0", "system, raid 1", "unused", ...) and > > rows for the underlying devices? Something like this, copying the > > numbers from your example. And I''m using colon here rather than comma, > > because I believe that it better captures the intent. > > > > Data: RAID 0 System: RAID 1 Unused > > /dev/vdb 307.25 MB - 2.23 GB > > /dev/vdc 307.25 MB 8 MB 2.69 GB > > /dev/vdd 307.25 MB 8 MB 2.24 GB > > ============ ============== ===========> > TOTAL 921.75 MB 16 MB 7.16 GB > > Hmmm, good idea. I like it this way around. > > It would scale better with the number of drives and there is a good way to > place the totals. > > I wonder about how to possibly include the used part of each tree. With > mostly 5 columns it might be doable.Note that this could get arbitrarily wide in the presence of the (planned) per-object replication config. Otherwise, it works. The width is probably likely to grow more slowly than the length, though, so this way round is probably the better option. IMO. Eggshell blue is good enough. :) Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Some days, it''s just not worth gnawing through the straps. ---
On 27 Oct 2012 23:38 +0100, from hugo@carfax.org.uk (Hugo Mills):>>> Data: RAID 0 System: RAID 1 Unused >>> /dev/vdb 307.25 MB - 2.23 GB >>> /dev/vdc 307.25 MB 8 MB 2.69 GB >>> /dev/vdd 307.25 MB 8 MB 2.24 GB >>> ============ ============== ===========>>> TOTAL 921.75 MB 16 MB 7.16 GB >> >> It would scale better with the number of drives and there is a good way to >> place the totals. > > Note that this could get arbitrarily wide in the presence of the > (planned) per-object replication config. Otherwise, it works. The > width is probably likely to grow more slowly than the length, though, > so this way round is probably the better option. IMO. Eggshell blue is > good enough. :)Of course, but the suggestion in the mail I replied to can get equally arbitrarily wide in the presence of a large number of _drives_. In my experience, many times it''s better to put something together that works with the current status of the project and start using it, than trying to shoehorn every "we''d like to do this some day" feature into the original design. _Particularly_ when it''s UI one is talking about. I can think of a few ways it might be possible to restrict the growth of the width of a table like this even in the face of separate per-object replication settings, the most obvious probably being to keep a tally on disk for each of the replication types, and have columns for each replication configuration (so you might get one column for RAID 0 data, one for RAID 1 data, one for SINGLE data, and so on, but you''ll _never_ get more "data" columns than the filesystem itself supports replication methods for "data" data; the tally simply being an optimization so you don''t have to scan the whole file system for a simple "df"), but by the time that feature gets implemented, maybe someone can think of a better presentation. After all, UI aspects tend to be the easiest to fiddle with. Organizing the drives in rows also has the advantage that you don''t _have_ to read everything before you can start printing the results, if you can live with the constraint of supporting only one data and metadata replication strategy. Whether to implement it that way is another matter. With large storage systems and multi-CPU/multi-core systems, while a multithreaded approach might not provide consistent device ordering between executions depending on the exact thread execution order, it could provide a fair performance enhancement. And forget KISS; don''t we all _love_ a chance to do a little multithreaded programming before coffee if it saves the poor sysadmin a few dozen milliseconds per "df"? ;-) -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 27, 2012, at 1:55 PM, Michael Kjörling <michael@kjorling.se> wrote:> Data: RAID 0 System: RAID 1 Unused > /dev/vdb 307.25 MB - 2.23 GB > /dev/vdc 307.25 MB 8 MB 2.69 GB > /dev/vdd 307.25 MB 8 MB 2.24 GB > ============ ============== ===========> TOTAL 921.75 MB 16 MB 7.16 GBI kinda like this arrangement a little better. If the top left corner could be the volume label, that might be nice? Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-28 00:38, Hugo Mills wrote:> On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: >> Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: >>> On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (Martin >> Steigerwald): >>>> Possibly this could be done tabular as well, like:>>> >>> Data: RAID 0 System: RAID 1 Unused >>> /dev/vdb 307.25 MB - 2.23 GB >>> /dev/vdc 307.25 MB 8 MB 2.69 GB >>> /dev/vdd 307.25 MB 8 MB 2.24 GB >>> ============ ============== ===========>>> TOTAL 921.75 MB 16 MB 7.16 GB >> >> Hmmm, good idea. I like it this way around. >> >> It would scale better with the number of drives and there is a good way to >> place the totals. >> >> I wonder about how to possibly include the used part of each tree. With >> mostly 5 columns it might be doable. > > Note that this could get arbitrarily wide in the presence of the > (planned) per-object replication config. Otherwise, it works. The > width is probably likely to grow more slowly than the length, though, > so this way round is probably the better option. IMO. Eggshell blue is > good enough. :)I liked the Martin idea too. However I think that it is not applicable. Even on my simple test bed I got Data,Single: 8.00MB Data,RAID0: 307.25MB Metadata,Single: 8.00MB Metadata,RAID1: 460.94MB System,Single: 4.00MB System,RAID1: 8.00MB Plus we can have also Data+Metadata...> > Hugo. >-- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-10-25 21:21, Goffredo Baroncelli wrote:> Hi all, > > this is a new attempt to improve the output of the command "btrfs > fi df".Below you can see another iteration. I tried to address all the cwillu requests, which to me make sense. I thought a lot about chunk/disks vs disk/chunks, and I reached the conclusion that we could need both. In order to avoid a too long and redundant output, my idea is to have three different commands: 1) btrfs filesystem df <path> -> which show what it is called summary 2) btrfs filesystem disk-usage <path> -> which show what is called "Detail" 3) btrfs device disk-usage <path> -> which is like 2) but grouped by disk instead of chunks. This is an idea about which I want some comments. The example below show the latest results. I want to point out that I was not able to show the chunk usage per disk, because I don''t have this information. The patches are not shaped to be showed, however the code is pullable from http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git branch info-cmd (commit f90e55e7) $ sudo ./btrfs fi df /mnt/btrfs1/ Path: /mnt/btrfs1 Summary: Disk_size: 21.00GB Disk_allocated: 1.83GB Disk_unallocated: 19.17GB Used: 284.00KB Free_(Estimated): 15.76GB (Max: 20.54GB, min: 10.96GB) Data_to_disk_ratio: 75 % Allocated_area: Data,Single: Size:8.00MB, Used:0.00 /dev/vdb 8.00MB Data,RAID0: Size:921.75MB, Used:256.00KB /dev/vdb 307.25MB /dev/vdc 307.25MB /dev/vdd 307.25MB Metadata,Single: Size:8.00MB, Used:0.00 /dev/vdb 8.00MB Metadata,RAID1: Size:460.94MB, Used:24.00KB /dev/vdb 460.94MB /dev/vdd 460.94MB System,Single: Size:4.00MB, Used:0.00 /dev/vdb 4.00MB System,RAID1: Size:8.00MB, Used:4.00KB /dev/vdc 8.00MB /dev/vdd 8.00MB Unallocated: /dev/vdb 2.23GB /dev/vdc 2.69GB /dev/vdd 2.24GB /dev/vdf 12.00GB Disks: /dev/vdb 3.00GB Data,Single: 8.00MB Data,RAID0: 307.25MB Metadata,Single: 8.00MB Metadata,RAID1: 460.94MB System,Single: 4.00MB Unallocated: 2.23GB /dev/vdc 3.00GB Data,RAID0: 307.25MB System,RAID1: 8.00MB Unallocated: 2.69GB /dev/vdd 3.00GB Data,RAID0: 307.25MB Metadata,RAID1: 460.94MB System,RAID1: 8.00MB Unallocated: 2.24GB /dev/vdf 12.00GB Unallocated: 12.00GB - -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJQjPRpAAoJEA7amzeLguC1sUoP+gMhv1NWN48Sun/8fNxJxTg6 TCf65+TKDGKByUKjSLjS0hFEb5xcoWG7H1DEf4KzdZYGMkw5iq8IkZcejVlrj/fn Z+BRS02Zl0zMdYmMg7SH75BQz2oT5rpUsMI499qiE8gd6UJzeFD0DHjBWWBdptaA irba605QCtW7JRib3yZwmkcBXSl+oLX2bpELbGARxKWLSx7B5WcKLGNtV3f7whVH EBzZYXQi2LamRbxH7/la75GXvjSTAXHBrGy3p4TTiF2HAEQ3xALr0aVuYvyHDQhF 4EeledgW/Z26F5DUZcatTlvlse7oKl1R4XX0r5MQRJ4aBwwUAIK/qNpTxm3HNAC9 TSbhUsA/LtcMZyl7AEhKt5/B2MQgkqobpkDK2DiA+qZkONSfoTDBdCjMiwg/yMq8 N07MevdI7gzy3ZKb26v5N6W+wLImpA6mVceWrCksIRh5TP3aOjnsvrkH8uopNC2J rbb0CW+KF1o28iI/doHuh8BRhhpCfl0vd4oJQseUPDtCsRIqXlHqBu+i/CMwKWE0 fne8RZRia2EAa4fSrl3I7Uc8gEuYt1EgbXdoDvejB6vdVZ7DsHGVc2Pe9JwLi9rv iL9TU4L+Iaw4DHR+h68k0Eg1q+GX0u3LUxXvgLBmsoyb3JQRMrcVbT+gusvU6Mrr 1YgFwwLdi+KMBTWSHoo2 =iKFO -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli:> On 2012-10-25 21:21, Goffredo Baroncelli wrote: > > Hi all, > > > > this is a new attempt to improve the output of the command "btrfs > > fi df". > > Below you can see another iteration. I tried to address all the cwillu > requests, which to me make sense. > > I thought a lot about chunk/disks vs disk/chunks, and I reached the > conclusion that we could need both. In order to avoid a too long and > redundant output, my idea is to have three different commands: > > 1) btrfs filesystem df <path> -> which show what it is called > summary > 2) btrfs filesystem disk-usage <path> -> which show what is called > "Detail" > 3) btrfs device disk-usage <path> -> which is like 2) but grouped > by disk instead of chunks. > > > This is an idea about which I want some comments. The example below > show the latest results. I want to point out that I was not able to > show the chunk usage per disk, because I don''t have this information. > > The patches are not shaped to be showed, however the code is pullable > from > > http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git > branch > info-cmd (commit f90e55e7) > > > $ sudo ./btrfs fi df /mnt/btrfs1/ > Path: /mnt/btrfs1 > Summary: > Disk_size: 21.00GB > Disk_allocated: 1.83GB > Disk_unallocated: 19.17GB > Used: 284.00KB > Free_(Estimated): 15.76GB (Max: 20.54GB, min: 10.96GB) > Data_to_disk_ratio: 75 %Okay, so that btrfs fi df. Which is a summary. I think much more wouldn´t be a summary anymore, so thats okay. And the following> Allocated_area: > Data,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > Data,RAID0: Size:921.75MB, Used:256.00KB > /dev/vdb 307.25MB > /dev/vdc 307.25MB > /dev/vdd 307.25MB > > Metadata,Single: Size:8.00MB, Used:0.00 > /dev/vdb 8.00MB > > Metadata,RAID1: Size:460.94MB, Used:24.00KB > /dev/vdb 460.94MB > /dev/vdd 460.94MB > > System,Single: Size:4.00MB, Used:0.00 > /dev/vdb 4.00MB > > System,RAID1: Size:8.00MB, Used:4.00KB > /dev/vdc 8.00MB > /dev/vdd 8.00MB > > Unallocated: > /dev/vdb 2.23GB > /dev/vdc 2.69GB > /dev/vdd 2.24GB > /dev/vdf 12.00GBis detail shown by btrfs filesystem disk-usage? While> Disks: > /dev/vdb 3.00GB > Data,Single: 8.00MB > Data,RAID0: 307.25MB > Metadata,Single: 8.00MB > Metadata,RAID1: 460.94MB > System,Single: 4.00MB > Unallocated: 2.23GB > > /dev/vdc 3.00GB > Data,RAID0: 307.25MB > System,RAID1: 8.00MB > Unallocated: 2.69GB > > /dev/vdd 3.00GB > Data,RAID0: 307.25MB > Metadata,RAID1: 460.94MB > System,RAID1: 8.00MB > Unallocated: 2.24GB > > /dev/vdf 12.00GB > Unallocated: 12.00GBwill be btrfs device disk-usage? What was your reasoning for not using options to btrfs filesystem df? That df doesn´t show more than "disk free" as well? Then there is a little "inconsistency": "df" versus "disk-usage". I would use either "disk-free" and "disk-usage" or "df" and "du". While regular "du" is not disk-usage but a filesystem directory tree usage command. I will think about this a bit more. Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli:> On 2012-10-28 00:38, Hugo Mills wrote: > > On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: > >> Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: > >>> On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (Martin > >> > >> Steigerwald): > >>>> Possibly this could be done tabular as well, like: > >>> Data: RAID 0 System: RAID 1 Unused > >>> > >>> /dev/vdb 307.25 MB - 2.23 GB > >>> /dev/vdc 307.25 MB 8 MB 2.69 GB > >>> /dev/vdd 307.25 MB 8 MB 2.24 GB > >>> > >>> ============ ============== ===========> >>> > >>> TOTAL 921.75 MB 16 MB 7.16 GB > >> > >> Hmmm, good idea. I like it this way around. > >> > >> It would scale better with the number of drives and there is a good > >> way to place the totals. > >> > >> I wonder about how to possibly include the used part of each tree. > >> With mostly 5 columns it might be doable. > > > > > > Note that this could get arbitrarily wide in the presence of the > > > > (planned) per-object replication config. Otherwise, it works. The > > width is probably likely to grow more slowly than the length, though, > > so this way round is probably the better option. IMO. Eggshell blue > > is good enough. :) > > I liked the Martin idea too. However I think that it is not applicable. > Even on my simple test bed I got > > Data,Single: 8.00MB > Data,RAID0: 307.25MB > Metadata,Single: 8.00MB > Metadata,RAID1: 460.94MB > System,Single: 4.00MB > System,RAID1: 8.00MB > > Plus we can have also Data+Metadata...One could still use multi row approach in that case: Data: RAID 0 System: RAID 1 Unused /dev/vdb 307.25 MB - 2.23 GB Data: RAID 1 System: RAID 0 250.12 MB 128 MB Data: RAID 0 System: RAID 1 Unused /dev/vdc 307.25 MB 8 MB 2.69 GB Data: RAID 1 System: RAID 0 250.12 MB - […] But still if if can be arbitrarily long due to that per object replication config, a vertical output might and leaving graphical representation to a Qt Quick application or so might be better. -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-28 11:33, Martin Steigerwald wrote:> Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli:[...]>> $ sudo ./btrfs fi df /mnt/btrfs1/ >> Path: /mnt/btrfs1 >> Summary: >> Disk_size: 21.00GB >> Disk_allocated: 1.83GB >> Disk_unallocated: 19.17GB >> Used: 284.00KB >> Free_(Estimated): 15.76GB (Max: 20.54GB, min: 10.96GB) >> Data_to_disk_ratio: 75 % > > Okay, so that btrfs fi df. > > Which is a summary. > > I think much more wouldn´t be a summary anymore, so thats okay. > > > And the following > >> Allocated_area: >> Data,Single: Size:8.00MB, Used:0.00 >> /dev/vdb 8.00MB >> >> Data,RAID0: Size:921.75MB, Used:256.00KB >> /dev/vdb 307.25MB[...]> > is detail shown by btrfs filesystem disk-usage?Yes> > While > >> Disks: >> /dev/vdb 3.00GB >> Data,Single: 8.00MB >> Data,RAID0: 307.25MB >> Metadata,Single: 8.00MB >> Metadata,RAID1: 460.94MB[...]> > will be btrfs device disk-usage?yes> > What was your reasoning for not using options to btrfs filesystem df? That > df doesn´t show more than "disk free" as well?My feel is that a switch should change "a bit" a command. In this case there are very different outputs, for different purposes (how is used a disk ? where are the chunks ? how many free space I have ? ).> > Then there is a little "inconsistency": "df" versus "disk-usage". I would > use either "disk-free" and "disk-usage" or "df" and "du". While regular > "du" is not disk-usage but a filesystem directory tree usage command. > > I will think about this a bit more. > > Thanks,-- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Michael, please keep CCs. Its usual to keep them on kernel related lists. Thanks. Am Sonntag, 28. Oktober 2012 schrieb Michael Kjörling:> On 27 Oct 2012 23:38 +0100, from hugo@carfax.org.uk (Hugo Mills): > >>> Data: RAID 0 System: RAID 1 Unused > >>> > >>> /dev/vdb 307.25 MB - 2.23 GB > >>> /dev/vdc 307.25 MB 8 MB 2.69 GB > >>> /dev/vdd 307.25 MB 8 MB 2.24 GB > >>> > >>> ============ ============== ===========> >>> > >>> TOTAL 921.75 MB 16 MB 7.16 GB > >> > >> It would scale better with the number of drives and there is a good > >> way to place the totals. > >> > > Note that this could get arbitrarily wide in the presence of the > > > > (planned) per-object replication config. Otherwise, it works. The > > width is probably likely to grow more slowly than the length, though, > > so this way round is probably the better option. IMO. Eggshell blue > > is good enough. :) > > Of course, but the suggestion in the mail I replied to can get equally > arbitrarily wide in the presence of a large number of _drives_. > > In my experience, many times it''s better to put something together > that works with the current status of the project and start using it, > than trying to shoehorn every "we''d like to do this some day" feature > into the original design. _Particularly_ when it''s UI one is talking > about. I can think of a few ways it might be possible to restrict the > growth of the width of a table like this even in the face of separate > per-object replication settings, the most obvious probably being to > keep a tally on disk for each of the replication types, and have > columns for each replication configuration (so you might get one > column for RAID 0 data, one for RAID 1 data, one for SINGLE data, and > so on, but you''ll _never_ get more "data" columns than the filesystem > itself supports replication methods for "data" data; the tally simply > being an optimization so you don''t have to scan the whole file system > for a simple "df"), but by the time that feature gets implemented, > maybe someone can think of a better presentation.Good idea. Maybe its also possible to hide to subsum empty trees as "Empty areas" or "Unused" or so.> After all, UI aspects tend to be the easiest to fiddle with.I agree. Output is not set in stone and I still think, scripts shall not parse it. If output is needed for scripts, provide a switch for csv or json like output. fio - the flexible I/O tester - has got JSON support recently so some code is already there. Or provide direct API by libbtrfs or so.> Organizing the drives in rows also has the advantage that you don''t > _have_ to read everything before you can start printing the results, > if you can live with the constraint of supporting only one data and > metadata replication strategy. Whether to implement it that way is > another matter. With large storage systems and multi-CPU/multi-core > systems, while a multithreaded approach might not provide consistent > device ordering between executions depending on the exact thread > execution order, it could provide a fair performance enhancement. And > forget KISS; don''t we all _love_ a chance to do a little multithreaded > programming before coffee if it saves the poor sysadmin a few dozen > milliseconds per "df"? ;-)Well if tabular I am all for having drives in rows. Aside from all of this, I wonder how ZFS tools are doing it? Anyone who has access to ZFS and can provide some outputs? There are claims for it being better in that regard¹. I bet the new quota stuff provides for used space per subvolume, but still they claim they can now how much space is free exactly. If different things can have different replication strategies I think they can´t. Still I would like to see the output of something ZFS commands to show disk-usage. Maybe that gives some additional ideas. Hmmm, I got something: merkaba:/> zpool create -m /mnt/zeit zeit /dev/merkaba/zeit merkaba:/> zfs create zeit/test1 merkaba:/> zfs create zeit/test2 merkaba:/> zfs create zeit/test3 merkaba:/> mount | grep zfs kstat on /zfs-kstat type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) zeit on /mnt/zeit type fuse.zfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) zeit/test1 on /mnt/zeit/test1 type fuse.zfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) zeit/test2 on /mnt/zeit/test2 type fuse.zfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) zeit/test3 on /mnt/zeit/test3 type fuse.zfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) merkaba:/> zfs list NAME USED AVAIL REFER MOUNTPOINT zeit 168K 19,6G 24K /mnt/zeit zeit/test1 21K 19,6G 21K /mnt/zeit/test1 zeit/test2 21K 19,6G 21K /mnt/zeit/test2 zeit/test3 21K 19,6G 21K /mnt/zeit/test3 merkaba:/> dd if=/dev/zero of=/mnt/zeit/test2/schlumpf bs=1M count=10 10+0 Datensätze ein 10+0 Datensätze aus 10485760 Bytes (10 MB) kopiert, 0,022653 s, 463 MB/s merkaba:/> zfs list NAME USED AVAIL REFER MOUNTPOINT zeit 169K 19,6G 25K /mnt/zeit zeit/test1 21K 19,6G 21K /mnt/zeit/test1 zeit/test2 21K 19,6G 21K /mnt/zeit/test2 zeit/test3 21K 19,6G 21K /mnt/zeit/test3 merkaba:/> sync merkaba:/> zfs list NAME USED AVAIL REFER MOUNTPOINT zeit 169K 19,6G 25K /mnt/zeit zeit/test1 21K 19,6G 21K /mnt/zeit/test1 zeit/test2 21K 19,6G 21K /mnt/zeit/test2 zeit/test3 21K 19,6G 21K /mnt/zeit/test3 merkaba:/> dd if=/dev/urandom of=/mnt/zeit/test2/schlumpf bs=1M count=10 10+0 Datensätze ein 10+0 Datensätze aus 10485760 Bytes (10 MB) kopiert, 0,929913 s, 11,3 MB/s merkaba:/> zfs list NAME USED AVAIL REFER MOUNTPOINT zeit 10,2M 19,6G 25K /mnt/zeit zeit/test1 21K 19,6G 21K /mnt/zeit/test1 zeit/test2 10,0M 19,6G 10,0M /mnt/zeit/test2 zeit/test3 21K 19,6G 21K /mnt/zeit/test3 Hmmm. They just tell used/free space, but also for "subvolumes". Which we can only do by estimation in some cases. [1] http://rudd-o.com/linux-and-free-software/ways-in-which-zfs-is-better- than-btrfs under "ZFS tracks used space per file system" Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-28 11:38, Martin Steigerwald wrote:> Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: >> On 2012-10-28 00:38, Hugo Mills wrote: >>> On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: >>>> Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: >>>>> On 27 Oct 2012 18:43 +0200, from Martin@lichtvoll.de (Martin >>>> >>>> Steigerwald): >>>>>> Possibly this could be done tabular as well, like: >>>>> Data: RAID 0 System: RAID 1 Unused >>>>> >>>>> /dev/vdb 307.25 MB - 2.23 GB >>>>> /dev/vdc 307.25 MB 8 MB 2.69 GB >>>>> /dev/vdd 307.25 MB 8 MB 2.24 GB >>>>> >>>>> ============ ============== ===========>>>>> >>>>> TOTAL 921.75 MB 16 MB 7.16 GB >>>> >>>> Hmmm, good idea. I like it this way around. >>>> >>>> It would scale better with the number of drives and there is a good >>>> way to place the totals. >>>> >>>> I wonder about how to possibly include the used part of each tree. >>>> With mostly 5 columns it might be doable. >>> >>> >>> Note that this could get arbitrarily wide in the presence of the >>> >>> (planned) per-object replication config. Otherwise, it works. The >>> width is probably likely to grow more slowly than the length, though, >>> so this way round is probably the better option. IMO. Eggshell blue >>> is good enough. :) >> >> I liked the Martin idea too. However I think that it is not applicable. >> Even on my simple test bed I got >> >> Data,Single: 8.00MB >> Data,RAID0: 307.25MB >> Metadata,Single: 8.00MB >> Metadata,RAID1: 460.94MB >> System,Single: 4.00MB >> System,RAID1: 8.00MB >> >> Plus we can have also Data+Metadata... > > One could still use multi row approach in that case: > > Data: RAID 0 System: RAID 1 Unused > /dev/vdb 307.25 MB - 2.23 GB > Data: RAID 1 System: RAID 0 > 250.12 MB 128 MB > Data: RAID 0 System: RAID 1 Unused > /dev/vdc 307.25 MB 8 MB 2.69 GB > Data: RAID 1 System: RAID 0 > 250.12 MB - > […] > > But still if if can be arbitrarily long due to that per object replication > config, a vertical output might and leaving graphical representation to a > Qt Quick application or so might be better.Yes, this is my same feel: For console I prefer a text representation in rows, leaving to a graphical GUI to show the information in columns.. -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli:> > What was your reasoning for not using options to btrfs filesystem df? > > That df doesn´t show more than "disk free" as well? > > My feel is that a switch should change "a bit" a command. In this case > there are very different outputs, for different purposes (how is used a > disk ? where are the chunks ? how many free space I have ? ).Hmm. The question whether how these are all related to the question on how much space is free? Are they related? Then a single command makes sense. Are they completely different outputs? Then separate commands make sense. I think, before continuing to argue over the details of the output and options, it would be good to get the use cases clear. My first and most important use case would be: How much space is used and more importantly how much space is still free? A further use case would be: Is a some disk on my BTRFS RAID limiting the usable space by being too small in compared to other disks? And then: Is there a imbalance in disk allocation that prevents me from using all space and that I can have fixed by doing a balance operation? Maybe also: What would be a good way to extend the usable space in my BTRFS given the current disks, their sizes and the current allocation strategies? Could even be a simulation. btrfs filesystem df -a /dev/sdi1 and then it shows how usable size would change if /dev/sdi1 was added to the fs. And then subvolume related questions like: Which subvolumes are using up all this disk space? Anything else? With the use cases in mind we can work out what a good output would be by asking ourselves: Does the output(s) answer the questions from the use cases? Given above use cases, I think its most important to get the summary view into btrfs filesystem df and then go from there for additional details provided by options or different commands. Given that the summary view IMHO was quite uncontroversial I think it can go in already while we discuss the other stuff, can´t it? IMHO it would then be good to also fix regular df to provide a free space like in the summary view. Maybe by being conservative and using the smallest estimation. But mabye its just me who never really understood why a 100 GB BTRFS RAID1 shows 200GB of free space and a 100 MB file on it occupies 200MB. Currently the regular df output, while technically somewhat correct, IMHO is just of no good use for anyone using a BTRFS raid. Especially with applications that use df value to determine how much data to place on a volume like fscache or other caches. Hmmm, however these will work if using a percentage, cause that would still be correct, cause 180 GB of 200 GB is still 90% like 90 GB of 100GB. But they would over-allocate if being told to leave 10 GB free or so. Ciao, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 28 Oct 2012 11:59 +0100, from kreijack@gmail.com (Goffredo Baroncelli):> On 2012-10-28 11:38, Martin Steigerwald wrote: >> But still if if can be arbitrarily long due to that per object replication >> config, a vertical output might and leaving graphical representation to a >> Qt Quick application or so might be better. > > Yes, this is my same feel: For console I prefer a text representation in > rows, leaving to a graphical GUI to show the information in columns..So a sysadmin who logs on to a server (which uses btrfs) over plain SSH should need to install a good chunk of X11 plus a load of support libraries on the server, plus have a X11 server on the machine they are connecting from, simply to get a tabular view of the disk space? That _might_ fly in a strictly *nix environment. In a mixed *nix/Windows (and maybe even throw in some Mac OS X) environment, not necessarily as much so. And what about the sysadmin on the road who logs on using a smartphone SSH client to get a quick overview of the disk usage situation because they received an alert that disk space is starting to run low? It appears to me that what is being discussed here is, to some extent, an edge case of use of a file system feature that hasn''t even been implemented yet, but rather is only planned. Like Martin pointed out, scripts should not parse meant-for-human-consumption output anyway, and beyond trivial greps and that kind of parsing, tabular data by design lends itself rather poorly to script parsing anyway. I would personally really prefer a tabular view that perhaps does not show _all_ details when no explicit parameters are given, and a "--dump" or similar that will give a fuller data dump that might lend itself better to parsing by other tools for a more thorough presentation. The data is all right there; it''s basically a matter of deciding how much of it to show by default. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 27 Oct 2012 17:35 -0600, from lists@colorremedies.com (Chris Murphy):> On Oct 27, 2012, at 1:55 PM, Michael Kjörling <michael@kjorling.se> wrote: > >> Data: RAID 0 System: RAID 1 Unused >> /dev/vdb 307.25 MB - 2.23 GB >> /dev/vdc 307.25 MB 8 MB 2.69 GB >> /dev/vdd 307.25 MB 8 MB 2.24 GB >> ============ ============== ===========>> TOTAL 921.75 MB 16 MB 7.16 GB > > I kinda like this arrangement a little better. If the top left > corner could be the volume label, that might be nice?I see no reason why that couldn''t be done. It might be tricky if the volume label is long, but then it could just as easily as the device names be printed on a line of its own. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-28 12:18, Michael Kjörling wrote:> On 28 Oct 2012 11:59 +0100, from kreijack@gmail.com (Goffredo Baroncelli): >> On 2012-10-28 11:38, Martin Steigerwald wrote: >>> But still if if can be arbitrarily long due to that per object replication >>> config, a vertical output might and leaving graphical representation to a >>> Qt Quick application or so might be better. >> >> Yes, this is my same feel: For console I prefer a text representation in >> rows, leaving to a graphical GUI to show the information in columns.. > > So a sysadmin who logs on to a server (which uses btrfs) over plain > SSH should need to install a good chunk of X11 plus a load of support > libraries on the server, plus have a X11 server on the machine they > are connecting from, simply to get a tabular view of the disk space?As reported in another email, I like the tabular format. The point is that it is not usable because the combination of Chunk-type (Data, Metadata, System) and Profile (DUP, Single, RAID1, RAID10, RAID0...RAID5/6) doesn''t fit in 80 columns width. Even tough the simple case (Only one profile, Data,Metadata,System chunk type) could fit, this solution doesn''t scale when will be possible to select different profiles per subvolume basis. [...]> > I would personally really prefer a tabular view that perhaps does not > show _all_ details when no explicit parameters are given, and a > "--dump" or similar that will give a fuller data dump that might lend > itself better to parsing by other tools for a more thorough > presentation. The data is all right there; it''s basically a matter of > deciding how much of it to show by default.In your opinion, which details we could omit ?>-- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 28 Oct 2012 13:25 +0100, from kreijack@inwind.it (Goffredo Baroncelli):> On 2012-10-28 12:18, Michael Kjörling wrote: >> I would personally really prefer a tabular view that perhaps does not >> show _all_ details when no explicit parameters are given, and a >> "--dump" or similar that will give a fuller data dump that might lend >> itself better to parsing by other tools for a more thorough >> presentation. The data is all right there; it''s basically a matter of >> deciding how much of it to show by default. > > In your opinion, which details we could omit ?For a _summary_, _I_ would want an overview that tells me: 1. how much data is actually currently stored on the file system (which might not be equal to the number of data bytes used on disk) 2. if I add a single file to the file system, how large can that file grow before I run out of disk space? ("how much free space is there on the file system?") In the case of btrfs, this should probably be calculated under a "worst case scenario" presumption but still assuming that all currently unused space is usable for data storage and no changed file system settings, and without rebalancing between data and metadata areas. For multi-device file systems, it seems logical to offer a per-device breakdown of the numbers as well as totals. In that case the totals should definitely be at the bottom of the output so you don''t need to pipe through "head" just to get the totals if you have a file system spread out over a large number of devices. Since unless I''ve missed something big per-object replication settings is a _planned_ feature and not currently implemented, I don''t quite see why the "btrfs fi df" output design _now_ has to accomodate that. I still feel that the format I mentioned in my post 27 Oct 19:55 UTC should work fairly well, given the _currently implemented_ btrfs feature set. It answers the question that "home" users are most likely to face ("how much more data can I store?") and can be supplemented by a more in-depth view meant for technically inclined users who really care about the technical breakdown of the data. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli:> On 2012-10-28 12:18, Michael Kjörling wrote: > > On 28 Oct 2012 11:59 +0100, from kreijack@gmail.com (GoffredoBaroncelli):> >> On 2012-10-28 11:38, Martin Steigerwald wrote: > >>> But still if if can be arbitrarily long due to that per object > >>> replication config, a vertical output might and leaving graphical > >>> representation to a Qt Quick application or so might be better. > >> > >> Yes, this is my same feel: For console I prefer a text > >> representation in rows, leaving to a graphical GUI to show the > >> information in columns.. > > > > > > > > So a sysadmin who logs on to a server (which uses btrfs) over plain > > SSH should need to install a good chunk of X11 plus a load of support > > libraries on the server, plus have a X11 server on the machine they > > are connecting from, simply to get a tabular view of the disk space? > > As reported in another email, I like the tabular format. The point is > that it is not usable because the combination of Chunk-type (Data, > Metadata, System) and Profile (DUP, Single, RAID1, RAID10, > RAID0...RAID5/6) doesn''t fit in 80 columns width. > Even tough the simple case (Only one profile, Data,Metadata,System > chunk type) could fit, this solution doesn''t scale when will be > possible to select different profiles per subvolume basis. > > [...]The vertical output makes it quite difficult to spot unbalanced drives tough. A simplified output might help: Just show how much is used/unused on each drive. This helps the usecase to spot unbalanced drives, I think. And it can fit in tabular output. I think its good to go from the use cases. What questions may the user have and how to answer them in a clear and concise way? Then for details the vertical output might be a good option. Anyway, I think first go for the summary thing… since I think that is clear. -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 28, 2012, at 5:16 AM, Martin Steigerwald <Martin@lichtvoll.de> wrote:> The question whether how these are all related to the question on how much > space is free?This is my main confusion with ''filesystem df'' is that it''s very circumspect about this. For a way longer time than I think is OK, I''d use ''fi show'' expecting to see single, raid0, raid1 information, but of course that''s in df. Whereas I find I get more useful, though still circumspect, usage information from which to infer free space from fi show than fi df. Small problem.> IMHO it would then be good to also fix regular df to provide a free space > like in the summary view. Maybe by being conservative and using the > smallest estimation. But mabye its just me who never really understood why > a 100 GB BTRFS RAID1 shows 200GB of free space and a 100 MB file on it > occupies 200MB.It''s a perspective. I think we''re all trained to see file size as absolute, even when the file is in effect duplicated. I think it''s possible we need to be retrained to look at this from a storage point of view. The storage "pool" is 200GB in size, and data allocation is 2x in raid1. I find new users intuit this, rather than thinking the file size is no different, there in a sense no second copy, but the file system is half the size. With conventional raid, this logic sorta works because the raid isn''t a file system, so the filesystem really wasn''t twice the size. But in btrfs it is. Same for ZFS for that matter. When considering how du will work down the road with per subvolume, and possibly per file (or folder) redundancy levels, I think everything should be treated literally. The volume/pool is the total size of storage. And then the "disk usage" hit is file size times a redundancy factor. For single and raid0, 1x. For raid1, 2x. If mixed raid5/6 is one day possible too, then it can have a variable factor depending on the stripe/parity ratio. In any case, the *allocation* required for each file is really its size in the file system. How many bytes are in the file is a different inquiry anyway due to metadata which is not part of the file contents anyway. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 28 Oct 2012 12:27 -0600, from lists@colorremedies.com (Chris Murphy):> I think we''re all trained to see file size as absolute, even when > the file is in effect duplicated. I think it''s possible we need to > be retrained to look at this from a storage point of view. The > storage "pool" is 200GB in size, and data allocation is 2x in raid1. > I find new users intuit this, rather than thinking the file size is > no different, there in a sense no second copy, but the file system > is half the size. With conventional raid, this logic sorta works > because the raid isn''t a file system, so the filesystem really > wasn''t twice the size. But in btrfs it is. Same for ZFS for that > matter.This makes a valid point. And there''s another issue potentially at play: file system level compression, _which btrfs already supports_. Say you store a highly compressible file (a web server access log, perhaps) on a compressed volume that is set up with RAID 1 across a set of physical drives. You know the pool size, and you know the data allocation ratio. What you _don''t_ know and can''t predict is the _compression_ ratio for any new data, which means that you still can''t calculate how much more payload data can be added to the file before you run out of storage space. You _can_ calculate a worst case scenario figure (no compression possible on the new payload data), but that''s about it. _However_, while thinking in terms of storage pool sizes, allocation ratios and so forth might work for technically inclined people, I''m not so sure it''ll work for ordinary users. But then again: * How many ordinary users are going to use multi-device file systems in the first place? Most ordinary users I know have only a single internal disk for primary storage. Even the tiny subset of those who use Linux aren''t magically going to add several disks to their PCs to create a storage pool encompassing multiple physical devices just because the file system supports it. * "btrfs" _is_ the file system administration tool. In a way, it makes sense that the data provided by it will be geared more toward technically minded people. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 28, 2012, at 1:06 PM, Michael Kjörling <michael@kjorling.se> wrote:> > This makes a valid point. And there''s another issue potentially at > play: file system level compression, _which btrfs already supports_. > Say you store a highly compressible file (a web server access log, > perhaps) on a compressed volume that is set up with RAID 1 across a > set of physical drives. You know the pool size, and you know the data > allocation ratio. What you _don''t_ know and can''t predict is the > _compression_ ratio for any new data, which means that you still can''t > calculate how much more payload data can be added to the file before > you run out of storage space. You _can_ calculate a worst case > scenario figure (no compression possible on the new payload data), but > that''s about it.Btrfs definitely makes absolute prediction of free space remaining difficult. I''m fine with a verbose feature proposing various "free space scenarios" based on compression and redundancy, however I think a summary version can do better. Better being defined as: relatively easy to understand, not a lie, but not the whole truth, and subject to change if you deviate from the ability of the system to estimate. As the file system is actually used, there is pretty good statistical information, which should get better the more full it gets, how it''s being used and therefore how much time to fullness. And I think it''s fine for this to vary somewhat based on use. You could have: Storage remaining current usage: 3 hours average usage: 3 weeks So that''s a file system with a lot being written to it currently, way more than average. Storage remaining current usage: 35 days average usage: 40 days You could even include a switch that allows enterprise to assume regularly added storage. Computing an average data allocation rate allows a prediction for when the file system will be full, assuming that average allocation trend. So an optional switch can include the assumption that storage will be grown at a certain rate: e.g. 3TB per month. Based on both allocation and growth assumptions, storage remaining as time becomes even more useful than a seemingly "raw" percentage that fluctuates. I think human''s are a lot more forgiving of time estimates being pliable than percentages anyway, although I don''t know why. Anyway, eventually the file system could stop just short of issuing its own purchase orders for the drives it needs to maintain a reasonable current/average usage value. Ergo, when the file system is young and mostly empty, the estimate may be really off. But once this statistic actually starts becoming important, on the other side of 50%, I think the trend is fairly well established. So it''s kindof a no consequence estimate early on.> _However_, while thinking in terms of storage pool sizes, allocation > ratios and so forth might work for technically inclined people, I''m > not so sure it''ll work for ordinary users. But then again:Agreed. I wouldn''t expose that to them. At least not in the summary. But underlying, the ratios are in play, and affect the summary information users are given rather than purely raw information, which I think is useless for ordinary users.> * How many ordinary users are going to use multi-device file systems > in the first place?It''s a very interesting question. What''s the role of mobile in all of this too, if Gartner is even remotely in the ballpark for latest estimates of (just) Android devices equalling (then surpassing) Windows devices in 2015? However, sticking to desktop for now, with SSD users can form factor wise much more easily have a raid1 setup; or much more likely, -d single profile to add on more storage, while nothing else file system wise changes. They just have more space. They don''t have to move things around, establish a new mount points, etc. Or even screw with LVM. It hardly gets easier than ''device add X Y'' done.> > * "btrfs" _is_ the file system administration tool. In a way, it makes > sense that the data provided by it will be geared more toward > technically minded people.It is a fair point. But this is not LVM or md raid. It''s way easier to understand and manage already, even with some trouble spots. But in the area of storage pool/volume management it''s perhaps potentially more complex due to subvolumes and actually usable snapshots. So, I must refuse part of the premise that sysadmins and storage experts aren''t also ordinary people. They can benefit too from useful "at a glance" information that, again, doesn''t mislead them. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 28, 2012, at 1:06 PM, Michael Kjörling <michael@kjorling.se> wrote:> > This makes a valid point. And there''s another issue potentially at > play: file system level compression, _which btrfs already supports_. > Say you store a highly compressible file (a web server access log, > perhaps) on a compressed volume that is set up with RAID 1 across a > set of physical drives. You know the pool size, and you know the data > allocation ratio. What you _don''t_ know and can''t predict is the > _compression_ ratio for any new data, which means that you still can''t > calculate how much more payload data can be added to the file before > you run out of storage space. You _can_ calculate a worst case > scenario figure (no compression possible on the new payload data), but > that''s about it.Btrfs definitely makes absolute prediction of free space remaining difficult. I''m fine with a verbose feature proposing various "free space scenarios" based on compression and redundancy, however I think a summary version can do better. Better being defined as: relatively easy to understand, not a lie, but not the whole truth, and subject to change if you deviate from the ability of the system to estimate. As the file system is actually used, there is pretty good statistical information, which should get better the more full it gets, how it''s being used and therefore how much time to fullness. And I think it''s fine for this to vary somewhat based on use. You could have: Storage remaining current usage: 3 hours average usage: 3 weeks So that''s a file system with a lot being written to it currently, way more than average. Storage remaining current usage: 35 days average usage: 40 days You could even include a switch that allows enterprise to assume regularly added storage. Computing an average data allocation rate allows a prediction for when the file system will be full, assuming that average allocation trend. So an optional switch can include the assumption that storage will be grown at a certain rate: e.g. 3TB per month. Based on both allocation and growth assumptions, storage remaining as time becomes even more useful than a seemingly "raw" percentage that fluctuates. I think human''s are a lot more forgiving of time estimates being pliable than percentages anyway, although I don''t know why. Anyway, eventually the file system could stop just short of issuing its own purchase orders for the drives it needs to maintain a reasonable current/average usage value. Ergo, when the file system is young and mostly empty, the estimate may be really off. But once this statistic actually starts becoming important, on the other side of 50%, I think the trend is fairly well established. So it''s kindof a no consequence estimate early on.> _However_, while thinking in terms of storage pool sizes, allocation > ratios and so forth might work for technically inclined people, I''m > not so sure it''ll work for ordinary users. But then again:Agreed. I wouldn''t expose that to them. At least not in the summary. But underlying, the ratios are in play, and affect the summary information users are given rather than purely raw information, which I think is useless for ordinary users.> * How many ordinary users are going to use multi-device file systems > in the first place?It''s a very interesting question. What''s the role of mobile in all of this too, if Gartner is even remotely in the ballpark for latest estimates of (just) Android devices equalling (then surpassing) Windows devices in 2015? However, sticking to desktop for now, with SSD users can form factor wise much more easily have a raid1 setup; or much more likely, -d single profile to add on more storage, while nothing else file system wise changes. They just have more space. They don''t have to move things around, establish a new mount points, etc. Or even screw with LVM. It hardly gets easier than ''device add X Y'' done.> > * "btrfs" _is_ the file system administration tool. In a way, it makes > sense that the data provided by it will be geared more toward > technically minded people.It is a fair point. But this is not LVM or md raid. It''s way easier to understand and manage already, even with some trouble spots. But in the area of storage pool/volume management it''s perhaps potentially more complex due to subvolumes and actually usable snapshots. So, I must refuse part of the premise that sysadmins and storage experts aren''t also ordinary people. They can benefit too from useful "at a glance" information that, again, doesn''t mislead them. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 28 Oct 2012 13:42 -0600, from chris@colorremedies.com (Chris Murphy):>> * "btrfs" _is_ the file system administration tool. In a way, it makes >> sense that the data provided by it will be geared more toward >> technically minded people. > > It is a fair point. But this is not LVM or md raid. It''s way easier > to understand and manage already, even with some trouble spots. But > in the area of storage pool/volume management it''s perhaps > potentially more complex due to subvolumes and actually usable > snapshots. > > So, I must refuse part of the premise that sysadmins and storage > experts aren''t also ordinary people. They can benefit too from > useful "at a glance" information that, again, doesn''t mislead them.I did not mean to imply that "sysadmins and storage experts aren''t also ordinary people" who can benefit from an "at a glance" view of their storage situation. Quite to the contrary, that''s one point I have raised in this thread: the usefulness of an "at a glance" view of the storage situation. But I can also certainly see the side of the argument that they could reason their way around the concept of raw storage capacity and allocation multipliers. After all, by the time you get down to the "a few days of ordinary added data" range of free space remaining, which would be when the exact numbers _really_ begin to matter, you should be on top of adding more storage capacity anyway. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 28, 2012, at 2:09 PM, Michael Kjörling <michael@kjorling.se> wrote:> I did not mean to imply that "sysadmins and storage experts aren''t > also ordinary people" who can benefit from an "at a glance" view of > their storage situation. Quite to the contrary, that''s one point I > have raised in this thread: the usefulness of an "at a glance" view of > the storage situation. But I can also certainly see the side of the > argument that they could reason their way around the concept of raw > storage capacity and allocation multipliers. After all, by the time > you get down to the "a few days of ordinary added data" range of free > space remaining, which would be when the exact numbers _really_ begin > to matter, you should be on top of adding more storage capacity > anyway.Agreed. Whether the free space is GB, %, or time - in any case if it jumps from a "comfortable" value to a "crisis" value inside of a few days, it''s a problem (for the user and the algorithm). Somehow for large storage I like the idea of time to "full." GB and TB of storage is not really what we care about, it just seems that way because it''s what we''re used to. (Not to imply that we don''t care about GB and TB at all, but I do think most people convert this into a "how many more movies, how much more database growth" and how much time remaining is just a generic variation on those things. GB/TB remaining doesn''t do that, *and* just by nature of the term it connotes an absolute value with inflexible meaning. But free space on btrfs is flexible/relative, not absolute. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 28 Oct 2012 14:19 -0600, from lists@colorremedies.com (Chris Murphy):> Somehow for large storage I like the idea of time to "full." GB and > TB of storage is not really what we care about, it just seems that > way because it''s what we''re used to. (Not to imply that we don''t > care about GB and TB at all, but I do think most people convert this > into a "how many more movies, how much more database growth" and how > much time remaining is just a generic variation on those things. > GB/TB remaining doesn''t do that, *and* just by nature of the term it > connotes an absolute value with inflexible meaning. But free space > on btrfs is flexible/relative, not absolute.There are two fairly big issues however that I can see having thought a little about this that will need careful consideration before deciding to go with a "time to full" scheme. First, what if disk usage is actually decreasing over time? It should be trivial to translate that to, say, something like "> 3 years", but it''s a case that needs to be taken into consideration. Second, there should still be a way to answer the question "will this amount of payload data fit here?". There are numerous valid reasons for wanting to know whether you can store a given number of bytes in a specific place. Of course, depending on various factors (compression etc.) the amount of available space may decrease by less than the size of the payload data, but that is only a nice bonus in that case. At work, it isn''t uncommon for my disk usage to vary up _and down_ by several GB (on the order of 10% or so of the total used is far from uncommon) inside the scope of a day. I''d like to see the statistical algorithm that can take such wild fluctuations and produce any meaningful metric of the amount of remaining space expressed as "time to full". *Perhaps*, to accomodate per-object replication settings, there could be a command like "display free space for this directory" which can take the specific directory''s replication settings into account once that has been implemented. It could display two figures: the amount of payload data that could be stored in that directory without touching any replication settings ("if I do ''cat /dev/random > ./here'', how big will the file become before I run out of space?"), as well as the number of data bytes available (think "how big will it become if I force SINGLE?"). Might that be a workable approach? -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/28/2012 03:06 PM, Michael Kjörling wrote:> You_can_ calculate a worst case > scenario figure (no compression possible on the new payload data), but > that''s about it.Well, there are those humorous edge cases where you compress a tiny amount of uncompressible data, and the "compressed" version ends up being larger. Not sure if that applies to the specific algorithms that btrfs uses, but I''ve seen it happen with tiny datasets before. Even worst case could be hard to get right perhaps! -- R -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all,
this is a new attempt to improve the output of the command "btrfs fi
df".
The previous attempt received a good reception. However there was no a
general consensus about the wording.
Moreover I still didn''t understand how btrfs was using the disks.
A my first attempt was to develop a new command which shows how the
disks are divided in chunks.
Then after some thoughts I made three commands:
$ btrfs filesystem df <path>
# btrfs device disk-usage <path>
# btrfs filesystem disk-usage <path>
The first one shows only the section that before I called "Summary":
$ ./btrfs filesystem df /mnt/btrfs1/
Disk_size: 21.00GB
Disk_allocated: 1.83GB
Disk_unallocated: 19.17GB
Used: 284.00KB
Free_(Estimated): 15.76GB (Max: 20.54GB, min: 10.96GB)
Data_to_disk_ratio: 75 %
The second one show the chunk list grouped by devices:
$ sudo ./btrfs device disk-usage /mnt/btrfs1/
/dev/vdb 3.00GB
Data,Single: 8.00MB
Data,RAID0: 307.25MB
Metadata,Single: 8.00MB
Metadata,RAID1: 460.94MB
System,Single: 4.00MB
Unallocated: 2.23GB
/dev/vdc 3.00GB
Data,RAID0: 307.25MB
System,RAID1: 8.00MB
Unallocated: 2.69GB
/dev/vdd 3.00GB
Data,RAID0: 307.25MB
Metadata,RAID1: 460.94MB
System,RAID1: 8.00MB
Unallocated: 2.24GB
/dev/vdf 12.00GB
Unallocated: 12.00GB
The third one shows the devices grouped by chunk type; it has two kind
of output the first kind of output is a linear one (as before); the
second kind of output is in a tabular form. I don''t like it because it
is suitable only when the filesystem uses few profile; otherwise this
output is larger than 80 columns.
$ sudo ./btrfs filesystem disk-usage /mnt/btrfs1/
Data,Single: Size:8.00MB, Used:0.00
/dev/vdb 8.00MB
Data,RAID0: Size:921.75MB, Used:256.00KB
/dev/vdb 307.25MB
/dev/vdc 307.25MB
/dev/vdd 307.25MB
Metadata,Single: Size:8.00MB, Used:0.00
/dev/vdb 8.00MB
Metadata,RAID1: Size:460.94MB, Used:24.00KB
/dev/vdb 460.94MB
/dev/vdd 460.94MB
System,Single: Size:4.00MB, Used:0.00
/dev/vdb 4.00MB
System,RAID1: Size:8.00MB, Used:4.00KB
/dev/vdc 8.00MB
/dev/vdd 8.00MB
Unallocated:
/dev/vdb 2.23GB
/dev/vdc 2.69GB
/dev/vdd 2.24GB
/dev/vdf 12.00GB
$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
Data Data Metadata System System
Single RAID0 Single Single RAID1 Unused
vdb 8.00MB 307.25MB 8.00MB 4.00MB - 2.23GB
vdc - 307.25MB - - 8.00MB 2.69GB
vdd - 307.25MB - - 8.00MB 2.24GB
vdf - - - - - 12.00GB
========= ========= ========= ========= ========= ========Total
8.00MB 921.75MB 8.00MB 4.00MB 8.00MB 19.17GB
Used 0.00 256.00KB 0.00 0.00 4.00KB
The latest output is a fake, because the original one requires 78-79
columns, so the email reader would rewrap it. I had to remove a column
(Metadata,RAID1) : this confirm me that this kind of output is
unusable as general case.
The code is pullable from
http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git
branch
info-cmd
I don''t publish the patches because aren''t in a good shape.
The
example is a filesystem based on three disks of 3GB and one disk of 12 GB.
Comments are welcome.
Known bugs:
- - the commands btrfs fi disk-usage and btrfs device disk-usage need
root capability (I use the BTRFS_IOC_TREE_SEARCH
to get the chunk info; so that is unavoidable)
- --
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBAgAGBQJQjwFeAAoJEA7amzeLguC1jtEP/1tzU9s2mfLhz1AVotYW3oWd
hnVNdBmz/E+uGJYTY99heB/Vmnu2dZUQyKlbUilAM7XtQw7GsEITGa+6peoPIPrI
8v0mYL+e/Bd24caMS7DlRR1cJ9lqWlJg6lxwGryVYCyYcFIkkRhw5PcFAxOyF2xe
Q0aESR8a4CA7NBMXH6QeTLD7oSZAnIFAK/EDYbExweJ1a+OSMzgJ+CMWmlaOf79v
R90c9qg309dUctHO0mB/D5XjJlFwNaGJpNsxEvBJdkZl0VgHxm1unsaxYLFU5ybk
/qJs7yqJIFk9BtV15zREtuC7n+CC5Mwo/gPOP24qyZazfLgMv72NaOTmDdiXuNpV
vjVgAgNSRL5JSI6RTQEgJwy2xgPJA/k0z4R20xpXRn0+E2OHDSKk/+gqmCE3Iieb
3LcJBuEPSQxpx6R5RubNhwauu3h6Dn4qBIUSagH6zYz+AUwkfcUTGdS3vgrXfEUf
UNGn3wEcz8veIr66cOm3t6pTcRInFOIvOf/32Bei1SZyEiiX0pzxmSWAutaOiRNe
m+sjWhFF3QW6wKX25QRXFgqnFJQwpx+9sP90dQvTh5oOio+xbi+eGZJ8WEMVY6b1
L4iiqY8DtnfufQF2XUGGIa8b8Wxdp15gUoeqPKh2Oy8srEsUjV+g4vwXOMrRtYfR
d4o8YssrtjlpSBQCYHpZ
=TUa5
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 29, 2012, at 3:04 AM, Michael Kjörling <michael@kjorling.se> wrote:> There are two fairly big issues however that I can see having thought > a little about this that will need careful consideration before > deciding to go with a "time to full" scheme. > > First, what if disk usage is actually decreasing over time? It should > be trivial to translate that to, say, something like "> 3 years", but > it''s a case that needs to be taken into consideration.Yes it should.> > Second, there should still be a way to answer the question "will this > amount of payload data fit here?".I agree. Time to full is just one possible way of being more forward thinking with large file systems.> At work, it isn''t uncommon for my disk usage to vary up _and down_ by > several GB (on the order of 10% or so of the total used is far from > uncommon) inside the scope of a day. I''d like to see the statistical > algorithm that can take such wild fluctuations and produce any > meaningful metric of the amount of remaining space expressed as "time > to full".I think you''d something like CPU load balancing stats. An estimate based on today''s usage, vs the past 2 weeks, or something. You wouldn''t get just one estimate. It''d supply a context and thus give you a range of times to full depending on context, which hopefully the user knows something about.> > *Perhaps*, to accomodate per-object replication settings, there could > be a command like "display free space for this directory" which can > take the specific directory''s replication settings into account once > that has been implemented. It could display two figures: the amount of > payload data that could be stored in that directory without touching > any replication settings ("if I do ''cat /dev/random > ./here'', how big > will the file become before I run out of space?"), as well as the > number of data bytes available (think "how big will it become if I > force SINGLE?"). > > Might that be a workable approach?Seems reasonable. Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Michael Kjörling
2012-Oct-30 09:42 UTC
Re: [RFC][V2] New attempt to a better "btrfs fi df"
On 29 Oct 2012 23:21 +0100, from kreijack@gmail.com (Goffredo Baroncelli):> Hi all, > > this is a new attempt to improve the output of the command "btrfs fi df". > > The previous attempt received a good reception. However there was no a > general consensus about the wording.In general, I like it. And I think that especially with today''s large terminals (with frame buffers and X11), one doesn''t need to be too fixated at an 80 column terminal width. One thing, though; what is the difference between "unused" and "unallocated"? If there is no difference, I feel the same word should be used throughout. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Goffredo Baroncelli
2012-Oct-30 18:15 UTC
Re: [RFC][V2] New attempt to a better "btrfs fi df"
On 2012-10-30 10:42, Michael Kjörling wrote:> On 29 Oct 2012 23:21 +0100, from kreijack@gmail.com (Goffredo Baroncelli): >> Hi all,[...]> One thing, though; what is the difference between "unused" and > "unallocated"? If there is no difference, I feel the same word should > be used throughout.I had to use "Unused" instead of "Unallocated" because the first one is shorter. In the tabular output of the command "btrfs fi disk-usage" I have 10 columns available for every fields. 1 columns is used by the space character, so I have only 9 characters available for the description. "Unused" is 6 chars, "Unallocated" is 11 chars.... The only options is to use "Unalloc." (8 chars)... -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Michael Kjörling
2012-Oct-30 18:32 UTC
Re: [RFC][V2] New attempt to a better "btrfs fi df"
On 30 Oct 2012 19:15 +0100, from kreijack@gmail.com (Goffredo Baroncelli):> On 2012-10-30 10:42, Michael Kjörling wrote: >> what is the difference between "unused" and "unallocated"? If there >> is no difference, I feel the same word should be used throughout. > > I had to use "Unused" instead of "Unallocated" because the first one is > shorter. In the tabular output of the command "btrfs fi disk-usage" I > have 10 columns available for every fields. 1 columns is used by the > space character, so I have only 9 characters available for the > description. "Unused" is 6 chars, "Unallocated" is 11 chars.... > The only options is to use "Unalloc." (8 chars)...Or to simply use "unused" (or some other word like "free", "available", ...) throughout. I''m not so much concerned about the exact word being used as I feel the same word should be used throughout a UI to describe the same concept. Whether it''s called "free" space, "unused" space, "unallocated" space or "fuzzbar''d" space is really beside the point. -- Michael Kjörling • http://michael.kjorling.se • michael@kjorling.se “People who think they know everything really annoy those of us who know we don’t.” (Bjarne Stroustrup) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Oct 30, 2012, at 12:32 PM, Michael Kjörling <michael@kjorling.se> wrote:> > > I''m not so much concerned about the exact word being used as I feel > the same word should be used throughout a UI to describe the same > concept. Whether it''s called "free" space, "unused" space, > "unallocated" space or "fuzzbar''d" space is really beside the point.Agreed. If there''s some meaningful distinction, then different words should be used. There is the small vagueness of what exactly is "used". So if you say "50GB used" what exactly does this mean? Because btrfs fi show regularly shows me two completely different used values and they''re significantly different (in value) so while the word "used" may be applicable, what''s missing is the "what". Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html