These are the btrfs-tools versions on Debian: squeeze: kernel: 2.6.32 tools: 0.19+20100601-3 squeeze-backports: kernel: 2.6.39 tools: nothing (so user ends up with 0.19+20100601-3) wheezy/testing/sid: kernel: 3.1.6-1 tools: 0.19+20111105-2 Using the 2.6.39 kernel from squeeze-backports, do I need a newer btrfs-tools and is there a particular reason it is not in squeeze-backports too? Or should I not be trying to use the versions in squeeze at all - should I be on testing/wheezy or unstable? The Debian btrfs wiki and the regular btrfs wiki don''t really suggest a good starting point (other than suggesting the btrfs has been in Debian since squeeze) http://wiki.debian.org/Btrfs If I try to use the version from testing or unstable, I get this error on a squeeze setup: btrfs-tools depends on e2fslibs (>= 1.41.99); however: Version of e2fslibs on system is 1.41.12-4stable1. http://packages.debian.org/search?keywords=btrfs-tools -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 02, 2012 at 03:29:49PM +0100, Daniel Pocock wrote:> These are the btrfs-tools versions on Debian: > > squeeze: > kernel: 2.6.32 > tools: 0.19+20100601-3 > > squeeze-backports: > kernel: 2.6.39 > tools: nothing (so user ends up with 0.19+20100601-3) > > wheezy/testing/sid: > kernel: 3.1.6-1 > tools: 0.19+20111105-2 > > Using the 2.6.39 kernel from squeeze-backports,Don''t do that. It''s pretty old, and there''s been a large number of fixes in btrfs since then. I would recommend using the very latest kernel you can -- preferably an -rc series kernel (after -rc3 or so, though), or the latest release kernel.> do I need a newer btrfs-toolsThe only real reason to upgrade your btrfs tools is to get access to the newer features of the FS.> and is there a particular reason it is not in squeeze-backports too?You''ll have to ask the backports people about that. The btrfs developers don''t have any control over what ends up in most distributions.> Or should I not be trying to use the versions in squeeze at all - should > I be on testing/wheezy or unstable?You should definitely be on 3.1 or 3.2 kernel at the moment.> The Debian btrfs wiki and the regular btrfs wiki don''t really suggest a > good starting point (other than suggesting the btrfs has been in Debian > since squeeze) > http://wiki.debian.org/Btrfs > > If I try to use the version from testing or unstable, I get this error > on a squeeze setup: > > btrfs-tools depends on e2fslibs (>= 1.41.99); however: > Version of e2fslibs on system is 1.41.12-4stable1.In that case, I''d suggest either building from the git source tree for tools (see the wiki[1] for instructions), or, if you insist on having .deb packages, doing your own backport: $ apt-get build-dep btrfs-tools $ apt-get source -t unstable btrfs-tools $ cd btrfs-tools-0.19+20111105 $ fakeroot debian/rules $ sudo dpkg -i ../btrfs-tools-0.19+20111105.deb # This may be a different name Hugo. [1] http://btrfs.ipv5.de/index.php?title=Btrfs_source_repositories#Official_repository -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Anyone using a computer to generate random numbers is, of --- course, in a state of sin.
On Mon, Jan 2, 2012 at 8:29 AM, Daniel Pocock <daniel@pocock.com.au> wrote:> > > These are the btrfs-tools versions on Debian: > > squeeze: > kernel: 2.6.32 > tools: 0.19+20100601-3 > > squeeze-backports: > kernel: 2.6.39 > tools: nothing (so user ends up with 0.19+20100601-3) > > wheezy/testing/sid: > kernel: 3.1.6-1 > tools: 0.19+20111105-2 > > Using the 2.6.39 kernel from squeeze-backports, do I need a newer > btrfs-tools and is there a particular reason it is not in > squeeze-backports too? > > Or should I not be trying to use the versions in squeeze at all - should > I be on testing/wheezy or unstable? > > The Debian btrfs wiki and the regular btrfs wiki don''t really suggest a > good starting point (other than suggesting the btrfs has been in Debian > since squeeze) > http://wiki.debian.org/Btrfs > > If I try to use the version from testing or unstable, I get this error > on a squeeze setup: > > btrfs-tools depends on e2fslibs (>= 1.41.99); however: > Version of e2fslibs on system is 1.41.12-4stable1.0.19+20111105-2 should be sufficiently up to date for most day-to-day purposes; the particular dependency you''re running up against is probably a quirk of the packaging. Note that you really want to be running the latest kernel possible if using btrfs; since 2.6.39 there have been several major performance fixes, stability fixes, crash-corruption fixes, which users did hit on a somewhat regular basis. Btrfs is not yet stable for the typical user who just wants things to work, even when things don''t. I don''t know of any major distros that offer support services for btrfs filesystems, for instance. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Note that you really want to be running the latest kernel possible if > using btrfs; since 2.6.39 there have been several major performance > fixes, stability fixes, crash-corruption fixes, which users did hit on > a somewhat regular basis. Btrfs is not yet stable for the typical > user who just wants things to work, even when things don''t. I don''t > know of any major distros that offer support services for btrfs > filesystems, for instance.I''m not planning to run my whole system on btrfs just yet - but I was keen to start running one or two test filesystems on a server that I currently have running Debian squeeze. Everything else on the server has to remain as-is if possible. Thanks for the feedback - I will start looking into the newer kernels and see if I can use one of them with squeeze, or maybe I will just set up a VM for btrfs One thing I''ve already noticed in 2.6.39 (and both versions of the tools) is that df results are misleading. E.g. if I run regular df (not btrfs fi df), I am seeing the same amount of available space for all filesystems. Is there currently a way to see space used by each subvolume and snapshot and which kernel and tools versions might be needed? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 02, 2012 at 04:01:48PM +0100, Daniel Pocock wrote:> One thing I''ve already noticed in 2.6.39 (and both versions of the > tools) is that df results are misleading. E.g. if I run regular df (not > btrfs fi df), I am seeing the same amount of available space for all > filesystems. Is there currently a way to see space used by each > subvolume and snapshot and which kernel and tools versions might be needed?It''s actually not possible in general. Since it''s possible to have different bits of the FS (data vs metadata) with different replication levels, one byte written to the FS could take up either 1 or 2 bytes of raw disk storage, and there''s no way of predicting what the overall usage will be, so it''s not possible to give an accurate estimate of free space. Similarly, if you have a 10G subvolume, and a snapshot of it, then how much space does each one take up? If you said 10G, then you''re wrong, because the total storage space for the two together is only 10G. If, on the basis of that, you said 5G each, you''re wrong, because if you delete one of them, you get no free space back. If, on the basis of that, you said 0G each, you''re wrong, because if you delete both of them you get 10G of free space back. The best you can ask is "if I delete this subvolume, how much space will I get back?", but even that''s non-trivial: Arne Jansen is working on that feature as a side-effect of his quota work, and it will be coming at some point, but we can''t do it right now. See the FAQ on filesystem usage[1] for how to interpret the btrfs tools for examining space usage, and the mis-named "sysadmin''s guide"[2] for more horrible details of what''s going on underneath. Hugo. [1] http://btrfs.ipv5.de/index.php?title=FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F [2] http://btrfs.ipv5.de/index.php?title=SysadminGuide -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I''m on a 30-day diet. So far I''ve lost 18 days. ---
> It''s actually not possible in general. Since it''s possible to have > different bits of the FS (data vs metadata) with different replication > levels, one byte written to the FS could take up either 1 or 2 bytes > of raw disk storage, and there''s no way of predicting what the overall > usage will be, so it''s not possible to give an accurate estimate of > free space. > > Similarly, if you have a 10G subvolume, and a snapshot of it, then > how much space does each one take up?Yes, I agree it is not simple However, the lvs command from LVM does provide a useful way to show how full a snapshot volume is. In the btrfs situation, the snapshot is within the FS, so it is not quite the same metric I am looking at what metrics are needed to monitor btrfs in production. I actually look after the ganglia-modules-linux package, which includes some FS space metrics, but I figured that btrfs throws all that out the window. Can you suggest metrics that would be meaningful, do I look in /proc or with syscalls, is there any code I should look at for an example of how to extract them with C? Ideally, Ganglia runs without root privileges too, so please let me know if btrfs will allow me to access them -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 02, 2012 at 05:25:41PM +0100, Daniel Pocock wrote:> > > It''s actually not possible in general. Since it''s possible to have > > different bits of the FS (data vs metadata) with different replication > > levels, one byte written to the FS could take up either 1 or 2 bytes > > of raw disk storage, and there''s no way of predicting what the overall > > usage will be, so it''s not possible to give an accurate estimate of > > free space. > > > > Similarly, if you have a 10G subvolume, and a snapshot of it, then > > how much space does each one take up? > > Yes, I agree it is not simple > > However, the lvs command from LVM does provide a useful way to show how > full a snapshot volume is. In the btrfs situation, the snapshot is > within the FS, so it is not quite the same metricThe way that LVM works is very different to the way that btrfs works. If you try drawing parallels, you will probably go very wrong. LVM snapshots are (IIRC) given a fixed-size pool of blocks on creation, from which blocks are taken when modifications are made to the original, so it''s possible to know how much of the pool has been taken. With btrfs, as you''ve spotted, the pool is the entirety of the unused space in the filesystem, so there''s no limit on snapshot size or the divergence of a snapshot from its original subvolume.> I am looking at what metrics are needed to monitor btrfs in production. > I actually look after the ganglia-modules-linux package, which includes > some FS space metrics, but I figured that btrfs throws all that out the > window. > > Can you suggest metrics that would be meaningful, do I look in /proc or > with syscalls, is there any code I should look at for an example of how > to extract them with C? Ideally, Ganglia runs without root privileges > too, so please let me know if btrfs will allow me to access themIt depends on what you want to know, really. If you want "how close am I to a full filesystem?", then the output of df will give you a measure, even if it could be up to a factor of 2 out -- you can use it for predictive planning, though, as it''ll be near zero when the FS runs out of space. If you really want to, you can get your hands into the filesystem structures on a read-only (and non-locked) basis using the BTRFS_IOC_TREE_SEARCH ioctl, and the FS structures are documented at [1]. However, that''s generally going to be pretty ugly, and most likely pretty slow for many operations at the subvolume level. If you want anything on a per-subvolume basis, then you''ll have to wait for Arne to finish the work on quotas. Hugo. [1] http://btrfs.ipv5.de/index.php?title=Data_Structures -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- The Tao that is seen / Is not the true Tao, until / You --- bring fresh toner.
>> I am looking at what metrics are needed to monitor btrfs in production. >> I actually look after the ganglia-modules-linux package, which includes >> some FS space metrics, but I figured that btrfs throws all that out the >> window. >> >> Can you suggest metrics that would be meaningful, do I look in /proc or >> with syscalls, is there any code I should look at for an example of how >> to extract them with C? Ideally, Ganglia runs without root privileges >> too, so please let me know if btrfs will allow me to access them > > It depends on what you want to know, really. If you want "how close > am I to a full filesystem?", then the output of df will give you a > measure, even if it could be up to a factor of 2 out -- you can use it > for predictive planning, though, as it''ll be near zero when the FS > runs out of space.Maybe if you look at it from the point of the sysadmin and think about what questions he might want to ask: a) how much space would I reclaim if I deleted snapshot X? b) how much space would I reclaim if I deleted all snapshots? c) how much space would I need if I start making 4 snapshots a day and keeping them for 48 hours? (Ganglia also sums data across the enterprise, so if such metrics are implemented, the admin can quickly see the sum of all snapshot usage on his grid/cluster) and also: d) what metrics would be useful for someone developing or testing some solution involving btrfs? The Ganglia framework uses rrdtool to graph metric data and present it alongside other system data (e.g. to see disk IO rates, CPU load, cache activity all on the same graph) so it could provide some useful insights into any performance testing of btrfs. Even better, using rrdtool, you can overlay some btrfs metric on the same graph as a system metric (e.g. IO request sizes)> If you really want to, you can get your hands into the filesystem > structures on a read-only (and non-locked) basis using the > BTRFS_IOC_TREE_SEARCH ioctl, and the FS structures are documented at > [1]. However, that''s generally going to be pretty ugly, and most > likely pretty slow for many operations at the subvolume level. > > If you want anything on a per-subvolume basis, then you''ll have to > wait for Arne to finish the work on quotas. >Initially, I could just start with some simple metric (even just retrieving the btrfs UUID) as a proof-of-concept, and then add more stuff later, for example, when Arne has the quota work in a stable form. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 02.01.2012 16:01, Daniel Pocock wrote:> One thing I''ve already noticed in 2.6.39 (and both versions of the > tools) is that df results are misleading. E.g. if I run regular df (not > btrfs fi df), I am seeing the same amount of available space for all > filesystems. Is there currently a way to see space used by each > subvolume and snapshot and which kernel and tools versions might be needed?If you''re interested (and brave), you might give the subvolume quota patches a try. Arne sent them last October, subject [PATCH v0 00/18] btfs: Subvolume Quota Groups Be warned that this is a v0 patch, it''s not been tested too much and will very likely be reworked in the future. But that kind of functionality will hopefully be added to btrfs, consequently. -Jan -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jan 4, 2012 at 3:48 AM, Daniel Pocock <daniel@pocock.com.au> wrote:> >>> I am looking at what metrics are needed to monitor btrfs in production. >>> I actually look after the ganglia-modules-linux package, which includes >>> some FS space metrics, but I figured that btrfs throws all that out the >>> window. >>> >>> Can you suggest metrics that would be meaningful, do I look in /proc or >>> with syscalls, is there any code I should look at for an example of how >>> to extract them with C? Ideally, Ganglia runs without root privileges >>> too, so please let me know if btrfs will allow me to access them >> >> It depends on what you want to know, really. If you want "how close >> am I to a full filesystem?", then the output of df will give you a >> measure, even if it could be up to a factor of 2 out -- you can use it >> for predictive planning, though, as it''ll be near zero when the FS >> runs out of space. > > > Maybe if you look at it from the point of the sysadmin and think about > what questions he might want to ask: > > a) how much space would I reclaim if I deleted snapshot X? > > b) how much space would I reclaim if I deleted all snapshots? > > c) how much space would I need if I start making 4 snapshots a day and > keeping them for 48 hours?chiming in on the discussion - what I''d like to personally see: First, probably easiest: Display per subvol the space used that is "unique" (not used by other subvolumes), and shared (the opposite - all blocks that appear in other subvolumes as well). From there on, one could potentially create a matrix: (proportional font art, apologies): | subvol1 | subvol2 | subvol3 | ----------+----------+----------+----------+ subvol1 | 200M | 20M | 50M | ----------+----------+----------+----------+ subvol2 | 20M | 350M | 22M | ----------+----------+----------+----------+ subvol3 | 50M | 22M | 634M | ----------+----------+----------+----------+ The diagonal obviously shows the "unique" blocks, subvol2 and subvol1 share 20M data, etc. Missing from this plot would be "how much is shared between subvol1, subvol2, and subvol3" together, but it''s a start and not something that hard to understand. One might add a column for "total size" of each subvol, which may obviously not be an addition of the rest of the columns in this diagram. Anyway, something like this would be high on my list of `df` numbers I''d like to see - since I think they are useful numbers. Cheers, Auke -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> > From there on, one could potentially create a matrix: (proportional > font art, apologies): > > | subvol1 | subvol2 | subvol3 | > ----------+----------+----------+----------+ > subvol1 | 200M | 20M | 50M | > ----------+----------+----------+----------+ > subvol2 | 20M | 350M | 22M | > ----------+----------+----------+----------+ > subvol3 | 50M | 22M | 634M | > ----------+----------+----------+----------+ > > The diagonal obviously shows the "unique" blocks, subvol2 and subvol1 > share 20M data, etc. Missing from this plot would be "how much is > shared between subvol1, subvol2, and subvol3" together, but it''s a > start and not something that hard to understand. One might add a > column for "total size" of each subvol, which may obviously not be an > addition of the rest of the columns in this diagram. > > Anyway, something like this would be high on my list of `df` numbers > I''d like to see - since I think they are useful numbers. >This is an interesting way to look at it Ganglia typically records time series data, it is quite conceivable to create a metric for every permutation in each and store that in rrdtool The challenge would then be in reporting on the data: the rrdtool graphs use time as an X-axis, and then it can display multiple Y values However, now that I''ve started thinking about the type of data generated from btrfs, I was wondering if some kind of rr3dtool is needed - a 3D graphing solution - or potentially making graphs that do not include time on any axis? Has anyone seen anything similar for administering ZFS, for example? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html