Hi again, Having numerous snapshots, I prefer to ask rather than take the risk of exploding my storage space, better safe than sorry ;-) "man btrfs" states : « NOTE: defragmenting with kernels up to 2.6.37 will unlink COW-ed copies of data, don''t use it if you use snapshots, have deduplicated your data or made copies with cp --reflink. » I use : # uname -r 3.5.0-24-generic ...So should I expect that defraging my BTRFS will be smart enough not to uncow my snapshots ? Is it actually able to defrag both the file and its snapshots altogether, keeping all this as a single physical copy of the defragged data ? TIA. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 21, 2013 at 04:46:14PM +0100, Swâmi Petaramesh wrote:> Hi again, > > Having numerous snapshots, I prefer to ask rather than take the risk of > exploding my storage space, better safe than sorry ;-) > > "man btrfs" states : > > « NOTE: defragmenting with kernels up to 2.6.37 will unlink COW-ed > copies of data, don''t use it if you use snapshots, have > deduplicated your data or made copies with cp --reflink. » > > I use : > # uname -r > 3.5.0-24-generic > > ...So should I expect that defraging my BTRFS will be smart enough not > to uncow my snapshots ? Is it actually able to defrag both the file and > its snapshots altogether, keeping all this as a single physical copy of > the defragged data ?Well, there is already a patch which addresses your concern and it''s ''snapshot-aware defrag'' feature and now in v6, it''s not merged yet. thanks, liubo> > TIA. > > -- > Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E > Ne cherchez pas : Je ne suis pas sur Facebook. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2013-02-21 at 16:46 +0100, Swâmi Petaramesh wrote:> Hi again, > > Having numerous snapshots, I prefer to ask rather than take the risk of > exploding my storage space, better safe than sorry ;-) > > "man btrfs" states : > > « NOTE: defragmenting with kernels up to 2.6.37 will unlink COW-ed > copies of data, don''t use it if you use snapshots, have > deduplicated your data or made copies with cp --reflink. » > > I use : > # uname -r > 3.5.0-24-generic > > ...So should I expect that defraging my BTRFS will be smart enough not > to uncow my snapshots ? Is it actually able to defrag both the file and > its snapshots altogether, keeping all this as a single physical copy of > the defragged data ?The message in the btrfs man page is a bit out of date - defragmenting files on btrfs will uncow files on all currently released kernels. There''s a patch available to fix this, but it hasn''t been merged yet. It might show up in 3.9 or 3.10. You really should upgrade your kernel, however. 3.5.0 is rather old in btrfs-years! Lots of fixes have gone into newer kernels. -- Calvin Walton <calvin.walton@kepstin.ca> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 21/02/2013 16:50, Liu Bo a écrit :> Well, there is already a patch which addresses your concern and it''s > ''snapshot-aware defrag'' feature and now in v6, it''s not merged yet. > thanks, liuboHi Liu, So should I understand that, even though the manpage states that the issue is for kernels <= 2.6.37, I should however not defrag until the patch you mention is actually merged in a future kernel release ? TIA. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 21/02/2013 16:54, Calvin Walton a écrit :> You really should upgrade your kernel, however. 3.5.0 is rather old in > btrfs-years! Lots of fixes have gone into newer kernels.Hi Calvin, I expect Ubuntu 13.04 to come with kernel 3.7 in April. Having Ubuntu kernel upgrades every 6 months (and several machines), I try to stick as much as I can to my distro''s kernel. I''m no developper myself and can hardly consider that the kernel that came with the latest distro 4 months ago is actually _that_ old... So I''ll live with it for 2 more months - unless I''m pretty sure that upgrading to some more recent kernel than the one that Ubuntu 13.04 will have would give me huge speed improvements or features I need much - (and I assume that being able to defrag, with my snapshots, would be a good reason for me...) Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 02/21/2013 08:01 AM, Swâmi Petaramesh wrote:> Le 21/02/2013 16:54, Calvin Walton a écrit : >> You really should upgrade your kernel, however. 3.5.0 is rather old in >> btrfs-years! Lots of fixes have gone into newer kernels. > > Hi Calvin, > > I expect Ubuntu 13.04 to come with kernel 3.7 in April.13.04 has 3.8, they just updated to the final 3.8 release. Blair -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 21, 2013 at 05:01:30PM +0100, Swâmi Petaramesh wrote:> Le 21/02/2013 16:54, Calvin Walton a écrit : > > You really should upgrade your kernel, however. 3.5.0 is rather old in > > btrfs-years! Lots of fixes have gone into newer kernels. > > Hi Calvin, > > I expect Ubuntu 13.04 to come with kernel 3.7 in April. Having Ubuntu > kernel upgrades every 6 months (and several machines), I try to stick as > much as I can to my distro''s kernel. I''m no developper myself and can > hardly consider that the kernel that came with the latest distro 4 > months ago is actually _that_ old... So I''ll live with it for 2 more > months - unless I''m pretty sure that upgrading to some more recent > kernel than the one that Ubuntu 13.04 will have would give me huge speed > improvements or features I need much - (and I assume that being able to3.7 is quite a bit faster. Plus, if something does go wrong with your FS, and you''re running an older kernel, you''ll get limited amounts of sympathy, because quite a lot of the problems people encounter with older kernels have already been fixed in newer ones. Finally, Ubuntu publish the latest kernels in a PPA[1], so there''s not really much excuse for not keeping up with them.> defrag, with my snapshots, would be a good reason for me...)Hugo. [1] http://kernel.ubuntu.com/~kernel-ppa/mainline/ -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Nostalgia isn''t what it used to be. ---
Le 21/02/2013 17:38, Hugo Mills a écrit :> Plus, if something does go wrong with your FS, and you''re running an > older kernel, you''ll get limited amounts of sympathy, because quite a > lot of the problems people encounter with older kernels have already > been fixed in newer ones.The matter, as usual, is that what may be a "new" kernel from a user''s standpoint, may be an "old" kernel from a developer''s standpoint. Should this exclude all standard users from developer''s "sympathy", or reduce it to very limited amounts ? Users are supposed to use released material, not bleeding-edge compile-it-yourself devs toys...> Finally, Ubuntu publish the latest kernels in a PPA[1], so there''s not > really much excuse for not keeping up with them.Yep. AFAIK the MAINLINE PPA is a Ubuntu-packaged STANDARD kernel without any of the specific Ubuntu kernel patches. That makes it very inappropriate for daily use, and the Ubuntu devs ask for trying it ONLY when reporting kernel bugs, to check if said bug may have been fixed upstream in latest kernel, or not. Last time I tried this standard kernel, on one of my machines, it just wouldn''t boot ; on another one, it couldn''t manage power (battery etc) correctly, and on the machine on which I write this email, it couldn''t see my SD card reader anymore - for this one is managed by a driver which still didn''t make it into the standard kernel. I use Ubuntu on several -dekstop- machines that benefit and need the Ubuntu patches on their Ubuntus. I''m not (currently) running a headless generic server for which my main and only concern would be BTRFS... So I try to stick to the very old - albeit maximum 6 months old ! - stock Ubuntu kernel... Well... Sometimes I upgrade to next Beta :-] Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 21, 2013 at 06:03:17PM +0100, Swâmi Petaramesh wrote:> Le 21/02/2013 17:38, Hugo Mills a écrit : > > Plus, if something does go wrong with your FS, and you''re running an > > older kernel, you''ll get limited amounts of sympathy, because quite a > > lot of the problems people encounter with older kernels have already > > been fixed in newer ones. > > The matter, as usual, is that what may be a "new" kernel from a user''s > standpoint, may be an "old" kernel from a developer''s standpoint. > > Should this exclude all standard users from developer''s "sympathy", or > reduce it to very limited amounts ?For an filesystem still under heavy development, and still marked "experimental", yes, it should. In fact, *particularly* for a filesystem, where a bug can cause persistent damage which isn''t easily fixable by upgrading to a later kernel (which is the case with most kernel bugs).> Users are supposed to use released material, not bleeding-edge > compile-it-yourself devs toys...Correct. But btrfs isn''t at that stage yet. It''s getting visibly closer, but it''s not quite there. Hence the very strong recommendation to keep up with the latest code. Hugo.> > Finally, Ubuntu publish the latest kernels in a PPA[1], so there''s not > > really much excuse for not keeping up with them. > > Yep. AFAIK the MAINLINE PPA is a Ubuntu-packaged STANDARD kernel without > any of the specific Ubuntu kernel patches. > > That makes it very inappropriate for daily use, and the Ubuntu devs ask > for trying it ONLY when reporting kernel bugs, to check if said bug may > have been fixed upstream in latest kernel, or not. > > Last time I tried this standard kernel, on one of my machines, it just > wouldn''t boot ; on another one, it couldn''t manage power (battery etc) > correctly, and on the machine on which I write this email, it couldn''t > see my SD card reader anymore - for this one is managed by a driver > which still didn''t make it into the standard kernel. > > I use Ubuntu on several -dekstop- machines that benefit and need the > Ubuntu patches on their Ubuntus. I''m not (currently) running a headless > generic server for which my main and only concern would be BTRFS... > > So I try to stick to the very old - albeit maximum 6 months old ! - > stock Ubuntu kernel...-- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I get nervous when I see words like ''mayhaps'' in a novel, --- because I fear that just round the corner is lurking ''forsooth''
Le 21/02/2013 18:25, Hugo Mills a écrit :> Correct. But btrfs isn''t at that stage yet. It''s getting visibly > closer, but it''s not quite there. Hence the very strong recommendation > to keep up with the latest code. Hugo.The matter is that BTRFS had many early adopters just because it is - and has been for long now - in the mainline Linux kernel, so supposed stable and good choice for the future. To be honest (and not wanting to troll, promised) this is the only single reason for which I use BTRFS on 5 of my 6 machines at home - just because I thought that "Just upgrade the distro every 6 months and it will become better and better over time, no hassle, make my life easy". OTOH my 6th machine runs native ZFS on Linux, and I have to tell that it shows orders of magnitude better performance and never gave me a single problem in several (3 ?) years. Only upgrading the distro is always a big frightening and problematic. And initial installation was a bit tricky. Probably a lot of BTRS early adopters choosed it for the same reason why I did : Included in the standard kernel, easy to install, and *expected* to improve quickly - yes, I already made the move back and forth twice, between ext4 and BTRFS, on 2 machines... About one year ago I choosed to jump "for good" and stay, but performance degrades so quickly and so much that every couple of days I wonder if I won''t rollback to ex4 again (and again), or shift to ZFS once for all and just forget about it... Everytime I show my Linux machines to friends and say : “Hey, I got the most advanced filesystem on earth !” I soon get the answer “Oh boy, that''s the slowest machine boot and FS I''ve ever seen since I was reading floppy disks on my 386SX in 1991 ! Can you really live with this ?” So, for "not quite there" and the return codes "+20" that have been a minor pain in the arse for a couple years but the line is still in the code... I can understand developer''s PoV, been there, done that, but still, BTRFS might in the end lose a numer of its early adopters if it keeps being "not quite there" too long. Shitfing to ZFS is just a PPA and 2 apt-get install commands away... It will definitely be easier than start playing with mainline PPA Ubuntu kernels... Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 21, 2013 at 06:47:28PM +0100, Swâmi Petaramesh wrote:> Le 21/02/2013 18:25, Hugo Mills a écrit : > > Correct. But btrfs isn''t at that stage yet. It''s getting visibly > > closer, but it''s not quite there. Hence the very strong recommendation > > to keep up with the latest code. Hugo. > > The matter is that BTRFS had many early adopters just because it is - > and has been for long now - in the mainline Linux kernel, so supposed > stable and good choice for the future.I''m afraid that''s not actually a valid conclusion. Just because something makes it into the mainline kernel, it doesn''t mean that it''s considered stable -- just that any instabilities aren''t going to affect the rest of the kernel.> To be honest (and not wanting to troll, promised) this is the only > single reason for which I use BTRFS on 5 of my 6 machines at home - just > because I thought that "Just upgrade the distro every 6 months and it > will become better and better over time, no hassle, make my life easy".That approach means you''re still somewhere between 1 and 3 kernel versions out of date, on something which is still undergoing significant changes on most kernel releases.> OTOH my 6th machine runs native ZFS on Linux, and I have to tell that it > shows orders of magnitude better performance and never gave me a single > problem in several (3 ?) years. Only upgrading the distro is always a > big frightening and problematic. And initial installation was a bit tricky.ZFS has had something like an extra 5 years of development on it, so it''s not surprising that it''s more stable.> Probably a lot of BTRS early adopters choosed it for the same reason why > I did : Included in the standard kernel, easy to install, and *expected* > to improve quicklyBut you don''t get the improvements if you don''t get the kernels...> - yes, I already made the move back and forth twice, > between ext4 and BTRFS, on 2 machines... About one year ago I choosed to > jump "for good" and stay, but performance degrades so quickly and so > much that every couple of days I wonder if I won''t rollback to ex4 again > (and again), or shift to ZFS once for all and just forget about it...Performance was greatly improved in 3.7. You may also find performance problems with a very full filesystem, or with workloads which write heavily to one file (databases, and bitcoin''s DB in particular). The latter issue can largely be dealt with by judicious use of nocow attributes. Plus go for 3.7 or later anyway.> Everytime I show my Linux machines to friends and say : “Hey, I got the > most advanced filesystem on earth !” I soon get the answer “Oh boy, > that''s the slowest machine boot and FS I''ve ever seen since I was > reading floppy disks on my 386SX in 1991 ! Can you really live with this ?” > > So, for "not quite there" and the return codes "+20" that have been a > minor pain in the arse for a couple years but the line is still in the > code...There''s been about one question (not even complaint) about that return code every... maybe 3-6 months? It''s not high on the list of priorities of things to fix, compared to some of the other problems that have shown up, or even the repeated questions (like mis-interpretation of the output of df, btrfs fi df and btrfs fi show).> I can understand developer''s PoV, been there, done that, but > still, BTRFS might in the end lose a numer of its early adopters if it > keeps being "not quite there" too long. > > Shitfing to ZFS is just a PPA and 2 apt-get install commands away... It > will definitely be easier than start playing with mainline PPA Ubuntu > kernels...Well, given that btrfs is still getting at least one significant fix to a major bug or set of fixes in every release, it would still seem sensible to keep up with the latest developments. If that''s not what you expected, then I''m sorry if we''ve failed to get the point across. We do try to manage expectations on the btrfs website, and we''re quite up-front about the need for keeping up with releases there. If that''s more than you want to do, that''s OK -- I wouldn''t expect it to be something that everyone would want to do. Many people do keep up with the kernels through the Ubuntu PPA (or other, similar repositories for other distributions), and don''t have problems. If it doesn''t work out for you, then parting company without rancour or prejudice may be the best thing. Temporarily, I hope, of course. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I get nervous when I see words like ''mayhaps'' in a novel, --- because I fear that just round the corner is lurking ''forsooth''
On 02/21/2013 06:47 PM, Swâmi Petaramesh wrote:> Le 21/02/2013 18:25, Hugo Mills a écrit : >> Correct. But btrfs isn''t at that stage yet. It''s getting visibly >> closer, but it''s not quite there. Hence the very strong recommendation >> to keep up with the latest code. Hugo. > > The matter is that BTRFS had many early adopters just because it is - > and has been for long now - in the mainline Linux kernel, so supposed > stable and good choice for the future. > > To be honest (and not wanting to troll, promised) this is the only > single reason for which I use BTRFS on 5 of my 6 machines at home - just > because I thought that "Just upgrade the distro every 6 months and it > will become better and better over time, no hassle, make my life easy". >Unfortunately many distros don''t make it obvious, but Btrfs is still hidden behind a giant EXPERIMENTAL label in the kernel configuration. Regards, -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 21 Feb 2013 18:47:28 +0100 Swâmi Petaramesh <swami@petaramesh.org> wrote:> Le 21/02/2013 18:25, Hugo Mills a écrit : > > Correct. But btrfs isn''t at that stage yet. It''s getting visibly > > closer, but it''s not quite there. Hence the very strong > > recommendation to keep up with the latest code. Hugo. > > The matter is that BTRFS had many early adopters just because it is - > and has been for long now - in the mainline Linux kernel, so supposed > stable and good choice for the future.And it''s marked as EXPERIMENTAL. So if you want to join the game, you have to accept the rules.> OTOH my 6th machine runs native ZFS on Linux, and I have to tell that > it shows orders of magnitude better performance and never gave me a > single problem in several (3 ?) years. Only upgrading the distro is > always a big frightening and problematic. And initial installation > was a bit tricky.You didn''t, many other had. I remember a lot of threads in the OpenSolaris forum, where the solution for problems was: recreate your filesystem, replay your backup.> Everytime I show my Linux machines to friends and say : “Hey, I got > the most advanced filesystem on earth !” I soon get the answer “Oh > boy, that''s the slowest machine boot and FS I''ve ever seen since I was > reading floppy disks on my 386SX in 1991 ! Can you really live with > this ?”Did you presented it while (re)creating the inode_cache? Sounds a little like that.> So, for "not quite there" and the return codes "+20" that have been a > minor pain in the arse for a couple years but the line is still in the > code... I can understand developer''s PoV, been there, done that, but > still, BTRFS might in the end lose a numer of its early adopters if it > keeps being "not quite there" too long. > > Shitfing to ZFS is just a PPA and 2 apt-get install commands away... > It will definitely be easier than start playing with mainline PPA > Ubuntu kernels...So why do you bother with btrfs, if ZFS fit your needs? regards, Johannes -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 21, 2013 at 09:58:16PM +0100, Bardur Arantsson wrote:> On 02/21/2013 06:47 PM, Swâmi Petaramesh wrote: > > Le 21/02/2013 18:25, Hugo Mills a écrit : > >> Correct. But btrfs isn''t at that stage yet. It''s getting visibly > >> closer, but it''s not quite there. Hence the very strong recommendation > >> to keep up with the latest code. Hugo. > > > > The matter is that BTRFS had many early adopters just because it is - > > and has been for long now - in the mainline Linux kernel, so supposed > > stable and good choice for the future. > > > > To be honest (and not wanting to troll, promised) this is the only > > single reason for which I use BTRFS on 5 of my 6 machines at home - just > > because I thought that "Just upgrade the distro every 6 months and it > > will become better and better over time, no hassle, make my life easy". > > > > Unfortunately many distros don''t make it obvious, but Btrfs is still > hidden behind a giant EXPERIMENTAL label in the kernel configuration.Removed in 3.9-rc1 as a part of a broad Kconfig cleanup http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commit;h=38db331b578005d32155bb6f6a80654ef127cff5 The CONFIG_EXPERIMENTAL config item has not carried much meaning for a while now and is almost always enabled by default. As agreed during the Linux kernel summit, remove it from any "depends on" lines in Kconfigs. --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -1,6 +1,5 @@ config BTRFS_FS - tristate "Btrfs filesystem (EXPERIMENTAL) Unstable disk format" - depends on EXPERIMENTAL + tristate "Btrfs filesystem Unstable disk format" select LIBCRC32C select ZLIB_INFLATE select ZLIB_DEFLATE --- is it still experimental then? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 02/21/2013 10:56 PM, David Sterba wrote:> On Thu, Feb 21, 2013 at 09:58:16PM +0100, Bardur Arantsson wrote: >> On 02/21/2013 06:47 PM, Swâmi Petaramesh wrote: >>> Le 21/02/2013 18:25, Hugo Mills a écrit : >>>> Correct. But btrfs isn''t at that stage yet. It''s getting visibly >>>> closer, but it''s not quite there. Hence the very strong recommendation >>>> to keep up with the latest code. Hugo. >>> >>> The matter is that BTRFS had many early adopters just because it is - >>> and has been for long now - in the mainline Linux kernel, so supposed >>> stable and good choice for the future. >>> >>> To be honest (and not wanting to troll, promised) this is the only >>> single reason for which I use BTRFS on 5 of my 6 machines at home - just >>> because I thought that "Just upgrade the distro every 6 months and it >>> will become better and better over time, no hassle, make my life easy". >>> >> >> Unfortunately many distros don''t make it obvious, but Btrfs is still >> hidden behind a giant EXPERIMENTAL label in the kernel configuration. > > Removed in 3.9-rc1 as a part of a broad Kconfig cleanup >[--snip--]> > is it still experimental then?Interesting, but IMO having the experimental label taken off is a necessary, but not sufficient condition for $X to be considered stable :). Regards, -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html