-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Is there any particular reason why I can''t use DUP for data? When I try to set it with balance there is a kernel message: btrfs: dup for data is not allowed The glossary at https://btrfs.wiki.kernel.org/index.php/Glossary says: Regular data cannot be assigned DUP level. It is somewhat baffling that code and documentation exists to prevent this! My current use case is an older hard drive I am putting backups on. Since my data fits in less than half of the drive, and drives get bad sectors (this has had several it has reallocated) using DUP would be useful. (I realise making two partitions and RAID-1 with them would work, which makes the DUP restriction even sillier.) Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFy978ACgkQmOOfHg372QTCXwCdGuWbRFY8cZoWNFZb1k3mFeZM nrkAoK9c5KddsJ1R57pAR1Lk89fPSo9Q =ENfT -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Apr 20, 2013 at 01:17:06PM -0700, Roger Binns wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Is there any particular reason why I can''t use DUP for data?Technically, no. Performance is likely to suck if you use rotational disks, and you may find some SSDs deduplicate blocks, making it fairly pointless for those devices.> When I try to set it with balance there is a kernel message: > > btrfs: dup for data is not allowedWhat kernel and userspace versions are you using? I thought the restriction had been removed at some point (but possibly I''m just misremembering it).> The glossary at https://btrfs.wiki.kernel.org/index.php/Glossary says: > > Regular data cannot be assigned DUP level. > > It is somewhat baffling that code and documentation exists to prevent this!I think it did get fixed at some point. The documentation doesn''t always get updated everywhere -- it can be quite hard to find every single implication of a feature when that feature gets changd/updated. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- UNIX: Spanish manufacturer of fire extinguishers. ---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/04/13 13:48, Hugo Mills wrote:> On Sat, Apr 20, 2013 at 01:17:06PM - -0700, Roger Binns wrote:>> Is there any particular reason why I can''t use DUP for data? > > Technically, no. Performance is likely to suck if you use rotational > disks, and you may find some SSDs deduplicate blocks, making it fairly > pointless for those devices.Wanting dup is because I care about resilience in the face of errors over performance, so I don''t actually care how bad performance sucks. I only put data on SSDs that I can afford to lose (usually backed up to Dropbox/github/spinning disks).>> When I try to set it with balance there is a kernel message: >> >> btrfs: dup for data is not allowed > > What kernel and userspace versions are you using? I thought the > restriction had been removed at some point (but possibly I''m just > misremembering it).Whatever Ubuntu 12.10 ships with. Kernel package is 3.5.0.27.43 and btrfs-tools is 0.19+20120328-7ubuntu1. Note the message came from the kernel so it would appear to be solely to blame for refusing my request.> it can be quite hard to find every single implication of a feature when > that feature gets changd/updated.I''m more amused that someone went to the trouble of putting in kernel detection and messages plus updating the documentation in order to prevent using DUP for data! Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFzBWEACgkQmOOfHg372QT5twCeI5KOgHYT1wcPNPtni9TYKQaG aeIAmQGw/nL/ziTDAu4gUJIFPwCLI5td =ABih -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Apr 20, 2013 at 02:15:14PM -0700, Roger Binns wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 20/04/13 13:48, Hugo Mills wrote:> On Sat, Apr 20, 2013 at 01:17:06PM > - -0700, Roger Binns wrote: > >> When I try to set it with balance there is a kernel message: > >> > >> btrfs: dup for data is not allowed > > > > What kernel and userspace versions are you using? I thought the > > restriction had been removed at some point (but possibly I''m just > > misremembering it). > > Whatever Ubuntu 12.10 ships with. Kernel package is 3.5.0.27.43 and > btrfs-tools is 0.19+20120328-7ubuntu1. Note the message came from the > kernel so it would appear to be solely to blame for refusing my request.OK, I''m not 100% certain, but that''s probably too early for the fix I was thinking of. You should upgrade anyway -- there''s been a number of serious bugs in btrfs fixed since then. Ubuntu has new kernels in a PPA at [1].> > it can be quite hard to find every single implication of a feature when > > that feature gets changd/updated. > > I''m more amused that someone went to the trouble of putting in kernel > detection and messages plus updating the documentation in order to prevent > using DUP for data!It''d have been a mere couple of lines of code, and probably there to match historical usage of mkfs (which I believe has also been dealt with). Hugo. [1] https://launchpad.net/~yofel/+archive/btrfs -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- A linked list is still a binary tree. Just a --- very unbalanced one.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/04/13 14:23, Hugo Mills wrote:> You should upgrade anyway -- there''s been a number of serious bugs in > btrfs fixed since then.13.04 is imminent so I''ll pick up a newer kernel as part of that anyway. (Also Tanglu which I hope to move to intends to use the Ubuntu kernel anyway.) In any event I am not worried. Bug fixes get backported. The probability of hitting any serious bug is low (others would likely be victims first) and worst case I have backups (snapshots, other machines, Google/Dropbox, DVDs & hard drives at other people''s houses etc). There are two major reasons I switched from ext4. The first is that everything is online, including adding and removing devices, checking data integrity etc. The second is that data not silently lost. I had some bad sectors develop on ext4 spinning disk and the only way to properly recover was to do offline checks that would have taken ~ 24 hours! Finding out which filenames were involved was far too much effort. My philosophy is that we have machines with billions of processor cycles per second. They can figure things out for themselves without requiring me to baby sit them! I also run btrfs in a variety of configurations - raid0, raid1, sata, usb, hdd, ssd, single device, multi-device, bare, dmcrypt, machines on all the time, laptop, frequent suspend/resume, frequent power on/off. I''ve never experienced any problems with btrfs and use scrub for reassurance. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlFzHkwACgkQmOOfHg372QSBBwCfYYJQm9l9ts9eWOvZIRUlTCWJ KsMAoLVsd1fRdV+T7KO7nVVGuFCGYN5a =Y4by -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Apr 20, 2013 at 01:17:06PM -0700, Roger Binns wrote:> Is there any particular reason why I can''t use DUP for data?IIRC separate DUP for metadata and data is not possible due to some technical reason (I don''t remember nor can find the mail where this was mentioned). DUP data is possible only in the --mixed mode so this mkfs.btrfs --mixed -d dup -m dup /dev/ice works. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, 2013-04-20 at 16:01 -0700, Roger Binns wrote:> On 20/04/13 14:23, Hugo Mills wrote: > > You should upgrade anyway -- there''s been a number of serious bugs in > > btrfs fixed since then. > > 13.04 is imminent so I''ll pick up a newer kernel as part of that anyway. > (Also Tanglu which I hope to move to intends to use the Ubuntu kernel anyway.) > > In any event I am not worried. Bug fixes get backported. The probability > of hitting any serious bug is low (others would likely be victims first) > and worst case I have backups (snapshots, other machines, Google/Dropbox, > DVDs & hard drives at other people''s houses etc).Unfortunately, bugfixes in btrfs have tended to be *not* backported; aside from a few special cases, btrfs changes only go into new kernel 3.x releases. A few of the Linux distributions that officially support btrfs (Possibly Oracle, Red Hat, Suse? You would have to contact them for details...) may be backporting fixes themselves. As a result, we recommend that btrfs users should generally run whatever the latest upstream kernel release is - in particular, you should try the latest kernel before reporting bugs in btrfs. -- Calvin Walton <calvin.walton@kepstin.ca>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 27/04/13 14:40, Calvin Walton wrote:> Unfortunately, bugfixes in btrfs have tended to be *not* backported; > aside from a few special cases, ...Your efforts to scare me are admirable, but have failed :-) As btrfs development has progressed, the probability of a random user like me hitting bugs keeps decreasing. This is a reflection of the maturity of the code base, the increasing number of users, improved test suites, more eyes on the code, more diversity in uses etc. As far as I can see, backported bugfixes are made when the probability of a bug being encountered is significantly higher than the current probabilities, and that is why they are rare. As for the severity of the rarely hit bugs, the COW nature of the data means there is unlikely to be corruption, and if there is then of the most recent activity. Additionally the checksums mean it is possible to proactively verify (online) that unexpected corruption hasn''t been creeping in. And if all that fails, I have multiple layers of backups. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlF8X14ACgkQmOOfHg372QT5EQCfSbzZBRdm1KyljpoNpP5kHy15 iFoAn2eyair54JQdhnL2dfvqAJrVMWe4 =9zYA -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Roger Binns wrote:> On 27/04/13 14:40, Calvin Walton wrote: >> Unfortunately, bugfixes in btrfs have tended to be *not* backported; >> aside from a few special cases, ... > > Your efforts to scare me are admirable, but have failed :-) > > As btrfs development has progressed, the probability of a random user like > me hitting bugs keeps decreasing. This is a reflection of the maturity of > the code base, the increasing number of users, improved test suites, more > eyes on the code, more diversity in uses etc. As far as I can see, > backported bugfixes are made when the probability of a bug being > encountered is significantly higher than the current probabilities, and > that is why they are rare. > > As for the severity of the rarely hit bugs, the COW nature of the data > means there is unlikely to be corruption, and if there is then of the most > recent activity. Additionally the checksums mean it is possible to > proactively verify (online) that unexpected corruption hasn''t been > creeping in. > > And if all that fails, I have multiple layers of backups.I would be *very* hard-pressed to see it as an attempt to scare you - he''s just saying what is (very consistently) said to others on this list: When using btrfs, run a recent kernel :P. Honestly, even leaving aside the lack of backporting, there are other benefits to a recent kernel - things like cross-subvolume reflinks, btrfs device replace support being far more efficient than add/balance/remove/balance, and a bunch more. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 27/04/13 19:53, Alex Elsayed wrote:> When using btrfs, run a recent kernel :P.Every software developer says that of what they produce. Newer is almost always better in many different axes.> Honestly, even leaving aside the lack of backporting, there are other > benefits to a recent kernel - things like cross-subvolume reflinks, > btrfs device replace support being far more efficient than > add/balance/remove/balance, and a bunch more.Those are all "features", none of which I use or have had to use yet. If it will make you feel better I did upgrade some systems today to the most recent Ubuntu release which meant going from kernel 3.5 to 3.8. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlF83dUACgkQmOOfHg372QQeDACgz0oBnrYeg6fO5tFxUy9qonE9 HYIAoJWjT8z2sJ356YAph1NAyLKhcEBz =ySAg -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html