Hallo, linux-btrfs, I''ve (once more) created my test system: mkfs.btrfs -d raid0 -m raid1 /dev/sdb1 /dev/sdc1 73 GB + 146 GB. Then I mounted it and copied about 150 GByte onto it. But copying was incomplete, the job ended with "no space on ..." # btrfs fi show Label: ''Scsi'' uuid: e30586e9-a903-4d17-8ec0-1781457212c6 Total devices 2 FS bytes used 134.86GB devid 1 size 68.37GB used 68.37GB path /dev/sdb1 devid 2 size 136.73GB used 68.35GB path /dev/sdc1 Btrfs Btrfs v0.19 # btrfs fi df <mountpoint> Data, RAID0: total=134.68GB, used=134.67GB Data: total=8.00MB, used=8.00MB System, RAID1: total=8.00MB, used=16.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=1.00GB, used=185.39MB Metadata: total=8.00MB, used=0.00 # df <mountpoint> Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc1 215059324 141597364 8724 100% /mnt/btr # fdisk -l Disk /dev/sdb: 73.4 GB, 73407868928 bytes Disk /dev/sdc: 146.8 GB, 146814976000 bytes ----------------------------------- Where is the problem, how can I use the full space? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Saturday 17 March 2012 23:01:00 Helmut Hullen wrote:> Where is the problem, how can I use the full space?Which kernel was this with Helmut? cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
Hallo, Chris, Du meintest am 17.03.12:>> Where is the problem, how can I use the full space?> Which kernel was this with Helmut?Kernel 3.2.9 (self made) btrfs-progs-20111030 Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Mar 17, 2012 at 01:01:00PM +0100, Helmut Hullen wrote:> Hallo, linux-btrfs, > > I''ve (once more) created my test system: > > mkfs.btrfs -d raid0 -m raid1 /dev/sdb1 /dev/sdc1 > > 73 GB + 146 GB. > > Then I mounted it and copied about 150 GByte onto it. But copying was > incomplete, the job ended with "no space on ..." > > > # btrfs fi show > > Label: ''Scsi'' uuid: e30586e9-a903-4d17-8ec0-1781457212c6 > Total devices 2 FS bytes used 134.86GB > devid 1 size 68.37GB used 68.37GB path /dev/sdb1 > devid 2 size 136.73GB used 68.35GB path /dev/sdc1 > > Btrfs Btrfs v0.19 > > # btrfs fi df <mountpoint> > > Data, RAID0: total=134.68GB, used=134.67GB > Data: total=8.00MB, used=8.00MB > System, RAID1: total=8.00MB, used=16.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=1.00GB, used=185.39MB > Metadata: total=8.00MB, used=0.00 > > # df <mountpoint> > > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdc1 215059324 141597364 8724 100% /mnt/btr > > # fdisk -l > > Disk /dev/sdb: 73.4 GB, 73407868928 bytes > Disk /dev/sdc: 146.8 GB, 146814976000 bytes > > ----------------------------------- > > Where is the problem, how can I use the full space?You can''t. btrfs requires RAID-0 to be at least two devices wide (otherwise it''s not striped at all, which is the point of RAID-0). If you want to use the full capacity of both disks and don''t care about the performance gain from striping, use -d single (which is the default). If you do care about the performance gain from striping, then you''re going to have to lose some usable space. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I am but mad north-north-west: when the wind is southerly, I --- know a hawk from a handsaw.
Helmut Hullen <Hullen <at> t-online.de> writes:>> > Where is the problem, how can I use the full space? > > Viele Gruesse! > HelmutEffectively it''s missing the trigger to rebalance when the ''primary'' device starts to get full, or just to randomly spread the data between the devices. Isn''t this going to be a problem for anyone restoring from a backup? Lots of data heading to btrfs without a pause. Helmut, can you tell everyone which version of the kernel & patches you''re running pls. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Mar 17, 2012 at 12:19:54PM +0000, Alex wrote:> Helmut Hullen <Hullen <at> t-online.de> writes: > > > > > > > > Where is the problem, how can I use the full space? > > > > Viele Gruesse! > > Helmut > > Effectively it''s missing the trigger to rebalance when the ''primary'' device > starts to get full, or just to randomly spread the data between the devices.No, a balance isn''t going to help here. RAID-0 requires a minimum of 2 chunks in a block group. With two disks, you''re only going to be able to fill the smallest one before you run out of space.> Isn''t this going to be a problem for anyone restoring from a backup? Lots of > data heading to btrfs without a pause.Again, not the issue here. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- I am but mad north-north-west: when the wind is southerly, I --- know a hawk from a handsaw.
> > No, a balance isn''t going to help here. RAID-0 requires a minimum > of 2 chunks in a block group. With two disks, you''re only going to be > able to fill the smallest one before you run out of space. > > > Isn''t this going to be a problem for anyone restoring from a backup? Lots of > > data heading to btrfs without a pause. > > Again, not the issue here. > > Hugo. >Thank you HH for the explanation. What''s the ''solution'' though to Hugo''s situation? By ''solution'' I mean the highest-utility way of dealing with unequal devices problem in a two or more -up setting. (I understand that we''re violating the expectations of RAID0 and have read the http://btrfs.ipv5.de/index.php?title=FAQ#How_much_space_do_I_get_with_unequal_devices_in_RAID-1_mode.3F about RAID1.) Disks are (were) getting cheaper per MB faster than I could run out of space! So I have increasing size array of drives - apparently the similar situation as Hugo (I have three btrfs-ready drives). I''m struggling how to mkfs.btrfs with this new knowledge whilst resolving it with the btrfs storage pool concept. I stress there is a difficulty with my understanding not btrfs. Kind regards. Al. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hallo, Hugo, Du meintest am 17.03.12: [no space left on device ...]>>> Where is the problem, how can I use the full space?>> Effectively it''s missing the trigger to rebalance when the ''primary'' >> device starts to get full, or just to randomly spread the data >> between the devices.> No, a balance isn''t going to help here. RAID-0 requires a minimum > of 2 chunks in a block group. With two disks, you''re only going to be > able to fill the smallest one before you run out of space.Ok - it happens only with 2 disks/Partitions? I''ve continued playing; added a 3rd partition/device and then balanced: # btrfs device add /dev/sdd1 /mnt/btr ## 73 + 146 + 146 GByte # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc1 358432300 225400136 59842800 80% /mnt/btr # added about 70 GByte (all together more than 3 times the smallest partition) # btrfs fi show Label: ''Scsi'' uuid: e30586e9-a903-4d17-8ec0-1781457212c6 Total devices 3 FS bytes used 214.67GB devid 1 size 68.37GB used 68.37GB path /dev/sdb1 devid 3 size 136.73GB used 41.01GB path /dev/sdd1 devid 2 size 136.73GB used 109.36GB path /dev/sdc1 Btrfs Btrfs v0.19 # btrfs fi df /mnt/btr Data, RAID0: total=216.71GB, used=214.38GB System, RAID1: total=8.00MB, used=24.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=1.00GB, used=295.13MB ================================================================ # btrfs fi balance /mnt/btr # btrfs fi show Label: ''Scsi'' uuid: e30586e9-a903-4d17-8ec0-1781457212c6 Total devices 3 FS bytes used 214.67GB devid 1 size 68.37GB used 67.34GB path /dev/sdb1 devid 3 size 136.73GB used 40.85GB path /dev/sdd1 devid 2 size 136.73GB used 107.85GB path /dev/sdc1 Btrfs Btrfs v0.19 # btrfs fi df /mnt/btr Data, RAID0: total=215.01GB, used=214.38GB System, RAID1: total=8.00MB, used=24.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=512.00MB, used=294.88MB ---------------------------------------------------------------------- Looks as desired, the 3-disks-system contains more than 3 times the smallest disk. Balancing hasn''t redistributed the contents - no problem. By the way: you should name the prefixes in the NIST way for powers of 2: KiB, MiB, GiB. Or change to decimal prefixes. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Mar 17, 2012 at 01:36:04PM +0000, Alex wrote:> > No, a balance isn''t going to help here. RAID-0 requires a minimum > > of 2 chunks in a block group. With two disks, you''re only going to be > > able to fill the smallest one before you run out of space. > > > > > Isn''t this going to be a problem for anyone restoring from a backup? Lots of > > > data heading to btrfs without a pause. > > > > Again, not the issue here. > > Thank you HH for the explanation. > > What''s the ''solution'' though to Hugo''s situation? > By ''solution'' I mean the highest-utility way of dealing with unequal devices > problem in a two or more -up setting.Use mkfs.btrfs -d single, as I said in another part of this thread. We have a bit of a problem right now with adding a second device to an existing FS with -d single, where the FS "upgrades" the single to RAID-0. However, there''s patches from Ilya that stop that behaviour, and the restriper can be used to switch back to single after the "upgrade".> (I understand that we''re violating the expectations of RAID0 and > have read the > http://btrfs.ipv5.de/index.php?title=FAQ#How_much_space_do_I_get_with_unequal_devices_in_RAID-1_mode.3F > about RAID1.)I''m working on a comprehensive explanation of the limits on btrfs''s space usage, and a little JavaScript tool to help plan/explain disk usage.> Disks are (were) getting cheaper per MB faster than I could run out > of space! So I have increasing size array of drives - apparently the > similar situation as Hugo (I have three btrfs-ready drives).I don''t have this problem personally. I just know something about the way btrfs allocates chunks. I think you meant Helmut has the problem. :)> I''m struggling how to mkfs.btrfs with this new knowledge whilst resolving it > with the btrfs storage pool concept. > > I stress there is a difficulty with my understanding not btrfs.Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- UNIX: Spanish manufacturer of fire extinguishers. ---
On Sat, Mar 17, 2012 at 02:46:00PM +0100, Helmut Hullen wrote:> Hallo, Hugo, > > Du meintest am 17.03.12: > > [no space left on device ...] > > >>> Where is the problem, how can I use the full space? > > >> Effectively it''s missing the trigger to rebalance when the ''primary'' > >> device starts to get full, or just to randomly spread the data > >> between the devices. > > > No, a balance isn''t going to help here. RAID-0 requires a minimum > > of 2 chunks in a block group. With two disks, you''re only going to be > > able to fill the smallest one before you run out of space. > > Ok - it happens only with 2 disks/Partitions?Not quite. With the current RAID-0 implementation, you will lose the difference between the largest disk and the second-largest. There''s a proposal from November that may help with that, but I haven''t had a chance to look at it yet to see what the implications are.> I''ve continued playing; added a 3rd partition/device and then balanced: > > > # btrfs device add /dev/sdd1 /mnt/btr > ## 73 + 146 + 146 GByteOK, so in this case the largest and second-largest devices are the same, so you will lose no space. [snip]> Looks as desired, the 3-disks-system contains more than 3 times the > smallest disk. > > Balancing hasn''t redistributed the contents - no problem. > > By the way: you should name the prefixes in the NIST way for powers of > 2: KiB, MiB, GiB. Or change to decimal prefixes.I know. It''s something I submitted patches for a long time ago, and they never got taken up. It''s on my list of things to fix. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- UNIX: Spanish manufacturer of fire extinguishers. ---
Hallo, Hugo, Du meintest am 17.03.12:>> What''s the ''solution'' though to Hugo''s situation? >> By ''solution'' I mean the highest-utility way of dealing with unequal >> devices problem in a two or more -up setting.> Use mkfs.btrfs -d single, as I said in another part of this > thread.Does "single" allow adding new (bigger) disks and removing old (smaller) disks? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Mar 17, 2012 at 03:24:00PM +0100, Helmut Hullen wrote:> Hallo, Hugo, > > Du meintest am 17.03.12: > > >> What''s the ''solution'' though to Hugo''s situation? > >> By ''solution'' I mean the highest-utility way of dealing with unequal > >> devices problem in a two or more -up setting. > > > Use mkfs.btrfs -d single, as I said in another part of this > > thread. > > Does "single" allow adding new (bigger) disks and removing old (smaller) > disks?Yes. "single" is effectively "RAID linear" -- just adding the disks together, effectively linearly(*). It''s called "single" because it''s single (i.e. no) redundancy. Hugo. (*) actually the allocator will kind of "stripe" the data in 1GiB-wide stripes, but that''s incidental really. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- A diverse working environment: Di longer you vork here, di --- verse it gets.
Hugo Mills <hugo <at> carfax.org.uk> writes:> > On Sat, Mar 17, 2012 at 01:36:04PM +0000, Alex wrote:> > I don''t have this problem personally. I just know something about > the way btrfs allocates chunks. I think you meant Helmut has the > problem. :) > > > I''m struggling how to mkfs.btrfs with this new knowledge whilst resolving it > > with the btrfs storage pool concept. > > > > I stress there is a difficulty with my understanding not btrfs. > > Hugo. >Yes, I meant Hugo .. damn .. Helmut, sorry! However deed-poll is a cost effective measure which would allow you to avoid the coincident first letter with Helmut. ;-) Since we''re still experimental in the non-SUSE and Oracle world and the space meta demands sb lower can I squeak by: mkfs.btrfs -d single -m raid1 ? Don''t laugh. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> I''m working on a comprehensive explanation of the limits on btrfs''s > space usage, and a little JavaScript tool to help plan/explain disk > usage.Thank you Hugo. Given the kernel 3.3 release yesterday [1] we''ve gained the ability to restripe to different raid levels after the fact. So I guess I don''t have too much to worry about with decent backups. The captcha word for this post is "baptizes". Quite apt (no pun intended) really! [1] http://thread.gmane.org/gmane.linux.kernel/1240436 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Mar 19, 2012 at 03:14:19PM +0000, Alex wrote:> > > I''m working on a comprehensive explanation of the limits on btrfs''s > > space usage, and a little JavaScript tool to help plan/explain disk > > usage. > > Thank you Hugo. > > Given the kernel 3.3 release yesterday [1] we''ve gained the ability to restripe > to different raid levels after the fact. So I guess I don''t have too much to > worry about with decent backups.Yes, you do. RAID is not a backup: What happens if you accidentally write zeroes over all your superblocks? Or rm -rf /home? RAID will not help you here. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- In the future, terrorists won''t be carrying their ID cards. --- They''ll be carrying yours. -- Henry Porter, Suspect Nation
Hi Hugo, I did mention backups sir in my last sentence, but thank you anyway. No harm in repeating the point. Blackberry PIN:268a2953 Sent from my BlackBerry® -----Original Message----- From: Hugo Mills <hugo@carfax.org.uk> Date: Mon, 19 Mar 2012 17:58:50 To: Alex<alex@bpmit.com> Cc: <linux-btrfs@vger.kernel.org> Subject: Re: "not enough space" with "data raid0" On Mon, Mar 19, 2012 at 03:14:19PM +0000, Alex wrote:> > > I'm working on a comprehensive explanation of the limits on btrfs's > > space usage, and a little JavaScript tool to help plan/explain disk > > usage. > > Thank you Hugo. > > Given the kernel 3.3 release yesterday [1] we've gained the ability to restripe > to different raid levels after the fact. So I guess I don't have too much to > worry about with decent backups.Yes, you do. RAID is not a backup: What happens if you accidentally write zeroes over all your superblocks? Or rm -rf /home? RAID will not help you here. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- In the future, terrorists won't be carrying their ID cards. --- They'll be carrying yours. -- Henry Porter, Suspect Nation
On 3/17/2012 8:19 AM, Hugo Mills wrote:>> Where is the problem, how can I use the full space? > > You can''t. btrfs requires RAID-0 to be at least two devices wide > (otherwise it''s not striped at all, which is the point of RAID-0). If > you want to use the full capacity of both disks and don''t care about > the performance gain from striping, use -d single (which is the > default). If you do care about the performance gain from striping, > then you''re going to have to lose some usable space.So currently btrfs''s concept of raid0 is "stripe across as many disks as possible, with a minimum of 2 disks". Is there any reason for that minimum? I don''t see why it can''t allow only one if that''s the best it can manage. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Phillip Susi <psusi <at> ubuntu.com> writes:> > So currently btrfs''s concept of raid0 is "stripe across as many disks as > possible, with a minimum of 2 disks". Is there any reason for that > minimum? I don''t see why it can''t allow only one if that''s the best it > can manage. >That''s called "Single". http://btrfs.ipv5.de/index.php?title=Mkfs.btrfs -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Alex posted on Wed, 28 Mar 2012 22:11:01 +0000 as excerpted:> Phillip Susi <psusi <at> ubuntu.com> writes: > > >> So currently btrfs''s concept of raid0 is "stripe across as many disks >> as possible, with a minimum of 2 disks". Is there any reason for that >> minimum? I don''t see why it can''t allow only one if that''s the best it >> can manage. >> > That''s called "Single". http://btrfs.ipv5.de/index.php?title=Mkfs.btrfsIt''s called "single" if you deliberately create it that way, yes. However, Phillip''s point was, I believe, that if a striped set can start with say five devices of varying size and drop one at a time as the smallest devices run out of space, there''s no reason it should stop at two devices. If the goal is maximum performance, it should insist on using only the space available in parallel across all devices and should refuse to drop even a single one. If OTOH, the goal is maximum utilization of available space, with a secondary goal of performance if at all possible, then it should drop devices all the way down to a single device, not stopping at two. Since btrfs apparently is already content to drop from say five devices to two despite the performance drop on the way, there''s no fundamental reason it should stop there; it should continue all the way down to a single device, even tho when it gets to that point it''s no longer actually striped. Of course, that''s defining the behavior I''d expect of a mature btrfs. Right now, it''s still experimental, with many missing features. If at this point there''s and arbitrary line drawn at two devices minimum, that''s just the way it is. But I''d not expect that line to be there when the filesystem is declared mature and fully ready for use. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html