hello, i have some difficulty in understanding multi-device handling in depth. as http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices tells, btrfs can span over multiple devices at the same time. (great feature, btw !) ok then: mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs spanning over /dev/sdb and sdc mount /dev/sdb /btrfs - mounts it btrfs-vol -b /btrfs - does a rebalancing of all data and metadata btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!) but what if i want to remove /dev/sdb ? (as that one is in use for the mount) regards roland __________________________________________________________________________ Verschicken Sie SMS direkt vom Postfach aus - in alle deutschen und viele ausländische Netze zum gleichen Preis! https://produkte.web.de/webde_sms/sms
devzero@web.de wrote:> hello, > > i have some difficulty in understanding multi-device handling in depth. > > as http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices > tells, > > btrfs can span over multiple devices at the same time. (great feature, btw !) > > ok then: > > mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs spanning > over /dev/sdb and sdc > > mount /dev/sdb /btrfs - mounts it > > btrfs-vol -b /btrfs - does a rebalancing of all data and metadata > > btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any > extends in use on sdc to sdb (killer feature!!!) > > but what if i want to remove /dev/sdb ? (as that one is in use for the > mount) >Devices in btrfs are equal, so you can do this. The only glitch is /proc/mounts and mount(8) get confused. Regards Yan Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, 27 Dec 2008 10:15:18 am devzero@web.de wrote:> http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_DevicesI''ll add two questions that''re not answered by the Wiki too.. :-) 1) If you add a second disk to an existing btrfs filesystem, can you get it to set it up as a RAID-1 arrangement rather than just rebalancing the metadata and then striping ? 2) With the concerns that people have about SSD reliability (hi Val :-) !) would it make sense to set up two equal sized partitions on the SSD and use RAID-1 across them, or can you tell btrfs to keep multiple copies of the data, a la ZFS ? cheers! Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
>> i have some difficulty in understanding multi-device handling in depth. >> >> as >> http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices >> tells, >> >> btrfs can span over multiple devices at the same time. (great feature, >> btw !) >> >> ok then: >> >> mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs >> spanning >> over /dev/sdb and sdc >> >> mount /dev/sdb /btrfs - mounts it >> >> btrfs-vol -b /btrfs - does a rebalancing of all data and metadata >> >> btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any >> extends in use on sdc to sdb (killer feature!!!) >> >> but what if i want to remove /dev/sdb ? (as that one is in use for the >> mount) >> > > Devices in btrfs are equal, so you can do this. The only glitch is > /proc/mounts and mount(8) get confused.so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and then re-add - all while mount telling me, that /dev/sdb is still in use !? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , > replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and~~~~~~~~~~~~~~~~~~~~~~~ This step will fail, you will get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it''s ok.> then re-add - all while mount telling me, that /dev/sdb is still in use !?2008/12/27 Roland <devzero@web.de>:>>> i have some difficulty in understanding multi-device handling in depth. >>> >>> as >>> http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices >>> tells, >>> >>> btrfs can span over multiple devices at the same time. (great feature, >>> btw !) >>> >>> ok then: >>> >>> mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs >>> spanning >>> over /dev/sdb and sdc >>> >>> mount /dev/sdb /btrfs - mounts it >>> >>> btrfs-vol -b /btrfs - does a rebalancing of all data and metadata >>> >>> btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any >>> extends in use on sdc to sdb (killer feature!!!) >>> >>> but what if i want to remove /dev/sdb ? (as that one is in use for the >>> mount) >>> >> >> Devices in btrfs are equal, so you can do this. The only glitch is >> /proc/mounts and mount(8) get confused. > > so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , > replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and > then re-add - all while mount telling me, that /dev/sdb is still in use !? > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >-- Regards, Zhu Yanhai -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sun, 28 Dec 2008 21:26:11 +0800 "yanhai zhu" <zhu.yanhai@gmail.com> wrote:> > so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , > > replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and > ~~~~~~~~~~~~~~~~~~~~~~~ > This step will fail, you will > get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it''s ok.Really, what average-joe-user needs btrfs for his not-always-mounted data-partition? Why shouldn''t he just use any other fs for that? All users needing _uptime_ of their fs cannot simply unmount and wait hours for some cool feature to (hopefully) complete (without failures). Why does nobody talk about the real world? What''s the use of features that cannot complete their job in users'' lifetime due to everybody filling up TBs of data on their harddisks. How long does it really take to replace a full 300 GB hd with a 800 GB in a multi volume btrfs ? Please play with real amounts of data and judge for yourself the use of offline-fs-features. The times of classical PC use are gone, today people have just about everything on their hds, music collection, dvd collection, mails, documents of all kinds, sourcecode trees, name-one. 500 GB hds are very cheap. A _new_ fs not able to cope with these new usage-patterns is in fact _useless_. Regards, Stephan> > then re-add - all while mount telling me, that /dev/sdb is still in use !? > > 2008/12/27 Roland <devzero@web.de>: > >>> i have some difficulty in understanding multi-device handling in depth. > >>> > >>> as > >>> http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices > >>> tells, > >>> > >>> btrfs can span over multiple devices at the same time. (great feature, > >>> btw !) > >>> > >>> ok then: > >>> > >>> mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs > >>> spanning > >>> over /dev/sdb and sdc > >>> > >>> mount /dev/sdb /btrfs - mounts it > >>> > >>> btrfs-vol -b /btrfs - does a rebalancing of all data and metadata > >>> > >>> btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any > >>> extends in use on sdc to sdb (killer feature!!!) > >>> > >>> but what if i want to remove /dev/sdb ? (as that one is in use for the > >>> mount) > >>> > >> > >> Devices in btrfs are equal, so you can do this. The only glitch is > >> /proc/mounts and mount(8) get confused. > > > > so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , > > replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and > > then re-add - all while mount telling me, that /dev/sdb is still in use !? > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > -- > Regards, > Zhu Yanhai > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, 27 Dec 2008 5:45:42 pm Chris Samuel wrote:> I''ll add two questions that''re not answered by the Wiki too.. :-)Looking at the source implies to me:> 1) If you add a second disk to an existing btrfs filesystem, can you get it > to set it up as a RAID-1 arrangement rather than just rebalancing the > metadata and then striping ?No, you can''t, the data stripe numbers seem to be set at mkfs time. The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.> 2) With the concerns that people have about SSD reliability (hi Val :-) !) > would it make sense to set up two equal sized partitions on the SSD and use > RAID-1 across them, or can you tell btrfs to keep multiple copies of the > data, a la ZFS ?Again it would appear that you need to have two partitions and that btrfs cannot (at present) keep multiple data stripes on the same partition. Could someone who actually knows the code to comment on whether my inferences are accurate or not please ? :-) cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
>> so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk >> , >> replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk >> and > ~~~~~~~~~~~~~~~~~~~~~~~ > This step will fail, you > will > get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it''s ok.isn`t this sort of a design issue then? no way to work around this ? i think online data migration and disk replacement is a killer feature of btrfs (even zfs doesn`t have this!), but if you always have one out of X disks which can`t be used for this, then it`s of no real value, imho. regards roland> >> then re-add - all while mount telling me, that /dev/sdb is still in use >> !? > > 2008/12/27 Roland <devzero@web.de>: >>>> i have some difficulty in understanding multi-device handling in depth. >>>> >>>> as >>>> http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices >>>> tells, >>>> >>>> btrfs can span over multiple devices at the same time. (great feature, >>>> btw !) >>>> >>>> ok then: >>>> >>>> mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs >>>> spanning >>>> over /dev/sdb and sdc >>>> >>>> mount /dev/sdb /btrfs - mounts it >>>> >>>> btrfs-vol -b /btrfs - does a rebalancing of all data and metadata >>>> >>>> btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes >>>> any >>>> extends in use on sdc to sdb (killer feature!!!) >>>> >>>> but what if i want to remove /dev/sdb ? (as that one is in use for the >>>> mount) >>>> >>> >>> Devices in btrfs are equal, so you can do this. The only glitch is >>> /proc/mounts and mount(8) get confused. >> >> so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk >> , >> replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk >> and >> then re-add - all while mount telling me, that /dev/sdb is still in use >> !? >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > > > -- > Regards, > Zhu Yanhai-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2008/12/29 Chris Samuel <chris@csamuel.org>:> On Sat, 27 Dec 2008 5:45:42 pm Chris Samuel wrote: > >> I''ll add two questions that''re not answered by the Wiki too.. :-) > > Looking at the source implies to me: > >> 1) If you add a second disk to an existing btrfs filesystem, can you get it >> to set it up as a RAID-1 arrangement rather than just rebalancing the >> metadata and then striping ? > > No, you can''t, the data stripe numbers seem to be set at mkfs time. > > The rebalancing code does appear (from a naive read of the code) to be able to > rebalance over stripes, but I have no idea if the disk format currently > supports changing that on the fly. >The rebalancing moves data/metadata to newly created chunks. If there are two devices, the new chunks will be set up as RAID-1 by default.>> 2) With the concerns that people have about SSD reliability (hi Val :-) !) >> would it make sense to set up two equal sized partitions on the SSD and use >> RAID-1 across them, or can you tell btrfs to keep multiple copies of the >> data, a la ZFS ? > > Again it would appear that you need to have two partitions and that btrfs > cannot (at present) keep multiple data stripes on the same partition. >Yes. I think this is due to performance reason. Changing the code to support data duplication in single spindle configuration is easy. Regards Yan Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2008/12/29 Roland <devzero@web.de>:>>> so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk >>> , >>> replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk >>> and >> >> ~~~~~~~~~~~~~~~~~~~~~~~ >> This step will fail, you >> will >> get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it''s ok. > > isn`t this sort of a design issue then? > no way to work around this ? >No, this is a (easy to fix) bug. Regards Yan Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hello Yan, On Mon, 29 Dec 2008 11:33:18 pm Yan Zheng wrote:> 2008/12/29 Chris Samuel <chris@csamuel.org>: > > > The rebalancing code does appear (from a naive read of the code) to be > > able to rebalance over stripes, but I have no idea if the disk format > > currently supports changing that on the fly. > > The rebalancing moves data/metadata to newly created chunks. If there > are two devices, the new chunks will be set up as RAID-1 by default.Very interesting! I didn''t realise that - I was presuming that the BTRFS_BLOCK_GROUP_RAID1 ioctl needed to be passed through (as at mkfs time) to change it into a RAID1 array. Could you point me at the code where this change is detected please ? Just so I can try and wrap my mind around it!> > Again it would appear that you need to have two partitions and that btrfs > > cannot (at present) keep multiple data stripes on the same partition. > > Yes. I think this is due to performance reason.Understood.> Changing the code to support data duplication in single spindle configuration > is easy.OK, that''s encouraging (though for my immediate purposes creating two partitions is fine, but it might make life easier for others). Again thanks so much for the response! All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
2008/12/29 Chris Samuel <chris@csamuel.org>:> Hello Yan, > > On Mon, 29 Dec 2008 11:33:18 pm Yan Zheng wrote: > >> 2008/12/29 Chris Samuel <chris@csamuel.org>: >> >> > The rebalancing code does appear (from a naive read of the code) to be >> > able to rebalance over stripes, but I have no idea if the disk format >> > currently supports changing that on the fly. >> >> The rebalancing moves data/metadata to newly created chunks. If there >> are two devices, the new chunks will be set up as RAID-1 by default. > > Very interesting! I didn''t realise that - I was presuming that the > BTRFS_BLOCK_GROUP_RAID1 ioctl needed to be passed through (as at mkfs time) to > change it into a RAID1 array. >See __btrfs_reserve_extent and __btrfs_alloc_chunk. There is a mistake in my previous reply. Only metadata chunks are set up as RAID-1 by default when there are two devices. Regards Yan Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 2008-12-29 at 23:16 +0800, Yan Zheng wrote:> 2008/12/29 Chris Samuel <chris@csamuel.org>: > > Hello Yan, > > > > On Mon, 29 Dec 2008 11:33:18 pm Yan Zheng wrote: > > > >> 2008/12/29 Chris Samuel <chris@csamuel.org>: > >> > >> > The rebalancing code does appear (from a naive read of the code) to be > >> > able to rebalance over stripes, but I have no idea if the disk format > >> > currently supports changing that on the fly. > >> > >> The rebalancing moves data/metadata to newly created chunks. If there > >> are two devices, the new chunks will be set up as RAID-1 by default. > > > > Very interesting! I didn''t realise that - I was presuming that the > > BTRFS_BLOCK_GROUP_RAID1 ioctl needed to be passed through (as at mkfs time) to > > change it into a RAID1 array. > > > > See __btrfs_reserve_extent and __btrfs_alloc_chunk. There is a mistake in > my previous reply. Only metadata chunks are set up as RAID-1 by default > when there are two devices.This gets confusing in a hurry, but the idea is to duplicate metadata by default. So, if you''re using the default mount options on a single drive and add a second drive, it should switch metadata to raid1. I''ve always planned on adding an option to btrfs-vol -b to have it change the data or metadata allocation policies (raidX to raidY, restripe etc). The kernel side code is all there, we just need something to wire it up to userland. data duplication on a single spindle could work, I just haven''t hooked it up because I couldn''t think of a really strong reason to do it ;) -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 2008-12-29 at 20:35 +0800, Yan Zheng wrote:> 2008/12/29 Roland <devzero@web.de>: > >>> so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk > >>> , > >>> replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk > >>> and > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~ > >> This step will fail, you > >> will > >> get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it''s ok. > > > > isn`t this sort of a design issue then? > > no way to work around this ? > > > > No, this is a (easy to fix) bug.Right, the check for a mounted FS came after we made the device add/removal work ;) mkfs.btrfs should be changed to do the mounted FS check differently. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 31 Dec 2008 8:25:08 am Chris Mason wrote:> This gets confusing in a hurry, but the idea is to duplicate metadata by > default. So, if you''re using the default mount options on a single > drive and add a second drive, it should switch metadata to raid1.Yup, that''s what I inferred from your earlier posts and the code. :-)> I''ve always planned on adding an option to btrfs-vol -b to have it > change the data or metadata allocation policies (raidX to raidY, > restripe etc). The kernel side code is all there, we just need > something to wire it up to userland.Understood. If I get some Copious Spare Time (tm) I might take a look at that (though don''t let that stop anyone else trying first!).> data duplication on a single spindle could work, I just haven''t hooked > it up because I couldn''t think of a really strong reason to do it ;)I think for SSD''s it could be really handy, especially for people who have a standard distro install and decide they want to convert their ext3 partition to btrfs using the tools. It''s just a bit easier than getting them to backup, repartition, make the filesystem and restore. Downside of course is that those original blocks never get mirrored (unless something like btrfsck was extended to fix those things up). cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP