Right now, the majority of Linux users probably have LVM on their SAN devices (i.e those being iSCSI targets). Using LVM on a SAN device is easy: just create a new logical volume or its snapshot, make it a target to iSCSI initiators, done. I was wondering how btrfs would fit here and if it could replace LVM. I see the following benefits of using btrfs instead of LVM: - you can create sparse files which would grow as iSCSI initators use more space (you can do it with ext3 now as well) - you can use btrfs compression, to further reduce used space and perhaps increase speed (SANs are mostly IO bound, not CPU bound) - LVM has a big performance hit when using snapshots; btrfs doesn''t However, with btrfs, I''m not sure about: - what happens if SAN machine crashes while the iSCSI file images were being written to; with LVM and its block devices, I''m somehow more confident it wouldn''t make more data loss than necessary - taking snapshots of individual files (file images on SAN) is not possible with btrfs? Probably they would have to be placed in separate directories first to make snapshots - some minor manageability issue -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Apr 23, 2009 at 5:20 AM, Tomasz Chmielewski <mangoo@wpkg.org> wrote:> - what happens if SAN machine crashes while the iSCSI file images were being > written to; with LVM and its block devices, I''m somehow more confident it > wouldn''t make more data loss than necessaryI would hope that COW applies to such a setting. It certainly does for ZFS, making it an excellent backend for iSCSI. At least, having it on btrfs shouldn''t make it any less reliable than on LVM, as long as btrfs does its job correctly. -- Dmitri Nikulin Centre for Synchrotron Science Monash University Victoria 3800, Australia -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 2009-04-22 at 21:20 +0200, Tomasz Chmielewski wrote:> Right now, the majority of Linux users probably have LVM on their SAN > devices (i.e those being iSCSI targets). > > Using LVM on a SAN device is easy: just create a new logical volume or > its snapshot, make it a target to iSCSI initiators, done. >This is definitely one of the target use cases for btrfs.> I was wondering how btrfs would fit here and if it could replace LVM. > > > I see the following benefits of using btrfs instead of LVM: > > - you can create sparse files which would grow as iSCSI initators use > more space (you can do it with ext3 now as well) > > - you can use btrfs compression, to further reduce used space and > perhaps increase speed (SANs are mostly IO bound, not CPU bound) > > - LVM has a big performance hit when using snapshots; btrfs doesn''t > > > > However, with btrfs, I''m not sure about: > > - what happens if SAN machine crashes while the iSCSI file images were > being written to; with LVM and its block devices, I''m somehow more > confident it wouldn''t make more data loss than necessaryIf iscsi is writing with O_DIRECT|O_SYNC it should work. But, tuning for this config is something we have to concentrate more on.> > - taking snapshots of individual files (file images on SAN) is not > possible with btrfs? Probably they would have to be placed in separate > directories first to make snapshots - some minor manageability issueBtrfs can''t snapshot a single dir, but it can snapshot a single file. See the bcp command included with btrfs-progs. I''d also suggest using preallocated files (via fallocate) instead of sparse files. I will perform better in general. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Chris Mason schrieb:>> However, with btrfs, I''m not sure about: >> >> - what happens if SAN machine crashes while the iSCSI file images were >> being written to; with LVM and its block devices, I''m somehow more >> confident it wouldn''t make more data loss than necessary > > If iscsi is writing with O_DIRECT|O_SYNC it should work.What if it doesn''t?> But, tuning > for this config is something we have to concentrate more on. > >> - taking snapshots of individual files (file images on SAN) is not >> possible with btrfs? Probably they would have to be placed in separate >> directories first to make snapshots - some minor manageability issue > > Btrfs can''t snapshot a single dir, but it can snapshot a single file. > See the bcp command included with btrfs-progs. > > I''d also suggest using preallocated files (via fallocate) instead of > sparse files. I will perform better in general.When performance is needed - sure. -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 2009-04-24 at 01:34 +0200, Tomasz Chmielewski wrote:> Chris Mason schrieb: > > >> However, with btrfs, I''m not sure about: > >> > >> - what happens if SAN machine crashes while the iSCSI file images were > >> being written to; with LVM and its block devices, I''m somehow more > >> confident it wouldn''t make more data loss than necessary > > > > If iscsi is writing with O_DIRECT|O_SYNC it should work. > > What if it doesn''t? >Writes would go to the page cache only (not O_DIRECT) and the metadata wouldn''t get flushed with each write (not O_SYNC). -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Chris Mason wrote:> On Fri, 2009-04-24 at 01:34 +0200, Tomasz Chmielewski wrote: >> Chris Mason schrieb: >> >>>> However, with btrfs, I''m not sure about: >>>> >>>> - what happens if SAN machine crashes while the iSCSI file images were >>>> being written to; with LVM and its block devices, I''m somehow more >>>> confident it wouldn''t make more data loss than necessary >>> If iscsi is writing with O_DIRECT|O_SYNC it should work. >> What if it doesn''t? >> > > Writes would go to the page cache only (not O_DIRECT) and the metadata > wouldn''t get flushed with each write (not O_SYNC).What about the "administrative" part of using btrfs as a volume manager? For example, I can partition (fdisk + kpartx), extend volumes in LVM easily, on the host. Are there any tools to "partition" or "extend" file images? qemu-nbd and nbd-client come to mind, but it''s a bit of an overhead (in typing, I mean). -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Jul 02, 2009 at 05:22:24PM +0200, Tomasz Chmielewski wrote:> Chris Mason wrote: >> On Fri, 2009-04-24 at 01:34 +0200, Tomasz Chmielewski wrote: >>> Chris Mason schrieb: >>> >>>>> However, with btrfs, I''m not sure about: >>>>> >>>>> - what happens if SAN machine crashes while the iSCSI file images >>>>> were being written to; with LVM and its block devices, I''m >>>>> somehow more confident it wouldn''t make more data loss than >>>>> necessary >>>> If iscsi is writing with O_DIRECT|O_SYNC it should work. >>> What if it doesn''t? >>> >> >> Writes would go to the page cache only (not O_DIRECT) and the metadata >> wouldn''t get flushed with each write (not O_SYNC). > > What about the "administrative" part of using btrfs as a volume manager?In this case it depends on exactly how you do it. If you''re just using files and putting filesystems on those files (not partitioning inside the image), then you can use truncate to shrink the image and fallocate (or truncate if you want it sparse) to extend it. If you want to make partition tables inside the volumes, then you need to do the truncate/fallocate game and also change the partition table. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html