Hi, I think it would be great if there is a lvm volume or zfs zvol type support in btrfs. As far as I can tell, there''s nobody actively working on this feature. I want to know what the core developers think of this feature, is it technically possible? any strong opinions? implementation ideas? I''d be happy to work towards this feature, but want your feedback before proceeding. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote:> Hi, > > I think it would be great if there is a lvm volume or zfs zvol type > support in btrfs. As far as I can tell, there''s nobody actively > working on this feature. I want to know what the core developers think > of this feature, is it technically possible? any strong opinions? > implementation ideas? > > I''d be happy to work towards this feature, but want your feedback > before proceeding.Btrfs already has capabilities to add and remove block devices on the fly. Data can be stripped or mirrored or both. Raid 5/6 is in testing at the moment. https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices https://btrfs.wiki.kernel.org/index.php/UseCases#RAID Which specific features do you think btrfs is lacking? Mike -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Feb 26, 2013 at 11:59 AM, Mike Fleetwood <mike.fleetwood@googlemail.com> wrote:> On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote: >> Hi, >> >> I think it would be great if there is a lvm volume or zfs zvol type >> support in btrfs.> Btrfs already has capabilities to add and remove block devices on the > fly. Data can be stripped or mirrored or both. Raid 5/6 is in > testing at the moment. > https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices > https://btrfs.wiki.kernel.org/index.php/UseCases#RAID > > Which specific features do you think btrfs is lacking?I think he''s talking about zvol-like feature. In zfs, instead of creating a filesystem-that-is-accessible-as-a-directory, you can create a zvol which behaves just like any other standard block device (e.g. you can use it as swap, or create ext4 filesystem on top of it). But it would also have most of the benefits that a normal zfs filesystem has, like: - thin provisioning (sparse allocation, snapshot & clone) - compression - integrity check (via checksum) Typical use cases would be: - swap in a pure-zfs system - virtualization (xen, kvm, etc) - NAS which exports the block device using iscsi/AoE AFAIK no such feature exist in btrfs yet. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Yes, zvol like feature where a btrfs subvolume like construct can be made available as a LUN/block device. This device can then be used by any application that wants a raw block device. iscsi is another obvious usecase. Having thin provisioning support would make it pretty awesome. Suman On Mon, Feb 25, 2013 at 5:46 PM, Fajar A. Nugraha <list@fajar.net> wrote:> On Tue, Feb 26, 2013 at 11:59 AM, Mike Fleetwood > <mike.fleetwood@googlemail.com> wrote: >> On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote: >>> Hi, >>> >>> I think it would be great if there is a lvm volume or zfs zvol type >>> support in btrfs. > > >> Btrfs already has capabilities to add and remove block devices on the >> fly. Data can be stripped or mirrored or both. Raid 5/6 is in >> testing at the moment. >> https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices >> https://btrfs.wiki.kernel.org/index.php/UseCases#RAID >> >> Which specific features do you think btrfs is lacking? > > > I think he''s talking about zvol-like feature. > > In zfs, instead of creating a > filesystem-that-is-accessible-as-a-directory, you can create a zvol > which behaves just like any other standard block device (e.g. you can > use it as swap, or create ext4 filesystem on top of it). But it would > also have most of the benefits that a normal zfs filesystem has, like: > - thin provisioning (sparse allocation, snapshot & clone) > - compression > - integrity check (via checksum) > > Typical use cases would be: > - swap in a pure-zfs system > - virtualization (xen, kvm, etc) > - NAS which exports the block device using iscsi/AoE > > AFAIK no such feature exist in btrfs yet. > > -- > Fajar-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Can''t thus be done with a regular file and a loop back device? Remco On 26 Feb 2013, at 06:35, Suman C <schakrava@gmail.com> wrote:> Yes, zvol like feature where a btrfs subvolume like construct can be > made available as a LUN/block device. This device can then be used by > any application that wants a raw block device. iscsi is another > obvious usecase. Having thin provisioning support would make it pretty > awesome. > > Suman > > On Mon, Feb 25, 2013 at 5:46 PM, Fajar A. Nugraha <list@fajar.net> wrote: >> On Tue, Feb 26, 2013 at 11:59 AM, Mike Fleetwood >> <mike.fleetwood@googlemail.com> wrote: >>> On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote: >>>> Hi, >>>> >>>> I think it would be great if there is a lvm volume or zfs zvol type >>>> support in btrfs. >> >> >>> Btrfs already has capabilities to add and remove block devices on the >>> fly. Data can be stripped or mirrored or both. Raid 5/6 is in >>> testing at the moment. >>> https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices >>> https://btrfs.wiki.kernel.org/index.php/UseCases#RAID >>> >>> Which specific features do you think btrfs is lacking? >> >> >> I think he''s talking about zvol-like feature. >> >> In zfs, instead of creating a >> filesystem-that-is-accessible-as-a-directory, you can create a zvol >> which behaves just like any other standard block device (e.g. you can >> use it as swap, or create ext4 filesystem on top of it). But it would >> also have most of the benefits that a normal zfs filesystem has, like: >> - thin provisioning (sparse allocation, snapshot & clone) >> - compression >> - integrity check (via checksum) >> >> Typical use cases would be: >> - swap in a pure-zfs system >> - virtualization (xen, kvm, etc) >> - NAS which exports the block device using iscsi/AoE >> >> AFAIK no such feature exist in btrfs yet. >> >> -- >> Fajar > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 25 Feb 2013 21:35:08 -0800 Suman C <schakrava@gmail.com> wrote:> Yes, zvol like feature where a btrfs subvolume like construct can be > made available as a LUN/block device. This device can then be used by > any application that wants a raw block device. iscsi is another > obvious usecase. Having thin provisioning support would make it pretty > awesome.I think what you are missing is that btrfs is a filesystem, not a block device management mechanism. For your use case can simply create a snapshot and then make a sparse file inside of it. btrfs sub create foobar dd if=/dev/zero of=foobar/100GB.img bs=1 count=1 seek=100G If you need this to be a block device, use ''losetup'' to make foobar/100GB.img appear as one (/dev/loopX). But iSCSI/AoE/NBD can export files as well as block devices, so this is not even necessary. -- With respect, Roman
Thanks for the sparse file idea, I am actually using that solution already. I am not sure if its the best way, however. Suman On Mon, Feb 25, 2013 at 9:57 PM, Roman Mamedov <rm@romanrm.ru> wrote:> On Mon, 25 Feb 2013 21:35:08 -0800 > Suman C <schakrava@gmail.com> wrote: > >> Yes, zvol like feature where a btrfs subvolume like construct can be >> made available as a LUN/block device. This device can then be used by >> any application that wants a raw block device. iscsi is another >> obvious usecase. Having thin provisioning support would make it pretty >> awesome. > > I think what you are missing is that btrfs is a filesystem, not a block device > management mechanism. > > For your use case can simply create a snapshot and then make a sparse file > inside of it. > > btrfs sub create foobar > dd if=/dev/zero of=foobar/100GB.img bs=1 count=1 seek=100G > > If you need this to be a block device, use ''losetup'' to make foobar/100GB.img > appear as one (/dev/loopX). But iSCSI/AoE/NBD can export files as well as block > devices, so this is not even necessary. > > -- > With respect, > Roman-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
would be really cool if a TRIM to the loopback device would do a ''hole punch'' on the file Remco On Feb 26, 2013, at 7:25 AM, Suman C <schakrava@gmail.com> wrote:> Thanks for the sparse file idea, I am actually using that solution > already. I am not sure if its the best way, however. > > Suman > > On Mon, Feb 25, 2013 at 9:57 PM, Roman Mamedov <rm@romanrm.ru> wrote: >> On Mon, 25 Feb 2013 21:35:08 -0800 >> Suman C <schakrava@gmail.com> wrote: >> >>> Yes, zvol like feature where a btrfs subvolume like construct can be >>> made available as a LUN/block device. This device can then be used by >>> any application that wants a raw block device. iscsi is another >>> obvious usecase. Having thin provisioning support would make it pretty >>> awesome. >> >> I think what you are missing is that btrfs is a filesystem, not a block device >> management mechanism. >> >> For your use case can simply create a snapshot and then make a sparse file >> inside of it. >> >> btrfs sub create foobar >> dd if=/dev/zero of=foobar/100GB.img bs=1 count=1 seek=100G >> >> If you need this to be a block device, use ''losetup'' to make foobar/100GB.img >> appear as one (/dev/loopX). But iSCSI/AoE/NBD can export files as well as block >> devices, so this is not even necessary. >> >> -- >> With respect, >> Roman > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Remco Hosman - Yerf IT wrote:> would be really cool if a TRIM to the loopback device would do a ''hole > punch'' on the fileThere are patches on the scsi target mailing list to make this happen for the FILEIO backend. This has the added benefit that if you set it up via LIO, it appears as a full SCSI device, and can be exported via iSCSI, firewire, FC, USB, etc as well as being visible as a local disk. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Feb 26, 2013 at 07:28:43AM +0100, Remco Hosman - Yerf IT wrote:> would be really cool if a TRIM to the loopback device would do a > ''hole punch'' on the filedfaa2ef loop: add discard support for loop devices -- Dave Chinner david@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Dienstag, 26. Februar 2013 schrieb Fajar A. Nugraha:> On Tue, Feb 26, 2013 at 11:59 AM, Mike Fleetwood > > <mike.fleetwood@googlemail.com> wrote: > > On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote: > >> Hi, > >> > >> I think it would be great if there is a lvm volume or zfs zvol type > >> support in btrfs. > > > > Btrfs already has capabilities to add and remove block devices on the > > fly. Data can be stripped or mirrored or both. Raid 5/6 is in > > testing at the moment. > > https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devic > > es https://btrfs.wiki.kernel.org/index.php/UseCases#RAID > > > > Which specific features do you think btrfs is lacking? > > I think he''s talking about zvol-like feature. > > In zfs, instead of creating a > filesystem-that-is-accessible-as-a-directory, you can create a zvol > which behaves just like any other standard block device (e.g. you can > use it as swap, or create ext4 filesystem on top of it). But it would > also have most of the benefits that a normal zfs filesystem has, like: > - thin provisioning (sparse allocation, snapshot & clone) > - compression > - integrity check (via checksum) > > Typical use cases would be: > - swap in a pure-zfs system > - virtualization (xen, kvm, etc) > - NAS which exports the block device using iscsi/AoE > > AFAIK no such feature exist in btrfs yet.Sounds like the RADOS block device stuff for Ceph. -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Feb 26, 2013 at 9:30 PM, Martin Steigerwald <Martin@lichtvoll.de> wrote:> Am Dienstag, 26. Februar 2013 schrieb Fajar A. Nugraha: >> On Tue, Feb 26, 2013 at 11:59 AM, Mike Fleetwood >> >> <mike.fleetwood@googlemail.com> wrote: >> > On 25 February 2013 23:35, Suman C <schakrava@gmail.com> wrote: >> >> Hi, >> >> >> >> I think it would be great if there is a lvm volume or zfs zvol type >> >> support in btrfs. >> > >> > Btrfs already has capabilities to add and remove block devices on the >> > fly. Data can be stripped or mirrored or both. Raid 5/6 is in >> > testing at the moment. >> > https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devic >> > es https://btrfs.wiki.kernel.org/index.php/UseCases#RAID >> > >> > Which specific features do you think btrfs is lacking? >> >> I think he''s talking about zvol-like feature. >> >> In zfs, instead of creating a >> filesystem-that-is-accessible-as-a-directory, you can create a zvol >> which behaves just like any other standard block device (e.g. you can >> use it as swap, or create ext4 filesystem on top of it). But it would >> also have most of the benefits that a normal zfs filesystem has, like: >> - thin provisioning (sparse allocation, snapshot & clone) >> - compression >> - integrity check (via checksum) >> >> Typical use cases would be: >> - swap in a pure-zfs system >> - virtualization (xen, kvm, etc) >> - NAS which exports the block device using iscsi/AoE >> >> AFAIK no such feature exist in btrfs yet. > > Sounds like the RADOS block device stuff for Ceph.Exactly. While using files + loopback device mostly works, there were problems regarding performance and data integrity. Not to mention the hassle in accessing the data if it resides on a partition inside the file (e.g. you need losetup + kpartx to access it, and you must remember to do the reverse when you''re finished with it). In zfsonlinux it''s very easy to do so since a zvol is treated pretty much like a disk, and whenever there''s a partition inside a zvol, a coressponding device noed is also created automatically. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 27 Feb 2013 13:23:23 +1100 "Fajar A. Nugraha" <list@fajar.net> wrote:> Not to mention the hassle in accessing the data if it resides on a > partition inside the file (e.g. you need losetup + kpartx to access it, > and you must remember to do the reverse when you''re finished with it).> > In zfsonlinux it''s very easy to do so since a zvol is treated pretty > much like a disk, and whenever there''s a partition inside a zvol, a > coressponding device noed is also created automatically.So I''d say what you (we) need is a generic Linux kernel framework that would allow treating any regular file pretty much like a disk. Not some filesystem-specific block device emulation kludge. Btw some years ago there was a patchset adding proper automatic partition support to ''loop''; but it seems like that went nowhere, and I have no idea why something this useful did not end up being added into the mainline kernel. -- With respect, Roman
Am Mittwoch, 27. Februar 2013 schrieb Roman Mamedov:> On Wed, 27 Feb 2013 13:23:23 +1100 > > "Fajar A. Nugraha" <list@fajar.net> wrote: > > Not to mention the hassle in accessing the data if it resides on a > > partition inside the file (e.g. you need losetup + kpartx to access it, > > and you must remember to do the reverse when you''re finished with it). > > > > > > > > In zfsonlinux it''s very easy to do so since a zvol is treated pretty > > much like a disk, and whenever there''s a partition inside a zvol, a > > coressponding device noed is also created automatically. > > So I''d say what you (we) need is a generic Linux kernel framework that > would allow treating any regular file pretty much like a disk. Not some > filesystem-specific block device emulation kludge. > > Btw some years ago there was a patchset adding proper automatic partition > support to ''loop''; but it seems like that went nowhere, and I have no > idea why something this useful did not end up being added into the > mainline kernel.Are you sure about the partition support? I thought something related to loop partition support has gone into some not so recent kernel. -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Roman Mamedov wrote:> On Wed, 27 Feb 2013 13:23:23 +1100 > "Fajar A. Nugraha" <list@fajar.net> wrote: > >> Not to mention the hassle in accessing the data if it resides on a >> partition inside the file (e.g. you need losetup + kpartx to access it, >> and you must remember to do the reverse when you''re finished with it). > >> >> In zfsonlinux it''s very easy to do so since a zvol is treated pretty >> much like a disk, and whenever there''s a partition inside a zvol, a >> coressponding device noed is also created automatically. > > So I''d say what you (we) need is a generic Linux kernel framework that > would allow treating any regular file pretty much like a disk. Not some > filesystem-specific block device emulation kludge. > > Btw some years ago there was a patchset adding proper automatic partition > support to ''loop''; but it seems like that went nowhere, and I have no idea > why something this useful did not end up being added into the mainline > kernel. >I mentioned it in another branch of the thread, but the SCSI Target stack *does* this. If you use the FILEIO backend and the ''loopback'' transport, you get a file that is visible to the local system as Just Another SCSI Disk. There are patches on the scsi target list to add UNMAP and WRITE SAME w/UNMAP=1 support to the FILEIO backend, at which point it will do FL_PUNCH_HOLE operations appropriately. The commands are: ---cut--- # $IDX = a preferably-unique index # $NAME = a useful name for humans # $IMGFILE = a disk image # $SIZE = the size of $IMGFILE in bytes # $UUID = a random v4 UUID # ${NAA[x]} = a valid naa WWN, targetcli generates this as # "naa.6001405" + the first 10 digits of a random v4 UUID # (not the same uuid as $UUID). When x varies, it''s a different # WWN, since more than one gets used. # Load the scsi target core and backends modprobe target_core_mod # Tell it where the file is and how big it is tcm_node --establishdev "fileio_$IDX/$NAME" \ "fd_dev_name=$IMGFILE,fd_dev_size=$SIZE" # Give it a unique serial tcm_node --setunitserialwithmd fileio_$IDX/$NAME $UUID # Load the local scsi frontend transport modprobe tcm_loop mkdir -p /sys/kernel/config/target/loopback # Some setup so you have a place to put LUNs mkdir -p /sys/kernel/config/target/loopback/${NAA[1]}/tpgt_1 echo ${NAA[2]} > /sys/kernel/config/target/loopback/${NAA[1]}/tpgt_1/nexus # Create a fresh LUN... mkdir -p /sys/kernel/config/target/loopback/${NAA[1]}/tpgt_1/lun/lun_$IDX # ...and map the file to it. ln -s /sys/kernel/config/target/core/fileio_$IDX/$NAME \ /sys/kernel/config/target/loopback/${NAA[1]}/tpgt_1/lun/lun_$IDX ---cut--- This could be pretty easily put into a shell script that uses du -b and manually pokes configfs instead of calling tcm_node, and it''d be able to run without any nonstandard userspace dependencies. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 27 Feb 2013 09:42:02 +0100 Martin Steigerwald <Martin@lichtvoll.de> wrote:> Are you sure about the partition support? I thought something related to > loop partition support has gone into some not so recent kernel.Sorry, you are correct, this was in fact added since 2.6.26. Just tried out making a partitioned loop device: dd if=/dev/zero of=100GB bs=1 count=1 seek=100G modprobe loop max_part=8 losetup /dev/loop7 100GB cfdisk /dev/loop7 [ made two partitions, commit, quit cfdisk ] [ ...but partitions did not automatically appear as /dev/loop7pX ] blockdev --rereadpt /dev/loop7 [ OK, now here they are: ] $ ls -la /dev/loop7p? brw-rw---T 1 root disk 7, 113 Feb 27 15:10 /dev/loop7p1 brw-rw---T 1 root disk 7, 114 Feb 27 15:10 /dev/loop7p2 -- With respect, Roman
Alex Elsayed wrote:> Roman Mamedov wrote: > >> On Wed, 27 Feb 2013 13:23:23 +1100 >> "Fajar A. Nugraha" <list@fajar.net> wrote:<snip>> This could be pretty easily put into a shell script that uses du -b and > manually pokes configfs instead of calling tcm_node, and it''d be able to > run without any nonstandard userspace dependencies.Just for fun, I decided to put my money where my mouth is and implement a quick scsi-target-losetup that actually worked, both for creation and deletion. Here it is: ---cut--- #!/bin/bash gen_naa() { local UUID="$( uuidgen -r )" UUID="${UUID//-/}" UUID="${UUID:0:9}" echo "naa.6001405${UUID}" } setup() { local FILE local INPUT_NAME local NAME local BACKEND_IDX local TRANSPORT_IDX declare -a NAA FILE="${1}" INPUT_NAME="${2}" NAME="${INPUT_NAME//\//_}" BACKEND_IDX=''-1'' TRANSPORT_IDX=''-1'' if [[ $UID -ne 0 ]]; then echo "You must be root in order to set up a lioloop device" >&2 exit 1 fi if [[ "${NAME}" != "${INPUT_NAME}" ]]; then echo "The chosen name ''${INPUT_NAME}'' contained slashes, using ''${NAME}'' instead" >&2 fi declare SIZE="$(du -b "${FILE}")" SIZE="${SIZE/[^0123456789]*}" # Load the scsi target core and backends modprobe target_core_mod >/dev/null 2>&1 while BACKEND_IDX=$((BACKEND_IDX + 1)); do if [[ -d "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" ]]; then echo "A backstore with the name ''${NAME}'' already exists" >&2 exit 1 elif ! [[ -d /sys/kernel/config/target/core/fileio_${BACKEND_IDX} ]]; then # mkdir -p "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" # Tell it where the file is and how big it is tcm_node --establishdev "fileio_${BACKEND_IDX}/${NAME}" \ "fd_dev_name=${FILE},fd_dev_size=${SIZE}" # Give it a unique serial tcm_node --setunitserialwithmd "fileio_${BACKEND_IDX}/${NAME}" "$(uuidgen -r)" break fi done # Load the local scsi frontend transport modprobe tcm_loop >/dev/null 2>&1 mkdir -p /sys/kernel/config/target/loopback NAA=( "$(gen_naa)" "$(gen_naa)" ) # Some setup so you have a place to put LUNs mkdir -p "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1" echo "${NAA[1]}" > "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/nexus" # Create a fresh LUN... while TRANSPORT_IDX=$((TRANSPORT_IDX + 1)); do if ! [[ -d "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" ]]; then mkdir -p "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" # ...and map the file to it. ln -s "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" \ "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" break fi done } teardown() { local INPUT_NAME="${1}" local NAME="${INPUT_NAME//\//_}" if [[ $UID -ne 0 ]]; then echo "You must be root in order to tear down a lioloop device" >&2 exit 1 fi if [[ "${NAME}" != "${INPUT_NAME}" ]]; then echo "The chosen name ''${INPUT_NAME}'' contained slashes, using ''${NAME}'' instead" >&2 fi local FOUND='''' for LUN in /sys/kernel/config/target/loopback/*/tpgt_1/lun/lun_*; do if [[ -L "${LUN}/${NAME}" ]]; then rm -f "${LUN}/${NAME}" FOUND=1 fi done if [[ -z "${FOUND}" ]]; then echo "No lioloop with the name ''${NAME}'' was found" >&2 return fi for BACKSTORE in /sys/kernel/config/target/core/fileio_*; do if [[ -d "${BACKSTORE}/${NAME}" ]]; then rmdir "${BACKSTORE}/${NAME}" fi done } if [[ "$1" == ''-d'' ]]; then shift; for name in "$@"; do teardown "${name}" done else setup "$@" fi -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Alex Elsayed wrote:
...and a second version, because I forgot to actually remove the calls to
tcm_node in favor of poking configfs like I said.
---cut---
#!/bin/bash
gen_naa() {
local UUID="$( uuidgen -r )"
UUID="${UUID//-/}"
UUID="${UUID:0:9}"
echo "naa.6001405${UUID}"
}
setup() {
local FILE
local INPUT_NAME
local NAME
local BACKEND_IDX
local TRANSPORT_IDX
declare -a NAA
FILE="${1}"
INPUT_NAME="${2}"
NAME="${INPUT_NAME//\//_}"
BACKEND_IDX=''-1''
TRANSPORT_IDX=''-1''
if [[ $UID -ne 0 ]]; then
echo "You must be root in order to set up a lioloop device"
>&2
exit 1
fi
if [[ "${NAME}" != "${INPUT_NAME}" ]]; then
echo "The chosen name ''${INPUT_NAME}'' contained
slashes, using ''${NAME}'' instead" >&2
fi
declare SIZE="$(du -b "${FILE}")"
SIZE="${SIZE/[^0123456789]*}"
# Load the scsi target core and backends
modprobe target_core_mod >/dev/null 2>&1
while BACKEND_IDX=$((BACKEND_IDX + 1)); do
if [[ -d
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" ]];
then
echo "A backstore with the name ''${NAME}''
already exists" >&2
exit 1
elif ! [[ -d /sys/kernel/config/target/core/fileio_${BACKEND_IDX} ]];
then
# Create the backstore
mkdir -p
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}"
# Tell it where the file is and how big it is
echo "fd_dev_name=${FILE},fd_dev_size=${SIZE}" > \
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}/control"
# Turn it on
echo 1 >
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}/enable"
# Give it a unique serial
echo "$(uuidgen -r)" >
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}/wwn/vpd_unit_serial"
break
fi
done
# Load the local scsi frontend transport
modprobe tcm_loop >/dev/null 2>&1
mkdir -p /sys/kernel/config/target/loopback
NAA=( "$(gen_naa)" "$(gen_naa)" )
# Some setup so you have a place to put LUNs
mkdir -p "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1"
echo "${NAA[1]}" >
"/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/nexus"
# Create a fresh LUN...
while TRANSPORT_IDX=$((TRANSPORT_IDX + 1)); do
if ! [[ -d
"/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}"
]]; then
mkdir -p
"/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}"
# ...and map the file to it.
ln -s
"/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" \
"/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}"
break
fi
done
}
teardown() {
local INPUT_NAME="${1}"
local NAME="${INPUT_NAME//\//_}"
if [[ $UID -ne 0 ]]; then
echo "You must be root in order to tear down a lioloop device"
>&2
exit 1
fi
if [[ "${NAME}" != "${INPUT_NAME}" ]]; then
echo "The chosen name ''${INPUT_NAME}'' contained
slashes, using ''${NAME}'' instead" >&2
fi
local FOUND=''''
for LUN in /sys/kernel/config/target/loopback/*/tpgt_1/lun/lun_*; do
if [[ -L "${LUN}/${NAME}" ]]; then
rm -f "${LUN}/${NAME}"
FOUND=1
fi
done
if [[ -z "${FOUND}" ]]; then
echo "No lioloop with the name ''${NAME}'' was
found" >&2
return
fi
for BACKSTORE in /sys/kernel/config/target/core/fileio_*; do
if [[ -d "${BACKSTORE}/${NAME}" ]]; then
rmdir "${BACKSTORE}/${NAME}"
fi
done
}
if [[ "$1" == ''-d'' ]]; then
shift;
for name in "$@"; do
teardown "${name}"
done
else
setup "$@"
fi
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
That''s great, but the issue is that usually the block device version performs better than just creating a file and using it as a raw image or loop device. Creating a file, then running it through a SCSI target seems like it''s going in the opposite direction. On Wed, Feb 27, 2013 at 2:57 AM, Alex Elsayed <eternaleye@gmail.com> wrote:> Alex Elsayed wrote: > >> Roman Mamedov wrote: >> >>> On Wed, 27 Feb 2013 13:23:23 +1100 >>> "Fajar A. Nugraha" <list@fajar.net> wrote: > > <snip> > >> This could be pretty easily put into a shell script that uses du -b and >> manually pokes configfs instead of calling tcm_node, and it''d be able to >> run without any nonstandard userspace dependencies. > > Just for fun, I decided to put my money where my mouth is and implement > a quick scsi-target-losetup that actually worked, both for creation and > deletion. Here it is: > > ---cut--- > > #!/bin/bash > > gen_naa() { > local UUID="$( uuidgen -r )" > UUID="${UUID//-/}" > UUID="${UUID:0:9}" > echo "naa.6001405${UUID}" > } > > setup() { > local FILE > local INPUT_NAME > local NAME > local BACKEND_IDX > local TRANSPORT_IDX > declare -a NAA > FILE="${1}" > INPUT_NAME="${2}" > NAME="${INPUT_NAME//\//_}" > BACKEND_IDX=''-1'' > TRANSPORT_IDX=''-1'' > > if [[ $UID -ne 0 ]]; then > echo "You must be root in order to set up a lioloop device" >&2 > exit 1 > fi > > if [[ "${NAME}" != "${INPUT_NAME}" ]]; then > echo "The chosen name ''${INPUT_NAME}'' contained slashes, using ''${NAME}'' instead" >&2 > fi > > declare SIZE="$(du -b "${FILE}")" > SIZE="${SIZE/[^0123456789]*}" > > # Load the scsi target core and backends > modprobe target_core_mod >/dev/null 2>&1 > > while BACKEND_IDX=$((BACKEND_IDX + 1)); do > if [[ -d "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" ]]; then > echo "A backstore with the name ''${NAME}'' already exists" >&2 > exit 1 > elif ! [[ -d /sys/kernel/config/target/core/fileio_${BACKEND_IDX} ]]; then > # mkdir -p "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" > # Tell it where the file is and how big it is > tcm_node --establishdev "fileio_${BACKEND_IDX}/${NAME}" \ > "fd_dev_name=${FILE},fd_dev_size=${SIZE}" > # Give it a unique serial > tcm_node --setunitserialwithmd "fileio_${BACKEND_IDX}/${NAME}" "$(uuidgen -r)" > break > fi > done > > # Load the local scsi frontend transport > modprobe tcm_loop >/dev/null 2>&1 > mkdir -p /sys/kernel/config/target/loopback > > NAA=( "$(gen_naa)" "$(gen_naa)" ) > # Some setup so you have a place to put LUNs > mkdir -p "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1" > echo "${NAA[1]}" > "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/nexus" > > # Create a fresh LUN... > while TRANSPORT_IDX=$((TRANSPORT_IDX + 1)); do > if ! [[ -d "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" ]]; then > mkdir -p "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" > # ...and map the file to it. > ln -s "/sys/kernel/config/target/core/fileio_${BACKEND_IDX}/${NAME}" \ > "/sys/kernel/config/target/loopback/${NAA[0]}/tpgt_1/lun/lun_${TRANSPORT_IDX}" > break > fi > done > } > > teardown() { > local INPUT_NAME="${1}" > local NAME="${INPUT_NAME//\//_}" > > if [[ $UID -ne 0 ]]; then > echo "You must be root in order to tear down a lioloop device" >&2 > exit 1 > fi > > if [[ "${NAME}" != "${INPUT_NAME}" ]]; then > echo "The chosen name ''${INPUT_NAME}'' contained slashes, using ''${NAME}'' instead" >&2 > fi > > local FOUND='''' > for LUN in /sys/kernel/config/target/loopback/*/tpgt_1/lun/lun_*; do > if [[ -L "${LUN}/${NAME}" ]]; then > rm -f "${LUN}/${NAME}" > FOUND=1 > fi > done > > if [[ -z "${FOUND}" ]]; then > echo "No lioloop with the name ''${NAME}'' was found" >&2 > return > fi > > for BACKSTORE in /sys/kernel/config/target/core/fileio_*; do > if [[ -d "${BACKSTORE}/${NAME}" ]]; then > rmdir "${BACKSTORE}/${NAME}" > fi > done > } > > if [[ "$1" == ''-d'' ]]; then > shift; > for name in "$@"; do > teardown "${name}" > done > else > setup "$@" > fi > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi, On Wed, Feb 27, 2013 at 02:12:56AM -0800, Alex Elsayed wrote:> Alex Elsayed wrote: > > ...and a second version, because I forgot to actually remove the calls to > tcm_node in favor of poking configfs like I said.I''d like to use your script to setup scsi devices for testing, however I''ve encountered 2 things did not work with the scsi emulation. Setup: There''s a 10G file image created by dd and exported via your script as /dev/sdl. $ df -h /dev/sdl 10G 312K 8.0G 1% /mnt/sdl mkfs is ok, mount is ok. Testing: $ cat /dev/zero > file ^C problem 1: size of ''file'' was 77 GB problem 2: umount was fine, remounted again and now size of ''file'' is 0 Both of them look quite serious to me :) david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Apr 08, 2013 at 02:01:09PM +0200, David Sterba wrote:> problem 1: size of ''file'' was 77 GB > problem 2: umount was fine, remounted again and now size of ''file'' is 0Works fine with a recent kernel (3.9). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 02/25/2013 10:25 PM, Suman C wrote:> Thanks for the sparse file idea, I am actually using that solution > already. I am not sure if its the best way, however.(Sorry to respond to such an old thread.) Hi Suman, I think zvol-like functionality would be very nice for btrfs to have. It would be more natural to manage btrvols than looped-back files I think, and removing those sw layers may also increase performance, but who knows by how much. It would let btrfs really act as "just" the LVM, if desired. Do we have any sense for how much work adding this would be? Regards -- Andy p.s. some interesting stuff on zvols http://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html