Hello Everyone, I am new to Lustre and to this list, and hope you might be able to point me in the right direction. I have built a test Lustre cluster and we are performing some feasibility studies to determine how well it may work as a storage solution for a project we have. As part of this I would like to server out iSCSI luns using Lustre. From what I am seeing so far this may not be the simplest thing to do. I would like to know if any one is running some combination of Luster and iSCSI and if so what method are they using to do so. Can you just run iSCSI on the same CentOS MDS server and server out the luns? Or perhaps use a separate iSCSI server that might access the Lustre file system and be able to format and use it, etc. Also, does the next version of Lustre plan to have any kind of iSCSI integration? Thank you very much for any input you may have. Cheers, Pete
On Mon, 2009-09-28 at 12:10 -0400, Pete Eby wrote:> Hello Everyone,Hi,> I have built a test Lustre cluster and we are performing some > feasibility studies to determine how well it may work as a storage > solution for a project we have. As part of this I would like to server > out iSCSI luns using Lustre.By this do you mean that you have iscsi luns which you have configured lustre to use for it''s storage and serve that to lustre clients? Or do you mean that you have configured a lustre installation and now want to re-export the filesystem from lustre clients to other clients via iscsi?> I would like to know if any one is running some combination of Luster > and iSCSI and if so what method are they using to do so.If you mean iscsi as lustre''s storage, if it''s a block device, lustre can use it. So if you configure your iscsi luns on the lustre servers, you should be able to configure those luns into the lustre filesystem.> Can you just > run iSCSI on the same CentOS MDS server and server out the luns?I''m starting to feel like you want lustre to be an iscsi "service". That''s not possible. Lustre presents a filesystem, not storage devices.> Also, does the next version of Lustre plan to have any kind of iSCSI > integration?For what you seem to want, no. There are no plans of that nature at all in the roadmap as far as I know. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090928/a8b16e2d/attachment.bin
Brian J. Murrell wrote:> > By this do you mean that you have iscsi luns which you have configured > lustre to use for it''s storage and serve that to lustre clients? Or do > you mean that you have configured a lustre installation and now want to > re-export the filesystem from lustre clients to other clients via iscsi? >Hi Brian - thanks very much for your reply - sorry I was no clear on what I would like to do. Your are correct in that I have configured a luster installation and want to re-export the file system from luster to other clients via iSCSI - and I can''t quite seem to grasp how this can be done.> I''m starting to feel like you want lustre to be an iscsi "service". > That''s not possible. Lustre presents a filesystem, not storage devices.Yea, I don''t expect Luster itself to present iSCSI, but rather how take advantage of an presented luster FS and re-export as iSCSI. My goal (like others before me I see from Google) is to use the benefits of Lustre as a clustered, scalable storage solution and re-export it through iSCSI (which will eventually be used by VMs). It seems like this is possible, but I am missing how to bridge between say a Luster client, connecting to an exported Luster FS and re-offering that file systems through iSCSI. We have plenty of hardware to use and can provide redundancy on the lower layers and can scale this well, so this is very much of interest to me. Thanks for your time Brian, I appreciate it. Pete
Pete Eby wrote:> Brian J. Murrell wrote: > >> By this do you mean that you have iscsi luns which you have configured >> lustre to use for it''s storage and serve that to lustre clients? Or do >> you mean that you have configured a lustre installation and now want to >> re-export the filesystem from lustre clients to other clients via iscsi? >> > > Hi Brian - thanks very much for your reply - sorry I was no clear on > what I would like to do. Your are correct in that I have configured a > luster installation and want to re-export the file system from luster to > other clients via iSCSI - and I can''t quite seem to grasp how this can > be done. > >> I''m starting to feel like you want lustre to be an iscsi "service". >> That''s not possible. Lustre presents a filesystem, not storage devices. > > Yea, I don''t expect Luster itself to present iSCSI, but rather how take > advantage of an presented luster FS and re-export as iSCSI. > > My goal (like others before me I see from Google) is to use the benefits > of Lustre as a clustered, scalable storage solution and re-export it > through iSCSI (which will eventually be used by VMs). > > It seems like this is possible, but I am missing how to bridge between > say a Luster client, connecting to an exported Luster FS and re-offering > that file systems through iSCSI. We have plenty of hardware to use and > can provide redundancy on the lower layers and can scale this well, so > this is very much of interest to me. > > Thanks for your time Brian, I appreciate it. > > Pete >Pete, iSCSI allows access to block devices. You put filesystems on block devices. Lustre is a filesystem. It uses block devices in the backend, but you don''t use iSCSI to connect to it. In your VMs you would load the Lustre client, which provides its own protocol, to mount the filesystem. Craig> _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- Craig Tierney (craig.tierney at noaa.gov)
On Tue, 2009-09-29 at 10:24 -0400, Pete Eby wrote:> > Your are correct in that I have configured a > luster installation and want to re-export the file system from luster to > other clients via iSCSI - and I can''t quite seem to grasp how this can > be done.That''s because it can''t. :-) Lustre is a filesystem.> Yea, I don''t expect Luster itself to present iSCSI, but rather how take > advantage of an presented luster FS and re-export as iSCSI.Well, if you can come with any other way of providing normal filesystem namespace as an iscsi device, you might have a chance. But until you can point to something in say /export/ and make it an iscsi target, you won''t have any hope of lustre doing this for you. But TBH, I think you are barking up the wrong tree.> My goal (like others before me I see from Google) is to use the benefits > of Lustre as a clustered, scalable storage solution and re-export it > through iSCSI (which will eventually be used by VMs).Why not just make the clients Lustre clients instead of iscsi consumers? What are these iscsi consumers (VMs did you say?) going to do with the block device once they get it? Put a filesystem on it? What OS will these VMs be running?> It seems like this is possible,I doubt it.> but I am missing how to bridge between > say a Luster client, connecting to an exported Luster FS and re-offering > that file systems through iSCSI.As I said. Unless you can make this work with any old filesystem namespace on Linux, you won''t get it working with Lustre either. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090929/c2317709/attachment.bin
On Sep 29, 2009 10:24 -0400, Pete Eby wrote:> ... I want to re-export the file system from luster to other clients > via iSCSI - and I can''t quite seem to grasp how this can be done. > > > I''m starting to feel like you want lustre to be an iscsi "service". > > That''s not possible. Lustre presents a filesystem, not storage devices. > > Yea, I don''t expect Luster itself to present iSCSI, but rather how take > advantage of an presented luster FS and re-export as iSCSI.I expect that the Linux kernel already has some sort of iSCSI target support. It may be possible to use this to export files from Lustre, though the iSCSI target may depend on access to a Linux block device, which isn''t really supported by Lustre (since there are not necessarily even disks on the Lustre clients). You might be able to use a loop device to create a block device from a Lustre file, and then re-export that via iSCSI. There is also UNSUPPORTED work that was done to add a block device interface to a Lustre file. I don''t think it works very well at the moment, but if you have programming skills and time for testing and development I could probably help you with debugging this code.> It seems like this is possible, but I am missing how to bridge between > say a Luster client, connecting to an exported Luster FS and re-offering > that file systems through iSCSI.This is something that probably already exists in the Linux kernel. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.