Hi all. Again. I am still fighting that "VMware esxi cannot use striped gluster volumes" thing, and a couple of crazy ideas are coming to mind. One of them is using iSCSI WITH gluster, and esxi connecting via iSCSI. My experience with iSCSI is limited to a couple of FreeNAS test installs, and some tuning on FreeNAS and esxi in order to implement multipathing, but nothing dead serious. I remember that after creating a volume and formating it (zvol), THEN space was allocated to iSCSI. Makes some sense, since iSCIS is a block device, and after it is available, the operating system will actually use it. But it is a bit foggy. I am trying to bypass the present limitation on Gluster, which refuses to talk to esxi using a striped volume. So, here is the question: anyone here uses gluster and iSCSI ? Would anyone care to comment on performance of this kind of solution, pros and cons ? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140305/45847962/attachment.html>
Can you please post some logs (the client logs which is exporting ISCSI)? It is hard to diagnose issues without logs. thanks, Avati On Wed, Mar 5, 2014 at 9:28 AM, Carlos Capriotti <capriotti.carlos at gmail.com> wrote:> Hi all. Again. > > I am still fighting that "VMware esxi cannot use striped gluster volumes" > thing, and a couple of crazy ideas are coming to mind. > > One of them is using iSCSI WITH gluster, and esxi connecting via iSCSI. > > My experience with iSCSI is limited to a couple of FreeNAS test installs, > and some tuning on FreeNAS and esxi in order to implement multipathing, but > nothing dead serious. > > I remember that after creating a volume and formating it (zvol), THEN > space was allocated to iSCSI. Makes some sense, since iSCIS is a block > device, and after it is available, the operating system will actually use > it. But it is a bit foggy. > > I am trying to bypass the present limitation on Gluster, which refuses to > talk to esxi using a striped volume. > > So, here is the question: anyone here uses gluster and iSCSI ? > > Would anyone care to comment on performance of this kind of solution, pros > and cons ? > > Thanks. > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140305/b92d64f1/attachment.html>
Hope you have seen this: https://www.gluster.org/2013/12/libgfapi-and-the-linux-target-driver/ Anand On 03/05/2014 10:58 PM, Carlos Capriotti wrote:> Hi all. Again. > > I am still fighting that "VMware esxi cannot use striped gluster > volumes" thing, and a couple of crazy ideas are coming to mind. > > One of them is using iSCSI WITH gluster, and esxi connecting via iSCSI. > > My experience with iSCSI is limited to a couple of FreeNAS test > installs, and some tuning on FreeNAS and esxi in order to implement > multipathing, but nothing dead serious. > > I remember that after creating a volume and formating it (zvol), THEN > space was allocated to iSCSI. Makes some sense, since iSCIS is a block > device, and after it is available, the operating system will actually > use it. But it is a bit foggy. > > I am trying to bypass the present limitation on Gluster, which refuses > to talk to esxi using a striped volume. > > So, here is the question: anyone here uses gluster and iSCSI ? > > Would anyone care to comment on performance of this kind of solution, > pros and cons ? > > Thanks. > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140306/cbf2c1f2/attachment.html>
I have done quite a bit with iSCSI and gluster recently. A wiki page on iSCSI and gluster is below. Its not clear what you mean by "cannot use striped gluster volumes". Do you mean a striped volume cannot grow by adding bricks? About the performance, I have taken some measurements for small (4 node) clusters and am working on some larger ones (96). Some of the results are posted on the gluster blog. I did not encounter any major performance problems. Generally, the likely user of iSCSI would not run gluster on their client, so would not see the performance benefits of fan-out. If you have data replicated over 2 nodes, the iSCSI connection will only send I/O down one path to a single node and not utilize the second one for load balancing. Another thing of note is a libgfapi plug-in to the iSCSI driver is working its way through review now, which will help performance significantly by bypassing FUSE. Hope this helps, feel free to ping me if you want to dig into this further. Dan http://www.gluster.org/community/documentation/index.php/GlusterFS_iSCSI http://www.gluster.org/2013/11/a-gluster-block-interface-performance-and-configuration/ https://www.gluster.org/category/iscsi/ ----- Original Message ----- From: "Carlos Capriotti" <capriotti.carlos at gmail.com> To: gluster-users at gluster.org Sent: Wednesday, March 5, 2014 12:28:35 PM Subject: [Gluster-users] iSCSI and Gluster Hi all. Again. I am still fighting that "VMware esxi cannot use striped gluster volumes" thing, and a couple of crazy ideas are coming to mind. One of them is using iSCSI WITH gluster, and esxi connecting via iSCSI. My experience with iSCSI is limited to a couple of FreeNAS test installs, and some tuning on FreeNAS and esxi in order to implement multipathing, but nothing dead serious. I remember that after creating a volume and formating it (zvol), THEN space was allocated to iSCSI. Makes some sense, since iSCIS is a block device, and after it is available, the operating system will actually use it. But it is a bit foggy. I am trying to bypass the present limitation on Gluster, which refuses to talk to esxi using a striped volume. So, here is the question: anyone here uses gluster and iSCSI ? Would anyone care to comment on performance of this kind of solution, pros and cons ? Thanks. _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users