Hi folks, I recently got hands on a Infiniband hardware (2 40G cards with a Mellanox switch). I've been able to configure the base infiniband system with Centos 6.3 and OFED 1.5.4. Both nodes see each other and network values are the expected of the scenario. However, when trying to use gluster on the test scenario, I faced lots of troubles. I created a volume on an additional disk formatted as ext4 and exported to a gluster client. I've tried 3.3.1, 3.4-rc1,3.2.7 and only been able to mount the volume with 3.2.7. With the other gluster versions, the following error appears on the gluster logs: tcp connect to failed (Connection refused) My question is whether, as I've read, rdma transport is not available in 3.3 and I have to use 3.2.7. I've also tried 3.4.rc1 with the same problems as 3.3.... Thanks in advance, Samuel. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121019/6e4b72e3/attachment.html>
Unfortunately we have had similar problems. RDMA in GlusterFS 3.3.x is not well (or even at all, maybe?) supported. Although I was able to mount a volume with "transport rdma" using "mount -t glusterfs masterib:/volume.rdma /volume" Note the ".rdma" after volume name. Due to the lack of rdma support in 3.3.x we decided to stick with plain IPoIB. Regards -- Bartek Krawczyk network and system administrator
I have an existing volume configured to use GbE and just got two Infiniband cards. How can I reconfigure the peers to use the IPoIB? On 10/19/12 2:48 PM, Bartek Krawczyk wrote:> Due to the lack of rdma support in 3.3.x we decided to stick with plain IPoIB. >
* samuel <samu60 at gmail.com> [2012 10 19, 11:44]:> I've tried 3.3.1, 3.4-rc1,3.2.7 and only been able to mount the volume > with 3.2.7. With the other gluster versions, the following error appears > on the gluster logs:In the Admin-Guide the following can be read (taken from 3.3.1 changelog): Command Reference - The Gluster Console Manager "NOTE: with 3.3.0 release, transport type 'rdma' and 'tcp,rdma' are not fully supported" Regards