Gluster users - Jul 2013 - RDMA Volume Mount Not Functioning for Debian 3.4.beta4

Anyone ele having these issues mounting RDMA only volumes under Ubuntu? Install
of Gluster via: https://launchpad.net/~semiosis/+archive/ubuntu-glusterfs-3.4

I'm guessing this build is looking for an include somewhere that is just
misplaced?

glusterfs 3.4.0beta4 built on Jun 28 2013 16:16:07
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General
Public License.

Specifically:

root at cs3-p:~# mount -t glusterfs cs1-i:/test2.rdma /tmp/test2/
[2013-07-02 21:04:51.407554] I [glusterfsd.c:1878:main] 0-/usr/sbin/glusterfs:
Started running /usr/sbin/glusterfs version 3.4.0beta4 (/usr/sbin/glusterfs
--volfile-id=/test2.rmda --volfile-server=cs1-i /tmp/test2/)
[2013-07-02 21:04:51.408841] I [socket.c:3480:socket_init] 0-glusterfs: SSL
support is NOT enabled
[2013-07-02 21:04:51.408872] I [socket.c:3495:socket_init] 0-glusterfs: using
system polling thread
[2013-07-02 21:04:51.414278] W [rdma.c:4197:__gf_rdma_ctx_create]
0-rpc-transport/rdma: rdma_cm event channel creation failed (No such file or
directory)
[2013-07-02 21:04:51.414309] E [rdma.c:4485:init] 0-test2-client-1: Failed to
initialize IB Device
[2013-07-02 21:04:51.414324] E [rpc-transport.c:320:rpc_transport_load]
0-rpc-transport: 'rdma' initialization failed
[2013-07-02 21:04:51.414390] W [rpc-clnt.c:972:rpc_clnt_connection_init]
0-test2-client-1: loading of new rpc-transport failed
[2013-07-02 21:04:51.414402] I [mem-pool.c:541:mem_pool_destroy]
0-test2-client-1: size=2236 max=0 total=0
[2013-07-02 21:04:51.414471] I [mem-pool.c:541:mem_pool_destroy]
0-test2-client-1: size=124 max=0 total=0
[2013-07-02 21:04:51.414483] E [client.c:2283:client_init_rpc] 0-test2-client-1:
failed to initialize RPC
[2013-07-02 21:04:51.414527] E [xlator.c:390:xlator_init] 0-test2-client-1:
Initialization of volume 'test2-client-1' failed, review your volfile
again
[2013-07-02 21:04:51.414534] E [graph.c:292:glusterfs_graph_init]
0-test2-client-1: initializing translator failed
[2013-07-02 21:04:51.414540] E [graph.c:479:glusterfs_graph_activate] 0-graph:
init failed
[2013-07-02 21:04:51.414668] W [glusterfsd.c:970:cleanup_and_exit]
(-->/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)
[0x7fa820b56bd5] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x31b)
[0x7fa82120bd1b] (-->/usr/sbin/glusterfs(glusterfs_process_volfp+0x103)
[0x7fa821207473]))) 0-: received signum (0), shutting down
[2013-07-02 21:04:51.414687] I [fuse-bridge.c:5212:fini] 0-fuse: Unmounting
'/tmp/test2/'.
Mount failed. Please check the log file for more details.

Volume Details:

root at cs1-p:~# gluster volume info test2

Volume Name: test2
Type: Stripe
Volume ID: fd2a2fc2-c5b3-4241-9e02-e2d936972357
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: rdma
Bricks:
Brick1: cs1-i:/gluster/test2
Brick2: cs2-i:/gluster/test2


Other basic stuff:

root at cs3-p:~# ibv_devinfo
hca_id: mlx4_0
        transport:                      InfiniBand (0)
        fw_ver:                         2.9.1000
        node_guid:                      0002:c903:000c:1e82
        sys_image_guid:                 0002:c903:000c:1e85
        vendor_id:                      0x02c9
        vendor_part_id:                 26428
        hw_ver:                         0xB0
        board_id:                       IBM0D90000009
        phys_port_cnt:                  1
                port:   1
                        state:                  PORT_ACTIVE (4)
                        max_mtu:                4096 (5)
                        active_mtu:             4096 (5)
                        sm_lid:                 125
                        port_lid:               604
                        port_lmc:               0x00

root at cs3-p:~# dpkg --get-selections | grep ibv
ibverbs-utils                                   install
libibverbs-dev                                  install
libibverbs1                                     install

root at cs3-p:~# dpkg --get-selections | grep rdma
librdmacm-dev                                   install
librdmacm1                                      install
librdmacm1-dbg                                  install
rdmacm-utils                                    install
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130702/385b5e65/attachment.html>

On 03/07/2013, at 12:52 AM, Ryan Aydelott wrote:
<snip>
> [2013-07-02 21:04:51.414278] W [rdma.c:4197:__gf_rdma_ctx_create]
0-rpc-transport/rdma: rdma_cm event channel creation failed (No such file or
directory)
Note, I don't use Ubuntu. :)

As an off chance thought, the above line from the error message looks
like there's a problem with your IPoIB's Connected Mode.

Is there any chance the adapters aren't running in Connected Mode for
some reason?

Regards and best wishes,

Justin Clift

--
Open Source and Standards @ Red Hat

twitter.com/realjustinclift