Gluster users - Apr 2012 - IPoIB Volume (3.3b3) started but not online, not mountable

I was successfully running a IPoIB gluster testbed (V3.3b3 on Ubuntu 
10.04.04) and brought it down smoothly to adjust some parameters.  
It now looks like this (the options reconfigured were just added):

# gluster volume info
 
Volume Name: gli
Type: Distribute
Volume ID: 76cc5e88-0ac4-42ac-a4a3-31bf2ba611d4
Status: Started
Number of Bricks: 5
Transport-type: tcp,rdma
Bricks:
Brick1: pbs1ib:/bducgl
Brick2: pbs2ib:/bducgl
Brick3: pbs2ib:/bducgl1
Brick4: pbs3ib:/bducgl
Brick5: pbs4ib:/bducgl
Options Reconfigured:
performance.io-cache: on
performance.quick-read: on
performance.io-thread-count: 64
auth.allow: 10.255.77.*, 128.200.15.*, 10.255.78.*, 10.255.89.*


however, a status query gives this:

# gluster volume status

Status of volume: gli
Gluster process                                         Port    Online  
Pid
------------------------------------------------------------------------------
Brick pbs1ib:/bducgl                                    24016   N       
N/A
Brick pbs2ib:/bducgl                                    24023   N       
N/A
Brick pbs2ib:/bducgl1                                   24025   N       
N/A
Brick pbs3ib:/bducgl                                    24016   N       
N/A
Brick pbs4ib:/bducgl                                    24016   N       
N/A
NFS Server on localhost                                 38467   N       
N/A
NFS Server on pbs4ib                                    38467   N       
N/A
NFS Server on pbs3ib                                    38467   N       
N/A
NFS Server on pbs2ib                                    38467   N       
N/A

(I didn't want the NFS Server options - is that a default to start 
it?)

But the operative bit is that it's not online, despite being started.  
What could give this situation?


As might be expected, clients can't mount the gluster vol.

The last part of the /etc-glusterfs-glusterd.vol.log/ is many lines 
like this:
[2012-04-18 11:36:57.456318] E [socket.c:2115:socket_connect] 0-
management: connection attempt failed (Connection refused)
and the last lines before are a number of stanzas like this:

[2012-04-18 11:31:14.698184] I [glusterd-op-
sm.c:2222:glusterd_op_ac_send_commit_op] 0-management: Sent op req to 
3 peers
[2012-04-18 11:31:14.698379] I [glusterd-rpc-
ops.c:1294:glusterd3_1_commit_op_cbk] 0-glusterd: Received ACC from 
uuid: 2a593581-bf45-446c-8f7c-212c53297803
[2012-04-18 11:31:14.698496] I [glusterd-rpc-
ops.c:1294:glusterd3_1_commit_op_cbk] 0-glusterd: Received ACC from 
uuid: c79c4084-d6b9-4af9-b975-40dd6aa99b42
[2012-04-18 11:31:14.698581] I [glusterd-rpc-
ops.c:1294:glusterd3_1_commit_op_cbk] 0-glusterd: Received ACC from 
uuid: 26de63bd-c5b7-48ba-b81d-5d77a533d077
[2012-04-18 11:31:14.698834] I [glusterd-rpc-
ops.c:606:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received ACC 
from uuid: 2a593581-bf45-446c-8f7c-212c53297803
[2012-04-18 11:31:14.698879] I [glusterd-rpc-
ops.c:606:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received ACC 
from uuid: 26de63bd-c5b7-48ba-b81d-5d77a533d077
[2012-04-18 11:31:14.698910] I [glusterd-rpc-
ops.c:606:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received ACC 
from uuid: c79c4084-d6b9-4af9-b975-40dd6aa99b42
[2012-04-18 11:31:14.698929] I [glusterd-op-
sm.c:2491:glusterd_op_txn_complete] 0-glusterd: Cleared local lock
[2012-04-18 11:31:15.410106] E [socket.c:2115:socket_connect] 0-
management: connection attempt failed (Connection refused)

-- 
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  Google Voice Multiplexer: (949) 478-4487
415 South Circle View Dr, Irvine, CA, 92697 [shipping]
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120418/ed7544e7/attachment.html>

With JoeJulian's help, tracked this down to what looks like a bug in 
the IP# format which causes glusterfsd to crash.  The bug is: 
<https://bugzilla.redhat.com:443/show_bug.cgi?id=813937>
 
If anyone has an immediate workaround or correction, be glad to hear 
of it.

hjm
-- 
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  Google Voice Multiplexer: (949) 478-4487
415 South Circle View Dr, Irvine, CA, 92697 [shipping]
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120418/e1fbda47/attachment.html>