Gluster users - Jul 2011 - Possible to bind to multiple addresses?

I am trying to run GlusterFS on only my internal interfaces.  I have 
setup two bricks and have a replicated volume that is started.

Everything works fine when I run with no transport.socket.bind-address 
defined in the /etc/glusterfs/glusterd.vol file, but when I add it I get:

Transport endpoint is not connected

My configuration looks like this:

volume management
     type mgmt/glusterd
     option working-directory /etc/glusterd
     option transport-type socket,rdma
     option transport.socket.bind-address 192.168.1.1
     option transport.socket.keepalive-time 10
     option transport.socket.keepalive-interval 2
end-volume

and the same on the other brick binding to it's own address.  When I 
attempt to start up glusterd this is what I get in the logs:

[2011-07-08 08:44:03.778665] I [glusterd.c:564:init] 0-management: Using 
/etc/glusterd as working directory
[2011-07-08 08:44:03.779276] I [glusterd.c:88:glusterd_uuid_init] 
0-glusterd: retrieved UUID: 6bf087ca-e5a2-4dcc-964c-7f17d21bb17f
[2011-07-08 08:44:03.780841] E 
[glusterd-store.c:1567:glusterd_store_retrieve_volume] 0-: Unknown key: 
brick-0
[2011-07-08 08:44:03.780861] E 
[glusterd-store.c:1567:glusterd_store_retrieve_volume] 0-: Unknown key: 
brick-1
[2011-07-08 08:44:03.917480] I 
[glusterd-handler.c:3399:glusterd_friend_add] 0-glusterd: connect returned 0
[2011-07-08 08:44:03.917709] I 
[glusterd-utils.c:1092:glusterd_volume_start_glusterfs] 0-: About to 
start glusterfs for brick 192.168.1.1:/glusterback/reports
Given volfile:
+------------------------------------------------------------------------------+
   1: volume management
   2:     type mgmt/glusterd
   3:     option working-directory /etc/glusterd
   4:     option transport-type socket
   5:     option transport.socket.bind-address 192.168.1.1
   6:     option transport.socket.keepalive-time 10
   7:     option transport.socket.keepalive-interval 2
   8: end-volume
   9:

+------------------------------------------------------------------------------+
[2011-07-08 08:44:03.938185] E [socket.c:1685:socket_connect_finish] 
0-management: connection to  failed (Connection reset by peer)
[2011-07-08 08:44:03.938593] I 
[glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using 
Program glusterd clnt mgmt, Num (1238433), Version (1)
[2011-07-08 08:44:07.923034] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2011-07-08 08:44:10.926009] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2011-07-08 08:44:13.928965] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2011-07-08 08:44:16.932929] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2011-07-08 08:44:19.935893] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2011-07-08 08:44:22.938846] E [socket.c:2080:socket_connect] 
0-management: connection attempt failed (Connection refused)

Does anyone have any ideas as to what might be going on?

Thank you,
James

Hi,

just spotted your problem in google search, I assume you solved it in 
the meanwhile :)

The problem with transport.socket.bind-address parameter is that the 
glusterfs processes started by glusterd (nfs and self-heal daemons) have 
hardcoded localhost address (--volfile-server/-s parameter). If you 
check /var/log/glusterfs/nfs.log you will notice that nfs glusterfs 
process failed because glusterd is not on localhost. Since nfs glusterfs 
process dies glusterd keeps on complaining with the message:
  E [socket.c:2080:socket_connect] 0-management: connection attempt 
failed (Connection refused)

I tried setting nfs.disable ON but it doesn't help in this case.

Cheers,
emir

On 8.7.2011. 14:48, James FLatten wrote:
> I am trying to run GlusterFS on only my internal interfaces.  I have
> setup two bricks and have a replicated volume that is started.
>
> Everything works fine when I run with no transport.socket.bind-address
> defined in the /etc/glusterfs/glusterd.vol file, but when I add it I get:
>
> Transport endpoint is not connected
>
> My configuration looks like this:
>
> volume management
>      type mgmt/glusterd
>      option working-directory /etc/glusterd
>      option transport-type socket,rdma
>      option transport.socket.bind-address 192.168.1.1
>      option transport.socket.keepalive-time 10
>      option transport.socket.keepalive-interval 2
> end-volume
>
> and the same on the other brick binding to it's own address.  When I
> attempt to start up glusterd this is what I get in the logs:
>
> [2011-07-08 08:44:03.778665] I [glusterd.c:564:init] 0-management: Using
> /etc/glusterd as working directory
> [2011-07-08 08:44:03.779276] I [glusterd.c:88:glusterd_uuid_init]
> 0-glusterd: retrieved UUID: 6bf087ca-e5a2-4dcc-964c-7f17d21bb17f
> [2011-07-08 08:44:03.780841] E
> [glusterd-store.c:1567:glusterd_store_retrieve_volume] 0-: Unknown key:
> brick-0
> [2011-07-08 08:44:03.780861] E
> [glusterd-store.c:1567:glusterd_store_retrieve_volume] 0-: Unknown key:
> brick-1
> [2011-07-08 08:44:03.917480] I
> [glusterd-handler.c:3399:glusterd_friend_add] 0-glusterd: connect
> returned 0
> [2011-07-08 08:44:03.917709] I
> [glusterd-utils.c:1092:glusterd_volume_start_glusterfs] 0-: About to
> start glusterfs for brick 192.168.1.1:/glusterback/reports
> Given volfile:
>
+------------------------------------------------------------------------------+
>
>    1: volume management
>    2:     type mgmt/glusterd
>    3:     option working-directory /etc/glusterd
>    4:     option transport-type socket
>    5:     option transport.socket.bind-address 192.168.1.1
>    6:     option transport.socket.keepalive-time 10
>    7:     option transport.socket.keepalive-interval 2
>    8: end-volume
>    9:
>
>
+------------------------------------------------------------------------------+
>
> [2011-07-08 08:44:03.938185] E [socket.c:1685:socket_connect_finish]
> 0-management: connection to  failed (Connection reset by peer)
> [2011-07-08 08:44:03.938593] I
> [glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using
> Program glusterd clnt mgmt, Num (1238433), Version (1)
> [2011-07-08 08:44:07.923034] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2011-07-08 08:44:10.926009] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2011-07-08 08:44:13.928965] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2011-07-08 08:44:16.932929] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2011-07-08 08:44:19.935893] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2011-07-08 08:44:22.938846] E [socket.c:2080:socket_connect]
> 0-management: connection attempt failed (Connection refused)
>
> Does anyone have any ideas as to what might be going on?
>
> Thank you,
> James
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>

-- 
Emir Imamagic
SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
Emir.Imamagic at srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559