I'm asking a lot of questions--sorry about that. I have a cluster setup with most of the nodes having two NICs, one for a local network running over an internal GigE switch and the other to our institutional network (also GigE). I have configured four bricks to run in unify on the internal network. The resulting file system is mounted on each node via the local network: volume remote1 type protocol/client option transport-type tcp/client option remote-host 192.168.8.104 option remote-subvolume brick end-volume volume remote2 type protocol/client option transport-type tcp/client option remote-host 192.168.8.102 option remote-subvolume brick end-volume ..... volume unify0 type cluster/unify option scheduler rr # round robin option namespace remote-ns #option block-size *:1MB subvolumes remote1 remote2 remote3 remote4 end-volume I have a second set of machines that are not on the internal network but communicate over the institutional network. I have modified the client configuration to use the IP addresses for the institutional network for each of the bricks. However, when I try to mount the file system using this configuration, I get this in the glusterfs.log. 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote2: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote2: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote3: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote3: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote4: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote4: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote-ns: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote-ns: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] glusterfs-fuse: 6: (34) / => -1 (2) 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote2: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote2: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote3: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote3: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote4: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote4: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] remote-ns: not connected at the moment to submit frame type(1) op(34) 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] remote-ns: no proper reply from server, returning ENOTCONN 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] glusterfs-fuse: 6: (34) / => -1 (2) Any suggestions as to what is going on? I didn't an answer to this anywhere in the docs and google came up pretty dry, also. Thanks again, Sean
Hi Sean, Is glusterfs servers started on all 'remote' nodes? (I see its started fine on just 192.168.8.104, on other nodes, it may not be started, or a firewall is preventing a connection. check the status of connection by 'netstat -nt' Regards, Amar 2008/12/10 Sean Davis <sdavis2 at mail.nih.gov>> I'm asking a lot of questions--sorry about that. > > I have a cluster setup with most of the nodes having two NICs, one for > a local network running over an internal GigE switch and the other to > our institutional network (also GigE). I have configured four bricks > to run in unify on the internal network. The resulting file system is > mounted on each node via the local network: > > volume remote1 > type protocol/client > option transport-type tcp/client > option remote-host 192.168.8.104 > option remote-subvolume brick > end-volume > > volume remote2 > type protocol/client > option transport-type tcp/client > option remote-host 192.168.8.102 > option remote-subvolume brick > end-volume > > ..... > > volume unify0 > type cluster/unify > option scheduler rr # round robin > option namespace remote-ns > #option block-size *:1MB > subvolumes remote1 remote2 remote3 remote4 > end-volume > > I have a second set of machines that are not on the internal network > but communicate over the institutional network. I have modified the > client configuration to use the IP addresses for the institutional > network for each of the bricks. However, when I try to mount the file > system using this configuration, I get this in the glusterfs.log. > > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote2: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote2: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote3: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote3: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote4: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote4: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote-ns: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote-ns: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] > glusterfs-fuse: 6: (34) / => -1 (2) > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote2: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote2: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote3: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote3: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote4: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote4: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] > remote-ns: not connected at the moment to submit frame type(1) op(34) > 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] > remote-ns: no proper reply from server, returning ENOTCONN > 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] > glusterfs-fuse: 6: (34) / => -1 (2) > > > Any suggestions as to what is going on? I didn't an answer to this > anywhere in the docs and google came up pretty dry, also. > > Thanks again, > Sean > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users > >-- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Super Storage! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081210/9e6169e6/attachment.html>
On Wed, Dec 10, 2008 at 1:41 PM, Amar S. Tumballi <amar at zresearch.com> wrote:> Hi Sean, > > Is glusterfs servers started on all 'remote' nodes? (I see its started fine > on just 192.168.8.104, on other nodes, it may not be started, or a firewall > is preventing a connection. check the status of connection by 'netstat -nt'That was the issue. Thanks, Amar. Sean> 2008/12/10 Sean Davis <sdavis2 at mail.nih.gov> >> >> I'm asking a lot of questions--sorry about that. >> >> I have a cluster setup with most of the nodes having two NICs, one for >> a local network running over an internal GigE switch and the other to >> our institutional network (also GigE). I have configured four bricks >> to run in unify on the internal network. The resulting file system is >> mounted on each node via the local network: >> >> volume remote1 >> type protocol/client >> option transport-type tcp/client >> option remote-host 192.168.8.104 >> option remote-subvolume brick >> end-volume >> >> volume remote2 >> type protocol/client >> option transport-type tcp/client >> option remote-host 192.168.8.102 >> option remote-subvolume brick >> end-volume >> >> ..... >> >> volume unify0 >> type cluster/unify >> option scheduler rr # round robin >> option namespace remote-ns >> #option block-size *:1MB >> subvolumes remote1 remote2 remote3 remote4 >> end-volume >> >> I have a second set of machines that are not on the internal network >> but communicate over the institutional network. I have modified the >> client configuration to use the IP addresses for the institutional >> network for each of the bricks. However, when I try to mount the file >> system using this configuration, I get this in the glusterfs.log. >> >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote2: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote2: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote3: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote3: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote4: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote4: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote-ns: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote-ns: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] >> glusterfs-fuse: 6: (34) / => -1 (2) >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote2: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote2: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote3: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote3: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote4: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote4: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 W [client-protocol.c:332:client_protocol_xfer] >> remote-ns: not connected at the moment to submit frame type(1) op(34) >> 2008-12-10 13:22:32 E [client-protocol.c:4430:client_lookup_cbk] >> remote-ns: no proper reply from server, returning ENOTCONN >> 2008-12-10 13:22:32 E [fuse-bridge.c:468:fuse_entry_cbk] >> glusterfs-fuse: 6: (34) / => -1 (2) >> >> >> Any suggestions as to what is going on? I didn't an answer to this >> anywhere in the docs and google came up pretty dry, also. >> >> Thanks again, >> Sean >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >> > > > > -- > Amar Tumballi > Gluster/GlusterFS Hacker > [bulde on #gluster/irc.gnu.org] > http://www.zresearch.com - Commoditizing Super Storage! >