Bryan McGuire
2010-May-06 23:26 UTC
[Gluster-users] New setup - Transport endpoint is not connected
Hello, New setup of two boxes CentOS - Clean install no problems. Both are servers and clients replicating a volume for mail storage After all is done I get Transport endpoint is not connected when trying to access the shared space. Here are the debug logs. Please help.... [root at msvr1 mnt]# glusterfsd --debug -f /etc/glusterfs/glusterfs- server.vol [2010-05-06 18:14:39] D [glusterfsd.c:432:_get_specfp] glusterfs: loading volume file /etc/glusterfs/glusterfs-server.vol [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/features/ locks.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: dumpops -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/features/ locks.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: dumpops -- neglecting = = = = = = = = =======================================================================Version : glusterfs 3.0.4 built on May 6 2010 15:26:26 git: v3.0.4 Starting Time: 2010-05-06 18:14:39 Command line : glusterfsd --debug -f /etc/glusterfs/glusterfs-server.vol PID : 5400 System name : Linux Nodename : msvr1.newnet66.org Kernel Release : 2.6.18-164.15.1.el5 Hardware Identifier: x86_64 Given volfile: + ------------------------------------------------------------------------------+ 1: # file: /etc/glusterfs/glusterfs-server.vol 2: 3: volume posix 4: type storage/posix 5: option directory /fs/data/export 6: end-volume 7: 8: volume locks 9: type features/locks 10: subvolumes posix 11: end-volume 12: 13: volume brick 14: type performance/io-threads 15: option thread-count 8 16: subvolumes locks 17: end-volume 18: 19: volume posix-ns 20: type storage/posix 21: option directory /fs/data/export-ns 22: end-volume 23: 24: volume locks-ns 25: type features/locks 26: subvolumes posix-ns 27: end-volume 28: 29: volume brick-ns 30: type performance/io-threads 31: option thread-count 8 32: subvolumes locks-ns 33: end-volume 34: 35: volume server 36: type protocol/server 37: option transport-type tcp 38: option auth.addr.brick.allow * 39: option auth.addr.brick-ns.allow * 40: subvolumes brick brick-ns 41: end-volume 42: + ------------------------------------------------------------------------------+ [2010-05-06 18:14:39] D [glusterfsd.c:1382:main] glusterfs: running in pid 5400 [2010-05-06 18:14:39] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:14:39] D [name.c:553:server_fill_address_family] server: option address-family not specified, defaulting to inet/inet6 [2010-05-06 18:14:39] D [io-threads.c:2841:init] brick-ns: io-threads: Autoscaling: off, min_threads: 8, max_threads: 8 [2010-05-06 18:14:39] D [io-threads.c:2841:init] brick: io-threads: Autoscaling: off, min_threads: 8, max_threads: 8 [2010-05-06 18:14:39] N [glusterfsd.c:1408:main] glusterfs: Successfully started [root at msvr1 mnt]# glusterfsd -f /etc/glusterfs/glusterfs-server.vol [root at msvr1 mnt]# glusterfs --debug -f /etc/glusterfs/glusterfs- client.vol /mnt/mail [2010-05-06 18:16:45] D [glusterfsd.c:432:_get_specfp] glusterfs: loading volume file /etc/glusterfs/glusterfs-client.vol [2010-05-06 18:16:45] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/legacy/cluster/ unify.so: undefined symbol: dumpops -- neglecting = = = = = = = = =======================================================================Version : glusterfs 3.0.4 built on May 6 2010 15:26:26 git: v3.0.4 Starting Time: 2010-05-06 18:16:45 Command line : glusterfs --debug -f /etc/glusterfs/glusterfs- client.vol /mnt/mail PID : 5445 System name : Linux Nodename : msvr1.newnet66.org Kernel Release : 2.6.18-164.15.1.el5 Hardware Identifier: x86_64 Given volfile: + ------------------------------------------------------------------------------+ 1: ### Add client feature and attach to remote subvolume of server1 2: volume brick1 3: type protocol/client 4: option transport-type tcp/client 5: option remote-host 192.168.1.15 # IP address of the remote brick 6: option remote-subvolume brick # name of the remote volume 7: end-volume 8: 9: ### Add client feature and attach to remote subvolume of server2 10: volume brick2 11: type protocol/client 12: option transport-type tcp/client 13: option remote-host 192.168.1.16 # IP address of the remote brick 14: option remote-subvolume brick # name of the remote volume 15: end-volume 16: 17: ### The file index on server1 18: volume brick1-ns 19: type protocol/client 20: option transport-type tcp/client 21: option remote-host 192.168.1.15 # IP address of the remote brick 22: option remote-subvolume brick-ns # name of the remote volume 23: end-volume 24: 25: ### The file index on server2 26: volume brick2-ns 27: type protocol/client 28: option transport-type tcp/client 29: option remote-host 192.168.1.16 # IP address of the remote brick 30: option remote-subvolume brick-ns # name of the remote volume 31: end-volume 32: 33: #The replicated volume with data 34: volume afr1 35: type cluster/afr 36: subvolumes brick1 brick2 37: end-volume 38: 39: #The replicated volume with indexes 40: volume afr-ns 41: type cluster/afr 42: subvolumes brick1-ns brick2-ns 43: end-volume 44: 45: #The unification of all afr volumes (used for > 2 servers) 46: volume unify 47: type legacy/cluster/unify 48: option scheduler rr # round robin 49: option namespace afr-ns 50: subvolumes afr1 51: end-volume 52: + ------------------------------------------------------------------------------+ [2010-05-06 18:16:45] D [glusterfsd.c:1382:main] glusterfs: running in pid 5445 [2010-05-06 18:16:45] D [unify.c:4347:init] unify: namespace node specified as afr-ns [2010-05-06 18:16:45] D [scheduler.c:53:get_scheduler] scheduler: attempt to load file rr.so [2010-05-06 18:16:45] D [unify.c:4379:init] unify: Child node count is 1 [2010-05-06 18:16:45] C [unify.c:4389:init] unify: WARNING: You have defined only one "subvolumes" for unify volume. It may not be the desired config, review your volume volfile. If this is how you are testing it, you may hit some performance penalty [2010-05-06 18:16:45] D [rr-options.c:188:rr_options_validate] rr: using scheduler.limits.min-free-disk = 15 [default] [2010-05-06 18:16:45] D [rr-options.c:216:rr_options_validate] rr: using scheduler.refresh-interval = 10 [default] [2010-05-06 18:16:45] D [client-protocol.c:6603:init] brick2-ns: defaulting frame-timeout to 30mins [2010-05-06 18:16:45] D [client-protocol.c:6614:init] brick2-ns: defaulting ping-timeout to 42 [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [client-protocol.c:6603:init] brick1-ns: defaulting frame-timeout to 30mins [2010-05-06 18:16:45] D [client-protocol.c:6614:init] brick1-ns: defaulting ping-timeout to 42 [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [client-protocol.c:6603:init] brick2: defaulting frame-timeout to 30mins [2010-05-06 18:16:45] D [client-protocol.c:6614:init] brick2: defaulting ping-timeout to 42 [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [client-protocol.c:6603:init] brick1: defaulting frame-timeout to 30mins [2010-05-06 18:16:45] D [client-protocol.c:6614:init] brick1: defaulting ping-timeout to 42 [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick1: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [name.c:155:client_fill_address_family] brick1: address-family not specified, guessing it to be inet/inet6 [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick1: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [name.c:155:client_fill_address_family] brick1: address-family not specified, guessing it to be inet/inet6 [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick2: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [name.c:155:client_fill_address_family] brick2: address-family not specified, guessing it to be inet/inet6 [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick2: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [name.c:155:client_fill_address_family] brick2: address-family not specified, guessing it to be inet/inet6 [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick1: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick1: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick2: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] D [client-protocol.c:7027:notify] brick2: got GF_EVENT_PARENT_UP, attempting connect on transport [2010-05-06 18:16:45] N [glusterfsd.c:1408:main] glusterfs: Successfully started [2010-05-06 18:16:45] D [client-protocol.c:7041:notify] brick2: got GF_EVENT_CHILD_UP [2010-05-06 18:16:45] D [client-protocol.c:7041:notify] brick2: got GF_EVENT_CHILD_UP [2010-05-06 18:16:45] D [client-protocol.c:7041:notify] brick1: got GF_EVENT_CHILD_UP [2010-05-06 18:16:45] D [client-protocol.c:7041:notify] brick1: got GF_EVENT_CHILD_UP [2010-05-06 18:16:45] N [client-protocol.c:6246:client_setvolume_cbk] brick2: Connected to 192.168.1.16:6996, attached to remote volume 'brick'. [2010-05-06 18:16:45] N [afr.c:2632:notify] afr1: Subvolume 'brick2' came back up; going online. [2010-05-06 18:16:45] N [client-protocol.c:6246:client_setvolume_cbk] brick1: Connected to 192.168.1.15:6996, attached to remote volume 'brick'. [2010-05-06 18:16:45] D [fuse-bridge.c:3100:fuse_thread_proc] fuse: pthread_cond_timedout returned non zero value ret: 0 errno: 0 [2010-05-06 18:16:45] N [client-protocol.c:6246:client_setvolume_cbk] brick1: Connected to 192.168.1.15:6996, attached to remote volume 'brick'. [2010-05-06 18:16:45] N [fuse-bridge.c:2950:fuse_init] glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.10 [2010-05-06 18:16:45] N [client-protocol.c:6246:client_setvolume_cbk] brick2: Connected to 192.168.1.16:6996, attached to remote volume 'brick'. [root at msvr1 mnt]# glusterfsd --debug -f /etc/glusterfs/glusterfs- server.vol [2010-05-06 18:14:39] D [glusterfsd.c:432:_get_specfp] glusterfs: loading volume file /etc/glusterfs/glusterfs-server.vol [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/features/ locks.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: dumpops -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/features/ locks.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:740:xlator_set_type] xlator: dlsym(notify) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: notify -- neglecting [2010-05-06 18:14:39] D [xlator.c:745:xlator_set_type] xlator: dlsym(dumpops) on /usr/local/lib/glusterfs/3.0.4/xlator/performance/io- threads.so: undefined symbol: dumpops -- neglecting = = = = = = = = =======================================================================Version : glusterfs 3.0.4 built on May 6 2010 15:26:26 git: v3.0.4 Starting Time: 2010-05-06 18:14:39 Command line : glusterfsd --debug -f /etc/glusterfs/glusterfs-server.vol PID : 5400 System name : Linux Nodename : msvr1.newnet66.org Kernel Release : 2.6.18-164.15.1.el5 Hardware Identifier: x86_64 Given volfile: + ------------------------------------------------------------------------------+ 1: # file: /etc/glusterfs/glusterfs-server.vol 2: 3: volume posix 4: type storage/posix 5: option directory /fs/data/export 6: end-volume 7: 8: volume locks 9: type features/locks 10: subvolumes posix 11: end-volume 12: 13: volume brick 14: type performance/io-threads 15: option thread-count 8 16: subvolumes locks 17: end-volume 18: 19: volume posix-ns 20: type storage/posix 21: option directory /fs/data/export-ns 22: end-volume 23: 24: volume locks-ns 25: type features/locks 26: subvolumes posix-ns 27: end-volume 28: 29: volume brick-ns 30: type performance/io-threads 31: option thread-count 8 32: subvolumes locks-ns 33: end-volume 34: 35: volume server 36: type protocol/server 37: option transport-type tcp 38: option auth.addr.brick.allow * 39: option auth.addr.brick-ns.allow * 40: subvolumes brick brick-ns 41: end-volume 42: + ------------------------------------------------------------------------------+ [2010-05-06 18:14:39] D [glusterfsd.c:1382:main] glusterfs: running in pid 5400 [2010-05-06 18:14:39] D [transport.c:123:transport_load] transport: attempt to load file /usr/local/lib/glusterfs/3.0.4/transport/socket.so [2010-05-06 18:14:39] D [name.c:553:server_fill_address_family] server: option address-family not specified, defaulting to inet/inet6 [2010-05-06 18:14:39] D [io-threads.c:2841:init] brick-ns: io-threads: Autoscaling: off, min_threads: 8, max_threads: 8 [2010-05-06 18:14:39] D [io-threads.c:2841:init] brick: io-threads: Autoscaling: off, min_threads: 8, max_threads: 8 [2010-05-06 18:14:39] N [glusterfsd.c:1408:main] glusterfs: Successfully started Bryan McGuire Senior Network Engineer NewNet 66 918.231.8063 bmcguire at newnet66.org -------------- next part --------------