Tohru_Kao
2015-Sep-08 03:45 UTC
[Gluster-users] [Q] Successful Gluster Peer Probe via GRE tunnel, but State returns Accepted peer request (Connected/Disconnected)
Hi all, This is regarding using Gluster 3.7.4 with GRE tunnel between 2 Ubuntu (14.04.3) VMs. ### Observation After Gluster Peer Probe success, Status returns few thing strange: #1: the State as "Accepted?peer?request (Connected/Disconnected)?.#2: IP address return is the Tunnel IP, not probed host IP. * Environments and GRE commands I applied are listed below.* GRE tunnel settings are done by referring?http://ask.xmodulo.com/create-gre-tunnel-linux.html ### Questions Q1:? Do I need to add/set additional parameters in glusterd.vol ?because of GRE ? Q2:? Are my GRE tunnel command settings wrong ? or missing some commands? Any idea? Appreciate any comments or pointers. ### Note:* Peer Probe via external IPs works properly. Thank in Advance.-JaCoder ############## ?Environment and Commands?##############?Environment:? U1: external ip address: ?172.16.213.128 ? internal network: 169.254.0.0/24? U2: external ip address: ?172.16.213.129 ? internal network: 169.254.1.0/24 ### Output of ?gluster peer probe? and status On U1#?gluster?peer?probe?169.254.1.1peer?probe:?success.? #?gluster?peer?statusNumber?of?Peers:?1 Hostname:?169.254.1.1Uuid:?b6519618-e3aa-4307-afce-8f3d0dae39fcState:?Accepted?peer?request?(Connected)---On?U2 #?gluster?peer?statusNumber?of?Peers:?1 Hostname:?10.10.10.1Uuid:?2306591b-25d6-41cf-ba50-db30ef0687bbState:?Accepted?peer?request?(Disconnected) ### glusterd.vol FilesOn?U1 #?cat?/etc/glusterfs/glusterd.vol?volume?management? ? type?mgmt/glusterd? ? option?working-directory?/var/lib/glusterd? ? option?transport.socket.bind-address?169.254.0.1? ? option?transport-type?socket,rdma? ? option?transport.socket.keepalive-time?10? ? option?transport.socket.keepalive-interval?2? ? option?transport.socket.read-fail-log?off? ? option?ping-timeout?30#? ?option?base-port?49152end-volume ---On?U2 #?cat?/etc/glusterfs/glusterd.vol?volume?management? ? type?mgmt/glusterd? ? option?working-directory?/var/lib/glusterd? ? option?transport.socket.bind-address?169.254.1.1? ? option?transport-type?socket,rdma? ? option?transport.socket.keepalive-time?10? ? option?transport.socket.keepalive-interval?2? ? option?transport.socket.read-fail-log?off? ? option?ping-timeout?30#? ?option?base-port?49152end-volume ### Output of ?ifconfig | inet' On U1# ifconfig | grep inet? ? ? ? ? inet addr:172.16.213.128 ?Bcast:172.16.213.255 ?Mask:255.255.255.0? ? ? ? ? inet addr:169.254.0.1 ?Bcast:169.254.0.255 ?Mask:255.255.255.0? ? ? ? ? inet addr:127.0.0.1 ?Mask:255.0.0.0? ? ? ? ? inet addr:10.10.10.1 ?P-t-P:10.10.10.1 ?Mask:255.255.255.0 On U2 # ifconfig | grep inet? ? ? ? ? inet addr:172.16.213.129 ?Bcast:172.16.213.255 ?Mask:255.255.255.0? ? ? ? ? inet addr:169.254.1.1 ?Bcast:169.254.1.255 ?Mask:255.255.255.0 ? ? ? ? ? inet addr:127.0.0.1 ?Mask:255.0.0.0? ? ? ? ? inet addr:10.10.10.2 ?P-t-P:10.10.10.2 ?Mask:255.255.255.0 ### Applied GRE Tunnel CommandsOn?U1 modprobe?ip_greip?tunnel?add test-gre?mode?gre?remote?172.16.213.129 local?172.16.213.128 ttl?255ip?link?set test-gre?upip?addr?add?10.10.10.1/24?dev test-greip?route?add?169.254.1.0/24?dev test-gre On?U2 modprobe?ip_greip?tunnel?add test-gre?mode?gre?remote?172.16.213.128 local?172.16.213.129 ttl?255ip?link?set test-gre?upip?addr?add 10.10.10.2/24?dev test-greip?route?add?169.254.0.0/24?dev test-gre ### Output of 'ip route show'On?U1 #?ip?route?showdefault?via?172.16.213.2?dev?eth0?10.10.10.0/24?dev?test-gre? proto?kernel? scope?link? src?10.10.10.1?169.254.0.0/24?dev ux-br0? proto?kernel? scope?link? src?169.254.0.1?169.254.1.0/24?dev?test-gre? scope?link?172.16.213.0/24?dev?eth0? proto?kernel? scope?link? src?172.16.213.128? #?ping?169.254.1.1PING?169.254.1.1?(169.254.1.1)?56(84)?bytes?of?data.64?bytes?from?169.254.1.1:?icmp_seq=1?ttl=64?time=0.340?ms64?bytes?from?169.254.1.1:?icmp_seq=2?ttl=64?time=0.305?ms ---On?U2 #?ip?route?showdefault?via?172.16.213.2?dev?eth0?10.10.10.0/24?dev?test-gre? proto?kernel? scope?link? src?10.10.10.2?169.254.0.0/24?dev?test-gre? scope?link?169.254.1.0/24?dev ux-br0? proto?kernel? scope?link? src?169.254.1.1?172.16.213.0/24?dev?eth0? proto?kernel? scope?link? src?172.16.213.100 #?ping?169.254.0.1PING?169.254.0.1?(169.254.0.1)?56(84)?bytes?of?data.64?bytes?from?169.254.0.1:?icmp_seq=1?ttl=64?time=0.720?ms64?bytes?from?169.254.0.1:?icmp_seq=2?ttl=64?time=0.298?ms === End ==-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150908/8101970c/attachment.html>
Atin Mukherjee
2015-Sep-08 04:43 UTC
[Gluster-users] [Q] Successful Gluster Peer Probe via GRE tunnel, but State returns Accepted peer request (Connected/Disconnected)
+ Humble Considering you had recently faced a similar problem, you could share the work around. On 09/08/2015 09:15 AM, Tohru_Kao wrote:> Hi all, > > This is regarding using Gluster 3.7.4 with GRE tunnel between 2 Ubuntu > (14.04.3) VMs. > > ### Observation > > After Gluster Peer Probe success, Status returns few thing strange: > > #1: the State as "Accepted peer request (Connected/Disconnected)?. > #2: IP address return is the Tunnel IP, not probed host IP. > > * Environments and GRE commands I applied are listed below. > * GRE tunnel settings are done by > referring http://ask.xmodulo.com/create-gre-tunnel-linux.html > > ### Questions > > Q1: > Do I need to add/set additional parameters in glusterd.vol because of > GRE ? > > Q2: > Are my GRE tunnel command settings wrong ? or missing some commands? > > Any idea? > > Appreciate any comments or pointers. > > ### Note: > * Peer Probe via external IPs works properly. > > Thank in Advance. > -JaCoder > > ############## Environment and Commands ############## > Environment: > U1: external ip address: 172.16.213.128 internal network: > 169.254.0.0/24 > U2: external ip address: 172.16.213.129 internal network: > 169.254.1.0/24 > > ### Output of ?gluster peer probe? and status > > On U1 > # gluster peer probe 169.254.1.1 > peer probe: success. > > # gluster peer status > Number of Peers: 1 > > Hostname: 169.254.1.1 > Uuid: b6519618-e3aa-4307-afce-8f3d0dae39fc > State: Accepted peer request (Connected) > --- > On U2 > > # gluster peer status > Number of Peers: 1 > > Hostname: 10.10.10.1 > Uuid: 2306591b-25d6-41cf-ba50-db30ef0687bb > State: Accepted peer request (Disconnected) > > > ### glusterd.vol Files > On U1 > > # cat /etc/glusterfs/glusterd.vol > volume management > type mgmt/glusterd > option working-directory /var/lib/glusterd > option transport.socket.bind-address 169.254.0.1 > option transport-type socket,rdma > option transport.socket.keepalive-time 10 > option transport.socket.keepalive-interval 2 > option transport.socket.read-fail-log off > option ping-timeout 30 > # option base-port 49152 > end-volume > > --- > On U2 > > # cat /etc/glusterfs/glusterd.vol > volume management > type mgmt/glusterd > option working-directory /var/lib/glusterd > option transport.socket.bind-address 169.254.1.1 > option transport-type socket,rdma > option transport.socket.keepalive-time 10 > option transport.socket.keepalive-interval 2 > option transport.socket.read-fail-log off > option ping-timeout 30 > # option base-port 49152 > end-volume > > ### Output of ?ifconfig | inet' > > On U1 > # ifconfig | grep inet > inet addr:172.16.213.128 Bcast:172.16.213.255 Mask:255.255.255.0 > inet addr:169.254.0.1 Bcast:169.254.0.255 Mask:255.255.255.0 > inet addr:127.0.0.1 Mask:255.0.0.0 > inet addr:10.10.10.1 P-t-P:10.10.10.1 Mask:255.255.255.0 > > On U2 > # ifconfig | grep inet > inet addr:172.16.213.129 Bcast:172.16.213.255 Mask:255.255.255.0 > inet addr:169.254.1.1 Bcast:169.254.1.255 Mask:255.255.255.0 > inet addr:127.0.0.1 Mask:255.0.0.0 > inet addr:10.10.10.2 P-t-P:10.10.10.2 Mask:255.255.255.0 > > > ### Applied GRE Tunnel Commands > On U1 > > modprobe ip_gre > ip tunnel add test-gre mode gre remote 172.16.213.129 > local 172.16.213.128 ttl 255 > ip link set test-gre up > ip addr add 10.10.10.1/24 dev test-gre > ip route add 169.254.1.0/24 dev test-gre > > On U2 > > modprobe ip_gre > ip tunnel add test-gre mode gre remote 172.16.213.128 > local 172.16.213.129 ttl 255 > ip link set test-gre up > ip addr add 10.10.10.2/24 dev test-gre > ip route add 169.254.0.0/24 dev test-gre > > ### Output of 'ip route show' > On U1 > > # ip route show > default via 172.16.213.2 dev eth0 > 10.10.10.0/24 dev test-gre proto kernel scope link src 10.10.10.1 > 169.254.0.0/24 dev ux-br0 proto kernel scope link src 169.254.0.1 > 169.254.1.0/24 dev test-gre scope link > 172.16.213.0/24 dev eth0 proto kernel scope link src 172.16.213.128 > > # ping 169.254.1.1 > PING 169.254.1.1 (169.254.1.1) 56(84) bytes of data. > 64 bytes from 169.254.1.1: icmp_seq=1 ttl=64 time=0.340 ms > 64 bytes from 169.254.1.1: icmp_seq=2 ttl=64 time=0.305 ms > > --- > On U2 > > # ip route show > default via 172.16.213.2 dev eth0 > 10.10.10.0/24 dev test-gre proto kernel scope link src 10.10.10.2 > 169.254.0.0/24 dev test-gre scope link > 169.254.1.0/24 dev ux-br0 proto kernel scope link src 169.254.1.1 > 172.16.213.0/24 dev eth0 proto kernel scope link src 172.16.213.100 > > # ping 169.254.0.1 > PING 169.254.0.1 (169.254.0.1) 56(84) bytes of data. > 64 bytes from 169.254.0.1: icmp_seq=1 ttl=64 time=0.720 ms > 64 bytes from 169.254.0.1: icmp_seq=2 ttl=64 time=0.298 ms > > > === End ==> > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >