Gluster users - Oct 2014 - how to restrict client connection to server to only one IP address

Hello,

I am new to this list and new to GlusterFS, so I would be grateful if 
you could help me.

I am trying to do this setup:

client1(10.75.2.45)
    |
    |   MTU 1500
    V
(10.75.2.41)
   gluster1                gluster2
(10.75.2.43)  ------->  (10.75.2.44)
               <-------
               MTU 9000

In words, I have two glusterfs servers (in replication): gluster1 and 
gluster2 and a glusterfs client client1.
The gluster1 has two network interfaces: 10.75.2.41 and 10.75.2.43.
I would like gluster1 to communicate with gluster2 using jumbo frames 
and connection would be between interfaces 10.75.2.43 and 10.75.2.44.
Since the client1 can only use default packet size (MTU 1500) I would 
like it to connect with gluster1 using only other network interface: 
10.75.2.41.

Is it possible?

At the moment on gluster1 I have:

eno16780032: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
         inet 10.75.2.43  netmask 255.255.255.0  broadcast 10.75.2.255
eno33559296: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet 10.75.2.41  netmask 255.255.255.0  broadcast 10.75.2.255

and when I mount from client1 using:
mount -t glusterfs 10.75.2.41:/vol1 /mnt/glusterfs

it still uses connection to 10.75.2.43:
# netstat -natup | egrep '(2.41|2.43)'
tcp        0      0 10.75.2.45:1020 10.75.2.43:49152        ESTABLISHED 
10856/glusterfs
tcp        0      0 10.75.2.45:1022 10.75.2.41:24007        ESTABLISHED 
10856/glusterfs

Is there a way to restrict communication from client1 to gluster1 using 
only one IP address: 10.75.2.41?

Any help would be much appreciated.

Best regards
Lukasz

PS
GlusterFS version on client:
glusterfs-3.5.2-1.el7.x86_64
glusterfs-fuse-3.5.2-1.el7.x86_64

GlusterFS version on server:
glusterfs-server-3.5.2-1.el7.x86_64
glusterfs-3.5.2-1.el7.x86_64

On 10/16/2014 2:48 PM, ?ukasz Zygma?ski wrote:
> Hello,
>
> I am new to this list and new to GlusterFS, so I would be grateful if you 
> could help me.
>
> I am trying to do this setup:
>
> client1(10.75.2.45)
>    |
>    |   MTU 1500
>    V
> (10.75.2.41)
>   gluster1                gluster2
> (10.75.2.43)  ------->  (10.75.2.44)
>               <-------
>               MTU 9000
>
> In words, I have two glusterfs servers (in replication): gluster1 and 
> gluster2 and a glusterfs client client1.
> The gluster1 has two network interfaces: 10.75.2.41 and 10.75.2.43.
> I would like gluster1 to communicate with gluster2 using jumbo frames and 
> connection would be between interfaces 10.75.2.43 and 10.75.2.44.
> Since the client1 can only use default packet size (MTU 1500) I would like 
> it to connect with gluster1 using only other network interface: 10.75.2.41.
>
> Is it possible?
>
> At the moment on gluster1 I have:
>
> eno16780032: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
>         inet 10.75.2.43  netmask 255.255.255.0  broadcast 10.75.2.255
> eno33559296: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 10.75.2.41  netmask 255.255.255.0  broadcast 10.75.2.255
>
> and when I mount from client1 using:
> mount -t glusterfs 10.75.2.41:/vol1 /mnt/glusterfs
>
> it still uses connection to 10.75.2.43:
> # netstat -natup | egrep '(2.41|2.43)'
> tcp        0      0 10.75.2.45:1020 10.75.2.43:49152 ESTABLISHED 
> 10856/glusterfs
> tcp        0      0 10.75.2.45:1022 10.75.2.41:24007 ESTABLISHED 
> 10856/glusterfs
>
> Is there a way to restrict communication from client1 to gluster1 using 
> only one IP address: 10.75.2.41?
>
> Any help would be much appreciated.
>
> Best regards
> Lukasz
>
> PS
> GlusterFS version on client:
> glusterfs-3.5.2-1.el7.x86_64
> glusterfs-fuse-3.5.2-1.el7.x86_64
>
> GlusterFS version on server:
> glusterfs-server-3.5.2-1.el7.x86_64
> glusterfs-3.5.2-1.el7.x86_64
Since no one has answered this in a few days, I will try to do so, or at 
least start the process.

You do not mention how the client connects.

1. If it is using gluster-fuse, what you are trying to do is futile, because 
the connections are not as you think.  The data does not flow from client1 ->
gluster1 -> gluster2.  The way it really works is that client1 connects 
directly to both gluster1 and gluster2, and sends the data to both of them at 
the same time.  The only time any volume of data transfers directly from 
gluster1 to gluster2 is during a heal operation.  Unfortunately, gluster does 
not understand the concept of a separate "storage network" that the
servers
use to talk to each other.  It only has one address, and that address is the 
one that the clients connect to.

2. If the client uses NFS, then you have something more like what you drew.  
The data passes client1 -> gluster1 via NFS, and then gluster1 ->
gluster2.
I am not using NFS, so I can't help you with if it is possible to have NFS
on
one network connection and gluster on a different connection, or what is 
required to accomplish this (if it can be done at all).

Ted Miller