Gluster users - Mar 2015 - Configure separate network for inter-node communication

Hello,

I have two gluster nodes in a replicated setup and have connected the two nodes
together directly through a 10 Gbit/s crossover cable. Now I would like to tell
gluster to use this seperate private network for any communications between the
two nodes. Does that make sense? Will this bring me any performance gain? and if
yes how do I configure that?

My clients are intended to access my gluster volumes using FUSE through the main
1 Gbit/s interface.

Regards
ML

On 03/04/2015 11:24 PM, ML mail wrote:
> Hello,
>
> I have two gluster nodes in a replicated setup and have connected the two
nodes together directly through a 10 Gbit/s crossover cable. Now I would like to
tell gluster to use this seperate private network for any communications between
the two nodes. Does that make sense? Will this bring me any performance gain?
and if yes how do I configure that?
>
> My clients are intended to access my gluster volumes using FUSE through the
main 1 Gbit/s interface.
>
> Regards
> ML
>
Makes a lot of sense, just give the 10gbe nic's a different ip subnet
and put these in the /etc/hosts file or dns.

Use these hostname.storage names to setup the volume. There should be
some way to convert an existing volume but I don't have the details at
the moment. Read this for some info:

http://www.ovirt.org/Change_network_interface_for_Gluster





Met vriendelijke groet, With kind regards,

Jorick Astrego

Netbulae Virtualization Experts 

----------------

	Tel: 053 20 30 270 	info at netbulae.eu 	Staalsteden 4-3A 	KvK 08198180
 	Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW NL821234584B01

----------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150305/e973db46/attachment.html>

> I have two gluster nodes in a replicated setup and have connected the two
> nodes together directly through a 10 Gbit/s crossover cable. Now I would
> like to tell gluster to use this seperate private network for any
> communications between the two nodes. Does that make sense? Will this bring
> me any performance gain? and if yes how do I configure that?
It is possible, but it's not likely to improve performance much (yet).

The easiest way to do this is to use a custom /etc/hosts on the servers,
so that *on a server* every other server's name resolves to its private
back-end address.  Meanwhile, clients resolve that same name to the
server's front-end address.  You can get a similar effect with explicit
host routes or iptables rules on the servers.

The reason this won't have much effect on performance is that the
servers do not (currently) replicate to one another.  Instead, clients
send data directly to every replica themselves.  The only time time a
private network would see much traffic would be when the "clients" are
actually the servers performing administrative operations - self heal,
rebalance, and so on.

In 4.0, both parts of this answer would be different.  First, we expect
to have better handling of multiple networks and multi-homed hosts,
including user specification of which networks to use for which
traffic[1].  Second, 4.0 will have a new form of replication which
*does* replicate directly between servers[2].  Parts of this second
feature are in fact likely to appear well before the rest of 4.0, using
the server-to-server data flow but retaining our current methods of
tracking changes and re-syncing servers after a failure.  In fact I'm
writing a blog post right now about this, including some performance
measurements.  I'll respond again here when it's done.

[1]
http://www.gluster.org/community/documentation/index.php/Features/SplitNetwork
[2]
http://www.gluster.org/community/documentation/index.php/Features/new-style-replication