We have a use case in our clusters where we're adding IB to some racks
and would like to use gluster storage over IB in that mix. If it
works out on the cluster case, it would be a good argument to scale
this out much further.
So, is there an elegant, best-practices-way to use a gluster volume
that was established using the IPoIB net to provide storage to the
IPoGBe clients without routing thru a single gateway (and therefore
losing the multipath advantage of gluster) or creating a DNS hairball?
ie: The IPoIB net is 1.2.3.0 (host names are referenced as 'pbsXib')
and the IPoGbe net is 1.2.4.0 (names are 'pbsXgb'). All gluster
storage nodes have 1 IB port on 1.2.3.0 and one Gbe port on 1.2.4.0)
The gluster volume was set up as follows and the host names are
defined in the local /etc/hosts files for now):
gluster volume create gl \
transport tcp,rdma \
pbs1ib:/bducgl \
pbs2ib:/bducgl \
pbs3ib:/bducgl \
pbs4ib:/bducgl
The IPoIB clients (which also have the same /etc/hosts info) have no
problem accessing the volume as 'pbs1ib:/gl' (and have good large-file
IO, maxing out at about 250MB/s on a single file copy)
The problem (from the gl.log (5.8K):
<http://moo.nac.uci.edu/~hjm/gl.log>
seems to be that the Gbe gluster client can contact the Gbe side of
the gluster volume but cannot resolve the gluster volume since the
gluster volume was created with pbsXib names.
I can bypass the DNS by pointing the pbsXib names -> pbsXgb addresses
on a client (in /etc/hosts) and in fact this works fine and I have the
gluster volume reading and writing clients from both IB and Gbe.
The problem is that this leaves a fragile, tangled nest of DNS
overrides and manual edits. Setting this up in the DNS tables would
solve most of this but the servers would still have to have overrides
set up in their /etc/hosts to override the DNS.
Is there a more elegant way of doing this?
This is 3.3b3, (which was fairly easy to set up).
Also, with dual-port IB cards, is it recommended to do channel-bonding
or to use different IP #s?
hjm
--
Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697 Google Voice Multiplexer: (949) 478-4487
415 South Circle View Dr, Irvine, CA, 92697 [shipping]
MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps)
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120416/8147fef3/attachment.html>