Gluster users - Feb 2010 - gigabit to "infiniband storage network"

I'm planning a storage solution for our HPC environment which is built on
both gigabit and infiniband switches. The "gluster storage nodes" are
connected to a "infiniband storage switch", and the question is how to
connect our gigabit based subclusters to this storage network.  As we see
it we could either:

(i) equip all gluster nodes with gigabit nodes and attach them to a
"gigabit storage network" (as well as the "infiniband storage
network").
Our gigabit subclusters are then attached to a "gigabit storage
network".

or

(ii) use a separate computer as router equipped with a bunch of gigabit
ports (for connecting with the gigabit subclusters) and a few infiniband
ports (for connecting with the "infiniband storage network"). This
computer could even play the double role as a gluster node?

Are there any praticular drawbacks with either of these ways of doing it?

As of today our gigabit subclusters are connected to single fileserver,
connected to a raided bunch of disks using FC, giving us sufficient I/O
performance. We are therefore inclined to believe that the second option
above would also be sufficient.

Regards, and thanks,

/jon

Your descriptions are very confusing so I just wanted to make sure you
know that you can export gluster over infiniband and over gigE
(tcp/ip) from the same gluster storage node as long as it has both
infiniband and gigE connectivity:

volume server
  type protocol/server
  option transport-type ib-verbs
  option auth.addr.iothreads.allow 10.0.0.*
  option auth.addr.iothreads-dist.allow 10.0.0.*
  subvolumes iothreads iothreads-dist
end-volume

volume server-tcp
   type protocol/server
   option transport-type tcp
   option auth.addr.iothreads.allow 10.0.1.*
   option auth.addr.iothreads-dist.allow 10.0.1.*
   subvolumes iothreads iothreads-dist
end-volume

HTH,
Sabuj Pattanayek

On Tue, Feb 2, 2010 at 1:49 AM,  <tegner at renget.se>
wrote:
> I'm planning a storage solution for our HPC environment which is built
on
> both gigabit and infiniband switches. The "gluster storage nodes"
are
> connected to a "infiniband storage switch", and the question is
how to
> connect our gigabit based subclusters to this storage network. ?As we see
> it we could either:
>
> (i) equip all gluster nodes with gigabit nodes and attach them to a
> "gigabit storage network" (as well as the "infiniband
storage network").
> Our gigabit subclusters are then attached to a "gigabit storage
network".
>
> or
>
> (ii) use a separate computer as router equipped with a bunch of gigabit
> ports (for connecting with the gigabit subclusters) and a few infiniband
> ports (for connecting with the "infiniband storage network").
This
> computer could even play the double role as a gluster node?
>
> Are there any praticular drawbacks with either of these ways of doing it?
>
> As of today our gigabit subclusters are connected to single fileserver,
> connected to a raided bunch of disks using FC, giving us sufficient I/O
> performance. We are therefore inclined to believe that the second option
> above would also be sufficient.
>
> Regards, and thanks,
>
> /jon
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>