Gluster users - Aug 2009 - is it possible to use ib-verbs and tcp transport-types on the same volume?

Hi,

I have some clients that have ib hca's and some that don't. Is it
possible to setup the glusterfsd.vol (server configuration) files to
serve the same volume over both ib-verbs and tcp? Glusterfsd didn't
seem to complain when I put :

volume server
  type protocol/server
  option transport-type ib-verbs
  option transport-type tcp
  option auth.ip.readahead-gluster1.allow 10.2.178.*
  subvolumes readahead-gluster1
end-volume

But I think it only "registered" the last transport-type line so that
only TCP became the valid transport type because it caused the
ib-verbs mount to hang on all the clients. I've also seen that it's
not possible to have two "volume server" blocks in the server
configuration.

Is it possible to run another glusterfsd process on the same server
which uses a configuration identical to the ib-verbs configuration
except that in the server block it uses only tcp as the transport-type
? Would this cause serious conflicts when clients going over ib-verbs
and tcp mount the same volume?

Thanks,
Sabuj

On Mon, 10 Aug 2009, Sabuj Pattanayek wrote:
> Hi,
>
> I have some clients that have ib hca's and some that don't. Is it
> possible to setup the glusterfsd.vol (server configuration) files to
> serve the same volume over both ib-verbs and tcp? Glusterfsd didn't
> seem to complain when I put :
>
> volume server
>  type protocol/server
>  option transport-type ib-verbs
>  option transport-type tcp
>  option auth.ip.readahead-gluster1.allow 10.2.178.*
>  subvolumes readahead-gluster1
> end-volume
>
> But I think it only "registered" the last transport-type line so
that
> only TCP became the valid transport type because it caused the
> ib-verbs mount to hang on all the clients. I've also seen that it's
> not possible to have two "volume server" blocks in the server
> configuration.
You want to break it out into two server volumes:

# Server settings
volume tcp-server
  type protocol/server
  option transport-type tcp/server
  subvolumes brick
  option auth.addr.brick.allow *
end-volume

volume ib-server
  type protocol/server
  option transport-type ib-verbs/server
  subvolumes brick
  option auth.addr.brick.allow *
end-volume
><>
Nathan Stratton                                CTO, BlinkMind, Inc.
nathan at robotics.net                         nathan at blinkmind.com
http://www.robotics.net                        http://www.blinkmind.com