Gluster users - Oct 2013 - Gluster 3.4 on RHCS with OCF resource agents

Hello,

Gluster 3.4 provides OCF resource agents which should enable nice 
integration with RHCS. We would like to deploy gluster on two RHCS nodes 
with storage provided via iSCSI. Volume will be distributed and the idea 
is to have RHCS migrate iSCSI LUN & gluster daemon to a single node in 
case of failure of one node. Due to a limited storage we cannot go for 
replicated volume.

We have a similar setup with earlier version of Gluster (3.0) that 
enabled specification of IP address that each gluster daemon should use. 
This enabled usage of virtual IPs and integration with RHCS with service 
resource agent.

Questions are:
1. Is there any documentation on how to use OCF resource agents? I 
checked the source code (glusterd and volume) and the metadata, but 
couldn't figure out how to get two glusterd instances on a same node.

2. Is it possible to have multiple glusterd instances on a same machine 
using virtual IP?

Thanks in advance
-- 
Emir Imamagic
SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
Emir.Imamagic at srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559

Hello,

by looking at the source code I managed to find answer to my second 
question - yes :)

In order to run multiple glusterd on the same host one needs to provide 
option transport.socket.bind-address in the volfile, e.g. 
/etc/glusterfs/glusterd-1.vol:
---
volume management
     type mgmt/glusterd
     # don't forget to change the working-directory
     option working-directory /var/lib/glusterd-1
     option transport-type socket,rdma
     option transport.socket.keepalive-time 10
     option transport.socket.keepalive-interval 2
     option transport.socket.read-fail-log off
     option transport.socket.bind-address XX.YY.ZZ.AA
end-volume
---
In addition to different address and working directory one has to define 
different PID file when starting the service:
  glusterd -f /etc/glusterfs/glusterd-1.vol -p /var/run/glusterd-1.pid

Finally when using the CLI one has to specify to which glusterd to connect:
  gluster --remote-host=XX.YY.ZZ.AA
After that business as usual by using virtual IPs.

With this knowledge I think I'll be able to make the OCF resource agents 
work. Even if not I can always use service resource agent with two 
modified init.d scripts.

Cheers,
emir

On 30.10.2013. 12:58, Emir Imamagic wrote:
> Hello,
>
> Gluster 3.4 provides OCF resource agents which should enable nice
> integration with RHCS. We would like to deploy gluster on two RHCS nodes
> with storage provided via iSCSI. Volume will be distributed and the idea
> is to have RHCS migrate iSCSI LUN & gluster daemon to a single node in
> case of failure of one node. Due to a limited storage we cannot go for
> replicated volume.
>
> We have a similar setup with earlier version of Gluster (3.0) that
> enabled specification of IP address that each gluster daemon should use.
> This enabled usage of virtual IPs and integration with RHCS with service
> resource agent.
>
> Questions are:
> 1. Is there any documentation on how to use OCF resource agents? I
> checked the source code (glusterd and volume) and the metadata, but
> couldn't figure out how to get two glusterd instances on a same node.
>
> 2. Is it possible to have multiple glusterd instances on a same machine
> using virtual IP?
>
> Thanks in advance

-- 
Emir Imamagic
SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
Emir.Imamagic at srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559