At 10:18 AM 6/28/2011, ocfs2-users-request at oracle.com
wrote:>Message: 1
>Date: Tue, 28 Jun 2011 10:18:07 -0500
>From: Richard Pickett <richard.pickett at csrtechnologies.com>
>Subject: Re: [Ocfs2-users] multiple cluster doesn't work
>To: S?rgio Surkamp <sergio at gruposinternet.com.br>
>Cc: ocfs2-users at oss.oracle.com
>Message-ID: <BANLkTimvrejSyEhn7nEJYv0OwrD5yENm2A at mail.gmail.com>
>Content-Type: text/plain; charset="iso-8859-1"
>
>That's good news. When I go to format them, do I give them all the same
>name:
>
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdb1*
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdc1*
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdd1*
>
>If "no", then how do I do it?
>
>If "yes", then how does o2cb know the difference between each
device when
>they are mounted? (how do the nodes distinguish each device when they are
>communicating to the other nodes in the cluster that they are making a
>change?)
I think you are missing something conceptually about OCFS2. Perhaps
I can help out by showing an example of a configuration I have which
has multiple systems (5), multiple volumes (2), but all within a
single cluster.
So there are 5 systems, let's call them n1 through n5.
There are 2 devices, which all systems can see as a block device,
let's call them /dev/d1 & /dev/d2
I will be mounting them at /mnt/v1 and /mnt/v2, but not necessarily
both on all systems as some only need one.
Here is what the cluster.conf looks like, identical on all 5 nodes:
cluster:
node_count = 5
name = foo
node:
ip_port = 7777
ip_address = 10.10.10.1
number = 1
name = n1
cluster = live
node:
ip_port = 7777
ip_address = 10.10.10.2
number = 2
name = n2
cluster = live
node:
ip_port = 7777
ip_address = 10.10.10.3
number = 3
name = n3
cluster = live
node:
ip_port = 7777
ip_address = 10.10.10.4
number = 4
name = n4
cluster = live
node:
ip_port = 7777
ip_address = 10.10.10.5
number = 5
name = n5
cluster = live
When creating the filesystems, I used a command along the lines of:
mkfs.ocfs2 -b 4k -C 32K -L "OCFS2v1" -N 2 /dev/d1
mkfs.ocfs2 -b 4k -C 32K -L "OCFS2v2" -N 2 /dev/d2
Those only need to be done from one node, of course.
And for easy mounting and automount at boot, /etc/fstab entries would
be like this:
LABEL=OCFS2v1 /mnt/v1 ocfs2 _netdev,noatime 0 0
LABEL=OCFS2v2 /mnt/v2 ocfs2 _netdev,noatime 0 0
Then you can mount as desired from whatever nodes, no requirement
that you mount any/all of them in all places:
mkdir /mnt/v1
mount /mnt/v1
mkdir /mnt/v2
mount /mnt/v2
So on my setup, I currently have volume 1 mounted on nodes 2 & 3 & 4
and volume 2 mounted on nodes 1 & 4 & 5. This can be seen with
mounted.ocfs2 commands:
root at n1:~# mounted.ocfs2 -f
Device FS Nodes
/dev/d2 ocfs2 n1, n4, n5
root at n3:~# mounted.ocfs2 -f
Device FS Nodes
/dev/d1 ocfs2 n2, n3, n4
root at n4:~# mounted.ocfs2 -f
Device FS Nodes
/dev/d1 ocfs2 n2, n3, n4
/dev/d2 ocfs2 n1, n4, n5
If I were to go on node 2 and do a "umount /mnt/v1" then run the
above command again on node 4, I would see:
root at n4:~# mounted.ocfs2 -f
Device FS Nodes
/dev/d1 ocfs2 n3, n4
/dev/d2 ocfs2 n1, n4, n5
Also note that the actual block device names may not be exactly the
same on all systems depending on how it allocates them. In my case
I'm actually using dm-multipath over multiple sdX created by iscsi to
volumes on a SAN. So on one box it could be /dev/sda & /dev/sdb
presented as /dev/mapper/mpath0, and on another box that same volume
could by /dev/sde & /dev/sdf presented as /dev/mapper/mpath3. And
that's where the -L labels come in handy -- no matter what the block
devices area called, I created the filesystem with a label that
relates to what the purpose of the volume is so I can still just
mount by label and know that I get the right device mounted at the
correct mount point. If you want to see what the currently mounted
volume labels relate to device wise, you can use: mounted.ocfs2 -d
Or if it's one that is not mounted, you can use: tunefs.ocfs2 -Q
"%V\n" /dev/d1