I have been using RHEL5+dm-multipath+OCFS2 for a while now without
problems, it works great ... even when I yank out a network cable or
two just for fun >:->
I have seen what you are describing many times, and had to deal with
it as well. The problem has to do with the order in which it finds
the actual block devices, of which there are more than one with the
same UUID. The blkid database sometimes ends up with the mpath
device listed first, and other times one/both of the actual block
devices listed first. When the block devices are listed first,
dm-multipath has issues like the one you describe.
Take a look at your /etc/blkid/blkid.tab file and find your mpath
device, and the other devices with the same UUID. Note the
order. If the mpath device comes after any of its underlying
devices, delete the underlying devices and then run blkid again to
have it re-generate the cache file. Since you left the mpath device
in there, it will tack on the underlying devices further down the list.
Once you do this, everything should be discovered nicely by
dm-multipath and such.
As for the autogenerated names matching up, you can't rely on that at
all--they are OS/driver assigned, and the order is not guaranteed
from one system to the next (or even on the same system from one boot
up to the next). The UUID will be consistend across boots, as well
as across nodes. If you want a more friendly name than the UUID, you
can leverage the alias feature of dm-multipath to ensure that the
block device name on all nodes matches up. See multipath.conf
documentation for more details, but here's a quick example:
multipaths {
multipath {
wwid 12345a018f0472a3bdf8a240c53d0c92e
alias shared1
}
multipath {
wwid 67890a018f0476ae77790844e5ad059d8
alias shared2
}
}
You can then reference them
as /dev/mapper/shared1 and /dev/mapper/shared2 respectively.
At 05:44 AM 2/28/2009, you wrote:>Date: Tue, 24 Feb 2009 18:42:22 -0700
>From: Kevin Masaryk <kevin.masaryk at gmail.com>
>Subject: [Ocfs2-users] Mounting mpath devices in RHEL 5
>
>I'm seeing some strange behavior from OCFS2 while trying to mount
>mpath/dm-multipath devices under RHEL5. Sometimes I can mount the
>EMC-connected, dual-pathed volumes just fine and sometimes I get
"device
>busy" errors. I've tried mounting by label and also by explicit
>/dev/mapper/mpathXpY name with the same unpredictable behavior. I've
>also noticed that sometimes when a device is successfully mounted on all
>nodes, each node may return different output from the mount command
>regarding the device mounted; e.g. /dev/mapper/mpath0p1 vs. /dev/dm-17.
>
>One consistent aspect I have noticed whenever I receive the "device
>busy" error is that the /dev/dm-X names don't match up on each
node. I
>also see that ocfs2console refers to each device by the /dev/dm-X name
>instead of the /dev/mapper/XX name.
>
>I guess my question is simply: Are dm-multipath devices supported under
>OCFS2? Are multipathed devices not recommended with OCFS2? Any
>documentation available on this?
I have been using RHEL5+dm-multipath+OCFS2 for a while now without
problems, it works great ... even when I yank out a network cable or
two just for fun >:->
I have seen what you are describing many times, and had to deal with
it as well. The problem has to do with the order in which it finds
the actual block devices, of which there are more than one with the
same UUID. The blkid database sometimes ends up with the mpath
device listed first, and other times one/both of the actual block
devices listed first. When the block devices are listed first,
dm-multipath has issues like the one you describe.
Take a look at your /etc/blkid/blkid.tab file and find your mpath
device, and the other devices with the same UUID. Note the
order. If the mpath device comes after any of its underlying
devices, delete the underlying devices and then run blkid again to
have it re-generate the cache file. Since you left the mpath device
in there, it will tack on the underlying devices further down the list.
Once you do this, everything should be discovered nicely by
dm-multipath and such.
As for the autogenerated names matching up, you can't rely on that at
all--they are OS/driver assigned, and the order is not guaranteed
from one system to the next (or even on the same system from one boot
up to the next). The UUID will be consistend across boots, as well
as across nodes. If you want a more friendly name than the UUID, you
can leverage the alias feature of dm-multipath to ensure that the
block device name on all nodes matches up. See multipath.conf
documentation for more details, but here's a quick example:
multipaths {
multipath {
wwid 12345a018f0472a3bdf8a240c53d0c92e
alias shared1
}
multipath {
wwid 67890a018f0476ae77790844e5ad059d8
alias shared2
}
}
You can then reference them
as /dev/mapper/shared1 and /dev/mapper/shared2 respectively.
At 05:44 AM 2/28/2009, you wrote:>Date: Tue, 24 Feb 2009 18:42:22 -0700
>From: Kevin Masaryk <kevin.masaryk at gmail.com>
>Subject: [Ocfs2-users] Mounting mpath devices in RHEL 5
>
>I'm seeing some strange behavior from OCFS2 while trying to mount
>mpath/dm-multipath devices under RHEL5. Sometimes I can mount the
>EMC-connected, dual-pathed volumes just fine and sometimes I get
"device
>busy" errors. I've tried mounting by label and also by explicit
>/dev/mapper/mpathXpY name with the same unpredictable behavior. I've
>also noticed that sometimes when a device is successfully mounted on all
>nodes, each node may return different output from the mount command
>regarding the device mounted; e.g. /dev/mapper/mpath0p1 vs. /dev/dm-17.
>
>One consistent aspect I have noticed whenever I receive the "device
>busy" error is that the /dev/dm-X names don't match up on each
node. I
>also see that ocfs2console refers to each device by the /dev/dm-X name
>instead of the /dev/mapper/XX name.
>
>I guess my question is simply: Are dm-multipath devices supported under
>OCFS2? Are multipathed devices not recommended with OCFS2? Any
>documentation available on this?