I am attempting to get multipath working with device-mapper (CentOS
4.2 and 4.3). It works on EVERY install of mine from RH (also v4.2,
4.3), but the same multipath.conf imported to all my installs of
CentOS do not work. Note that I have tested a working 4.2
configuration file from RH on CentOS 4.2 and a working 4.3
configuration (it changed slightly) on CentOS 4.3. Neither worked. Our
production systems are RH 4.2 so the 4.2 configuration file is below.
Configuration:
Promise vTrak 300i iSCSI
iscsi-initiator-utils from 4.2/4.3
device-mapper-multipath from 4.2/4.3
Procedure:
1. Configure iscsi.conf and multipath.conf
2. Start iscsi: service iscsi start
3. Start multipath: service multipathd start
4. Make sure no stale paths exist: multipath -F
5. Scan and create paths: multipath -v 3
Below are the relevant configuration files:
=======================================================My iscsi.conf is very
simple:
DiscoveryAddress=172.16.160.60
=======================================================>From iscsi-ls, the
output is:
*******************************************************************************
SFNet iSCSI Driver Version ...4:0.1.11(12-Jan-2005)
===================================================There are two sessions, this
creates two targets:
# ls /dev/sd*
/dev/sda /dev/sda2 /dev/sda5 /dev/sdb /dev/sdb2 /dev/sdb5
/dev/sda1 /dev/sda3 /dev/sda6 /dev/sdb1 /dev/sdb3 /dev/sdb6
====================================================The multipath.conf is as
follows:
defaults {
multipath_tool "/sbin/multipath -v0"
udev_dir /dev
polling_interval 10
default_selector "round-robin 0"
default_path_grouping_policy multibus
default_getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
default_prio_callout "/bin/true"
default_features "0"
rr_wmin_io 100
failback immediate
}
devnode_blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^hda[0-9]*"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}
multipaths {
multipath {
wwid 222ea000155eadda4
alias stor
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
failback immediate
}
}
=================================================The output from "multipath
-v 3" of my CentOS machine is below:
load path identifiers cache
#
# all paths in cache :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200
222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200
dm-0 blacklisted
dm-1 blacklisted
hda blacklisted
hdc blacklisted
md0 blacklisted
ram0 blacklisted
ram10 blacklisted
ram11 blacklisted
ram12 blacklisted
ram13 blacklisted
ram14 blacklisted
ram15 blacklisted
ram1 blacklisted
ram2 blacklisted
ram3 blacklisted
ram4 blacklisted
ram5 blacklisted
ram6 blacklisted
ram7 blacklisted
ram8 blacklisted
ram9 blacklisted
===== path info sda (mask 0x1f) ====bus = 1
dev_t = 8:0
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 0:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
uid = 222ea000155eadda4 (cache)
===== path info sdb (mask 0x1f) ====bus = 1
dev_t = 8:16
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 1:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
uid = 222ea000155eadda4 (cache)
#
# all paths :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200
222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200
pgpolicy = multibus (LUN setting)
selector = round-robin 0 (LUN setting)
features = 0 (internal default)
hwhandler = 0 (internal default)
rr_weight = 2 (config file default)
no_path_retry = -1 (config file default)
pg_timeout = NONE (internal default)
0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100
set ACT_CREATE: map does not exists
message stor 0 switch_group 1
create: stor (222ea000155eadda4)
[size=1 GB][features="0"][hwhandler="0"]
\_ round-robin 0
\_ 0:0:0:0 sda 8:0 [ready]
\_ 1:0:0:0 sdb 8:16 [ready]
reloading features
^^^^^^^^ Notice the 1GB in the map created. I have read in mailing
list posts that 1GB is returned when it is unable to find the size.
=================================================The output from "multipath
-v 3" of the working 4.2/4.3 machine is below:
hda blacklisted
hdc blacklisted
md0 blacklisted
ram0 blacklisted
ram10 blacklisted
ram11 blacklisted
ram12 blacklisted
ram13 blacklisted
ram14 blacklisted
ram15 blacklisted
ram1 blacklisted
ram2 blacklisted
ram3 blacklisted
ram4 blacklisted
ram5 blacklisted
ram6 blacklisted
ram7 blacklisted
ram8 blacklisted
ram9 blacklisted
path sda not found in pathvec
===== path sda ====bus = 1
dev_t = 8:0
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 0:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672?
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
getuid = /sbin/scsi_id -g -u -s /block/%n (internal default)
uid = 222ea000155eadda4 (callout)
path sdb not found in pathvec
===== path sdb ====bus = 1
dev_t = 8:16
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 1:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672?
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
getuid = /sbin/scsi_id -g -u -s /block/%n (internal default)
uid = 222ea000155eadda4 (callout)
#
# all paths :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [VTrak M300i ]
222ea000155eadda4 1:0:0:0 sdb 8:16 [VTrak M300i ]
pgpolicy = multibus (LUN setting)
selector = round-robin 0 (LUN setting)
features = 0 (internal default)
hwhandler = 0 (internal default)
0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000
action preset to 0
action set to 4
create: stor (222ea000155eadda4)
[size=1 TB][features="0"][hwhandler="0"]
\_ round-robin 0
\_ 0:0:0:0 sda 8:0
\_ 1:0:0:0 sdb 8:16
message stor 0 switch_group 1
^^^^^^^^ it detects the size correctly (it is over 1TB, it seems to do
some rounding when it reports it here)
===================================
note the multipath config lines:
not working: 0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100
working: 0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000
The primary difference is in the 2nd number. That is, according to the
multipath wiki, the "target length in 512-bytes blocks" for the
multipath device. The one that does not work is smaller by a factor of
1063 (close to 1024, maybe it is using different units?)
Has anyone run into any issue of this sort?
I am attempting to get multipath working with device-mapper (CentOS
4.2 and 4.3). It works on EVERY install of mine from RH (also v4.2,
4.3), but the same multipath.conf imported to all my installs of
CentOS do not work. Note that I have tested a working 4.2
configuration file from RH on CentOS 4.2 and a working 4.3
configuration (it changed slightly) on CentOS 4.3. Neither worked. Our
production systems are RH 4.2 so the 4.2 configuration file is below.
Configuration:
Promise vTrak 300i iSCSI
iscsi-initiator-utils from 4.2/4.3
device-mapper-multipath from 4.2/4.3
Procedure:
1. Configure iscsi.conf and multipath.conf
2. Start iscsi: service iscsi start
3. Start multipath: service multipathd start
4. Make sure no stale paths exist: multipath -F
5. Scan and create paths: multipath -v 3
Below are the relevant configuration files:
=======================================================My iscsi.conf is very
simple:
DiscoveryAddress=172.16.160.60
=======================================================>From iscsi-ls, the
output is:
*******************************************************************************
SFNet iSCSI Driver Version ...4:0.1.11(12-Jan-2005)
===================================================There are two sessions, this
creates two targets:
# ls /dev/sd*
/dev/sda /dev/sda2 /dev/sda5 /dev/sdb /dev/sdb2 /dev/sdb5
/dev/sda1 /dev/sda3 /dev/sda6 /dev/sdb1 /dev/sdb3 /dev/sdb6
====================================================The multipath.conf is as
follows:
defaults {
multipath_tool "/sbin/multipath -v0"
udev_dir /dev
polling_interval 10
default_selector "round-robin 0"
default_path_grouping_policy multibus
default_getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
default_prio_callout "/bin/true"
default_features "0"
rr_wmin_io 100
failback immediate
}
devnode_blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^hda[0-9]*"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}
multipaths {
multipath {
wwid 222ea000155eadda4
alias stor
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
failback immediate
}
}
=================================================The output from "multipath
-v 3" of my CentOS machine is below:
load path identifiers cache
#
# all paths in cache :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200
222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200
dm-0 blacklisted
dm-1 blacklisted
hda blacklisted
hdc blacklisted
md0 blacklisted
ram0 blacklisted
ram10 blacklisted
ram11 blacklisted
ram12 blacklisted
ram13 blacklisted
ram14 blacklisted
ram15 blacklisted
ram1 blacklisted
ram2 blacklisted
ram3 blacklisted
ram4 blacklisted
ram5 blacklisted
ram6 blacklisted
ram7 blacklisted
ram8 blacklisted
ram9 blacklisted
===== path info sda (mask 0x1f) ====bus = 1
dev_t = 8:0
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 0:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
uid = 222ea000155eadda4 (cache)
===== path info sdb (mask 0x1f) ====bus = 1
dev_t = 8:16
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 1:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
uid = 222ea000155eadda4 (cache)
#
# all paths :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200
222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200
pgpolicy = multibus (LUN setting)
selector = round-robin 0 (LUN setting)
features = 0 (internal default)
hwhandler = 0 (internal default)
rr_weight = 2 (config file default)
no_path_retry = -1 (config file default)
pg_timeout = NONE (internal default)
0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100
set ACT_CREATE: map does not exists
message stor 0 switch_group 1
create: stor (222ea000155eadda4)
[size=1 GB][features="0"][hwhandler="0"]
\_ round-robin 0
\_ 0:0:0:0 sda 8:0 [ready]
\_ 1:0:0:0 sdb 8:16 [ready]
reloading features
^^^^^^^^ Notice the 1GB in the map created. I have read in mailing
list posts that 1GB is returned when it is unable to find the size.
=================================================The output from "multipath
-v 3" of the working 4.2/4.3 machine is below:
hda blacklisted
hdc blacklisted
md0 blacklisted
ram0 blacklisted
ram10 blacklisted
ram11 blacklisted
ram12 blacklisted
ram13 blacklisted
ram14 blacklisted
ram15 blacklisted
ram1 blacklisted
ram2 blacklisted
ram3 blacklisted
ram4 blacklisted
ram5 blacklisted
ram6 blacklisted
ram7 blacklisted
ram8 blacklisted
ram9 blacklisted
path sda not found in pathvec
===== path sda ====bus = 1
dev_t = 8:0
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 0:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672?
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
getuid = /sbin/scsi_id -g -u -s /block/%n (internal default)
uid = 222ea000155eadda4 (callout)
path sdb not found in pathvec
===== path sdb ====bus = 1
dev_t = 8:16
size = 3503903232
vendor = Promise
product = VTrak M300i
rev = 2200
h:b:t:l = 1:0:0:0
tgt_node_name serial = 000000000000000000000000D46D0E4B486DC672?
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
getuid = /sbin/scsi_id -g -u -s /block/%n (internal default)
uid = 222ea000155eadda4 (callout)
#
# all paths :
#
222ea000155eadda4 0:0:0:0 sda 8:0 [VTrak M300i ]
222ea000155eadda4 1:0:0:0 sdb 8:16 [VTrak M300i ]
pgpolicy = multibus (LUN setting)
selector = round-robin 0 (LUN setting)
features = 0 (internal default)
hwhandler = 0 (internal default)
0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000
action preset to 0
action set to 4
create: stor (222ea000155eadda4)
[size=1 TB][features="0"][hwhandler="0"]
\_ round-robin 0
\_ 0:0:0:0 sda 8:0
\_ 1:0:0:0 sdb 8:16
message stor 0 switch_group 1
^^^^^^^^ it detects the size correctly (it is over 1TB, it seems to do
some rounding when it reports it here)
===================================
note the multipath config lines:
not working: 0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100
working: 0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000
The primary difference is in the 2nd number. That is, according to the
multipath wiki, the "target length in 512-bytes blocks" for the
multipath device. The one that does not work is smaller by a factor of
1063 (close to 1024, maybe it is using different units?)
Has anyone run into any issue of this sort?
Seemingly Similar Threads
- ISCSI SAN suggestion
- Promise VTrak and CentOS 5
- Experience with Promise Tech. arrays/jbod''s?
- multipath using defaults rather than multipath.conf contents for some devices (?) - why ?
- SUMMARY : multipath using defaults rather than multipath.conf contents for some devices (?) - why ?