CentOS - Jan 2011 - CentOS and Dell MD3200i / MD3220i iSCSI w/ multipath

We've been wrestling with this for ... rather longer than I'd care to
admit.

Host / initiator systems are a number of real and virtualized CentOS 5.5
boxes.  Storage arrays / targets are Dell MD3220i storage arrays.

CentOS is not a Dell-supported configuration, and we've had little helpful
advice from Dell.  There's been some amount of FUD in that Dell don't
seem
to know what Dell's own software installation (the md3

Dell doesn't seem to have much OS experience generally.

Their docs are pretty inconsistent.  I've noted omissions, terminology
differences, and procedural differences among the "Owner's
Manual",
"Deployment Guide", a professional services "Remote Services
Installation
Agreement" service description.  Some of the multipathing guidance
we've had
comes from their EqualLogic line of storage servers.

Questions:

1: Is there anyone out there running this configuration and are you
satisfied with it?

2: We get a set of error messages on the initator at target login.  These
appear to be benign, and web research suggests it's the result of a driver
configuration issue in trying to send instructions to a

         http://comments.gmane.org/gmane.linux.iscsi.open-iscsi/5970
         http://bit.ly/gguLl7

        MD3000i boxes have 1 controller that can execute IO and one that
        cannot execute READ/WRITE IO until a special command is sent. In
        older kernels, layers like the partition scanning and udev and
        hal would send down IO to those disabled paths and we would see
        IO errors like in your link.

        In newer kernels we have device handler modules (for MD3000i you
        would want scsi_dh_rdac (so do "modprobe scsi_dh_rdac" and
then
        lsmod to see if it is there)) that will detect if the path is
        not active and if so it will either not send the IO or it will
        not print a error message since we expect the IO to fail.

We'd prefer *not* seeing spurious I/O errors that we don't have to sift
through looking for real storage issues.

3: The MD32xxi series is a dual-controller array with multiple ports on each
controller.  Multiple targets can be logged into from an initiator, with the
pathways aggregated by the Linux multipath (device-mapper-multipath)
system.  There's very little clear documentation on multipath.  In
particular, any way to trigger alerting from multipath events / failures, or
iscsi session actions, would be helpful.  The MD32xxi series only supports
reporting from a GUI management utility (MDSM) which would be at best
problematic to run in a server environment.  The other question is: is
multipathing typical of iSCSI configuation?  Little of the iSCSI docs I've
found discusses multipath configurations at all:

http://www.open-iscsi.org/
http://www.cuddletech.com/articles/iscsi/index.html (good but very dated)
http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html (no
mention of multipathing)

4: Dell suggests a shutdown procedure including a flush of multipathing
paths ("multipath -F" -- in the "Owner's Manual"):

3 Flush the Device Mapper multipath maps list to remove any old or
  modified mappings
  # multipath ?F

Presumably this would go into one of the init scripts -- perhaps the
/etc/init.d/multipath script, as part of the "stop" sequence (after
the
multipath daemon is killed).  Anyone done this or know why the practice is
recommended?

Based on our experience with Dell I would NOT recommend this configuration
for others.  But we're stuck with it, and any help in getting things
configured would be very helpful and gratefully received.

I'm also hoping to get clearance to release docs we've generated, though
that's the subject of some internal negotiation.

-- 
Dr. Ed Morbius
Chief Scientist
Krell Power Systems Unlimited
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.centos.org/pipermail/centos/attachments/20110121/5a32ee5a/attachment-0002.html>

On Jan 21, 2011, at 6:41 PM, Edward Morbius <dredmorbius at gmail.com>
wrote:
> We've been wrestling with this for ... rather longer than I'd care
to admit.
> 
> Host / initiator systems are a number of real and virtualized CentOS 5.5
boxes.  Storage arrays / targets are Dell MD3220i storage arrays.
> 
> CentOS is not a Dell-supported configuration, and we've had little
helpful advice from Dell.  There's been some amount of FUD in that Dell
don't seem to know what Dell's own software installation (the md3
> 
> Dell doesn't seem to have much OS experience generally.
> 
> Their docs are pretty inconsistent.  I've noted omissions, terminology
differences, and procedural differences among the "Owner's
Manual", "Deployment Guide", a professional services "Remote
Services Installation Agreement" service description.  Some of the
multipathing guidance we've had comes from their EqualLogic line of storage
servers.
> 
> Questions:
> 
> 1: Is there anyone out there running this configuration and are you
satisfied with it?
> 
> 2: We get a set of error messages on the initator at target login.  These
appear to be benign, and web research suggests it's the result of a driver
configuration issue in trying to send instructions to a
> 
>          http://comments.gmane.org/gmane.linux.iscsi.open-iscsi/5970
>          http://bit.ly/gguLl7 
> 
>         MD3000i boxes have 1 controller that can execute IO and one that
>         cannot execute READ/WRITE IO until a special command is sent. In
>         older kernels, layers like the partition scanning and udev and
>         hal would send down IO to those disabled paths and we would see
>         IO errors like in your link.
> 
>         In newer kernels we have device handler modules (for MD3000i you
>         would want scsi_dh_rdac (so do "modprobe scsi_dh_rdac"
and then
>         lsmod to see if it is there)) that will detect if the path is
>         not active and if so it will either not send the IO or it will
>         not print a error message since we expect the IO to fail.
> 
> We'd prefer *not* seeing spurious I/O errors that we don't have to
sift through looking for real storage issues.
> 
> 3: The MD32xxi series is a dual-controller array with multiple ports on
each controller.  Multiple targets can be logged into from an initiator, with
the pathways aggregated by the Linux multipath (device-mapper-multipath) system.
There's very little clear documentation on multipath.  In particular, any
way to trigger alerting from multipath events / failures, or iscsi session
actions, would be helpful.  The MD32xxi series only supports reporting from a
GUI management utility (MDSM) which would be at best problematic to run in a
server environment.  The other question is: is multipathing typical of iSCSI
configuation?  Little of the iSCSI docs I've found discusses multipath
configurations at all:
> 
> http://www.open-iscsi.org/ 
> http://www.cuddletech.com/articles/iscsi/index.html (good but very dated)
> http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html (no
mention of multipathing)
> 
> 4: Dell suggests a shutdown procedure including a flush of multipathing
paths ("multipath -F" -- in the "Owner's Manual"):
> 
> 3 Flush the Device Mapper multipath maps list to remove any old or
>   modified mappings
>   # multipath F
> 
> Presumably this would go into one of the init scripts -- perhaps the
/etc/init.d/multipath script, as part of the "stop" sequence (after
the multipath daemon is killed).  Anyone done this or know why the practice is
recommended?
> 
> Based on our experience with Dell I would NOT recommend this configuration
for others.  But we're stuck with it, and any help in getting things
configured would be very helpful and gratefully received.
> 
> I'm also hoping to get clearance to release docs we've generated,
though that's the subject of some internal negotiation.
You need the RDAC kernel module installed, this handles asymmetric multipathing
to these devices.

You can get this from Dell's site.

Once this is installed you need to setup dm-multipath, look for multipathd.conf
in /etc, get the product id and vendor id from dmesg after making an initial
connection via open-iscsi and use that in the mutipath config. Your going to
need to use path utility 'rdac' in the config instead of tur.

Google is your friend here.

-Ross

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.centos.org/pipermail/centos/attachments/20110121/d6dd2a06/attachment-0002.html>