CentOS - Jan 2011 - Static assignment of SCSI device names?

Hello list members,

In CentOS-5.5  I'm trying to achieve static assignment of SCSI device 
names for a bunch of RAID-60 drives on a Supermicro motherboard.  The 
"scsi_id" command identifies all drives ok.

The board has one SATA controller and three SAS/SATA controllers ... 
standard on-board ICH-10 ATA channels, an on-board LSI SAS/SATA 
controller, and two add-on SAS/SATA contoller cards.  There are 13 
drives in all, spread across the four controllers, all configured for 
Linux software RAID.

The problem is in management of the drive names and figuring out which 
drive to pull in case of failure.  Unfortunately the BIOS scan detects 
only the three drives connected to the ICH-10 SATA controller.  That's 
ok because that's where the RAID-1 boot drives are.  However, when the 
kernel starts it assigns those drives last, not first.

For this reason I want to use a set of udev rules to assign specific 
names to the drives plugged into specific ports (to maintain my sanity 
:-) ).

Identifying drives by their ID string (which includes the drive's serial 
number) and assigning names in the rules works ok.   BUT, what happens 
when I have to swap out a failed drive?  The serial number (and possibly 
model number) changes, and the udev assignment should fail, probably 
assigning an unexpected /dev/sd? name.  RAID rebuild would choke until I 
change the MD device assignment.

Is it possible to assign SCSI drive names by hardware path instead?  I 
especially want the three RAID1+spare boot drives to always be assigned 
sda/sdb/sdc, because that sorts out other issues I'm having in CentOS-5.

In the udev rules file I tried piping the output of "scsi_id -g -i -u -s 
/block/..." through "cut" to extract the path, but I get no match
string
when I run "udevtest" against that block device.  Does the 
"PROGRAM==....." clause not recognize the pipe symbol?  I tried a
little
shellscript to provide the RESULT match string, but udevtest didn't like 
that.

Is there a supported way to predictably assign a drive name according to 
the hardware port it's plugged into ... it would make swapping drives a 
lot easier, since it becomes 'drive-id-string' agnostic.  Better yet, is
there any way to tell the kernel the order in which to scan the controllers?

I'm also hoping the problem doesn't radically change when I install 
CentOS-6 on this box.  I'm using CentOS-5 just to get practice in using 
KVM and RAID-60.

Thanks for any advice you can offer.

Chuck

On 1/27/2011 2:41 PM, Chuck Munro wrote:
>
> Identifying drives by their ID string (which includes the drive's
serial
> number) and assigning names in the rules works ok.   BUT, what happens
> when I have to swap out a failed drive?  The serial number (and possibly
> model number) changes, and the udev assignment should fail, probably
> assigning an unexpected /dev/sd? name.  RAID rebuild would choke until I
> change the MD device assignment.
If you can figure things out for the initial md device creation, 
subsequent assembly uses the uuid to match up the members and doesn't 
care if the disks have been moved around either physically or by 
detection order.  And if you are hot-swapping drives and rebuilding 
manually, you should be able to find the just-assigned name with
'dmesg'.

-- 
   Les Mikesell
    lesmikesell at gmail.com

on 12:41 Thu 27 Jan, Chuck Munro (chuckm at seafoam.net)
wrote:
> 
> Hello list members,
> 
> In CentOS-5.5  I'm trying to achieve static assignment of SCSI device 
> names for a bunch of RAID-60 drives on a Supermicro motherboard.  The 
> "scsi_id" command identifies all drives ok.
> 
> The board has one SATA controller and three SAS/SATA controllers ... 
> standard on-board ICH-10 ATA channels, an on-board LSI SAS/SATA 
> controller, and two add-on SAS/SATA contoller cards.  There are 13 
> drives in all, spread across the four controllers, all configured for 
> Linux software RAID.
> 
> The problem is in management of the drive names and figuring out which 
> drive to pull in case of failure.  Unfortunately the BIOS scan detects 
> only the three drives connected to the ICH-10 SATA controller.  That's 
> ok because that's where the RAID-1 boot drives are.  However, when the 
> kernel starts it assigns those drives last, not first.
> 
> For this reason I want to use a set of udev rules to assign specific 
> names to the drives plugged into specific ports (to maintain my sanity 
> :-) ).
> 
> Identifying drives by their ID string (which includes the drive's
serial
> number) and assigning names in the rules works ok.   BUT, what happens 
> when I have to swap out a failed drive?  The serial number (and possibly 
> model number) changes, and the udev assignment should fail, probably 
> assigning an unexpected /dev/sd? name.  RAID rebuild would choke until I 
> change the MD device assignment.
> 
> Is it possible to assign SCSI drive names by hardware path instead?  I 
> especially want the three RAID1+spare boot drives to always be assigned 
> sda/sdb/sdc, because that sorts out other issues I'm having in
CentOS-5.
> 
> In the udev rules file I tried piping the output of "scsi_id -g -i -u
-s
> /block/..." through "cut" to extract the path, but I get no
match string
> when I run "udevtest" against that block device.  Does the 
> "PROGRAM==....." clause not recognize the pipe symbol?  I tried a
little
> shellscript to provide the RESULT match string, but udevtest didn't
like
> that.
> 
> Is there a supported way to predictably assign a drive name according to 
> the hardware port it's plugged into ... it would make swapping drives a
> lot easier, since it becomes 'drive-id-string' agnostic.  Better
yet, is
> there any way to tell the kernel the order in which to scan the
controllers?
> 
> I'm also hoping the problem doesn't radically change when I install
> CentOS-6 on this box.  I'm using CentOS-5 just to get practice in using
> KVM and RAID-60.
Though I don't swear to understand it well, it's possible that multipath
(device-mapper-multipath) may work in your situation.  I've been using
it for iSCSI storage, where it provides multipathing capabilities,
including performance improvements, HA, and persistent device naming.

Whether this applies to hotplugged SCSI devices I'm not so sure, and
udev would be my first choice.

The multipath documentation is unfortunately atrocious.

-- 
Dr. Ed Morbius
Chief Scientist
Krell Power Systems Unlimited

> Les Mikesell kindly wrote:
>> >
>> >  Identifying drives by their ID string (which includes the
drive's serial
>> >  number) and assigning names in the rules works ok.   BUT, what
happens
>> >  when I have to swap out a failed drive?  The serial number (and
possibly
>> >  model number) changes, and the udev assignment should fail,
probably
>> >  assigning an unexpected /dev/sd? name.  RAID rebuild would choke
until I
>> >  change the MD device assignment.
> If you can figure things out for the initial md device creation,
> subsequent assembly uses the uuid to match up the members and doesn't
> care if the disks have been moved around either physically or by
> detection order.  And if you are hot-swapping drives and rebuilding
> manually, you should be able to find the just-assigned name with
'dmesg'.
>
That's how I've been doing it with the two RAID-6 servers I have now, 
but in my advanced age I'm getting lazy.  I did figure out how to tag 
the drive by path using the '-i' option to scsi_id and filtering by a 
"RESULT==<path> *" expression.  However, something has gone very
much
astray, because now the rules don't do anything ... the system assigns 
names in the order scanned despite having added the udev rules before 
the default '50-udev.rules' is executed.  I'm a bit mystified by
this.
Before I started messing around, filtering by drive id string did work, 
now it doesn't.  Is there a small database somewhere I need to trash?

Mike's suggestion is intriguing and makes me wonder if I can manually 
label a replacement drive with the same UUID as the failed one (assuming 
I keep records, of course).  Since all drives in the array are 
partitioned exactly the same way, I can have one or two spares on the 
shelf pre-partitioned, ready to be given a UUID.  I don't plan to hot swap.

Still, I'd love to be able to use udev rules to assign names, if only to 
make life easier ... inquiring minds need to know why I suddenly can't 
get any rules to work.  Maybe I need to re-install.  I have just 
downloaded a trial version of RHEL-6 which I should try, to see if it 
handles this stuff more gracefully.  It'll do for experimentation until 
CentOS-6 arrives.

Chuck