Where can I find a list of supported multipath drivers for ZFS? Keith McAndrew Senior Systems Engineer Northern California SUN Microsystems - Data Management Group <mailto:Keith.McAndrew at SUN.com> Keith.McAndrew at SUN.com 916 715 8352 Cell CONFIDENTIALITY NOTICE The information contained in this transmission may contain privileged and confidential information of SUN Microsystems. It is intended for review only by the person(s) named above. Dissemination, distribution or duplication of this communication is strictly prohibited by all recipients. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080208/e54d53f1/attachment.html>
James C. McPherson
2008-Feb-08 23:31 UTC
[zfs-discuss] List of supported multipath drivers
Keith McAndrew wrote:> Where can I find a list of supported multipath drivers for ZFS?Not quite sure what you mean. Do you mean "which hba drivers are supported with MPxIO?" ? If so, that list - as far as I know - is currently mpt (for SAS) emlxs qlc If instead you are asking "which drivers manage multipathing" then the list is scsi_vhci While there are other multipathing solutions out there such as PowerPath and VxDMP, I don''t think they work quite so well with ZFS. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
I was referring to HDLM/SDLM, powerpath, DMP. Specifically HDLM. Keith McAndrew Senior Systems Engineer Northern California SUN Microsystems - Data Management Group 916 715 8352 -----Original Message----- From: James.McPherson at Sun.COM [mailto:James.McPherson at Sun.COM] Sent: Friday, February 08, 2008 3:31 PM To: Keith.McAndrew at Sun.COM Cc: zfs-discuss at opensolaris.org Subject: Re: [zfs-discuss] List of supported multipath drivers Keith McAndrew wrote:> Where can I find a list of supported multipath drivers for ZFS?Not quite sure what you mean. Do you mean "which hba drivers are supported with MPxIO?" ? If so, that list - as far as I know - is currently mpt (for SAS) emlxs qlc If instead you are asking "which drivers manage multipathing" then the list is scsi_vhci While there are other multipathing solutions out there such as PowerPath and VxDMP, I don''t think they work quite so well with ZFS. James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
Hi, MPxIO is basically a failover protocol. The way MpXIO will handle a storage device is listed in /kernel/drv/scsi_vhci.conf If you have multiple paths to your device over Fibre or over IP for iSCSI and MPXIO is not configured right or disabled, or turned off on your storage array, then at the format command, you will get multiple entries for one LUN with short device names and the customer will be pretty unhappy. Any changes to this (scsi_vhci.conf) file will require a reboot. I always do a reboot -- -r to make sure the changes are taken. Then, watch the Solaris boot carefully for any error messages. For Sun supported hardware - 6130, 6140, 6540, 25xx et el, all you need to do is to set the storage device to MPxIO - for Solaris. Many of the Sun storage devices are already defaulted to this value. Just connect them up, use CAM to verify the setting is set to MPxIO and then check the /kernel/drv/scsi_vhci.conf file to make sure MPXIO is enabled. On Solaris 8 and 9 MPxIO was disabled by default. On Solaris 10, it is enabled and included in the distribution. If memory serves me correctly, for Solaris 8 and 9, you need to download MPxIO support and install it as a package with pkgadd or it is a huge shell script that installs the packages for you. You need to do this first on a virgin Solaris 8 or 9 system. Save the headache and just download Solaris 10 U5 and burn a DVD and install this for a new customer if possible. MpXIO will work over fibre channel or iSCSI connections as it is really a protocol. If your storage device is third party and not well known by Sun or tested well, then you need to enter the device into /kernel/drv/scsi_vhci.conf The file has an example on what to add, so look in the file first. To find out the exact string- Make sure your storage device has a LUN online that you can see with the format command from a Solaris prompt. Then, from format, select the device and from the format prompt use an inquiry command like this: format> inquiry This will send a SCSI_INQUIRY command to the device/lun. You will get back the VID/PID pair. Like: XYZ FAST_ARRAY_500 You need to put this exactly into the /kernel/drv/scsi_vhci.conf file to support the third party array and reboot.. Now, on build 79b all this is different. You put the same string into the scsi_vhci.conf file from the format> inquiry command, but you then have to call out the driver used. For example, if your device is a generic symmetric storage array, then the entry will be "XYZ FAST_ARRAY_500", "f_sym"; You need to have the _exact_ spaces in the above commands matching the comments in the /kernel/drv/scsi_vhci.conf file. Then, reboot -- -r When the system comes back up, run format and look for a single entry for your LUN with a LONG device name. When you see long device names and only one device in format for your one LUN, then you have succeeded. For EMC, you will need to probably put in the exact inquiry string to support it in /kernel/drv/scsi_vhci.conf - for Solaris 10 U5. For Opensolaris build 79b and above, there is an f_EMC driver in there that you need to pair with your EMC array. Again, the VID/PID is obtained from the format> inquiry command. Best to you, Jeff This message posted from opensolaris.org
I have a Hitachi SMS100 iSCSI array attached to my Nexenta 1.0 box. The storage array has dual ports/controllers and I have dual bge NICs on my server. I''m not using a iSCSI HBA in my setup. I''m unable to get MPxIO working, and the format command always shows my disk targets multiple times. Can you please tell me what I should put in scsi_vhci.conf? My config is below. # iscsiadm list target Target: iqn.1994-04.jp.co.hitachi:rsd.d8a.t.11089.1a001 Alias: - TPGT: 1 ISID: 4000002a0000 Connections: 1 Target: iqn.1994-04.jp.co.hitachi:rsd.d8a.t.11089.0a001 Alias: - TPGT: 1 ISID: 4000002a0000 Connections: 1 # format ... AVAILABLE DISK SELECTIONS: ... 2. c2t2d0 <DEFAULT cyl 2558 alt 2 hd 128 sec 32> /iscsi/disk at 0000iqn.1994-04.jp.co.hitachi%3Arsd.d8a.t.11089.0a0010001,1 3. c2t3d0 <DEFAULT cyl 2558 alt 2 hd 128 sec 32> /iscsi/disk at 0000iqn.1994-04.jp.co.hitachi%3Arsd.d8a.t.11089.1a0010001,1 ... Specify disk (enter its number): 2 selecting c2t2d0 ... format> inquiry Vendor: HITACHI Product: DF600F Revision: 0000 cat /kernel/drv/iscsi.conf # # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License, Version 1.0 only # (the "License"). You may not use this file except in compliance # with the License. # # You can obtain a copy of the license at src/sun_nws/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at src/sun_nws/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # # Copyright 2006 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # #ident "@(#)iscsi.conf 1.4 06/03/22 SMI" name="iscsi" parent="/" instance=0; # # I/O multipathing feature (MPxIO) can be enabled or disabled using # mpxio-disable property. Setting mpxio-disable="no" will activate # I/O multipathing; setting mpxio-disable="yes" disables the feature. # # Global mpxio-disable property: # # To globally enable MPxIO on all iscsi ports set: # mpxio-disable="no"; # # To globally disable MPxIO on all iscsi ports set: # mpxio-disable="yes"; # mpxio-disable="no"; cat /kernel/drv/scsi_vhci.conf # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License (the "License"). # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at usr/src/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # #pragma ident "@(#)scsi_vhci.conf 1.10 07/08/10 SMI" # name="scsi_vhci" class="root"; # # Load balancing global configuration: setting load-balance="none" will cause # all I/O to a given device (which supports multipath I/O) to occur via one # path. Setting load-balance="round-robin" will cause each path to the device # to be used in turn. # #load-balance="round-robin"; load-balance="none" # # Automatic failback configuration # possible values are auto-failback="enable" or auto-failback="disable" auto-failback="enable"; #BEGIN: FAILOVER_MODULE_BLOCK (DO NOT MOVE OR DELETE) # # Declare scsi_vhci failover module paths with ''ddi-forceload'' so that # they get loaded early enough to be available for scsi_vhci root use. # # NOTE: Correct operation depends on the value of ''ddi-forceload'', this # value should not be changed. The ordering of entries is from # most-specific failover modules (with a "probe" implementation that is # completely VID/PID table based), to most generic (failover modules that # are based on T10 standards like TPGS). By convention the last part of a # failover module path, after "/scsi_vhci_", is called the # "failover-module-name", which begins with "f_" (like "f_asym_sun"). The # "failover-module-name" is also used in the override mechanism below. ddi-forceload "misc/scsi_vhci/scsi_vhci_f_asym_sun", "misc/scsi_vhci/scsi_vhci_f_asym_lsi", "misc/scsi_vhci/scsi_vhci_f_asym_emc", "misc/scsi_vhci/scsi_vhci_f_sym_emc", "misc/scsi_vhci/scsi_vhci_f_sym", "misc/scsi_vhci/scsi_vhci_f_tpgs"; # # For a device that has a GUID, discovered on a pHCI with mpxio enabled, vHCI # access also depends on one of the scsi_vhci failover modules accepting the # device. The default way this occurs is by a failover module''s "probe" # implementation (sfo_device_probe) indicating the device is supported under # scsi_vhci. To override this default probe-oriented configuration in # order to # # 1) establish support for a device not currently accepted under scsi_vhci # # or 2) override the module selected by "probe" # # or 3) disable scsi_vhci support for a device # # # you can add a ''scsi-vhci-failover-override'' tuple, as documented in # scsi_get_device_type_string(9F). For each tuple, the first part provides # basic device identity information (vid/pid) and the second part selects # the failover module by "failover-module-name". If you want to disable # scsi_vhci support for a device, use the special failover-module-name "NONE". # Currently, for each failover-module-name in ''scsi-vhci-failover-override'' # (except "NONE") there needs to be a # "misc/scsi_vhci/scsi_vhci_<failover-module-name>" in ''ddi-forceload'' above. # # " 111111" # "012345670123456789012345", "failover-module-name" or "NONE" # "|-VID--||-----PID------|", # scsi-vhci-failover-override # "STK FLEXLINE 400", "f_asym_lsi", # "SUN T4", "f_tpgs", # "CME XIRTEMMYS", "NONE"; # #END: FAILOVER_MODULE_BLOCK (DO NOT MOVE OR DELETE) This message posted from opensolaris.org