thr3ads.net - zfs discuss - [zfs-discuss] "zpool create" behaviour [Jun 2008]

If this information is useful, please help other people find it:
Share via:

Cesare

2008-Jun-22 13:06 UTC

[zfs-discuss] "zpool create" behaviour

Hy,

I''m facing to a problem where I configure and create a zpool on my
test bed. The hardware is: T-5120 with Solaris10 with latest patch and
Clariion CX3 attached by 2 HBA. In this type of configuration every
LUN exported by Clariion is viewed 4 times by operating system.

If I configure the latest disk by using a controller the "zfs create"
doesn''t working telling me that there is a devices currently
unavailable. If I''ll use a different controller  (but is the same LUN
from the Clariion) I''ll not encountered the problem and the raidz pool
is created. I''m willing to use that controller for balance the I/O
between HBA and storage processor.

The output of "zpool create is":

- Not Working:
 zpool create -f tank raidz c2t5006016041E0222Ed3
c3t5006016141E0222Ed0 c2t5006016041E0222Ed1 c3t5006016141E0222Ed2
cannot create ''tank'': one or more devices is currently
unavailable

- Working
zpool create -f tank raidz c2t5006016041E0222Ed3 c3t5006016141E0222Ed0
c2t5006016041E0222Ed1 c2t5006016841E0222Ed2

zpool status
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME                       STATE     READ WRITE CKSUM
        tank                       ONLINE       0     0     0
          raidz1                   ONLINE       0     0     0
            c2t5006016041E0222Ed3  ONLINE       0     0     0
            c3t5006016141E0222Ed0  ONLINE       0     0     0
            c2t5006016041E0222Ed1  ONLINE       0     0     0
            c2t5006016841E0222Ed2  ONLINE       0     0     0

errors: No known data errors
rmims03#
-------------

Thanks for any suggestion.

Cesare

////////////////////////////
The disks and format output are:

-- powermt display dev=all
Pseudo name=emcpower2a
CLARiiON ID=CKXXXXXXXXXXXX [Storage (T-5120)]
Logical device ID=YYYYYYYYYYYYYY0002D230CB033EDD11 [LUN 12]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A       Array failover mode: 1
=============================================================================----------------
Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
=============================================================================3074
pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6041XXXXXXd3s0 SP A0
    active  alive      0      0
3074 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6841XXXXXXd3s0
SP B0
    active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6141XXXXXXd3s0 SP
A1     active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6941XXXXXXd3s0 SP
B1     active  alive      0      0

Pseudo name=emcpower0a
CLARiiON ID=CKXXXXXXXXXXXX [Storage (T-5120)]
Logical device ID=YYYYYYYYYYYYYY0070885B63033EDD11 [LUN 1]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A       Array failover mode: 1
=============================================================================----------------
Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
=============================================================================3074
pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6041XXXXXXd0s0 SP A0
    active  alive      0      0
3074 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6841XXXXXXd0s0
SP B0
    active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6141XXXXXXd0s0 SP
A1     active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6941XXXXXXd0s0 SP
B1     active  alive      0      0

Pseudo name=emcpower3a
CLARiiON ID=CKXXXXXXXXXXXX [Storage (T-5120)]
Logical device ID=YYYYYYYYYYYYYY009419CFDA033EDD11 [LUN 21]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A       Array failover mode: 1
=============================================================================----------------
Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
=============================================================================3074
pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6041XXXXXXd1s0 SP A0
    active  alive      0      0
3074 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6841XXXXXXd1s0
SP B0
    active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6141XXXXXXd1s0 SP
A1     active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6941XXXXXXd1s0 SP
B1     active  alive      0      0

Pseudo name=emcpower1a
CLARiiON ID=CKXXXXXXXXXXXX [Storage (T-5120)]
Logical device ID=YYYYYYYYYYYYYY00E45735E6033EDD11 [LUN 32]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
=============================================================================----------------
Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
=============================================================================3074
pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6041XXXXXXd2s0 SP A0
    active  alive      0      0
3074 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0 c2tXXXXXX6841XXXXXXd2s0
SP B0
    active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6141XXXXXXd2s0 SP
A1     active  alive      0      0
3072 pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0
c3tXXXXXX6941XXXXXXd2s0 SP
B1     active  alive      0      0

And "format" is:

--
AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci at 0/pci at 0/pci at 2/scsi at 0/sd at 0,0
       1. c1t1d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci at 0/pci at 0/pci at 2/scsi at 0/sd at 1,0
       2. c2tXXXXXX6041XXXXXXd0 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6041XXXXXX,0
       3. c2tXXXXXX6841XXXXXXd0 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6841XXXXXX,0
       4. c2tXXXXXX6841XXXXXXd1 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6841XXXXXX,1
       5. c2tXXXXXX6041XXXXXXd1 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6041XXXXXX,1
       6. c2tXXXXXX6841XXXXXXd2 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6841XXXXXX,2
       7. c2tXXXXXX6041XXXXXXd2 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6041XXXXXX,2
       8. c2tXXXXXX6841XXXXXXd3 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6841XXXXXX,3
       9. c2tXXXXXX6041XXXXXXd3 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0/fp at 0,0/ssd at
wXXXXXX6041XXXXXX,3
      10. c3tXXXXXX6941XXXXXXd0 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6941XXXXXX,0
      11. c3tXXXXXX6141XXXXXXd0 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6141XXXXXX,0
      12. c3tXXXXXX6141XXXXXXd1 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6141XXXXXX,1
      13. c3tXXXXXX6941XXXXXXd1 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6941XXXXXX,1
      14. c3tXXXXXX6141XXXXXXd2 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6141XXXXXX,2
      15. c3tXXXXXX6941XXXXXXd2 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6941XXXXXX,2
      16. c3tXXXXXX6941XXXXXXd3 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6941XXXXXX,3
      17. c3tXXXXXX6141XXXXXXd3 <DGC-RAID 5-0324-30.00GB>
          /pci at 0/pci at 0/pci at 9/SUNW,qlc at 0,1/fp at 0,0/ssd at
wXXXXXX6141XXXXXX,3
      18. emcpower0a <DGC-RAID 5-0324-30.00GB>
          /pseudo/emcp at 0
      19. emcpower1a <DGC-RAID 5-0324-30.00GB>
          /pseudo/emcp at 1
      20. emcpower2a <DGC-RAID 5-0324-30.00GB>
          /pseudo/emcp at 2
      21. emcpower3a <DGC-RAID 5-0324-30.00GB>
          /pseudo/emcp at 3
Specify disk (enter its number):
////////////////////////////////

Peter Tribble

2008-Jun-22 15:48 UTC

head link

[zfs-discuss] "zpool create" behaviour

On Sun, Jun 22, 2008 at 2:06 PM, Cesare <voltsz at gmail.com>
wrote:> Hy,
>
> I''m facing to a problem where I configure and create a zpool on my
> test bed. The hardware is: T-5120 with Solaris10 with latest patch and
> Clariion CX3 attached by 2 HBA. In this type of configuration every
> LUN exported by Clariion is viewed 4 times by operating system.
>
> If I configure the latest disk by using a controller the "zfs
create"
> doesn''t working telling me that there is a devices currently
> unavailable. If I''ll use a different controller  (but is the same
LUN
> from the Clariion) I''ll not encountered the problem and the raidz
pool
> is created. I''m willing to use that controller for balance the I/O
> between HBA and storage processor.
My experience is that zfs + powerpath + clariion doesn''t work.

(Try a ''zpool export'' followed by ''zpool
import'' - do you get your pool back?)

For this I''ve had to get rid of powerpath and use mpxio instead.

The problem seems to be that the clariion arrays are active/passive and
zfs trips up if it tries to use one of the passive links. Using mpxio hides
this and works fine. And powerpath on the (active/active) DMX-4 seems
to be OK too.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/

Cesare

2008-Jun-27 16:02 UTC

head link

[zfs-discuss] "zpool create" behaviour

Hy Peter and all,

after a couple of reboot the PowerPath and reconfiguring the LUN
exported by the Clariion now all seems to working fine.

Following the output of "zpool status":

---
machine# zpool status
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME                       STATE     READ WRITE CKSUM
        tank                       ONLINE       0     0     0
          raidz1                   ONLINE       0     0     0
            c3t5006016941E0222Ed3  ONLINE       0     0     0
            c3t5006016141E0222Ed1  ONLINE       0     0     0
            c2t5006016841E0222Ed2  ONLINE       0     0     0
            c3t5006016141E0222Ed0  ONLINE       0     0     0

errors: No known data errors
machine#
---

Meanwhile, I did some benchmarching on "raidz" zfs pool. Normally,
with application running on a zone where open, read and move thousand
of file (average dimension of that file is around 2k) I gave this
result:

--
machine# zpool iostat 2
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
tank        7.56G   111G      4    233  25.7K  1.37M
tank        7.56G   111G      0      0      0    254
tank        7.56G   111G      0    866      0  5.35M
tank        7.56G   111G      0      0      0      0
tank        7.56G   111G      0    389      0  1.23M
tank        7.56G   111G      0      0      0      0
tank        7.56G   111G      0     79      0   237K
tank        7.57G   111G      0    769      0  4.29M
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0    580      0  1.67M
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0    152      0   472K
tank        7.57G   111G      0    631      0  3.36M
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0     98      0   439K
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0    191      0   505K
tank        7.57G   111G      0    873      0  5.77M
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0    163      0   463K
tank        7.57G   111G      0      0      0      0
tank        7.57G   111G      0    252  64.2K   817K
tank        7.57G   111G      0    550      0  4.90M
^C
machine#
---

Can I achieve more performance?

Thanks.

Cesare
On Sun, Jun 22, 2008 at 5:48 PM, Peter Tribble <peter.tribble at
gmail.com> wrote:> On Sun, Jun 22, 2008 at 2:06 PM, Cesare <voltsz at gmail.com> wrote:
>> Hy,
>>
>> I''m facing to a problem where I configure and create a zpool
on my
>> test bed. The hardware is: T-5120 with Solaris10 with latest patch and
>> Clariion CX3 attached by 2 HBA. In this type of configuration every
>> LUN exported by Clariion is viewed 4 times by operating system.
>>
>> If I configure the latest disk by using a controller the "zfs
create"
>> doesn''t working telling me that there is a devices currently
>> unavailable. If I''ll use a different controller  (but is the
same LUN
>> from the Clariion) I''ll not encountered the problem and the
raidz pool
>> is created. I''m willing to use that controller for balance the
I/O
>> between HBA and storage processor.
>
> My experience is that zfs + powerpath + clariion doesn''t work.
>
> (Try a ''zpool export'' followed by ''zpool
import'' - do you get your pool back?)
>
> For this I''ve had to get rid of powerpath and use mpxio instead.
>
> The problem seems to be that the clariion arrays are active/passive and
> zfs trips up if it tries to use one of the passive links. Using mpxio hides
> this and works fine. And powerpath on the (active/active) DMX-4 seems
> to be OK too.

zfs discuss - Jun 2008 - "zpool create" behaviour

[zfs-discuss] "zpool create" behaviour

[zfs-discuss] "zpool create" behaviour

[zfs-discuss] "zpool create" behaviour