thr3ads.net - zfs discuss - [zfs-discuss] General recommendations on raidz groups of different sizes [Jul 2007]

If this information is useful, please help other people find it:
Share via:

David Smith

2007-Jul-18 21:20 UTC

[zfs-discuss] General recommendations on raidz groups of different sizes

What are your thoughts or recommendations on having a zpool made up of
raidz groups of different sizes?  Are there going to be performance issues?

For example:

  pool: testpool1
 state: ONLINE
 scrub: none requested
config:

        NAME                                        STATE     READ WRITE CKSUM
        testpool1                                   ONLINE       0     0     0

          raidz1                                    ONLINE       0     0     0
            c12t600A0B800029E5EA000007234685122Ad0  ONLINE       0     0     0
            c12t600A0B800029E5EA000007254685123Cd0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000072F46851256d0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000073146851266d0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000073746851278d0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000074146851292d0  ONLINE       0     0     0
            c12t600A0B800029E5EA00000747468512B6d0  ONLINE       0     0     0
            c12t600A0B800029E5EA00000749468512C2d0  ONLINE       0     0     0
          raidz1                                    ONLINE       0     0     0
            c12t600A0B800029E5EA0000074F468512E0d0  ONLINE       0     0     0
            c12t600A0B800029E5EA00000751468512E8d0  ONLINE       0     0     0
            c12t600A0B800029E5EA000007574685130Cd0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000075946851318d0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000075F4685132Ed0  ONLINE       0     0     0
            c12t600A0B800029E5EA0000076546851342d0  ONLINE       0     0     0


Thanks,

David
 
 
This message posted from opensolaris.org

Richard Elling

2007-Jul-19 18:30 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

After a cup of French coffee, I feel strong enough to recommend :-)

David Smith wrote:> What are your thoughts or recommendations on having a zpool made up of
> raidz groups of different sizes?  Are there going to be performance issues?
It is more complicated and, in general, more complicated is a bad thing.
But in your example, with only 2 top-level vdevs, it isn''t overly
complicated.

Performance issues will be difficult to predict because this hasn''t
been
studied.  With the gazillions of possible permutations, it is not likely
to be extensively characterized.  But it if it works for you, then be happy :-)
  -- richard
> For example:
> 
>   pool: testpool1
>  state: ONLINE
>  scrub: none requested
> config:
> 
>         NAME                                        STATE     READ WRITE
CKSUM
>         testpool1                                   ONLINE       0     0   
0
> 
>           raidz1                                    ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007234685122Ad0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007254685123Cd0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000072F46851256d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000073146851266d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000073746851278d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000074146851292d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000747468512B6d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000749468512C2d0  ONLINE       0     0   
0
>           raidz1                                    ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000074F468512E0d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000751468512E8d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007574685130Cd0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000075946851318d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000075F4685132Ed0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000076546851342d0  ONLINE       0     0   
0
> 
> 
> Thanks,
> 
> David
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Matthew Ahrens

2007-Jul-19 21:49 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

David Smith wrote:> What are your thoughts or recommendations on having a zpool made up of
> raidz groups of different sizes?  Are there going to be performance issues?
It should be fine.  Under some circumstances the performance could be similar 
to a pool with all raidz groups of the smallest size you used (less bandwidth).

--matt
> 
> For example:
> 
>   pool: testpool1
>  state: ONLINE
>  scrub: none requested
> config:
> 
>         NAME                                        STATE     READ WRITE
CKSUM
>         testpool1                                   ONLINE       0     0   
0
> 
>           raidz1                                    ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007234685122Ad0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007254685123Cd0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000072F46851256d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000073146851266d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000073746851278d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000074146851292d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000747468512B6d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000749468512C2d0  ONLINE       0     0   
0
>           raidz1                                    ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000074F468512E0d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA00000751468512E8d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA000007574685130Cd0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000075946851318d0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000075F4685132Ed0  ONLINE       0     0   
0
>             c12t600A0B800029E5EA0000076546851342d0  ONLINE       0     0   
0
> 
> 
> Thanks,
> 
> David
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

David W. Smith

2007-Jul-19 23:28 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

Matt,

Thank you for your reply.  I like to keep the raidz groups with the same
number of disks but it is not always that easily to have the numbers
work out way.

David


On Thu, 2007-07-19 at 14:49 -0700, Matthew Ahrens wrote:> David Smith wrote:
> > What are your thoughts or recommendations on having a zpool made up of
> > raidz groups of different sizes?  Are there going to be performance
issues?
> 
> It should be fine.  Under some circumstances the performance could be
similar
> to a pool with all raidz groups of the smallest size you used (less
bandwidth).
> 
> --matt
> 
> > 
> > For example:
> > 
> >   pool: testpool1
> >  state: ONLINE
> >  scrub: none requested
> > config:
> > 
> >         NAME                                        STATE     READ
WRITE CKSUM
> >         testpool1                                   ONLINE       0    
0     0
> > 
> >           raidz1                                    ONLINE       0    
0     0
> >             c12t600A0B800029E5EA000007234685122Ad0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA000007254685123Cd0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000072F46851256d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000073146851266d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000073746851278d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000074146851292d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA00000747468512B6d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA00000749468512C2d0  ONLINE       0    
0     0
> >           raidz1                                    ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000074F468512E0d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA00000751468512E8d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA000007574685130Cd0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000075946851318d0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000075F4685132Ed0  ONLINE       0    
0     0
> >             c12t600A0B800029E5EA0000076546851342d0  ONLINE       0    
0     0
> > 
> > 
> > Thanks,
> > 
> > David
> >  
> >  
> > This message posted from opensolaris.org
> > _______________________________________________
> > zfs-discuss mailing list
> > zfs-discuss at opensolaris.org
> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Haudy Kazemi

2007-Jul-21 22:21 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

How would one calculate system reliability estimates here? One is a RAIDZ 
set of 6 disks, the other a set of 8. The reliability of each RAIDZ set by 
itself isn''t too hard to calculate, but put together, especially since 
they''re different sizes, I don''t know.

On Jul 19 2007, Richard Elling wrote:
>After a cup of French coffee, I feel strong enough to recommend :-)
>
>David Smith wrote:
>> What are your thoughts or recommendations on having a zpool made up of 
>> raidz groups of different sizes? Are there going to be performance 
>> issues?
>
> It is more complicated and, in general, more complicated is a bad thing. 
> But in your example, with only 2 top-level vdevs, it isn''t overly 
> complicated.
>
> Performance issues will be difficult to predict because this
hasn''t been
> studied. With the gazillions of possible permutations, it is not likely 
> to be extensively characterized. But it if it works for you, then be 
> happy :-)
>  -- richard
>
>> For example:
>> 
>>   pool: testpool1
>>  state: ONLINE
>>  scrub: none requested
>> config:
>> 
>>         NAME STATE READ WRITE CKSUM
>>         testpool1 ONLINE 0 0 0
>> 
>>           raidz1 ONLINE 0 0 0
>>             c12t600A0B800029E5EA000007234685122Ad0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA000007254685123Cd0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000072F46851256d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000073146851266d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000073746851278d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000074146851292d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA00000747468512B6d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA00000749468512C2d0 ONLINE 0 0 0
>>           raidz1 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000074F468512E0d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA00000751468512E8d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA000007574685130Cd0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000075946851318d0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000075F4685132Ed0 ONLINE 0 0 0
>>             c12t600A0B800029E5EA0000076546851342d0 ONLINE 0 0 0
>> 
>> 
>> Thanks,
>> 
>> David
>>  
>>  
>> This message posted from opensolaris.org
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>_______________________________________________
>zfs-discuss mailing list
>zfs-discuss at opensolaris.org
>http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

Richard Elling

2007-Jul-23 04:21 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

Haudy Kazemi wrote:> How would one calculate system reliability estimates here? One is a 
> RAIDZ set of 6 disks, the other a set of 8. The reliability of each 
> RAIDZ set by itself isn''t too hard to calculate, but put together,
> especially since they''re different sizes, I don''t know.
We just weigh them accordingly.  MTTDL for a 6-disk set will be better
than for an 8-disk set, though that seems to be counter-intuitive for
some folks.  Let me see if I can put some numbers together later this
week...
  -- richard

Richard Elling

2007-Jul-23 23:39 UTC

head link

[zfs-discuss] General recommendations on raidz groups of different sizes

Richard Elling wrote:> Haudy Kazemi wrote:
>> How would one calculate system reliability estimates here? One is a 
>> RAIDZ set of 6 disks, the other a set of 8. The reliability of each 
>> RAIDZ set by itself isn''t too hard to calculate, but put
together,
>> especially since they''re different sizes, I don''t
know.
> 
> We just weigh them accordingly.  MTTDL for a 6-disk set will be better
> than for an 8-disk set, though that seems to be counter-intuitive for
> some folks.  Let me see if I can put some numbers together later this
> week...
OK, after some math we can get some idea...

Using the MTTDL[1] model for a default disk (500 GBytes, 800k hours MTBF,
24 hours logistical response, 60 GBytes/hr resync) we get:

	config                        MTTDL[1] (yrs)
	--------------------------------------------
  	6-disk raidz                    75,319
	8-disk raidz                    40,349
	2x 6-disk raidz                 37,659
	6-disk raidz + 8-disk raidz     26,274
	2x 8-disk raidz                 20,175

As you would expect, the MTTDL for the 6-disk scenario is better than for
the 8-disk scenario.  So it follows that the MTTDL for a pair of 6-disk
raidz sets is better than for a pair of 8-disk raidz sets and the 6+8
scenario is in between.

The MTTDL[1] model is described at:
	http://blogs.sun.com/relling/entry/a_story_of_two_mttdl

  -- richard

zfs discuss - Jul 2007 - General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes

[zfs-discuss] General recommendations on raidz groups of different sizes