zfs discuss - Apr 2006 - ZFS RAID-Z for Two-Disk Workstation Setup?

After reading the ZFS docs it does appear that RAID-Z can be used on a two-disk
system and I was wondering if the system would [i]basically [/i]work as
Intel''s Matrix RAID for two disks?

    [u] Intel Matrix RAID info:[/u]
     http://www.intel.com/design/chipsets/matrixstorage_sb.htm
     http://techreport.com/reviews/2005q1/matrix-raid/index.x?pg=1

My focus with this thread is some basic ZFS information on Solaris installations
in a two-disk workstation configuration.  Obviously mirrored or striped ZFS
pools seem quite straight forward, so my attention naturally turns to ZFS RAID-Z
in this application.

Can we expect a two-disk RAID-Z ZFS setup to have the following characteristics
for both storage space and overall performance?
          ZFS mirror  <  ZFS RAID-Z  <  ZFS stripe
          1 x             <  ~1? x         <  ~2 x
          (I know the numerical metric is a very rough approximation)

For my use, I would need to have a boot partition (until ZFS boot) and live
upgrade on each disk, then the remainder of each disk could be allocated to the
ZFS pool.  I don''t need mirrored disks in my workstations but
wouldn''t mind a slightly less performance for real-time data integrity.

Does anyone know if ZFS RAID-Z I/O performance "scales" well on only
two disks, or is it only slightly better than a single disk or mirrored pool? 
In other words, what''s the performance penalty running ZFS RAID-Z on
only two disks?

Disks in question are U320 SCSI (W2100z) or SATA (Ultra 20).

Any other insights or suggestions on this topic?

Thanks.
 
 
This message posted from opensolaris.org

On Fri, Apr 28, 2006 at 10:15:55AM -0700, Wes Williams
wrote:
> After reading the ZFS docs it does appear that RAID-Z can be used on a
two-disk system and I was wondering if the system would [i]basically [/i]work as
Intel''s Matrix RAID for two disks?
> 
>     [u] Intel Matrix RAID info:[/u]
>      http://www.intel.com/design/chipsets/matrixstorage_sb.htm
>      http://techreport.com/reviews/2005q1/matrix-raid/index.x?pg=1
I believe RAID-Z in a two-disk configuration is almost completely
identical (in terms of space and failure resistant) to mirroring, but
not an optimal implementation of it.

If you want mirroring, you should just use mirror vdevs.  Any ZFS folk
want to chime in?

Cheers,
- jonathan

-- 
Jonathan Adams, Solaris Kernel Development

On Fri, Apr 28, 2006 at 11:06:36AM -0700, Jonathan Adams
wrote:
> On Fri, Apr 28, 2006 at 10:15:55AM -0700, Wes Williams wrote:
> > After reading the ZFS docs it does appear that RAID-Z can be used on a
two-disk system and I was wondering if the system would [i]basically [/i]work as
Intel''s Matrix RAID for two disks?
> > 
> >     [u] Intel Matrix RAID info:[/u]
> >      http://www.intel.com/design/chipsets/matrixstorage_sb.htm
> >      http://techreport.com/reviews/2005q1/matrix-raid/index.x?pg=1
> 
> I believe RAID-Z in a two-disk configuration is almost completely
> identical (in terms of space and failure resistant) to mirroring, but
> not an optimal implementation of it.
> 
> If you want mirroring, you should just use mirror vdevs.  Any ZFS folk
> want to chime in?
That''s correct.  The Intel Matrix RAID thing looks like a scheme where
some data is RAID and some is single-copy.  They advertise it as
improving the performance of your single-copy data, which wouldn''t be
as
big of a deal for RAID-Z, since it is already much faster than
traditional RAID-4/5.  It would be an interesting idea though to have
the option of using less space to store things single-copy in either a
RAID-Z or a mirror.  We''ve discussed this idea a bit in the past, but
haven''t gotten around to implementing it.

That said, someday soon you''ll be able to do what you want (single-copy
and redundant filesystems in the same pool), using the same mechanism
("ditto" blocks) that we use today to store multiple copies of the
metadata (on different disks, if you have more than one),

--matt

> I believe RAID-Z in a two-disk configuration is
> almost completely identical (in terms of space and failure resistant)
> to mirroring, but not an optimal implementation of it.
> 
> If you want mirroring, you should just use mirror
> vdevs.  Any ZFS folk want to chime in?
> 
> Cheers,
> - jonathan
> 
Shame on me....the man for zpool has most everything I needed (SX 4/06) except
confirmation from a 3rd party.

========================================================-bash-3.00$ uname -a
Sun Microsystems Inc.   SunOS 5.11      snv_36  October 2007
-bash-3.00$ man zpool
Reformatting page.  Please Wait... done

System Administration Commands                          zpool(1M)

NAME
     zpool - configures ZFS storage pools

---snip---

     raidz

         A variation on RAID-5 that allows for  better  distribu-
         tion  of  parity  and eliminates the "RAID-5 write hole"
         (in which data and parity become  inconsistent  after  a
         power loss). Data and parity is striped across all disks
         within a raidz group.

         [b]A raidz group with N disks of size X can  hold  approxi-
         mately (N-1)*Xbytes[/b] and can withstand one device failing
         before data integrity is compromised. The minimum number
         of devices in a raidz group is 2. [b]The recommended number
         is between 3 and 9.[/b]
========================================================The emphasis above is
mine, but Jonathan appears correct in that there is no compelling reason to even
think about using RAID-Z for anything less than three physical disks, no matter
what way you slice up only two physical disks.

To summarize, if you only have two physical disks, forget about RAID-Z.
 
 
This message posted from opensolaris.org

To expand a bit on the description in the man page, the amount of data a
RAID-Z vdev can store actually varies quite a bit. One of the interesting
innovations of RAID-Z is that it only allocates chunks that are a multiple
of the minimum allocatable size (2 blocks -- 1 data, 1 parity) so that you
never end up with unallocatable holes. If you have a RAID-Z vdev with 3
100MB disks, you''ll be able to store anywhere between 150MB and 200MB
depending on the size of the actual I/Os.

Check it out:

# zpool status
  pool: pool
 state: ONLINE
 scrub: none requested
config:

        NAME            STATE     READ WRITE CKSUM
        pool            ONLINE       0     0     0
          raidz1        ONLINE       0     0     0
            /export/d0  ONLINE       0     0     0
            /export/d1  ONLINE       0     0     0
            /export/d2  ONLINE       0     0     0

# ls -lh /export/d[012]
-rw-rw-rw-   1 root     root        100M May  4 09:23 /export/d0
-rw-rw-rw-   1 root     root        100M May  4 09:23 /export/d1
-rw-rw-rw-   1 root     root        100M May  4 09:23 /export/d2
# zfs create pool/stuff
# cat < /dev/urandom > /pool/stuff/data
cat: output error (0/1040 characters written)
No space left on device
# ls -lh /pool/stuff/data
-rw-r--r--   1 root     root        169M May  4 09:23 /pool/stuff/data
# df -h /pool/stuff
Filesystem             size   used  avail capacity  Mounted on
pool/stuff             254M   254M     0K   100%    /pool/stuff

RAID-Z is great, but figuring out how much space you have left is a bit of
a crap shoot.

Adam

On Thu, May 04, 2006 at 06:48:25AM -0700, Wes Williams
wrote:
> > I believe RAID-Z in a two-disk configuration is
> > almost completely identical (in terms of space and failure resistant)
> > to mirroring, but not an optimal implementation of it.
> > 
> > If you want mirroring, you should just use mirror
> > vdevs.  Any ZFS folk want to chime in?
> > 
> > Cheers,
> > - jonathan
> > 
> 
> Shame on me....the man for zpool has most everything I needed (SX 4/06)
except confirmation from a 3rd party.
> 
> ========================================================> -bash-3.00$
uname -a
> Sun Microsystems Inc.   SunOS 5.11      snv_36  October 2007
> -bash-3.00$ man zpool
> Reformatting page.  Please Wait... done
> 
> System Administration Commands                          zpool(1M)
> 
> NAME
>      zpool - configures ZFS storage pools
> 
> ---snip---
> 
>      raidz
> 
>          A variation on RAID-5 that allows for  better  distribu-
>          tion  of  parity  and eliminates the "RAID-5 write hole"
>          (in which data and parity become  inconsistent  after  a
>          power loss). Data and parity is striped across all disks
>          within a raidz group.
> 
>          [b]A raidz group with N disks of size X can  hold  approxi-
>          mately (N-1)*Xbytes[/b] and can withstand one device failing
>          before data integrity is compromised. The minimum number
>          of devices in a raidz group is 2. [b]The recommended number
>          is between 3 and 9.[/b]
> ========================================================> The emphasis
above is mine, but Jonathan appears correct in that there is no compelling
reason to even think about using RAID-Z for anything less than three physical
disks, no matter what way you slice up only two physical disks.
> 
> To summarize, if you only have two physical disks, forget about RAID-Z.
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
-- 
Adam Leventhal, Solaris Kernel Development       http://blogs.sun.com/ahl