Hi All, I''ve been reading through the documentation for ZFS and have noted in several blogs that ZFS should support more advanced layouts like RAID1+0, RAID5+0, etc. I am having a little trouble getting these more advanced configurations to play nicely. I have two disk shelves, each with 9x 300GB SCSI drives attached to a Dell PowerEdge 1850 with dual XEON CPUs and 4GB RAM running the 64-bit Solaris OS. Ideally, I would like to have a RAIDz-2 on each disk shelf and have a mirror between the two disk shelves so that my pool would remain available even if I lost the entire shelf. So far, I''ve been able to configure a single pool with two RAID-z volumes (one per shelf) in a stripe, though this doesn''t help me if I lose one of the arrays. I''ve also been able to configure 4x 4-disk mirrors. e.g. zpool create zfsdata mirror c0t0d0 c0t1d0 c0t2d0 c0t3d0 mirror c0t4d0 c0t5d0 c0t8d0 c0t9d0 mirror c1t0d0 c1t1d0 c1t2d0 c1t3d0 mirror c1t4d0 c1t5d0 c1t8d0 c1t9d0 That gives me two mirrors per disk shelf (4 in total) but only 1.1TB of usable disk capacity. Is there any way to get the configuration I want, i.e. two raid-z2 volumes in a mirrored configuration? If anyone out there has some suggestions for a better configuration, please let me know :-). I''d like to be able to lose two disks per shelf before losing the shelf (if possible) but still be able to recover from a total array failure. Thanks in advance. Sincerely, Brett. This message posted from opensolaris.org
Brett wrote:> Hi All, > > I''ve been reading through the documentation for ZFS and have noted in several blogs that ZFS should support more advanced layouts like RAID1+0, RAID5+0, etc. I am having a little trouble getting these more advanced configurations to play nicely. > > I have two disk shelves, each with 9x 300GB SCSI drives attached to a Dell PowerEdge 1850 with dual XEON CPUs and 4GB RAM running the 64-bit Solaris OS. > > Ideally, I would like to have a RAIDz-2 on each disk shelf and have a mirror between the two disk shelves so that my pool would remain available even if I lost the entire shelf. > > So far, I''ve been able to configure a single pool with two RAID-z volumes (one per shelf) in a stripe, though this doesn''t help me if I lose one of the arrays. I''ve also been able to configure 4x 4-disk mirrors. > > e.g. > > zpool create zfsdata mirror c0t0d0 c0t1d0 c0t2d0 c0t3d0 mirror c0t4d0 c0t5d0 c0t8d0 c0t9d0 mirror c1t0d0 c1t1d0 c1t2d0 c1t3d0 mirror c1t4d0 c1t5d0 c1t8d0 c1t9d0 > > That gives me two mirrors per disk shelf (4 in total) but only 1.1TB of usable disk capacity. > > Is there any way to get the configuration I want, i.e. two raid-z2 volumes in a mirrored configuration? > > If anyone out there has some suggestions for a better configuration, please let me know :-). I''d like to be able to lose two disks per shelf before losing the shelf (if possible) but still be able to recover from a total array failure. > >How about 8 two way mirrors between shelves and a couple of hot spares? Ian
Sorry for singling you out, Ian; I meant "Reply to All". This list doesn''t set "reply-to"... On 5/30/07, Ian Collins <ian at ianshome.com> wrote:> How about 8 two way mirrors between shelves and a couple of hot spares?That''s fine and good, but then losing just one disk from each shelf fast enough means the whole array is gone. Then one strong enough power glitch could potentially kill the whole array, if your power configuration lets that happen. And if you unplug one shelf by accident (or to change FC switches, cables, whatever), you''re left with no redundancy whatsoever. Will
will.murnane at gmail.com said:> On 5/30/07, Ian Collins <ian at ianshome.com> wrote: > > How about 8 two way mirrors between shelves and a couple of hot spares? > > That''s fine and good, but then losing just one disk from each shelf fast > enough means the whole array is gone. Then one strong enough power glitch > could potentially kill the whole array, if your power configuration lets that > happen. And if you unplug one shelf by accident (or to change FC switches, > cables, whatever), you''re left with no redundancy whatsoever.You''d get the kind of protection you want with: zpool create mirror shelf1disk1 shelf2disk1 \ mirror shelf1disk2 shelf2disk2 \ . . . mirror shelf1diskn shelf2diskn mypool It''s not the same as raidz2+1, but you would have twice the disk space as your four-way mirror example. Regards, Marion
Will Murnane wrote:> Sorry for singling you out, Ian; I meant "Reply to All". This list > doesn''t set "reply-to"... > On 5/30/07, Ian Collins <ian at ianshome.com> wrote: >> How about 8 two way mirrors between shelves and a couple of hot spares? > That''s fine and good, but then losing just one disk from each shelf > fast enough means the whole array is gone.Only if you lost the same two drives in each shelf, same as any other striped mirror. I guess the ideal solution in this case would be the ability to use mirrors as raidz components. Ian
Will Murnane wrote:> Sorry for singling you out, Ian; I meant "Reply to All". This list > doesn''t set "reply-to"... > On 5/30/07, Ian Collins <ian at ianshome.com> wrote: >> How about 8 two way mirrors between shelves and a couple of hot spares? > That''s fine and good, but then losing just one disk from each shelf > fast enough means the whole array is gone.Only if you lost the same two drives in each shelf, same as any other striped mirror. I guess the ideal solution in this case would be the ability to use mirrors as raidz components. Ian
The reliability calculations for these scenarios are described in several articles on my blog. http://blogs.sun.com/relling You do get additional, mirror-like reliability for using the copies property, also described in my blog. Personally, I''d go with mirroring across the shelves. KISS. -- richard Ian Collins wrote:> Will Murnane wrote: >> Sorry for singling you out, Ian; I meant "Reply to All". This list >> doesn''t set "reply-to"... >> On 5/30/07, Ian Collins <ian at ianshome.com> wrote: >>> How about 8 two way mirrors between shelves and a couple of hot spares? >> That''s fine and good, but then losing just one disk from each shelf >> fast enough means the whole array is gone. > Only if you lost the same two drives in each shelf, same as any other > striped mirror. I guess the ideal solution in this case would be the > ability to use mirrors as raidz components. > > Ian > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss