On 9/4/07 4:34 PM, "Richard Elling" <Richard.Elling at Sun.COM>
wrote:
> Hi Andy,
> my comments below...
> note that I didn''t see zfs-discuss at opensolaris.org in the CC
for the
> original...
>
> Andy Lubel wrote:
>> Hi All,
>>
>> I have been asked to implement a zfs based solution using storedge 6130
and
>> im chasing my own tail trying to decide how best to architect this.
The
>> storage space is going to be used for database dumps/backups (nearline
>> storage). What is killing me is that I must mix hardware raid and
zfs..
>
> Why should that be killing you? ZFS works fine with RAID arrays.
What kills me is the fact that I have a choice and it was hard to decide on
which one was going to be at the top of the totem pole. From now on I only
want JBOD!
Works even better when I export each disk in my array as a single raid0 x14
then create the zpool :)
#zpool create -f vol0 c2t1d12 c2t1d11 c2t1d10 c2t1d9 c2t1d8 c2t1d7 c2t1d6
c2t1d5 c2t1d4 c2t1d3 c2t1d2 c2t1d1 c2t1d0 spare c2t1d13>
>> The storedge shelf has 14 FC 72gb disks attached to a solaris snv_68.
>>
>> I was thinking that since I cant export all the disks un-raided out to
the
>> solaris system that I would instead:
>>
>> (on the 6130)
>> Create 3 raid5 volumes of 200gb each using the "Sun_ZFS" pool
(128k segment
>> size, read ahead enabled 4 disk).
>>
>> (On the snv_68)
>> Create a raid0 using zfs of the 3 volumes from the 6130, using the same
128k
>> stripe size.
>
> OK
>
>> It seemed to me that if I was going to go for redundancy with a mixture
of
>> zfs and hardware raid that I would put the redundancy into the hardware
raid
>> and use striping at the zfs level, is that methodology the best way to
think
>> of it?
>
> The way to think about this is that ZFS can only correct errors when it has
> redundancy. By default, for dynamic stripes, only metadata is redundant.
> You can set the copies parameter to add redundancy on a per-file system
basis,
> so you could set a different policy for data you really care about.
>
Makes perfect sense. Since this is a nearline backup solution, I think we
will be OK with a dynamic stripe. Once I get approved for thumper im
definitely going to go raidz2. Since we are a huge Sun partner.. It should
be easier than its been :(
>> The only requirement ive gotten so far is that it can be written to and
read
>> from at a minimum of 72mb/s locally and 1gb/35sec via nfs. I suspect I
>> would need at least 600gb of storage.
>
> I hope you have a test case for this. It is difficult for us to predict
> that sort of thing because there are a large number of variables. But in
> general, to get high bandwidth, you need large I/Os. That implies the
> application
> is responsible for it''s use of the system, since the application
is the source
> of I/Os.
>
Its all going to be accessed via NFS and eventually iscsi, as soon as we
figure out how to backup iscsi targets from the SAN itself.
>> Anyone have any recommendations? The last time tried to create one 13
disk
>> raid5 with zfs filesystem the performance was terrible via nfs.. But
when I
>> shared an nfs filesystem via a raidz or mirror things were much
better.. So
>> im nervous about doing this with only one volume in the zfs pool.
>
> 13 disk RAID-5 will suck. Try to stick with fewer devices in the set.
>
> See also
> http://mail.opensolaris.org/pipermail/zfs-discuss/2006-December/024194.html
> http://blogs.digitar.com/jjww/?itemid=44
>
I cant find a santricity download that will work with a 6130, but
that''s
ok.. I just created 14 volumes per shelf :) hardware raid is so yesterday.
> That data is somewhat dated, as we now have the ability to put the ZIL
> on a different log device (Nevada b70 or later). This will be more obvious
> if the workload creates a lot of small files, less of a performance problem
> for large files.
> -- richard
>
Got my hands on a Ram-San SSD 64gb and I''m using that for the zil.. Its
crazy fast now.
-Andy
--
