Hi, Forgive my ignorance of ZFS, but I have a customer that would like to set up three 14+2 raidz2 groups on a new thor with 48 1TB drives (updated thumper) so that 42TB for data could be achieved. What performance or other technical issues with a stripe 14 disks wide would he likely see? He does not want a hot spare. Any advice appreciated in advance. -- This message posted from opensolaris.org
On Tue, Sep 2, 2008 at 15:39, Barton Fisk <bfisk at bellsouth.net> wrote:> Hi, > Forgive my ignorance of ZFS, but I have a customer that would like to set up three 14+2 raidz2 groups on a new thor with 48 1TB drives (updated thumper) so that 42TB for data could be achieved. What performance or other technical issues with a stripe 14 disks wide would he likely see? He does not want a hot spare.Well, first, you need a boot disk of some sort. Thor does have a CF slot, so that''s an option, but if the customer plans to do, say, home directories on the root directory, using disks might be a good place to start. Second, random read/write performance is going to suck a lot compared to narrower stripes. Raidz{,2} groups need to read from N-{1,2} disks so that they can verify on reads that the checksum matches. Thus, with a 16-disk raidz2 group, you must wait for 14 IOs to complete before you can decide whether you''ve got junk back from any disk. Smaller groups would help this quite a bit, both because there are more groups and because they''re smaller. Even four 10+2 groups (for a still-respectable 40 TB) would be better, and six 6+2 groups (for 36 TB) would be my recommendation. If space is really that much of a concern, or you''re doing sequential transfers to large files, 14+2 groups will probably be survivable... but I wouldn''t use them. Will
Sorry I omitted that CF will be the boot device. Thanks again. -- This message posted from opensolaris.org
Barton Fisk wrote:> Sorry I omitted that CF will be the boot device. Thanks again. >What are you using for redundancy of the boot device? Ian
Barton Fisk wrote:> Hi, > Forgive my ignorance of ZFS, but I have a customer that would like to set up three 14+2 raidz2 groups on a new thor with 48 1TB drives (updated thumper) so that 42TB for data could be achieved. What performance or other technical issues with a stripe 14 disks wide would he likely see? He does not want a hot spare. > Any advice appreciated in advance. >RAIDoptimizer was designed to help you work through the possibilities and make trade-offs. Note that the man page does not encourage such wide sets, you can see why when you run RAIDoptimizer. http://ras4sun.sfbay.sun.com/HTMLS/tools.html -- richard
Richard Elling wrote:> Barton Fisk wrote: > >> Hi, >> Forgive my ignorance of ZFS, but I have a customer that would like to set up three 14+2 raidz2 groups on a new thor with 48 1TB drives (updated thumper) so that 42TB for data could be achieved. What performance or other technical issues with a stripe 14 disks wide would he likely see? He does not want a hot spare. >> Any advice appreciated in advance. >> >> > > RAIDoptimizer was designed to help you work through the possibilities > and make trade-offs. Note that the man page does not encourage such > wide sets, you can see why when you run RAIDoptimizer. > http://ras4sun.sfbay.sun.com/HTMLS/tools.html >Silly me. It is still Monday, and I am coffee challenged. RAIDoptimizer is still an internal tool. However, for those who are interested in the results of a RAIDoptimizer run for 48 disks, see: http://blogs.sun.com/relling/entry/sample_raidoptimizer_output -- richard
On Tue, Sep 2, 2008 at 2:15 PM, Richard Elling <Richard.Elling at sun.com> wrote:> Silly me. It is still Monday, and I am coffee challenged. RAIDoptimizer > is still an internal tool. However, for those who are interested in the > results > of a RAIDoptimizer run for 48 disks, see: > http://blogs.sun.com/relling/entry/sample_raidoptimizer_outputRichard -- Is there a chance of RAIDoptimizer will be made available to the unwashed masses? Could you post the results for a few runs with other numbers of disks, such as 8 (which is the number of drives I plan to use) or 12 (the number of drives in the 2510, etc)? -B -- Brandon High bhigh at freaks.com "You can''t blow things up with schools and hospitals." -Stephen Dailey
Yeah, I''m looking at using 10 disks or 16 disks (depending on which chassis I get) - and I would like reasonable redundancy (not HA-crazy redundancy where I can suffer tons of failures, I can power this down and replace disks, it''s a home server) and maximize the amount of usable space. Putting up some page somewhere (if possible) or just exposing the algorithms so maybe one of us can try to hack together a page would be cool (I don''t have openoffice/staroffice and admit I am too lazy to download it to examine the file on Windows) On 9/2/08, Brandon High <bhigh at freaks.com> wrote:> On Tue, Sep 2, 2008 at 2:15 PM, Richard Elling <Richard.Elling at sun.com> wrote: > > Silly me. It is still Monday, and I am coffee challenged. RAIDoptimizer > > is still an internal tool. However, for those who are interested in the > > results > > of a RAIDoptimizer run for 48 disks, see: > > http://blogs.sun.com/relling/entry/sample_raidoptimizer_output > > > Richard -- > > Is there a chance of RAIDoptimizer will be made available to the > unwashed masses? > > Could you post the results for a few runs with other numbers of disks, > such as 8 (which is the number of drives I plan to use) or 12 (the > number of drives in the 2510, etc)? > > -B > > -- > Brandon High bhigh at freaks.com > "You can''t blow things up with schools and hospitals." -Stephen Dailey > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Brandon High wrote:> On Tue, Sep 2, 2008 at 2:15 PM, Richard Elling <Richard.Elling at sun.com> wrote: > >> Silly me. It is still Monday, and I am coffee challenged. RAIDoptimizer >> is still an internal tool. However, for those who are interested in the >> results >> of a RAIDoptimizer run for 48 disks, see: >> http://blogs.sun.com/relling/entry/sample_raidoptimizer_output >> > > > Richard -- > > Is there a chance of RAIDoptimizer will be made available to the > unwashed masses? >Yes, I''m in the process of open-sourcing it. -- richard> Could you post the results for a few runs with other numbers of disks, > such as 8 (which is the number of drives I plan to use) or 12 (the > number of drives in the 2510, etc)? > > -B > >