zfs discuss - Jan 2007 - need advice: ZFS config ideas for X4500 Thumper?

Hi:       (Warning, new zfs user question)

I am setting up an X4500 for our small engineering site file server.
It''s mostly for builds, images, doc archives, certain workspace 
archives, misc
data.

I''d like a trade off between space and safety of data.  I have not set 
up a large
ZFS system before, and have only played with simple raidz2 with 7 disks.
After reading  
"http://blogs.sun.com/relling/entry/raid_recommendations_space_vs_mttdl"
I am leaning toward a RAID-Z2 config with spares, for approx 15 
terabytes, but I
do not yet understand the nomenclature and exact config details.
For example, the graph/chart shows that 7+2 RAID-Z2  with spares would 
be a good
balance in capacity and data safety, but I do not know what to do with 
that number, how
it maps to an actual setup?   Does that type of config also provide a 
balance between
performance and data safety?

Can someone provide an actual example of how the config should look?
If I save two disks for the boot, how do the other 46 disks get configured
between spares and zfs groups? 

Thanks,

Neal

Hi Neal,

We''ve been getting pretty good performance out of RAID-Z2 with 3x
6-disk RAID-Z2 stripes. More stripes mean better performance all
around...particularly on random reads. But as a file-server that''s
probably not a concern. With RAID-Z2 it seems to me 2 hot-spares is
very sufficient, but I''ll defer to others with more knowledge.

Best Regards,
Jason

On 1/23/07, Neal Pollack <Neal.Pollack at sun.com>
wrote:
> Hi:       (Warning, new zfs user question)
>
> I am setting up an X4500 for our small engineering site file server.
> It''s mostly for builds, images, doc archives, certain workspace
> archives, misc
> data.
>
> I''d like a trade off between space and safety of data.  I have not
set
> up a large
> ZFS system before, and have only played with simple raidz2 with 7 disks.
> After reading
>
"http://blogs.sun.com/relling/entry/raid_recommendations_space_vs_mttdl"
> I am leaning toward a RAID-Z2 config with spares, for approx 15
> terabytes, but I
> do not yet understand the nomenclature and exact config details.
> For example, the graph/chart shows that 7+2 RAID-Z2  with spares would
> be a good
> balance in capacity and data safety, but I do not know what to do with
> that number, how
> it maps to an actual setup?   Does that type of config also provide a
> balance between
> performance and data safety?
>
> Can someone provide an actual example of how the config should look?
> If I save two disks for the boot, how do the other 46 disks get configured
> between spares and zfs groups?
>
> Thanks,
>
> Neal
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

On 1/23/07, Neal Pollack <Neal.Pollack at sun.com>
wrote:
>
> Hi:       (Warning, new zfs user question)
>
> I am setting up an X4500 for our small engineering site file server.
> It''s mostly for builds, images, doc archives, certain workspace
> archives, misc
> data.
...
> Can someone provide an actual example of how the config should look?
> If I save two disks for the boot, how do the other 46 disks get configured
> between spares and zfs groups?

What I ended up with was working with 8+2 raidz2 vdevs. It could have been
4+2, but 8+2 gives you more space, and that was more important than
performance. (The performance of the 8+2 is easily adequate for our needs.)
And with 46 drives to play with I can have 4 lots of that. At the moment I
have
6 hot-spares (I may take some of those out later, but at the moment I
don''t
need them).

So the config looks like:

zpool create images \
raidz2 c{0,1,4,6,7}t0d0 c{1,4,5,6,7}t1d0 \
raidz2 c{0,4,5,6,7}t2d0 c{0,1,5,6,7}t3d0 \
raidz2 c{0,1,4,6,7}t4d0 c{0,1,4,6,7}t5d0 \
raidz2 c{0,1,4,5,7}t6d0 c{0,1,4,5,6}t7d0 \
spare c0t1d0 c1t2d0 c4t3d0 c5t5d0 c6t6d0 c7t7d0

this spreads everything across all the controllers, and with no more than 2
disks on each controller I could survive the rather unlikely event of a
controller failure (unless it''s the controller with the boot
drives...).

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070123/2ba5efd2/attachment.html>

Hi Peter,

Perhaps I''m a bit dense, but I''ve been befuddled by the x+y
notation
myself. Is it X stripes consisting of Y disks?

Best Regards,
Jason

On 1/23/07, Peter Tribble <peter.tribble at gmail.com>
wrote:
> On 1/23/07, Neal Pollack <Neal.Pollack at sun.com> wrote:
> > Hi:       (Warning, new zfs user question)
> >
> > I am setting up an X4500 for our small engineering site file server.
> > It''s mostly for builds, images, doc archives, certain
workspace
> > archives, misc
> > data.
>
> ...
> > Can someone provide an actual example of how the config should look?
> > If I save two disks for the boot, how do the other 46 disks get
configured
> > between spares and zfs groups?
>
> What I ended up with was working with 8+2 raidz2 vdevs. It could have been
> 4+2, but 8+2 gives you more space, and that was more important than
> performance. (The performance of the 8+2 is easily adequate for our needs.)
> And with 46 drives to play with I can have 4 lots of that. At the moment I
> have
> 6 hot-spares (I may take some of those out later, but at the moment I
don''t
> need them).
>
> So the config looks like:
>
> zpool create images \
> raidz2 c{0,1,4,6,7}t0d0 c{1,4,5,6,7}t1d0 \
> raidz2 c{0,4,5,6,7}t2d0 c{0,1,5,6,7}t3d0 \
> raidz2 c{0,1,4,6,7}t4d0 c{0,1,4,6,7}t5d0 \
> raidz2 c{0,1,4,5,7}t6d0 c{0,1,4,5,6}t7d0 \
> spare c0t1d0 c1t2d0 c4t3d0 c5t5d0 c6t6d0 c7t7d0
>
> this spreads everything across all the controllers, and with no more than 2
> disks on each controller I could survive the rather unlikely event of a
> controller failure (unless it''s the controller with the boot
drives...).
>
> --
> -Peter Tribble
> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
>
>

On 1/23/07, Jason J. W. Williams <jasonjwwilliams at gmail.com>
wrote:
>
> Hi Peter,
>
> Perhaps I''m a bit dense, but I''ve been befuddled by the
x+y notation
> myself. Is it X stripes consisting of Y disks?
>
Sorry. Took a short cut on that bit. It''s x data disks + y parity. So
in the
case of raidz1, y=1; in the case of raidz2, y=2. And ideally x should
be a power of 2. (So 8+2 is a raidz2 stripe of 10 disks in total.)

I''ve always used this notation, but now I think about it I''m
not sure
how universal it is.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070123/12b36fc8/attachment.html>

Hi Peter,

Ah! That clears it up for me. Thank you.

Best Regards,
Jason

On 1/23/07, Peter Tribble <peter.tribble at gmail.com>
wrote:
> On 1/23/07, Jason J. W. Williams <jasonjwwilliams at gmail.com>
wrote:
> > Hi Peter,
> >
> > Perhaps I''m a bit dense, but I''ve been befuddled by
the x+y notation
> > myself. Is it X stripes consisting of Y disks?
> >
>
> Sorry. Took a short cut on that bit. It''s x data disks + y parity.
So in the
> case of raidz1, y=1; in the case of raidz2, y=2. And ideally x should
> be a power of 2. (So 8+2 is a raidz2 stripe of 10 disks in total.)
>
> I''ve always used this notation, but now I think about it
I''m not sure
> how universal it is.
>
> --
>
> -Peter Tribble
> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/