zfs discuss - Jun 2006 - Re: [osol-discuss] Re: I wish Sun would open-source "QFS"... /was:Re: Re: Distributed File System for Solaris

> Uhm... that''s the point where you are IMO slightly wrong. The
exact
> requirement is that inodes and data need to be seperated.
I find that difficult to believe.

What you need is performance.  Based on your experiences with
completely different, static-metadata architectures, you''ve
concluded (quite plausibly) that those architectures require
data and inodes to be split to perform well.  That I believe.

But ZFS is not at all like those architectures.  I say this not
as a marketing statement, but as practical on-the-ground truth.
We still have lots of performance work to do; hell, we still have
lots of performance work to do in UFS, and that''s 25 years old.
But the performance challenges in ZFS are qualitatively different
than the ones we had in UFS and its offspring (e.g. ext3 and VxFS).

In many cases ZFS will perform better already; in some cases it will
perform worse; but in almost all cases it will perform *differently*.
We''ll be much better able to help you reach your performance goals
if you can state them as performance goals.

Jeff

Jeff Bonwick wrote:
> In many cases ZFS will perform better already; in some cases it will
> perform worse; but in almost all cases it will perform *differently*.
and as a result many of the solutions to getting better performance for 
ZFS will be completely different to the types of things done in other 
file systems.
> We''ll be much better able to help you reach your performance goals
> if you can state them as performance goals.
Business level performance goals and minimal requirements is what is 
important here.  Keep the technology speak out of it.  This is what I 
find best gets us the real information.

--
Darren J Moffat

>We''ll be much better able to help you reach your performance goals
>if you can state them as performance goals.
In particular, knowing the latency requirements is important.

Uncompressed HD video runs at 1.5 Gbps; two streams would require 3 Gbps, or 375
MB/sec.  The requirement for real-time mixing, for instance, would be that we
can read one frame of data from each stream (roughly 6 MB) within 16ms. 
(Buffering allows a little leeway, but depends on the capability of the video
hardware. If the bandwidth to storage is much higher than the requirement, we
can "catch up" if we drop behind, as long as we do it before the
buffer, which is typically 1 or sometimes 2 frames, empties.)

The 16ms requirement generally precludes long seeks, as well as the error
correction that the drives do on bad ECC -- hence sites doing video typically
turn off the retries on the disks and set the flag that says "transfer data
even if it''s known to be bad", because it''s better to
have a frame with some garbage in it than to lose sync altogether. When the file
system supports it, the two files are normally stored on separate disks. When it
doesn''t, on separate file systems so that they can be on separate
disks.

That''s just one application, of course.  The requirements for on-demand
video streaming, for instance, are much weaker per stream (but we want as many
streams as possible).

Unfortunately, the performance required by storage often isn''t well
understood by either the customer or the developer of the video hardware.  They
simply want to achieve operation with no dropped frames, and usually
overengineer the storage accordingly (e.g. providing 750 MB/s instead of the
required 375 MB/s, which allows greater leeway for catching up after a seek).
Tuning these systems tends to be a bit of a black art. :-)
 
 
This message posted from opensolaris.org