zfs discuss - May 2007 - Multiple filesystem costs? Directory sizes?

While setting up my new system, I''m wondering whether I should go with
plain directories or use ZFS filesystems for specific stuff. About the cost of
ZFS filesystems, I read on some Sun blog in the past about something like 64k
kernel memory (or whatever) per active filesystem. What are however the
additional costs?

The reason I''m considering multiple filesystems is for instance easy
ZFS backups and snapshots, but also tuning the recordsizes. Like storing lots of
generic pictures from the web, smaller recordsizes may be appropriate to trim
down the waste once the filesize surpasses the record size, aswell as using
large recordsizes for video files on a seperate filesystem. Turning on and off
compression and access times for performance reasons are another thing.

Also, in this same message, I''d like to ask what sensible maximum
directory sizes are. As in amount of files.

Thanks.
 
 
This message posted from opensolaris.org

Mario Goebbels wrote:
> While setting up my new system, I''m wondering whether I should go
with plain
> directories or use ZFS filesystems for specific stuff. About the cost of
ZFS
> filesystems, I read on some Sun blog in the past about something like 64k
> kernel memory (or whatever) per active filesystem. What are however the
additional
> costs?
I don''t think the resource costs are well characterized, yet.
IMHO, you should only create file systems if you need to have different
policies for the file systems.  Search this forum for more discussion on
this topic.
> The reason I''m considering multiple filesystems is for instance
easy ZFS backups
> and snapshots, but also tuning the recordsizes. Like storing lots of
generic
> pictures from the web, smaller recordsizes may be appropriate to trim down
the
> waste once the filesize surpasses the record size, aswell as using large
recordsizes
> for video files on a seperate filesystem. Turning on and off compression
and access
> times for performance reasons are another thing.
compression and atime settings are policies.
recordsize could also be a policy, however, it seems to me that you are confused
about
ZFS and recordsize.  The reason it exists is for those applications (eg.
databases)
which use a fixed recordsize and we want to match that record size to avoid
doing
extra work.  For example, if the application recordsize is fixed at 8 kBytes,
then
we don''t want to prefetch 129 kBytes (or 56 kBytes) as that could be
wasted work.
By default, ZFS will dynamically adjust its recordsize, which is probably what
you
want.
> Also, in this same message, I''d like to ask what sensible maximum
directory sizes
> are. As in amount of files.
Dunno. In theory, you could go until you run out of space.  Several people have
commented on their usage, so you can look in the archives.
  -- richard

Mario,

For the reasons you mentioned, having a few different filesystems
(on the order of 5-10, I''d guess) can be handy.  Any time you want
different behavior for different types of data, multiple filesystems
are the way to go.

For maximum directory size, it turns out that the practical limits
aren''t in ZFS -- they''re in your favorite applications, like
ls(1)
and file browsers.  ZFS won''t mind if you put millions of files
in a directory, but ls(1) will be painfully slow.  Similarly, if
you''re using a mail program and you go to a big directory to grab
an attachment... you''ll wait and wait while it reads the first few
bytes of every file in the directory to determine its type.

Hope that helps,

Jeff

Mario Goebbels wrote:
> While setting up my new system, I''m wondering whether I should go
with plain directories or use ZFS filesystems for specific stuff. About the cost
of ZFS filesystems, I read on some Sun blog in the past about something like 64k
kernel memory (or whatever) per active filesystem. What are however the
additional costs?
> 
> The reason I''m considering multiple filesystems is for instance
easy ZFS backups and snapshots, but also tuning the recordsizes. Like storing
lots of generic pictures from the web, smaller recordsizes may be appropriate to
trim down the waste once the filesize surpasses the record size, aswell as using
large recordsizes for video files on a seperate filesystem. Turning on and off
compression and access times for performance reasons are another thing.
> 
> Also, in this same message, I''d like to ask what sensible maximum
directory sizes are. As in amount of files.
> 
> Thanks.
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss