zfs code - Jan 2011 - tips for optimizing writes in append mode

Hello,

I''m wondering if, besides batching writes (writing less often, but
with larger chunks), there are other ways to improve write performance
to files opened in append mode (under Solaris with ZFS). I tried
posix_fallocate, with the purpose of pre allocating space and then
issue pwrite calls, but it seems unsupported for ZFS.

thanks

-- 
Filipe David Manana,
fdmanana at gmail.com, fdmanana at apache.org

"Reasonable men adapt themselves to the world.
?Unreasonable men adapt the world to themselves.
?That''s why all progress depends on unreasonable men."

> Hello,
> 
> I''m wondering if, besides batching writes (writing
> less often, but
> with larger chunks), there are other ways to improve
> write performance
> to files opened in append mode (under Solaris with
> ZFS). I tried
> posix_fallocate, with the purpose of pre allocating
> space and then
> issue pwrite calls, but it seems unsupported for ZFS.
> 
> thanks
Since ZFS is copy-on-write, it seems to me that new storage
is allocated for every write, so even if you could pre-allocate,
that would only affect the amount of space used by the file,
not the speed of writes to previously unused space.

So for ZFS, I think that anything that would speed up writes in
general would have similar results for writes that grow a file,
and probably very little could be done that would only be of help
to the latter.

In other words, just do what you can to have a faster zpool (mirrors faster
than raidz, adding a dedicated SSD log device could speed up writes, etc).

There is other tuning that might apply in special circumstances, but I
don''t think much of that is specific to writes that append to a file.

Regarding posix_fallocate(), the spec says:
"The posix_fallocate() function is part of the Advisory Information option
and need not be provided on all implementations."

As of the older source on src.opensolaris.org, at least, it looks to me like
only UFS may actually support posix_fallocate().

So...if your program might be used on systems running both UFS and ZFS,
then call posix_fallocate() if you wish, but be sure that your program
behaves sensibly whether or not the call succeeds.
-- 
This message posted from opensolaris.org