zfs discuss - Oct 2007 - ZFS is very slow in our test, when the capacity is high

We are tesing ZFS in OpenSolairs, write TBs data to ZFS, But when the
capacity is close to 90%, ZFS went into slowly. We do ls, rm, and write
something, those operation is so terrible. for example, we do ls in a
Directory which have about 4000 Directories, the time is about 5-10s!
we''ve
checked the CPU, memory(and swap), IO, all those are normal and idle. So is
there any specialty in ZFS when capacity is high, like UFS? thanks.

follow is some information:

bash-3.00# uname -a

SunOS stor-test 5.11 snv_70 i86pc i386 i86pc



bash-3.00# df -lh

Filesystem             size   used  avail capacity  Mounted on

/dev/dsk/c0t0d0s0       50G   9.1G    40G    19%    /

/devices                 0K     0K     0K     0%    /devices

/dev                     0K     0K     0K     0%    /dev

ctfs                     0K     0K     0K     0%    /system/contract

proc                     0K     0K     0K     0%    /proc

mnttab                   0K     0K     0K     0%    /etc/mnttab

swap                   1.3G   920K   1.3G     1%    /etc/svc/volatile

objfs                    0K     0K     0K     0%    /system/object

sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab

/usr/lib/libc/libc_hwcap1.so.1

                        50G   9.1G    40G    19%    /lib/libc.so.1

fd                       0K     0K     0K     0%    /dev/fd

swap                   1.3G    40K   1.3G     1%    /tmp

swap                   1.3G    32K   1.3G     1%    /var/run

/dev/dsk/c0t0d0s7      150G   1.7G   147G     2%    /export/home

zfstest1               1.9T   1.7T   211G    89%    /zfstest1

-- 
eSX
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20071012/26a6b237/attachment.html>

Hi,

did you read the following?
http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
> Currently, pool performance can degrade when a pool is very full and
> filesystems are updated frequently, such as on a busy mail server.
> Under these circumstances, keep pool space under 80% utilization
> to maintain pool performance.
I wonder if defining a zfs quota of roughly 80% of the whole pool capacity would
help to keep performance up. Users always use all the space available.

Regards,
Tom
 
 
This message posted from opensolaris.org

eSX wrote:
> We are tesing ZFS in OpenSolairs, write TBs data to ZFS, But when the
> capacity is close to 90%, ZFS went into slowly. We do ls, rm, and write
> something, those operation is so terrible. for example, we do ls in a
> Directory which have about 4000 Directories, the time is about 5-10s!
> we''ve checked the CPU, memory(and swap), IO, all those are normal
and
> idle. So is there any specialty in ZFS when capacity is high, like UFS?
> thanks.
It''s insane to exhaust every bits in almost *any* file system IMHO,
because when this happens you end up with a lot of fragments (this will
affect ZFS more than UFS, as the difference in disk layout), which will
just hurt performance because the increased disk seek requests.

Cheers,
-- 
Xin LI <delphij at delphij.net>	http://www.delphij.net/
FreeBSD - The Power to Serve!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20071012/48fc698e/attachment.bin>