freebsd stable - Mar 2014 - ZFS UNMAP performance

I'm seeing very slow performance with certain operations on a ZFS
filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI
implementation).  The issue appears to be with BIO_DELETE operations.
Monitoring the system with gstat shows expected times for read and write
operations, but deletes are in the multiple hundreds of milliseconds
under normal operation.  Destroying a snapshot sends the times to
astronomical levels.  sysctl says the system is using UNMAP for deletes:

kern.cam.da.0.delete_method: UNMAP

I searched and found where Oracle issued a performance alert for Solaris
11.1 where ZFS using UNMAP was in use.  Here's a link to a blog
discussing it:

http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html


Is FreeBSD also impacted?  If so, is there a fix or a workaround?

Thanks

Danny Schales

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20140311/0f89dfe6/attachment.sig>

On Tue, Mar 11, 2014 at 3:28 PM, Danny Schales <dan at latech.edu>
wrote:
> I'm seeing very slow performance with certain operations on a ZFS
> filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI
> implementation).  The issue appears to be with BIO_DELETE operations.
> Monitoring the system with gstat shows expected times for read and write
> operations, but deletes are in the multiple hundreds of milliseconds
> under normal operation.  Destroying a snapshot sends the times to
> astronomical levels.  sysctl says the system is using UNMAP for deletes:
>
> kern.cam.da.0.delete_method: UNMAP
>
> I searched and found where Oracle issued a performance alert for Solaris
> 11.1 where ZFS using UNMAP was in use.  Here's a link to a blog
> discussing it:
>
> http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html
>
>
> Is FreeBSD also impacted?  If so, is there a fix or a workaround?
FreeBSD is affected the same way Solaris is. IMHO, neither are
affected, it is merely a consequence of using a device with a poor
TRIM/UNMAP algorithm.

You can trivially tell ZFS to not use TRIM, or tweak the tunings to
suit your devices. I'm afraid I don't know what they all mean
precisely..

vfs.zfs.vdev.trim_on_init: 1
vfs.zfs.vdev.trim_max_bytes: 2147483648
vfs.zfs.vdev.trim_max_pending: 64
vfs.zfs.trim.enabled: 1
vfs.zfs.trim.txg_delay: 32
vfs.zfs.trim.timeout: 30
vfs.zfs.trim.max_interval: 1

Of course, if you disable TRIM you will end up with new (but
different) poor performance characteristics. Probably.

Cheers

Tom