zfs discuss - Nov 2007 - SAN arrays with NVRAM cache : ZIL and zfs_nocacheflush

Hi,

   I read some articles on solarisinternals.com like
"ZFS_Evil_Tuning_Guide" on
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide .  They
clearly suggest to disable cache flush
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#FLUSH .

It seems to be the only serious article on the net about this subject.

Could someone here state on this tuning suggestion ?  
My cu is running a HDS SAN array with Oracle on ZFS, I''d like to be
clear in my brain.
 
 
This message posted from opensolaris.org

ndo at unikservice.com said:
> They clearly suggest to disable cache flush
http://www.solarisinternals.com/
> wiki/index.php/ZFS_Evil_Tuning_Guide#FLUSH . 
> 
> It seems to be the only serious article on the net about this subject.
> 
> Could someone here state on this tuning suggestion ?   My cu is running a
HDS
> SAN array with Oracle on ZFS, I''d like to be clear in my brain. 
There is a brief discussion (in which I participated) at the HDS user
forum site:
	http://forums.hds.com/index.php?showtopic=427

Note that we made the change on the array itself, instead of using the
ZFS kernel tunable parameter.  Our servers have other storage devices
besides our HDS array, and it''s only recently that the tunable has made
its way into standard Solaris-10 releases.

I can say that for us, telling the HDS array to ignore sync-cache requests
made a huge improvement in performance for our NFS clients.  The increase
was small for local ZFS operations, including for Oracle, so you''ll
want
to measure for yourself.

Regards,

Marion

Marion Hakanson wrote:
> ndo at unikservice.com said:
>   
>> They clearly suggest to disable cache flush
http://www.solarisinternals.com/
>> wiki/index.php/ZFS_Evil_Tuning_Guide#FLUSH . 
>>
>> It seems to be the only serious article on the net about this subject.
>>
>> Could someone here state on this tuning suggestion ?   My cu is running
a HDS
>> SAN array with Oracle on ZFS, I''d like to be clear in my
brain.
>>     
>
> There is a brief discussion (in which I participated) at the HDS user
> forum site:
> 	http://forums.hds.com/index.php?showtopic=427
>
> Note that we made the change on the array itself, instead of using the
> ZFS kernel tunable parameter.  Our servers have other storage devices
> besides our HDS array, and it''s only recently that the tunable has
made
> its way into standard Solaris-10 releases.
>
> I can say that for us, telling the HDS array to ignore sync-cache requests
> made a huge improvement in performance for our NFS clients.  The increase
> was small for local ZFS operations, including for Oracle, so
you''ll want
> to measure for yourself.
>   
Interesting. The HDS folks I talked to said the array no-ops the cache 
sync. Which models were you using? Midrange only, right?

tmcmahon2 at yahoo.com said:
> Interesting. The HDS folks I talked to said the array no-ops the cache 
sync.
> Which models were you using? Midrange only, right? 
HDS "modular" product -- ours is 9520V, which was the smallest
available.
It has a mix of FC and SATA drives (yes, really).

Check the HDS forum article I referred to, as well as:
  http://forums.hds.com/index.php?showtopic=497

Default is to honor the cache sync request unless array-based replication
(Synchronous TrueCopy) is installed.  Also, our before/after timings were
pretty conclusive.

Regards,

Marion

The info in that tuning guide depends on what Solaris version you are working
with.  Last I checked it was not current.

I use Solaris 10u4 and have zfs_nocacheflush set.  Haven''t played with
using alternate disks for ZIL yet not really sure what that does to my HA model.
I have mirrored LUNS between 2 3510FC arrays, each of them with
dual-controllers.  With dual battery-backed controllers I''m not really
worried about the cache flush thing but if you operate without this safety net,
might not be a good idea.
>From everything I have read, performance of Oracle to ZFS is not very good. 
The Oracle system I have is still using plain UFS on a RAID-10 LUN with noatime,
directio options.
 
 
This message posted from opensolaris.org