zfs discuss - Oct 2010 - Is there a way to limit ZFS File Data but maintain room for the ARC to cache metadata

I''m working on this scenario in which file system activity appears to 
cause the arc cache to evict meta data.  I would like to have a 
preference to keep the metadata in cache over ZFS File Data

What I''ve notice on import of a zpool the arc_meta_used goes up 
significantly.  ZFS meta data operations usually run pretty good.  
However over time with IO Operations the cache get''s evicted and 
arc_no_grow get set.


-bash-3.00# echo ::arc | mdb -k| grep arc
arc_no_grow               =         1
arc_tempreserve           =         0 MB
arc_meta_used             =       277 MB
arc_meta_limit            =      3789 MB
arc_meta_max              =      1951 MB

-bash-3.00# echo ::arc | mdb -k| grep arc
arc_no_grow               =         1
arc_tempreserve           =         0 MB
arc_meta_used             =        91 MB
arc_meta_limit            =      3789 MB
arc_meta_max              =      1951 MB

this will have the adverse affect on zfs commands taking longer.

-bash-3.00# echo ::memstat | mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    1185002              4628   29%
ZFS File Data             2186752              8542   53%
Anon                        41183               160    1%
Exec and libs                2378                 9    0%
Page cache                  13202                51    0%
Free (cachelist)           518567              2025   13%
Free (freelist)            195901               765    5%

Total                     4142985             16183
Physical                  4054870             15839

So, I would like to limit the amount of ZFS File Data that can be used 
and keep the arc cache warm with metadata.  Any suggestions?

Thanks

Dave

-- 




-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101001/bfb365ff/attachment.html>

On Fri, Oct 1, 2010 at 11:46 AM, David Blasingame Oracle
<david.blasingame at oracle.com> wrote:
> I''m working on this scenario in which file system activity appears
to cause
> the arc cache to evict meta data.? I would like to have a preference to
keep
> the metadata in cache over ZFS File Data
>
> What I''ve notice on import of a zpool the arc_meta_used goes up
> significantly.? ZFS meta data operations usually run pretty good.? However
> over time with IO Operations the cache get''s evicted and
arc_no_grow get
> set.
<snip>
> So, I would like to limit the amount of ZFS File Data that can be used and
> keep the arc cache warm with metadata.? Any suggestions?
Would adding a cache device (L2ARC) and setting primarycache=metadata
and secondarycache=all on the root dataset do what you need?

That way ARC is used strictly for metadata, and L2ARC is used for metadata+data.

-- 
Freddie Cash
fjwcash at gmail.com

Good idea.  Provides options, but it would be nice to be able to set a 
low water mark on what can be taken away from the arc metadata cache 
without having to have something like an SSD.

Dave

On 10/01/10 14:02, Freddie Cash wrote:
> On Fri, Oct 1, 2010 at 11:46 AM, David Blasingame Oracle
> <david.blasingame at oracle.com> wrote:
>   
>> I''m working on this scenario in which file system activity
appears to cause
>> the arc cache to evict meta data.  I would like to have a preference to
keep
>> the metadata in cache over ZFS File Data
>>
>> What I''ve notice on import of a zpool the arc_meta_used goes
up
>> significantly.  ZFS meta data operations usually run pretty good. 
However
>> over time with IO Operations the cache get''s evicted and
arc_no_grow get
>> set.
>>     
>
> <snip>
>
>   
>> So, I would like to limit the amount of ZFS File Data that can be used
and
>> keep the arc cache warm with metadata.  Any suggestions?
>>     
>
> Would adding a cache device (L2ARC) and setting primarycache=metadata
> and secondarycache=all on the root dataset do what you need?
>
> That way ARC is used strictly for metadata, and L2ARC is used for
metadata+data.
>
>   

-- 



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20101006/f2e6bff8/attachment.html>