[zfs-discuss] Partitioning ARC

Stuart Anderson
2011-02-01T05:59:28Z

On Jan 30, 2011, at 6:03 PM, Richard Elling wrote:

> On Jan 30, 2011, at 5:01 PM, Stuart Anderson wrote:
>> On Jan 30, 2011, at 2:29 PM, Richard Elling wrote:
>>
>>> On Jan 30, 2011, at 12:21 PM, stuart anderson wrote:
>>>
>>>> Is it possible to partition the global setting for the maximum
ARC size
>>>> with finer grained controls? Ideally, I would like to do this
on a per
>>>> zvol basis but a setting per zpool would be interesting as
well?
>>>
>>> While perhaps not perfect, see the primarycache and secondarycache
>>> properties of the zvol or file system.
>>
>> With primarycache I can turn off utilization of the ARC for some
zvol''s,
>> but instead I was hoping to use the ARC but limit the maximum amount
>> on a per zvol basis.
>
> Just a practical question, do you think the average storage admin will have
> any clue as to how to use this tunable?

Yes. I think the basic idea of partitioning a memory cache over different
storage objects is a straightforward concept.

> How could we be more effective in
> communicating the features and pitfalls of resource management at this
> level?

Document that this is normally handled dynamically based on the default
policy that all storage objects should be assigned ARC space on a fair
share basis. However, if different quality of service is required for different
storage objects this may be adjusted as follows...

>
>>>
>>>> The use case is to prioritize which zvol devices should be
fully cached
>>>> in DRAM on a server that cannot fit them all in memory.
>>>
>>> It is not clear to me that this will make sense in a world of
snapshots and dedup.
>>> Could you explain your requirements in more detail?
>>
>> I am using zvol''s to hold the metadata for another filesystem
(SAM-QFS).
>> In some circumstances I can fit enough of this into the ARC that
virtually
>> all metadata reads IOPS happen at DRAM performance rather than SSD
>> or slower.
>>
>> However, with a single server hosting multiple filesystem (hence
multiple
>> zvols) I would like to be able to prioritize the use of the ARC.
>
> I think there is merit to this idea. It can be especially useful in the
zone
> context. Please gather your thoughts and file an RFE at www.illumos.org

Not sure how to file an illumos RFE, but one simple model to think about
would is a 2 tiered system where by default ZFS datasets use the ARC is
currently the case, with no (to the best of my knowledge) relative priority,
but some objects could optionally specific a request for a minimum size,
e.g., add a companion attribute to primarycache named primarycachesize.
This would represent the minimum amount of ARC space that is available
for that object.

Some thought would have to be given as to how to indicate if the sum
of all primarycachesize settings is greater than zfs_arc_max, and
document what happens in this case, e.g., all values ignored?

Presumably something similar could also be considered for secondarycache.


Thanks.


--
Stuart Anderson  anderson at ligo.caltech.edu