Is it possible to partition the global setting for the maximum ARC size with finer grained controls? Ideally, I would like to do this on a per zvol basis but a setting per zpool would be interesting as well? The use case is to prioritize which zvol devices should be fully cached in DRAM on a server that cannot fit them all in memory. Thanks. -- This message posted from opensolaris.org
----- Original Message -----> Is it possible to partition the global setting for the maximum ARC > size > with finer grained controls? Ideally, I would like to do this on a per > zvol basis but a setting per zpool would be interesting as well? > > The use case is to prioritize which zvol devices should be fully > cached in DRAM on a server that cannot fit them all in memory.I''m not aware of such a possibility. ARC is global, and L2ARC is local to the zpool(s) to which it is attached. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
On Jan 30, 2011, at 12:21 PM, stuart anderson wrote:> Is it possible to partition the global setting for the maximum ARC size > with finer grained controls? Ideally, I would like to do this on a per > zvol basis but a setting per zpool would be interesting as well?While perhaps not perfect, see the primarycache and secondarycache properties of the zvol or file system.> The use case is to prioritize which zvol devices should be fully cached > in DRAM on a server that cannot fit them all in memory.It is not clear to me that this will make sense in a world of snapshots and dedup. Could you explain your requirements in more detail? -- richard
On Jan 30, 2011, at 2:29 PM, Richard Elling wrote:> On Jan 30, 2011, at 12:21 PM, stuart anderson wrote: > >> Is it possible to partition the global setting for the maximum ARC size >> with finer grained controls? Ideally, I would like to do this on a per >> zvol basis but a setting per zpool would be interesting as well? > > While perhaps not perfect, see the primarycache and secondarycache > properties of the zvol or file system.With primarycache I can turn off utilization of the ARC for some zvol''s, but instead I was hoping to use the ARC but limit the maximum amount on a per zvol basis.> >> The use case is to prioritize which zvol devices should be fully cached >> in DRAM on a server that cannot fit them all in memory. > > It is not clear to me that this will make sense in a world of snapshots and dedup. > Could you explain your requirements in more detail?I am using zvol''s to hold the metadata for another filesystem (SAM-QFS). In some circumstances I can fit enough of this into the ARC that virtually all metadata reads IOPS happen at DRAM performance rather than SSD or slower. However, with a single server hosting multiple filesystem (hence multiple zvols) I would like to be able to prioritize the use of the ARC. P.S. Since SAM-QFS metadata is highly compressible, O(10x), it would also be great if there was an option to cache the compressed blocks in DRAM (and not just the decompressed version). Thanks. -- Stuart Anderson anderson at ligo.caltech.edu http://www.ligo.caltech.edu/~anderson
On Jan 30, 2011, at 5:01 PM, Stuart Anderson wrote:> On Jan 30, 2011, at 2:29 PM, Richard Elling wrote: > >> On Jan 30, 2011, at 12:21 PM, stuart anderson wrote: >> >>> Is it possible to partition the global setting for the maximum ARC size >>> with finer grained controls? Ideally, I would like to do this on a per >>> zvol basis but a setting per zpool would be interesting as well? >> >> While perhaps not perfect, see the primarycache and secondarycache >> properties of the zvol or file system. > > With primarycache I can turn off utilization of the ARC for some zvol''s, > but instead I was hoping to use the ARC but limit the maximum amount > on a per zvol basis.Just a practical question, do you think the average storage admin will have any clue as to how to use this tunable? How could we be more effective in communicating the features and pitfalls of resource management at this level?>> >>> The use case is to prioritize which zvol devices should be fully cached >>> in DRAM on a server that cannot fit them all in memory. >> >> It is not clear to me that this will make sense in a world of snapshots and dedup. >> Could you explain your requirements in more detail? > > I am using zvol''s to hold the metadata for another filesystem (SAM-QFS). > In some circumstances I can fit enough of this into the ARC that virtually > all metadata reads IOPS happen at DRAM performance rather than SSD > or slower. > > However, with a single server hosting multiple filesystem (hence multiple > zvols) I would like to be able to prioritize the use of the ARC.I think there is merit to this idea. It can be especially useful in the zone context. Please gather your thoughts and file an RFE at www.illumos.org Thanks! -- richard
On Jan 30, 2011, at 6:03 PM, Richard Elling wrote:> On Jan 30, 2011, at 5:01 PM, Stuart Anderson wrote: >> On Jan 30, 2011, at 2:29 PM, Richard Elling wrote: >> >>> On Jan 30, 2011, at 12:21 PM, stuart anderson wrote: >>> >>>> Is it possible to partition the global setting for the maximum ARC size >>>> with finer grained controls? Ideally, I would like to do this on a per >>>> zvol basis but a setting per zpool would be interesting as well? >>> >>> While perhaps not perfect, see the primarycache and secondarycache >>> properties of the zvol or file system. >> >> With primarycache I can turn off utilization of the ARC for some zvol''s, >> but instead I was hoping to use the ARC but limit the maximum amount >> on a per zvol basis. > > Just a practical question, do you think the average storage admin will have > any clue as to how to use this tunable?Yes. I think the basic idea of partitioning a memory cache over different storage objects is a straightforward concept.> How could we be more effective in > communicating the features and pitfalls of resource management at this > level?Document that this is normally handled dynamically based on the default policy that all storage objects should be assigned ARC space on a fair share basis. However, if different quality of service is required for different storage objects this may be adjusted as follows...> >>> >>>> The use case is to prioritize which zvol devices should be fully cached >>>> in DRAM on a server that cannot fit them all in memory. >>> >>> It is not clear to me that this will make sense in a world of snapshots and dedup. >>> Could you explain your requirements in more detail? >> >> I am using zvol''s to hold the metadata for another filesystem (SAM-QFS). >> In some circumstances I can fit enough of this into the ARC that virtually >> all metadata reads IOPS happen at DRAM performance rather than SSD >> or slower. >> >> However, with a single server hosting multiple filesystem (hence multiple >> zvols) I would like to be able to prioritize the use of the ARC. > > I think there is merit to this idea. It can be especially useful in the zone > context. Please gather your thoughts and file an RFE at www.illumos.orgNot sure how to file an illumos RFE, but one simple model to think about would is a 2 tiered system where by default ZFS datasets use the ARC is currently the case, with no (to the best of my knowledge) relative priority, but some objects could optionally specific a request for a minimum size, e.g., add a companion attribute to primarycache named primarycachesize. This would represent the minimum amount of ARC space that is available for that object. Some thought would have to be given as to how to indicate if the sum of all primarycachesize settings is greater than zfs_arc_max, and document what happens in this case, e.g., all values ignored? Presumably something similar could also be considered for secondarycache. Thanks. -- Stuart Anderson anderson at ligo.caltech.edu http://www.ligo.caltech.edu/~anderson