We have a 32 GB RAM server running about 14 zones. There are multiple databases, application servers, web servers, and ftp servers running in the various zones. I understand that using ZFS will increase kernel memory usage, however I am a bit concerned at this point. root at servername:~/zonecfg #mdb -k Loading modules: [ unix krtld genunix specfs dtrace uppc pcplusmp ufs md mpt ip indmux ptm nfs ] ::memstat Page Summary Pages MB %Tot ---- Kernel 4108442 16048 49% Anon 3769634 14725 45% Exec and libs 9098 35 0% Page cache 29612 115 0% Free (cachelist) 99437 388 1% Free (freelist) 369040 1441 4% Total 8385263 32754 Physical 8176401 31939 Out of 32GB of RAM, 16GB is being used by the kernel. Is there a way to find out how much of that kernel memory is due to ZFS? It just seems an excessively high amount of our memory is going to the kernel, even with ZFS being used on the server. This message posted from opensolaris.org
Matt Cohen wrote:> We have a 32 GB RAM server running about 14 zones. There are multiple databases, application servers, web servers, and ftp servers running in the various zones. > > I understand that using ZFS will increase kernel memory usage, however I am a bit concerned at this point. > > root at servername:~/zonecfg #mdb -k > > Loading modules: [ unix krtld genunix specfs dtrace uppc pcplusmp ufs md mpt ip indmux ptm nfs ] > > ::memstat > Page Summary Pages MB %Tot > ---- > Kernel 4108442 16048 49% > Anon 3769634 14725 45% > Exec and libs 9098 35 0% > Page cache 29612 115 0% > Free (cachelist) 99437 388 1% > Free (freelist) 369040 1441 4% > > Total 8385263 32754 > Physical 8176401 31939 > > Out of 32GB of RAM, 16GB is being used by the kernel. Is there a way to find out how much of that kernel memory is due to ZFS? >The size of the ARC (cache) is available from kstat in the zfs module (kstat -m zfs). Neel wrote a nifty tool to track it over time called arcstat. See http://www.solarisinternals.com/wiki/index.php/Arcstat Remember that this is a cache and subject to eviction when memory pressure grows. The Solaris Internals books have more details on how the Solaris virtual memory system works and is recommended reading. -- richard> It just seems an excessively high amount of our memory is going to the kernel, even with ZFS being used on the server. > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Richard Elling wrote:> > The size of the ARC (cache) is available from kstat in the zfs > module (kstat -m zfs). Neel wrote a nifty tool to track it over > time called arcstat. See > http://www.solarisinternals.com/wiki/index.php/Arcstat > > Remember that this is a cache and subject to eviction when > memory pressure grows. The Solaris Internals books have > more details on how the Solaris virtual memory system works > and is recommended reading. > -- richard > >The arcsize is also displayed in sysstat, which additionally shows a lot more information in a ''top'' like fashion. Get it here: http://www.maier-komor.de/sysstat.html - Thomas
root at servername:~ #mdb -k
Loading modules: [ unix krtld genunix specfs dtrace uppc pcplusmp ufs md mpt ip
hook neti sctp arp usba uhci fcp fctl qlc nca lofs zfs random fcip crypto
logindmux ptm nfs ]
::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 4314678 16854 51%
Anon 3538066 13820 42%
Exec and libs 9249 36 0%
Page cache 29347 114 0%
Free (cachelist) 89647 350 1%
Free (freelist) 404276 1579 5%
Total 8385263 32754
Physical 8176401 31939
:quit
root at servername:~ #kstat -m zfs
module: zfs instance: 0
name: arcstats class: misc
c 12451650535
c_max 33272295424
c_min 1073313664
crtime 175.759605187
deleted 26773228
demand_data_hits 89284658
demand_data_misses 1995438
demand_metadata_hits 1139759543
demand_metadata_misses 5671445
evict_skip 5105167
hash_chain_max 15
hash_chains 296214
hash_collisions 75773190
hash_elements 995458
hash_elements_max 1576353
hits 1552496231
mfu_ghost_hits 4321964
mfu_hits 1263340670
misses 11984648
mru_ghost_hits 474500
mru_hits 57043004
mutex_miss 106728
p 9304845931
prefetch_data_hits 10792085
prefetch_data_misses 3571943
prefetch_metadata_hits 312659945
prefetch_metadata_misses 745822
recycle_miss 2775287
size 12451397120
snaptime 2410363.20494097
So it looks like our kernel is using 16GB and ZFS is using ~12GB for
it''s arc cache. Is a 4GB kernel for other stuff normal? It still
seems like a lot of memory to me, but I don''t know how all the zones
affect the amount of memory the kernel needs.
This message posted from opensolaris.org