(Please reply to me directly as I am not on the ZFS alias)
IHAC running BEA WebLogic on a T2000 with ZFS.
Here''s what he''s telling me.....
- He found himself running out of memory on the T2000 (16GB)
- He rebooted his T2000 and got all his memory back
- He ran his system for a while and then did a vmstat and showed he had
10G available. He copied what should have been ~ 5Gig to disk and
immediately re-ran vmstat and it showed he only had 2Gig memory left!!!
My guess is that ZFS is using the system memory for cache and is not
giving it back? OR maybe ZFS will give it back when asked? Are there
better commands to run to see actual memory available? Is there a way to
cap the amount of memory being used for cache by ZFS? Or better yet is
there a tuning guide I can give to my customer to help him better
understand and run his ZFS environment? (I think there used to be an
evil tuning guide...but I have been unable to find it)
RS
--
_/_/_/ _/ _/ _/ _/ |Ross Schaulis
_/ _/ _/ _/_/ _/ |Senior Systems Engineer
_/_/_/ _/ _/ _/ _/ _/ |State and Local Government
_/ _/ _/ _/ _/_/ |(877) 249-7441 office
_/_/_/ _/_/_/ _/ _/
M I C R O S Y S T E M S |ross.schaulis at sun.com
google "evil tuning guide" and you will find it. you can throw a "zfs" into the query too, or not. zfs will basically use as much ram as it can. see section 2.2, "limiting arc cache" On Mon, May 11, 2009 at 11:16 AM, Ross Schaulis <Ross.Schaulis at sun.com>wrote:> > (Please reply to me directly as I am not on the ZFS alias) > > IHAC running BEA WebLogic on a T2000 with ZFS. > Here''s what he''s telling me..... > > - He found himself running out of memory on the T2000 (16GB) > - He rebooted his T2000 and got all his memory back > - He ran his system for a while and then did a vmstat and showed he had 10G > available. He copied what should have been ~ 5Gig to disk and immediately > re-ran vmstat and it showed he only had 2Gig memory left!!! > > My guess is that ZFS is using the system memory for cache and is not giving > it back? OR maybe ZFS will give it back when asked? Are there better > commands to run to see actual memory available? Is there a way to cap the > amount of memory being used for cache by ZFS? Or better yet is there a > tuning guide I can give to my customer to help him better understand and run > his ZFS environment? (I think there used to be an evil tuning guide...but I > have been unable to find it) > > RS > > -- > _/_/_/ _/ _/ _/ _/ |Ross Schaulis > _/ _/ _/ _/_/ _/ |Senior Systems Engineer > _/_/_/ _/ _/ _/ _/ _/ |State and Local Government > _/ _/ _/ _/ _/_/ |(877) 249-7441 office > _/_/_/ _/_/_/ _/ _/ M I C R O S Y S T E M S | > ross.schaulis at sun.com > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090511/ff89ba34/attachment.html>
Ross Schaulis wrote:> > (Please reply to me directly as I am not on the ZFS alias) > > IHAC running BEA WebLogic on a T2000 with ZFS. > Here''s what he''s telling me..... > > - He found himself running out of memory on the T2000 (16GB)In most similar cases, it is a misunderstanding of how memory is used on a virtual memory OS such as Solaris. It is insufficient to determine a memory shortfall by observing only the amount of free memory reported by vmstat. There are quite a few well-written books, articles, and blogs which will help you determine if your workload suffer from a memory shortfall.> - He rebooted his T2000 and got all his memory back > - He ran his system for a while and then did a vmstat and showed he > had 10G available. He copied what should have been ~ 5Gig to disk and > immediately re-ran vmstat and it showed he only had 2Gig memory left!!!Free memory reported by vmstat should drop to around the lotsfree limit, which by default will be around 512 kBytes (!) Indeed, there is little need to force caches to shrink until the demand reaches that sort of level. For ZFS-specific accounting, the arcstat program will help show how the Adaptive Replacement Cache (ARC) is being adjusted over time. Later versions of OpenSolaris have even more ARC instrumentation available. -- richard> > My guess is that ZFS is using the system memory for cache and is not > giving it back? OR maybe ZFS will give it back when asked? Are there > better commands to run to see actual memory available? Is there a way > to cap the amount of memory being used for cache by ZFS? Or better yet > is there a tuning guide I can give to my customer to help him better > understand and run his ZFS environment? (I think there used to be an > evil tuning guide...but I have been unable to find it) > > RS >