zfs discuss - Aug 2009 - limiting the ARC cache during early boot, without /etc/system

Hi,

Other than modifying /etc/system, how can I keep the ARC cache low at boot time?

Can I somehow create an SMF service and wire it in at a very low level to put a
fence around ZFS memory usage before other services come up?

I have a deployment scenario where I will have some reasonably large memory
systems (1.7GByte) on Amazon EC2 where the application I''m running
needs a lot of memory, is using large pages and won''t use ZFS in any
significant way.  Therefore, I would like to limit ZFS''s use of memory
on the system.

I''d followed the evil tuning guide to modify /etc/system, however
I''ve just found by corresponding with the EC2 support folks that it is
not supported to modify /etc/system (and it doesn''t work... it keeps
the system from booting).

Thanks in advance,

- Matt
-- 
This message posted from opensolaris.org

Matt,

On Wed, Aug 05, 2009 at 07:06:06PM -0700, Matt Ingenthron
wrote:
> Hi,
> 
> Other than modifying /etc/system, how can I keep the ARC cache low at boot
time?
> 
> Can I somehow create an SMF service and wire it in at a very low level to
put a fence around ZFS memory usage before other services come up?
> 
> I have a deployment scenario where I will have some reasonably large memory
systems (1.7GByte) on Amazon EC2 where the application I''m running
needs a lot of memory, is using large pages and won''t use ZFS in any
significant way.  Therefore, I would like to limit ZFS''s use of memory
on the system.
> 
If ZFS is not beinng used significantly, then ARC should not grow. ARC grows
based on the usage (ie. amount of ZFS files/data accessed). Hence, if you are
sure that the ZFS usage is low, things should be fine.

Hope that helps.

Regards,
Sanjeev
> I''d followed the evil tuning guide to modify /etc/system, however
I''ve just found by corresponding with the EC2 support folks that it is
not supported to modify /etc/system (and it doesn''t work... it keeps
the system from booting).
> 
> Thanks in advance,
> 
> - Matt
> -- 
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
-- 
----------------
Sanjeev Bagewadi
Solaris RPE 
Bangalore, India

> If ZFS is not beinng used significantly, then ARC
> should not grow. ARC grows
> based on the usage (ie. amount of ZFS files/data
> accessed). Hence, if you are
> sure that the ZFS usage is low, things should be
> fine.
I understand that it won''t grow, but I want it to be smaller than the
default.  Like I said, I have a use case where I would like to pre-allocate as
many large pages as possible.  How can I constrain or shrink it before I start
my other applications?

Thanks in advance,

- Matt

p.s.: I just found there may not be any large pages on domUs, so maybe it
doesn''t matter so much
-- 
This message posted from opensolaris.org

On 08/06/09 14:28, Matt Ingenthron wrote:
>> If ZFS is not beinng used significantly, then ARC
>> should not grow. ARC grows
>> based on the usage (ie. amount of ZFS files/data
>> accessed). Hence, if you are
>> sure that the ZFS usage is low, things should be
>> fine.
> 
> I understand that it won''t grow, but I want it to be smaller than
the default.  Like I said, I have a use case where I would like to pre-allocate
as many large pages as possible.  How can I constrain or shrink it before I
start my other applications?
> 
> Thanks in advance,
> 
> - Matt
> 
> p.s.: I just found there may not be any large pages on domUs, so maybe it
doesn''t matter so much
Hi Matt!

Besides the /etc/system, you could also export all the pools, use mdb to 
set the same variable that /etc/system sets, and then import the pools 
again. Don''t know of any other mechanism to limit ZFS''s memory
foot print.

If you don''t do ZFS boot, manually import the pools after the 
application starts, so you get your pages first.

Steffen

> Besides the /etc/system, you could also export all
> the pools, use mdb to 
> set the same variable that /etc/system sets, and then
> import the pools 
> again. Don''t know of any other mechanism to limit
> ZFS''s memory foot print.
> 
> If you don''t do ZFS boot, manually import the pools
> after the 
> application starts, so you get your pages first.
Sounds good... except this is OpenSolaris distro we''re talking about so
I have ZFS root with no other options.  It''ll always have at least the
rpool.

Good thought though!

- Matt
-- 
This message posted from opensolaris.org

Matt,


On Thu, Aug 06, 2009 at 11:28:50AM -0700, Matt Ingenthron
wrote:
> > If ZFS is not beinng used significantly, then ARC
> > should not grow. ARC grows
> > based on the usage (ie. amount of ZFS files/data
> > accessed). Hence, if you are
> > sure that the ZFS usage is low, things should be
> > fine.
> 
> I understand that it won''t grow, but I want it to be smaller than
the default.  Like I said, I have a use case where I would like to pre-allocate
as many large pages as possible.  How can I constrain or shrink it before I
start my other applications?
> 
I can''t think if anything else other than what Steffen recommended.

Have you already tried the configuration and found problems with it ?
Because, looking at the ARC code I see that under memory pressure it should free
up buffers making space for other consumers. So, I am curious to know if this
has not worked in your case.

Thanks and regards,
Sanjeev

-- 
----------------
Sanjeev Bagewadi
Solaris RPE 
Bangalore, India