R devel - Sep 2017 - R Configuration Variable: Maximum Memory Allocation per R Instance

Dear R Developers,

In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have the
innodb_buffer_pool_size or the key_buffer_size for setting the maximum
amount of RAM which can be used by a Server Instance:

?Would it be possible to create an R Configuration Variable which fixes the
maximum amount of RAM memory to be used as Commit / Dynamic Memory
Allocation?

Thank you.
Juan

	[[alternative HTML version deleted]]

Dear Juan,

I'm not deeply familiar with the DB's you mention but it seems to me
that
me that 'memory.limits' does what you want on one OS and you can use
shell
commands to limit R's memory usage for *nix-alike systems (see
?memory.limits). Also, Jeroen Ooms wrote a nice article about this in the
JSS: https://www.jstatsoft.org/article/view/v055i07 . There's also a
package for it: RAppArmor.

-M




Op zo 17 sep. 2017 om 00:39 schreef Juan Telleria <jtelleriar at
gmail.com>:
> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have
the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance:
>
> ?Would it be possible to create an R Configuration Variable which fixes the
> maximum amount of RAM memory to be used as Commit / Dynamic Memory
> Allocation?
>
> Thank you.
> Juan
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
	[[alternative HTML version deleted]]

This variables already exist as I have been said, and are:
* memory.size
* memory.limit

R Documentation:

https://stat.ethz.ch/R-manual/R-devel/library/utils/html/memory.size.html

Thank you,
Juan

El 17/9/2017 12:39 a. m., "Juan Telleria" <jtelleriar at
gmail.com> escribi?:
> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have
the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance:
>
> ?Would it be possible to create an R Configuration Variable which fixes
> the maximum amount of RAM memory to be used as Commit / Dynamic Memory
> Allocation?
>
> Thank you.
> Juan
>
	[[alternative HTML version deleted]]

On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria <jtelleriar at gmail.com>
wrote:
> Dear R Developers,
>
> In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria have
the
> innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> amount of RAM which can be used by a Server Instance.
Memory is not controlled by R itself because packages may malloc()
directly. However most operating systems have features to limit
resources of a given process. The CRAN package 'unix' has wrappers for
posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This
works pretty well, however I found that the way memory is managed and
counted varies a lot per OS and malloc implementation.

You can also set rlimits on a single evaluation via the rlimit
parameter in sys::eval_safe().


[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/getrlimit.html

Very very interesting, if it is ok with it, I will post these observations
to Stack Overflow so that they are useful to other R Programmers, and make
more research on the topic, putting it all together. I could even do a
small article with my research.

I think this is a critical point if you want to have an R Instance and a
RDBMS in the same Server.

Thank you,
Juan

2017-09-17 14:14 GMT+02:00 Jeroen Ooms <jeroenooms at gmail.com>:
> On Sun, Sep 17, 2017 at 12:39 AM, Juan Telleria <jtelleriar at
gmail.com>
> wrote:
> > Dear R Developers,
> >
> > In the same way that MySQL/MariaDB's Engine InnoDB or MyISAM/Aria
have
> the
> > innodb_buffer_pool_size or the key_buffer_size for setting the maximum
> > amount of RAM which can be used by a Server Instance.
>
> Memory is not controlled by R itself because packages may malloc()
> directly. However most operating systems have features to limit
> resources of a given process. The CRAN package 'unix' has wrappers
for
> posix setrlimit [1] e.g. unix::rlimit_as() limits address space. This
> works pretty well, however I found that the way memory is managed and
> counted varies a lot per OS and malloc implementation.
>
> You can also set rlimits on a single evaluation via the rlimit
> parameter in sys::eval_safe().
>
>
> [1] http://pubs.opengroup.org/onlinepubs/009695399/
> functions/getrlimit.html
>
	[[alternative HTML version deleted]]