R help - Jan 2011 - "cannot allocate vector of size ..." in RHLE5 PAE kernel

Dear R community,

I'm running R 32 bits in a 64-bits machine (with 16Gb of Ram) using a
PAE kernel, as you can see here:

$ uname -a
Linux mymachine 2.6.18-238.el5PAE #1 SMP Sun Dec 19 14:42:44 EST 2010
i686 i686 i386 GNU/Linux


When I try to create a large matrix ( Q.obs <- matrix(NA, nrow=6940,
ncol=9000) ), I got the following error:

> Error: cannot allocate vector of size 238.3 Mb

However, the amount of free memory in my machine seems to be much
larger than this:

system("free")
\             total       used       free     shared    buffers     cached
Mem:      12466236    6354116    6112120          0      67596    2107556
-/+ buffers/cache:    4178964    8287272
Swap:     12582904          0   12582904


I tried to increase the memory limit available for R by using:

$ R --min-vsize=10M --max-vsize=5000M --min-nsize=500k --max-nsize=5000M


but it didn't work.


Any hint about how can I get R using all the memory available in the machine ?


Thanks in advance,

Mauricio

-- 
==============================Linux user #454569 -- Ubuntu user #17469

>>>>> "MZ" == Mauricio Zambrano <hzambran.newsgroups
at gmail.com>
>>>>>     on Mon, 17 Jan 2011 11:46:44 +0100 writes:
    MZ> Dear R community,
    MZ> I'm running R 32 bits in a 64-bits machine (with 16Gb of Ram)
using a
    MZ> PAE kernel, as you can see here:

    MZ> $ uname -a
    MZ> Linux mymachine 2.6.18-238.el5PAE #1 SMP Sun Dec 19 14:42:44 EST 2010
    MZ> i686 i686 i386 GNU/Linux


    MZ> When I try to create a large matrix ( Q.obs <- matrix(NA,
nrow=6940,
    MZ> ncol=9000) ), I got the following error:


    >> Error: cannot allocate vector of size 238.3 Mb


    MZ> However, the amount of free memory in my machine seems to be much
    MZ> larger than this:

    MZ> system("free")
    MZ> \             total       used       free     shared    buffers    
cached
    MZ> Mem:      12466236    6354116    6112120          0      67596   
2107556
    MZ> -/+ buffers/cache:    4178964    8287272
    MZ> Swap:     12582904          0   12582904


    MZ> I tried to increase the memory limit available for R by using:

    MZ> $ R --min-vsize=10M --max-vsize=5000M --min-nsize=500k
--max-nsize=5000M


    MZ> but it didn't work.


    MZ> Any hint about how can I get R using all the memory available in the
machine ?

Install a 64-bit version of Linux, i.e., ubuntu in your case
and work from there.
I don't think there's a way around that.

Martin

Thank you very much Hugo for your answer.

Yesterday I was out of my office and I couldn't test the advise you
gave me. Today I'll do it.

I have never used C, so I have to ask.

For creating an executable file with your code, I copied and pasted
the text into a text  file, and then tried from the command  line:

$ gcc memorytest.c  -o memorytes.out

and I got the following warning:

memorytest.c: In function ?main?:
memorytest.c:5: warning: return type of ?main? is not ?int?

Is this important ?

After your answer, I will execute this program just before the line
that rises the error message, and I'll let you know today.

Thank you very much Hugo,

Cheers,

-- 
==============================Linux user #454569 -- Ubuntu user #17469
==============================
2011/1/18 Hugo Mildenberger <Hugo.Mildenberger at
web.de>:
> Hello Mauricio,
>
> Today I lost several messages from r-help and also forgot to
> include you in the reply-to address list previously. I'm curious what
the test
> program I already proposed in my answer to r-help reveals. I modified it
> a bit to force the operating system to actually provide the memory. On
> an old 32 bit ?notebook with 512 MB of physical RAM having X running
> in parallel, this test program takes about a minute or so, but ?eventually
> succeeds.
>
> --- snip ---
>
> #include <stdlib.h>
> #include <stdio.h>
> #include <string.h>
>
> void main() {
> ? ? const size_t size = 1000000000LU;
> ? ? fprintf(stderr,"Trying to allocate %lu bytes ... ",(unsigned
long)size);
> ? ? fflush(stderr);
> ? ? void *p = malloc(size);
> ? ? if ( p ) {
> ? ? ? ? ?fprintf(stderr,"success. Now trying to commit memory ...
");
> ? ? ? ? ?fflush(stderr);
> ? ? ? ? ?memset(p,0,size);
> ? ? ? ? ?fprintf(stderr,"succeeded.\n");
> ? ? }else {
> ? ? ? ? ?fprintf(stderr,"allocation of %lu bytes
failed:%m\n",(unsigned long) size);
> ? ? }
> }
>
> --- snip ---
>
> put this into a file named, say, "tmalloc.c" and compile it using
>
> ? ? ?gcc tmalloc.c -o tmalloc
>
>
> Best regards
>
> Hugo Mildenberger
>

> I got the following warning:
>   memorytest.c: In function ?main?:
>   memorytest.c:5: warning: return type of ?main? is not ?int?
> Is this important ?
Hello Mauricio, 

No, your gcc version is unduly puristic here. The traditional return 
type of the main function in "C" should be "int", and if
that is given,
main has to return an integer number explicitly. By convention, 
a main function returning 0 indicates success, and anything other 
is considered as an error or warning condition. Specifying "void
main()"
normally implies returning zero. But if you are still worried simply 
change "void main" into "int main()" and add the statement
" return 0; "
just before the last curly bracket (without quotation marks, of course):

  int main() {       
       [...]  
       return 0; 
}

Best 

Hugo