R help - Dec 2004 - outer(-x, x, pmin) cannot allocate

R> x <- 0. + 1:8000
R> y <- outer(-x, x, pmin)
Error: cannot allocate vector of size 1000000 Kb

Why does R need to allocate a gigabyte to create an 8000 x 8000 matrix?
It doesn't have any trouble with outer(-x, x, "+").  Thanks.

-- David Brahm (brahm at alum.mit.edu)

Version:
 platform = i686-pc-linux-gnu
 arch = i686
 os = linux-gnu
 system = i686, linux-gnu
 status = 
 major = 2
 minor = 0.1
 year = 2004
 month = 11
 day = 15
 language = R

"Brahm, David" <David.Brahm at geodecapital.com> writes:
> R> x <- 0. + 1:8000
> R> y <- outer(-x, x, pmin)
> Error: cannot allocate vector of size 1000000 Kb
> 
> Why does R need to allocate a gigabyte to create an 8000 x 8000 matrix?
> It doesn't have any trouble with outer(-x, x, "+").  Thanks.
Hmm, it does run the process size to over 5GB on systems that do not
crash. An 8000x8000 matrix of doubles takes about 0.5 GB, and the way
outer() works, it expands both arguments to vectors of the same length
as the result.

So needing a GB is not too surprising, but something does seem to be a
bit memory hungry. Looking inside pmin, it is pretty clear that it
wasn't built for memory efficiency...

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907

David:

In general, this is not a good question to ask, as one needs to go into the
bowels of R to find an answer.

But note that 8000 x 8000 x 4 bytes for double precision  = 256 mb. Now look
at the code of outer(). Two vectors of this size are created = 512mb. Then
copies of these must be created to be passed into pmin, I believe, as R
passes by value. That's 1gb.

My guess is that "+", as an internal function, avoids the final
doubling.

Corrections/clarifications by knowledgeable R experts cheerfully welcomed.
I'm on thin ice here.

-- Bert Gunter
Genentech Non-Clinical Statistics
South San Francisco, CA
 
"The business of the statistician is to catalyze the scientific learning
process."  - George E. P. Box
 
 
> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Brahm, David
> Sent: Wednesday, December 22, 2004 12:11 PM
> To: r-help at stat.math.ethz.ch
> Subject: [R] outer(-x, x, pmin) cannot allocate
> 
> R> x <- 0. + 1:8000
> R> y <- outer(-x, x, pmin)
> Error: cannot allocate vector of size 1000000 Kb
> 
> Why does R need to allocate a gigabyte to create an 8000 x 
> 8000 matrix?
> It doesn't have any trouble with outer(-x, x, "+").  Thanks.
> 
> -- David Brahm (brahm at alum.mit.edu)
> 
> Version:
>  platform = i686-pc-linux-gnu
>  arch = i686
>  os = linux-gnu
>  system = i686, linux-gnu
>  status = 
>  major = 2
>  minor = 0.1
>  year = 2004
>  month = 11
>  day = 15
>  language = R
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
>