R help - Jul 2011 - memory problem; Error: cannot allocate vector of size 915.5 Mb

Dear all,

I am trying to make some matrix operations (whose size I think is smaller
than what R allows) but the operations are not feasible when they run in one
session but it is feasible if they run separately while each operation is
totally independent of the other. I run the code in one session the error
that appears is:

Error: cannot allocate vector of size 915.5 Mb
R(16467,0xa0421540) malloc: *** mmap(size=960004096) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
R(16467,0xa0421540) malloc: *** mmap(size=960004096) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

In the code that I run (next lines), if I do not include the last three
lines it runs perfectly, if I exclude operations to create the xMax again it
runs perfectly, if I include both G.search and xMax appears the error term.
Does anyone knows the solution of this problem or why this problem happens?

The code that I run is:

N<-250
x<-matrix(c(rnorm(N,-1.5,1), rnorm(N,1,1), rbinom(N,1,0.5)), ncol=3)
start<-(-1)
end<-3
step<-10^(-2)
n.steps<-(end-start)/step	
steps2  <-n.steps^2
grids<-seq(from=start+step, to=end, by=step)
xMax    <-matrix(0,N*steps2,3)
xMax[,1]<-rep(x[,1],steps2)	
xMax[,2]<-rep(x[,2],steps2)
xMax[,3]<-rep(x[,3],steps2)
G.search1<-as.matrix(rep(grids, n.steps, each=N))
G.search2<-as.matrix(rep(grids, N, each=n.steps))
G.search<-cbind(1,G.search1, G.search2)
 
Thank you

Dimitris


--
View this message in context:
http://r.789695.n4.nabble.com/memory-problem-Error-cannot-allocate-vector-of-size-915-5-Mb-tp3707943p3707943.html
Sent from the R help mailing list archive at Nabble.com.

My advice to you is to get a 64-bit version of R.  Here is what it
does on my 64-bit Windows 7 version:
> N<-250
> x<-matrix(c(rnorm(N,-1.5,1), rnorm(N,1,1), rbinom(N,1,0.5)), ncol=3)
> my.stats(1)
1 (1) - Rgui : 22:30:20 <0.7 78.6> 78.6 : 20.5MB
> start<-(-1)
> end<-3
> step<-10^(-2)
> n.steps<-(end-start)/step
> steps2  <-n.steps^2
> grids<-seq(from=start+step, to=end, by=step)
> xMax    <-matrix(0,N*steps2,3)
> my.stats(2)
2 (1) - Rgui : 22:30:23 <4.1 82.1> 82.1 : 935.5MB
> xMax[,1]<-rep(x[,1],steps2)
> xMax[,2]<-rep(x[,2],steps2)
> xMax[,3]<-rep(x[,3],steps2)
> my.stats(3)
3 (1) - Rgui : 22:30:35 <16.0 94.3> 94.3 :
1998.9MB
> G.search1<-as.matrix(rep(grids, n.steps, each=N))
> G.search2<-as.matrix(rep(grids, N, each=n.steps))
> G.search<-cbind(1,G.search1, G.search2)
> my.stats(3)
3 (1) - Rgui : 22:30:45 <25.2 103.7> 103.7 :
2456.6MB
>
> gc()
            used   (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells    143726    7.7     350000   18.7    350000   18.7
Vcells 320137296 2442.5  353288723 2695.4 320138039
2442.5
> my.ls()
                      Size        Mode
.my.env                 56 environment
.Random.seed         2,544     numeric
end                     48     numeric
G.search       960,000,200     numeric
G.search1      320,000,200     numeric
G.search2      320,000,200     numeric
grids                3,240     numeric
N                       48     numeric
n.steps                 48     numeric
start                   48     numeric
step                    48     numeric
steps2                  48     numeric
x                    6,200   character
xMax           960,000,200     numeric
**Total      2,560,013,128     -------
>
You have objects totaling 2.5GB which is probably larger than can be
handled on a 32-bit version, especially when copies have to be made.

On Sun, Jul 31, 2011 at 11:53 AM, Dimitris.Kapetanakis
<dimitrios.kapetanakis at gmail.com> wrote:
> Dear all,
>
> I am trying to make some matrix operations (whose size I think is smaller
> than what R allows) but the operations are not feasible when they run in
one
> session but it is feasible if they run separately while each operation is
> totally independent of the other. I run the code in one session the error
> that appears is:
>
> Error: cannot allocate vector of size 915.5 Mb
> R(16467,0xa0421540) malloc: *** mmap(size=960004096) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
> R(16467,0xa0421540) malloc: *** mmap(size=960004096) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
>
> In the code that I run (next lines), if I do not include the last three
> lines it runs perfectly, if I exclude operations to create the xMax again
it
> runs perfectly, if I include both G.search and xMax appears the error term.
> Does anyone knows the solution of this problem or why this problem happens?
>
> The code that I run is:
>
> N<-250
> x<-matrix(c(rnorm(N,-1.5,1), rnorm(N,1,1), rbinom(N,1,0.5)), ncol=3)
> start<-(-1)
> end<-3
> step<-10^(-2)
> n.steps<-(end-start)/step
> steps2 ?<-n.steps^2
> grids<-seq(from=start+step, to=end, by=step)
> xMax ? ?<-matrix(0,N*steps2,3)
> xMax[,1]<-rep(x[,1],steps2)
> xMax[,2]<-rep(x[,2],steps2)
> xMax[,3]<-rep(x[,3],steps2)
> G.search1<-as.matrix(rep(grids, n.steps, each=N))
> G.search2<-as.matrix(rep(grids, N, each=n.steps))
> G.search<-cbind(1,G.search1, G.search2)
>
> Thank you
>
> Dimitris
>
>
> --
> View this message in context:
http://r.789695.n4.nabble.com/memory-problem-Error-cannot-allocate-vector-of-size-915-5-Mb-tp3707943p3707943.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?

Thanks a lot for the help. 

Actually, I am using a mac which (R for Mac OS X GUI 1.40-devel Leopard
build 32-bit (5751)) but I think I can find access on windows 7 64-bit. What
I am trying to do is a maximization through grid search (because I am not
sure that any of the optim() methods works sufficiently to my case, at least
all of them provide quite different results), the reason that I want the
optimizing is because I want to use it for a Monte Carlo analysis for
Smoothed Maximum Score estimator, and for that reason I want the
optimization to be the most efficient possible, but given that I am kind of
amateur on R and on programming in general, I doubt that I can do that
sufficiently.

Thanks again for your help

Dimitris

--
View this message in context:
http://r.789695.n4.nabble.com/memory-problem-Error-cannot-allocate-vector-of-size-915-5-Mb-tp3707943p3709002.html
Sent from the R help mailing list archive at Nabble.com.