R help - Nov 2003 - 64-bit R on Opteron [was Re: Windows R 1.8.0 hangs when M em Usage >1.8GB]

> From: Douglas Bates [mailto:bates at bates4.stat.wisc.edu]
> 
> "Liaw, Andy" <andy_liaw at merck.com> writes:
> 
> > Sorry.  I need to retract my claim.  There seems to be a 3G 
> limit, even
> > though the OS could handle nearly 8G.  (I can have two 
> simultaneous R
> > processes each using near 3G.)
> > 
> > On another note, on our dual Opteron box R (compiled as 
> 64-bit) could easily
> > use nearly all the 16G in that box (that's one of the 
> reason for having that
> > box).
> 
> Does "could" mean you have verified that it did or is this a
> theoretical statement?  I.e., have you compiled and tested R on your
> dual Opteron?
Given my questionable memory of things, this question is very fair.
Here's the evidence:
> x <- matrix(0, 5e5, 5e5)
> x2 <- matrix(0, 5e5, 5e5)
> gc()
             used    (Mb) gc trigger    (Mb)
Ncells     413379    22.1     741108    39.6
Vcells 1783900405 13610.1 1784288128 13613.1

Best,
Andy

(Reply moved to R-devel where it may be more appropriate.)

"Liaw, Andy" <andy_liaw@merck.com> writes:
> > From: Douglas Bates [mailto:bates@bates4.stat.wisc.edu]
> > 
> > "Liaw, Andy" <andy_liaw@merck.com> writes:
> > 
> > > Sorry.  I need to retract my claim.  There seems to be a 3G 
> > > limit, even though the OS could handle nearly 8G.  (I can have
> > > two simultaneous R processes each using near 3G.)
> > > 
> > > On another note, on our dual Opteron box R (compiled as 64-bit)
> > > could easily use nearly all the 16G in that box (that's one
of
> > > the reason for having that box).
> > 
> > Does "could" mean you have verified that it did or is this a
> > theoretical statement?  I.e., have you compiled and tested R on your
> > dual Opteron?
> 
> Given my questionable memory of things, this question is very fair.
> Here's the evidence:
> 
> > x <- matrix(0, 5e5, 5e5)
> > x2 <- matrix(0, 5e5, 5e5)
> > gc()
>              used    (Mb) gc trigger    (Mb)
> Ncells     413379    22.1     741108    39.6
> Vcells 1783900405 13610.1 1784288128 13613.1
Amazing!

I wonder if R&R thought when they started out that they would one day
see something like that.

How does the Opteron perform on floating point?  Can you try something
like
> mm = matrix(rnorm(1e6), nc = 1e3)
> system.time(crossprod(mm))
[1] 0.51 0.02 0.53 0.00 0.00
> system.time(crossprod(mm))
[1] 0.37 0.03 0.40 0.00 0.00
> system.time(crossprod(mm))
[1] 0.38 0.02 0.40 0.00 0.00
> system.time(crossprod(mm))
[1] 0.38 0.02 0.40 0.00 0.00

(That was with R compiled to use Goto's BLAS on a 2.0 GHz P4.)

Are you using Goto's BLAS or Atlas?