R help - Sep 2012 - Question about R performance on UNIX/LINUX with different memory/swap configurations

Does anyone have any guidance on swap and memory configuration when
running R v2.15.1 on UNIX/LINUX?  Through some benchmarking across
multiple hardware (UNIX, LINUX, SPARC, x86, Windows, physical, virtual)
it "seems" that the smaller memory machines have an advantage.

Typically my organization builds their UNIX servers at a 1:1 physical
memory:swap configuration.  We plan on running some tests where we set
have swap at 1:1, 0:1 and 1/2:1 to see if there is any benefit and to
what degree.

My first assumption is that it would depend on exactly what I am doing
in R and that fact would need to be taken into account to the
observations and testing I am doing.  To be clear, I've not approached
writing any parallized code other than what R might do out of the box.
However, what testing I have done (using a standard deviation test as
well as a GBM model) seems to indicate that the Windows desktop (with
small/slow swap footprint) as well as a Solaris 11 X86 server with swap
set to half of physical memory seems to perform quicker for these
scenarios than an physical server with 16CPU's and 48GB memory.

I found a few articles searching the group, but they seem to factor
around Windows performance considerations (for example, post entitled
"Re: [R] Memory limit for Windows 64bit build of R" from a few months
ago.)

I do plan on running through these different configurations on my own to
test out R, but wondered if the community had any experience with swap &
memory configuration when running R on UNIX/LINUX configurations.


Thanks in advance.

Anthony


	[[alternative HTML version deleted]]

On 16 September 2012 at 13:30, Eberle, Anthony wrote:
| Does anyone have any guidance on swap and memory configuration when
| running R v2.15.1 on UNIX/LINUX?  Through some benchmarking across
| multiple hardware (UNIX, LINUX, SPARC, x86, Windows, physical, virtual)
| it "seems" that the smaller memory machines have an advantage.

Would you be able to provide some "empirical proofs" for that
conjecture?
Please demonstrate how, say, a X gb ram machine outperforms one with Y gb
where Y > X.

Less memory is /never/ better.  

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com

My first criteria is to make sure my application never swaps/pages due
to memory issues -- have enough physical memory so it never happens
and control what else is running on the machine.  Once you start
paging, performance takes a real hit.

On Sun, Sep 16, 2012 at 2:30 PM, Eberle, Anthony <aeber at allstate.com>
wrote:
> Does anyone have any guidance on swap and memory configuration when
> running R v2.15.1 on UNIX/LINUX?  Through some benchmarking across
> multiple hardware (UNIX, LINUX, SPARC, x86, Windows, physical, virtual)
> it "seems" that the smaller memory machines have an advantage.
>
> Typically my organization builds their UNIX servers at a 1:1 physical
> memory:swap configuration.  We plan on running some tests where we set
> have swap at 1:1, 0:1 and 1/2:1 to see if there is any benefit and to
> what degree.
>
> My first assumption is that it would depend on exactly what I am doing
> in R and that fact would need to be taken into account to the
> observations and testing I am doing.  To be clear, I've not approached
> writing any parallized code other than what R might do out of the box.
> However, what testing I have done (using a standard deviation test as
> well as a GBM model) seems to indicate that the Windows desktop (with
> small/slow swap footprint) as well as a Solaris 11 X86 server with swap
> set to half of physical memory seems to perform quicker for these
> scenarios than an physical server with 16CPU's and 48GB memory.
>
> I found a few articles searching the group, but they seem to factor
> around Windows performance considerations (for example, post entitled
> "Re: [R] Memory limit for Windows 64bit build of R" from a few
months
> ago.)
>
> I do plan on running through these different configurations on my own to
> test out R, but wondered if the community had any experience with swap
&
> memory configuration when running R on UNIX/LINUX configurations.
>
>
> Thanks in advance.
>
> Anthony
>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.