R help - Feb 2011 - Boundaries of R

Hello Everyone,

I'm pretty new to R and I'm trying to get some idea of the capabilities
of
the language. I work with some pretty large data sets and the consensus
seems to be that R does not work well with big data. I've started talking to
the guys at Revolution, but I need to get some outside opinions of what R
can actually handle. At about what size does R start to run into problems?

-- 
*Michael Holt*, *Database Administrator*  |  A.E.R.S. & TERAPEAK
2307-4464 Markham Street, Victoria, BC, CANADA V8Z 7X8
 +1 250 483 3271 (FAX)
 www.terapeak.com | www.aers.ca | [image:
facebook]<http://www.facebook.com/pages/Terapeak/21544372056>| [image:
twitter] <http://twitter.com/terapeak>

	[[alternative HTML version deleted]]

On 18/02/2011 5:44 AM, Michael Holt wrote:
> Hello Everyone,
>
> I'm pretty new to R and I'm trying to get some idea of the
capabilities of
> the language. I work with some pretty large data sets and the consensus
> seems to be that R does not work well with big data. I've started
talking to
> the guys at Revolution, but I need to get some outside opinions of what R
> can actually handle. At about what size does R start to run into problems?
>
Vectors are limited to about 2 billion entries (2^31 - 1).  Matrices are 
vectors, so that limit applies to the total count of entries.  
Dataframes are lists of vectors, so that limit applies separately to the 
numbers of rows and columns.

Simple R code keeps everything in memory, so you're likely to run into 
hardware limits if you start working with really big vectors.  There are 
a number of packages that alleviate that by paging data in and out, but 
it takes a bit of work on your part to use them.  As far as I know, 
Revolution offers nothing in this area that isn't on CRAN, but they can 
certainly give you advice.

Duncan Murdoch