R help - Aug 2011 - parallel rbind

As I am sitting here waiting for some R scripts to run...I was
wondering... is there any way to parallelize rbind in R?

I wait for this call to complete frequently as I deal with large
amounts of data.

do.call("rbind", LIST)

What exactly do you mean by "parallelize"?  What is wrong with the
approach that you are using now?  What is a "large amount of data"?
Can you give some specifics on the problem you are trying to solve and
why your present approach does not appear to be working?  What are
your expectations of a potential solution?

On Sun, Aug 28, 2011 at 8:36 PM, Steven Bauer <steven.bauer at gmail.com>
wrote:
> As I am sitting here waiting for some R scripts to run...I was
> wondering... is there any way to parallelize rbind in R?
>
> I wait for this call to complete frequently as I deal with large
> amounts of data.
>
> do.call("rbind", LIST)
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?

If you know much about what the elements of LIST look like
you can speed things up by not making R figure out what
you already know.  E.g., if you know that LIST consists of
p numeric vectors, all of the same length, n, then the following
might be faster
   matrix(unlist(LIST, use.names=FALSE), nrow=n)
If you are worried about row or column names then you can
add that information to the call to matrix().  (The above
will also work if LIST contains some matrix elements, as long as
they all have n rows.)

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com 
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org] On Behalf Of Steven Bauer
> Sent: Sunday, August 28, 2011 5:36 PM
> To: r-help at r-project.org
> Subject: [R] parallel rbind
> 
> As I am sitting here waiting for some R scripts to run...I was
> wondering... is there any way to parallelize rbind in R?
> 
> I wait for this call to complete frequently as I deal with large
> amounts of data.
> 
> do.call("rbind", LIST)
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Oops, I mixed up rbind and cbind.  If LIST consists
of n numeric vectors, each of length p, try
   matrix(unlilst(LIST, use.names=FALSE), nrow=n, byrow=TRUE)

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com 
> -----Original Message-----
> From: William Dunlap
> Sent: Sunday, August 28, 2011 6:57 PM
> To: 'Steven Bauer'; r-help at r-project.org
> Subject: RE: [R] parallel rbind
> 
> If you know much about what the elements of LIST look like
> you can speed things up by not making R figure out what
> you already know.  E.g., if you know that LIST consists of
> p numeric vectors, all of the same length, n, then the following
> might be faster
>    matrix(unlist(LIST, use.names=FALSE), nrow=n)
> If you are worried about row or column names then you can
> add that information to the call to matrix().  (The above
> will also work if LIST contains some matrix elements, as long as
> they all have n rows.)
> 
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com
> 
> > -----Original Message-----
> > From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org] On Behalf Of Steven Bauer
> > Sent: Sunday, August 28, 2011 5:36 PM
> > To: r-help at r-project.org
> > Subject: [R] parallel rbind
> >
> > As I am sitting here waiting for some R scripts to run...I was
> > wondering... is there any way to parallelize rbind in R?
> >
> > I wait for this call to complete frequently as I deal with large
> > amounts of data.
> >
> > do.call("rbind", LIST)
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.

Steven Bauer <steven.bauer <at> gmail.com> writes:
> 
> As I am sitting here waiting for some R scripts to run...I was
> wondering... is there any way to parallelize rbind in R?
> 
> I wait for this call to complete frequently as I deal with large
> amounts of data.
> 
> do.call("rbind", LIST)
  Perfectly reasonable question, but please don't cross-post
to StackOverflow and R-help -- pick one or the other.

  cheers
    Ben Bolker