R help - Dec 2010 - [Fwd: adding more columns in big.matrix object of bigmemory package]

Hi,
   With reference to the mail below, I have large datasets, coming from various
   different sources, which I can read into filebacked big.matrix using library
   bigmemory. I want to merge them all into one 'big.matrix' object.
(Later, I
   want to run regression using library 'biglm').
   I am unsuccessfully trying to do this from quite some time now. Can you
   please suggest some way? Am I missing some already available function?
   Even a functionality of the following will work for me:
     * Just  appending more columns in an existing big.matrix object (not
       merging).
     * If the individual datasets are small enough to be read in usual R, just
       the combined dataset is huge.

   Any thoughts are welcome.
   Thanks,
   Utkarsh
   -------- Original Message --------

   Subject: adding more columns in big.matrix object of bigmemory package
      Date: Thu, 16 Dec 2010 18:29:38 +0530
      From: utkarshsinghal [1]<utkarsh.singhal at global-analytics.com>
        To: r help [2]<r-help at stat.math.ethz.ch>

   Hi all,
   Is there any way I can add more columns to an existing filebacked big.matrix
   object.
   In general, I want a way to modify an existing big.matrix object, i.e., add
   rows/columns, rename colnames, etc.
   I tried the following:
   > library(bigmemory)
   > x   
read.big.matrix("test.csv",header=T,type="double",shared=T,backingfile="test
   .backup",descriptorfile="test.desc")
   > x[,"v4"] = "new"
   Error in mmap(j, colnames(x)) :
     Couldn't find a match to one of the arguments.
   (The above functionality is presently there in usual data.frames in R.)
   Thanks in advance,
   Utkarsh

References

   1. mailto:utkarsh.singhal at global-analytics.com
   2. mailto:r-help at stat.math.ethz.ch

For good reasons (having to do with avoiding copies of massive things)
we leave such merging to the user: create a new filebacking of the
proper size, and fill it (likely a column at a time, assuming you have
enough RAM to support that).

Jay

On Fri, Dec 17, 2010 at 2:16 AM, utkarshsinghal
<utkarsh.singhal at global-analytics.com> wrote:
>
> Hi,
>
> With reference to the mail below, I have large datasets, coming from
various different sources, which I can read into filebacked big.matrix using
library bigmemory. I want to merge them all into one 'big.matrix'
object. (Later, I want to run regression using library 'biglm').
>
> I am unsuccessfully trying to do this from quite some time now. Can you
please suggest some way? Am I missing some already available function?
>
> Even a functionality of the following will work for me:
>
> Just appending more columns in an existing big.matrix object (not merging).
> If the individual datasets are small enough to be read in usual R, just the
combined dataset is huge.
>
> Any thoughts are welcome.
>
> Thanks,
> Utkarsh
>
>
> -------- Original Message --------
> Subject: adding more columns in big.matrix object of bigmemory package
> Date: Thu, 16 Dec 2010 18:29:38 +0530
> From: utkarshsinghal <utkarsh.singhal at global-analytics.com>
> To: r help <r-help at stat.math.ethz.ch>
>
> Hi all,
>
> Is there any way I can add more columns to an existing filebacked
big.matrix object.
>
> In general, I want a way to modify an existing big.matrix object, i.e., add
rows/columns, rename colnames, etc.
> I tried the following:
>
> > library(bigmemory)
> > x =
read.big.matrix("test.csv",header=T,type="double",shared=T,backingfile="test.backup",descriptorfile="test.desc")
> > x[,"v4"] = "new"
> Error in mmap(j, colnames(x)) :
> ? Couldn't find a match to one of the arguments.
> (The above functionality is presently there in usual data.frames in R.)
>
>
> Thanks in advance,
> Utkarsh
>


--
John W. Emerson (Jay)
Associate Professor of Statistics
Department of Statistics
Yale University
http://www.stat.yale.edu/~jay