R help - Apr 2013 - Merging big data.frame

Hi all,

I am trying to merge 2 big data.frame. The problem is merge is memory
intensive so R is going out of memory error: cannot allocate vector of size
360.1 Mb. To overcome this, I am exploring option of using data.table
package. But its not helping in term of memory as merge in data.table is
fast but not memory efficient. Similar error is coming.
My inputs are
inp1
 V1 V2
1  a i1
2  a i2
3  a i3
4  a i4
5  b i5
6  c i6

inp2
  V1 V2
1  a  x
2  b  x
3  a  y
4  c  z

I want  merge(x=inp1, y=inp2, by.x="V1", by.y="V1")
so the output

 V1 V2.x V2.y
1   a   i1    x
2   a   i1    y
3   a   i2    x
4   a   i2    y
5   a   i3    x
6   a   i3    y
7   a   i4    x
8   a   i4    y
9   b   i5    x
10  c   i6    z

Is there a way to do this without using merge in data.table? or Is there
any other solution to do this in more efficient and less memory ?

thanks
avi

	[[alternative HTML version deleted]]

check out the sqldf package
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live
Go...
                                      Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
--------------------------------------------------------------------------- 
Sent from my phone. Please excuse my brevity.

avinash sahu <avinash.sahu at gmail.com> wrote:
>Hi all,
>
>I am trying to merge 2 big data.frame. The problem is merge is memory
>intensive so R is going out of memory error: cannot allocate vector of
>size
>360.1 Mb. To overcome this, I am exploring option of using data.table
>package. But its not helping in term of memory as merge in data.table
>is
>fast but not memory efficient. Similar error is coming.
>My inputs are
>inp1
> V1 V2
>1  a i1
>2  a i2
>3  a i3
>4  a i4
>5  b i5
>6  c i6
>
>inp2
>  V1 V2
>1  a  x
>2  b  x
>3  a  y
>4  c  z
>
>I want  merge(x=inp1, y=inp2, by.x="V1", by.y="V1")
>so the output
>
> V1 V2.x V2.y
>1   a   i1    x
>2   a   i1    y
>3   a   i2    x
>4   a   i2    y
>5   a   i3    x
>6   a   i3    y
>7   a   i4    x
>8   a   i4    y
>9   b   i5    x
>10  c   i6    z
>
>Is there a way to do this without using merge in data.table? or Is
>there
>any other solution to do this in more efficient and less memory ?
>
>thanks
>avi
>
>	[[alternative HTML version deleted]]
>
>______________________________________________
>R-help at r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.