[R] cbind alternate

David Winsemius
2012-01-06T18:53:59Z

On Jan 6, 2012, at 12:43 PM, Mary Kindall wrote:

> I have two one dimensional list of elements and want to perform
> cbind and
> then write into a file. The number of entries are more than a
> million in
> both lists. R is taking a lot of time performing this operation.
>
> Is there any alternate way to perform cbind?
>
> x > y = table2[1:1000000,5]
>
> z = cbind(x,y)   //hanging the machine

You should have been able to bypass the intermediate steps with just:

z = cbind( table1[1:1000000,1] table2[1:1000000,5])

Whether you will have sufficient contiguous memory for that object at
the moment or even after rm(x), rm(y) is in doubt, but had you not
created the unneeded x and y, you _might_ have succeeded in your
limited environment. (Real answer: Buy more RAM.)

I speculate that you are on Windows and so refer your to the R-Win FAQ
for further reading about memory limits.


>
> write.table(z,'out.txt)

I do not know of a way to bypass the requirement of a named object to
pass to write.table, but testing suggests that you could try:

write( t(cbind( table1[1:1000000,1] table2[1:1000000,5])).
"test.txt", 2)

write()  does not require a named object but is less inquisitive than
write table and will give you a transposed matrix with 5 columns by
default which will really mess up things, so you need to transpose and
specify the number of columns. (And that may not save any space over
creating a "z" object.)

So there is another thread today to which master R programmer Bill
Dunlap has offered this strategy (with minor modifications to your
situation by me):

###
f1 <- function (n, fileName) {
      unlink(fileName)
      system.time({
          fileConn <- file(fileName, "wt")
          on.exit(close(fileConn))
          for (i in seq_len(n)) cat( table1[i, 1], " ",
                                     table2[i, 5],
                                   "\n", file = fileConn)
      })
  }

f1(1000000, 'out.txt')

#------------
--

David Winsemius, MD
West Hartford, CT