R help - Sep 2011 - write.matrix row names vs sink vs capture.output

Dear R gurus,

I am trying to write several large matrices (~ 1GB) to separate files. I have
learned that write.table is simply too slow for this task and was attempting to
use write.matrix, but write.matrix does not have the ability to include row
names in the output. Anyone know why that's the case? I've seen a thread
stating that write.matrix is the way to go for large prints to files, but it
doesn't do what I need it to. Since write.matrix wasn't working I tried
both sink and capture.output, but then the output is printed to the file using
the same 'width' restrictions as the general "options(width=)"
limit.

Any ideas on how to print a large matrix with row names? I could write a perl
script to modify the files after the fact, but I shouldn't have to do that.

Thanks for your help!

Mark T. W. Ebbert

On 09/06/2011 06:24 AM, Mark Ebbert wrote:
> Dear R gurus,
>
> I am trying to write several large matrices (~ 1GB) to separate files. I
have learned that write.table is simply too slow for this task and was
attempting to use write.matrix, but write.matrix does not have the ability to
include row names in the output. Anyone know why that's the case? I've
seen a thread stating that write.matrix is the way to go for large prints to
files, but it doesn't do what I need it to. Since write.matrix wasn't
working I tried both sink and capture.output, but then the output is printed to
the file using the same 'width' restrictions as the general
"options(width=)" limit.
>
> Any ideas on how to print a large matrix with row names? I could write a
perl script to modify the files after the fact, but I shouldn't have to do
that.
>
> Thanks for your help!
>
> Mark T. W. Ebbert
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
Hi,

What do you want with the data? If you want to store an R matrix on disk
for later use in R, take a look at ?save. If it is for use in another
programming language, I would write the matrix in binary format
(?writebin). This saves a lot of space and prevents any (significant)
rounding errors. It is probably also quite a bit faster. If you really
need some more metadata (such as rownames), I would add a second text
file which stores this information. Sort of a binary file plus a header,
which is a quite common format for storing data. Maybe you can even find
a standard binary format which you can use. But it is impossible to
comment on this because you did not provide information as to what you
want to do with the saved data.

good luck!
Paul

-- 
Paul Hiemstra, Ph.D.
Global Climate Division
Royal Netherlands Meteorological Institute (KNMI)
Wilhelminalaan 10 | 3732 GK | De Bilt | Kamer B 3.39
P.O. Box 201 | 3730 AE | De Bilt
tel: +31 30 2206 494

http://intamap.geo.uu.nl/~paul
http://nl.linkedin.com/pub/paul-hiemstra/20/30b/770