R help - Jan 2009 - Write to multiple connections or multiple text files

Hi all,

I want to modify a large number of text files (ca 4000) by replacing a
value found on a particular line in them with a value from an R object.

For a single file I would normally use: 
	con<-file ("foo.txt", open="r+")			
	content<-readLines(con)
	content[n]<-"test"
	writeLines(content,con)
	close(con)

For repeating this for several files I can write a for loop around this.
However, my problem with this is that it is rather slow. I am therefore
wondering if there is any other way to write to multiple connections in
a similar way as one can e.g. write to a large number of rows in a
matrix simultaneously? 

(Note that seek() is not so practical for me since the number of bytes
before the specific line varies between the files, therefore I use
reanLines() and match the right line instead.)

My Systems:
OS: Windows Server 2003 & Linux Red Hat (interchangeably)
R version: 2.7.2

Thanks for any suggestions!

ooo
Jafet Andersson
Eawag - The Swiss Federal Institute of Aquatic Science and Technology
Ueberlandstrasse 133
P.O. Box 611
CH-8600 Duebendorf
Switzerland
Phone: +41 (0)44 823 5358
Fax:     +41 (0)44 823 5028
http://www.eawag.ch/index_EN

The solution you have seems to read in all the lines of data at once,
operate on them and then write them out as a whole chunck.  Having
multiple connections open won't really help since I/O is serial.  So
is the code you included the actual code, or just an example?  It
would help to see what the real code is to see if there is some way of
optimizing it.

On Fri, Jan 23, 2009 at 8:06 AM, Andersson, Jafet
<Jafet.Andersson at eawag.ch> wrote:
> Hi all,
>
> I want to modify a large number of text files (ca 4000) by replacing a
> value found on a particular line in them with a value from an R object.
>
> For a single file I would normally use:
>        con<-file ("foo.txt", open="r+")
>        content<-readLines(con)
>        content[n]<-"test"
>        writeLines(content,con)
>        close(con)
>
> For repeating this for several files I can write a for loop around this.
> However, my problem with this is that it is rather slow. I am therefore
> wondering if there is any other way to write to multiple connections in
> a similar way as one can e.g. write to a large number of rows in a
> matrix simultaneously?
>
> (Note that seek() is not so practical for me since the number of bytes
> before the specific line varies between the files, therefore I use
> reanLines() and match the right line instead.)
>
> My Systems:
> OS: Windows Server 2003 & Linux Red Hat (interchangeably)
> R version: 2.7.2
>
> Thanks for any suggestions!
>
> ooo
> Jafet Andersson
> Eawag - The Swiss Federal Institute of Aquatic Science and Technology
> Ueberlandstrasse 133
> P.O. Box 611
> CH-8600 Duebendorf
> Switzerland
> Phone: +41 (0)44 823 5358
> Fax:     +41 (0)44 823 5028
> http://www.eawag.ch/index_EN
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?

On Fri, 23 Jan 2009, Andersson, Jafet wrote:
> Hi all,
>
> I want to modify a large number of text files (ca 4000) by replacing a
> value found on a particular line in them with a value from an R object.
>
> For a single file I would normally use:
> 	con<-file ("foo.txt", open="r+")
> 	content<-readLines(con)
> 	content[n]<-"test"
> 	writeLines(content,con)
> 	close(con)

If you know 'n' before opening the connection or can easily figure it
out
using the system's grep command (on your linux side), you would probably 
be best off using two calls to the system() function along with your 
system's head, cat, and tail functions to copy the parts you do not alter 
to a temp file. Using pipe() with the system's head and tail commands you 
pick off the line you modify and cat() to append it in between the 
system() call that copies what preceeds it and and the call that copies 
what succeeds it. Once the temp file is complete, you replace the orignal 
with it.

HTH,

Chuck
>
> For repeating this for several files I can write a for loop around this.
> However, my problem with this is that it is rather slow. I am therefore
> wondering if there is any other way to write to multiple connections in
> a similar way as one can e.g. write to a large number of rows in a
> matrix simultaneously?
>
> (Note that seek() is not so practical for me since the number of bytes
> before the specific line varies between the files, therefore I use
> reanLines() and match the right line instead.)
>
> My Systems:
> OS: Windows Server 2003 & Linux Red Hat (interchangeably)
> R version: 2.7.2
>
> Thanks for any suggestions!
>
> ooo
> Jafet Andersson
> Eawag - The Swiss Federal Institute of Aquatic Science and Technology
> Ueberlandstrasse 133
> P.O. Box 611
> CH-8600 Duebendorf
> Switzerland
> Phone: +41 (0)44 823 5358
> Fax:     +41 (0)44 823 5028
> http://www.eawag.ch/index_EN
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Charles C. Berry                            (858) 534-2098
                                             Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu	            UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901