R help - Aug 2010 - Making a series of similar, but modified .r files - suggested method(s)?

Dear all:

Any suggestions are much appreciated. I am looking for a way to make a
series of similar, but slightly modified, .r files.

My issue is automating making 320 .r files that change the for(i in 1:x) in
my base .r file (as well as other elements, e.g., the load(...),
setwd(...)). For smaller jobs running on a single computer with batch files,
I have been manually changing the for(i in 1:x) line, etc..

Why does this matter to me? I am planning on running a simulation experiment
on a linux cluster as a serial job. Although not elegant, it has been
suggested I make 320 .r files so qsub runs one .r file and then selects
other jobs. Thus, the manual route I am currently using would take a very
long time (given multiple runs of 320 .r files, given experimental
replication).

Thank you,
Laura


-- 
" Genius is the summed production of the many with the names of the few
attached for easy recall, unfairly so to other scientists"

- E. O. Wilson (The Diversity of Life)

	[[alternative HTML version deleted]]

On Sat, Aug 21, 2010 at 6:48 PM, Laura S <leslaura at gmail.com>
wrote:
> Dear all:
>
> Any suggestions are much appreciated. I am looking for a way to make a
> series of similar, but slightly modified, .r files.
>
> My issue is automating making 320 .r files that change the for(i in 1:x) in
> my base .r file (as well as other elements, e.g., the load(...),
> setwd(...)). For smaller jobs running on a single computer with batch
files,
> I have been manually changing the for(i in 1:x) line, etc..
>
> Why does this matter to me? I am planning on running a simulation
experiment
> on a linux cluster as a serial job. Although not elegant, it has been
> suggested I make 320 .r files so qsub runs one .r file and then selects
> other jobs. Thus, the manual route I am currently using would take a very
> long time (given multiple runs of 320 .r files, given experimental
> replication).
 qsub? Are you using the Sun Grid Engine or some other queue
submission system? It should be possible to pass a parameter that gets
through to your R process. I wrote some docs on something like that,
geared for our local HPC which uses SGE:

http://www.maths.lancs.ac.uk/~rowlings/HPC/RJobs/

 The crux of which is to get the TASK_ID variable from the environment
and use that to do slightly different things in a batched submission.
You get a TASK_ID if you submit the job as a task array.

 If you really do have to make 320 .R files, then look into the brew
package, which is a simple templating system. Create a analysis.brew
file that has tagged template variables in it (its something like <%i %>)
and then run brew 320 times.

Barry

On Sat, 21 Aug 2010, Laura S wrote:
> Dear all:
>
> Any suggestions are much appreciated. I am looking for a way to make a
> series of similar, but slightly modified, .r files.
>
> My issue is automating making 320 .r files that change the for(i in 1:x) in
> my base .r file (as well as other elements, e.g., the load(...),
> setwd(...)). For smaller jobs running on a single computer with batch
files,
> I have been manually changing the for(i in 1:x) line, etc..
>
R is a scripting language (among other things!).

You can read in a template .R file and then substitute pieces of it inside 
a loop that writes the revised peices to a directory from which you can 
later source or BATCH them.

Here is a simple example with just one substitution:

Here is the template file where the pieces to be substituted is "#for
i#":

,----[ template.R ]
| #for i#{
| toupper(i)
| }
|
`----
> template <- readLines("template.R") # read it in!
> dir.create("temp") # make a place to save new .Rs
> # modify the template
> for (i in letters[1:5]){
+  cat( sub( "#for i#",
+    paste("i <- '",i,"'\n",sep=''),
+    template), sep='\n',
+    file=file.path("temp",paste(i,"R",sep='.')))
+ }
> # look at a.R
> readLines(file.path("temp","a.R"))
[1] "i <- 'a'"   "{"         
"toupper(i)" "}"         
""
> sapply(Sys.glob("temp/*.R"),source) # Run them all
         temp/a.R temp/b.R temp/c.R temp/d.R temp/e.R
value   "A"      "B"      "C"      "D"  
"E"
visible TRUE     TRUE     TRUE     TRUE     TRUE
>
HTH,

Chuck

> Why does this matter to me? I am planning on running a simulation
experiment
> on a linux cluster as a serial job. Although not elegant, it has been
> suggested I make 320 .r files so qsub runs one .r file and then selects
> other jobs. Thus, the manual route I am currently using would take a very
> long time (given multiple runs of 320 .r files, given experimental
> replication).
>
> Thank you,
> Laura
>
>
> -- 
> " Genius is the summed production of the many with the names of the
few
> attached for easy recall, unfairly so to other scientists"
>
> - E. O. Wilson (The Diversity of Life)
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
Charles C. Berry                            (858) 534-2098
                                             Dept of Family/Preventive Medicine
E mailto:cberry at tajo.ucsd.edu	            UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901