R help - Apr 2012 - Dependency-aware scripting tools for R

There are numerous tools like scons, make, ruffus, ant, rake, etc.
that can be used to build complex pipelines based on task
dependencies.  These tools are written in a variety of languages, but
I have not seen such a thing for R.  Is anyone aware of a package
available?  The goal is to be able to develop robust bioinformatic
pipelines driven by scripts written in R.

Thanks,
Sean

Sean Davis <sdavis2 <at> mail.nih.gov> writes:
> 
> There are numerous tools like scons, make, ruffus, ant, rake, etc.
> that can be used to build complex pipelines based on task
> dependencies.  These tools are written in a variety of languages, but
> I have not seen such a thing for R.  Is anyone aware of a package
> available?  The goal is to be able to develop robust bioinformatic
> pipelines driven by scripts written in R.
  This might (?) be better for r-devel (although it's a bit of a toss-up).
Most of the activity I've seen along these lines in the R community has used
make ... it's a tradeoff between (1) re-inventing (and polishing) the wheel
for the n^th time and (2) getting rid of another dependency. (In contrast
to the use of (e.g.) configure and make, consider the pipeline used for
installing R itself and constructing and testing packages -- this has moved
from a dependence on Perl to being completely self-contained.)

  Bottom line: I think you could do the scripting in R, but IMO it
probably makes the most sense to leave the dependency management to make.

shameless self-plug: we break out of R to do this, and after many painful
years developing and maintaining idiosyncratic Makefiles, we are now using
Taverna to (visually) glue together UNIX commands (including R scripts) --
the benefits of which (over make and brethren) is that you can actually
*see* the dependencies and overall workflow (nesting workflows also makes
it easier to manage complexity).

see TavernaPBS:

  http://cphg.virginia.edu/mackey/projects/sequencing-pipelines/tavernapbs/

while designed to automate job submission to a PBS queuing system, you can
also use it to simply execute non-PBS jobs.

--
Aaron J. Mackey, PhD
Assistant Professor
Center for Public Health Genomics
University of Virginia
amackey@virginia.edu
http://www.cphg.virginia.edu/mackey


On Thu, Apr 19, 2012 at 3:27 PM, Sean Davis <sdavis2@mail.nih.gov> wrote:
> There are numerous tools like scons, make, ruffus, ant, rake, etc.
> that can be used to build complex pipelines based on task
> dependencies.  These tools are written in a variety of languages, but
> I have not seen such a thing for R.  Is anyone aware of a package
> available?  The goal is to be able to develop robust bioinformatic
> pipelines driven by scripts written in R.
>
> Thanks,
> Sean
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
	[[alternative HTML version deleted]]