On 09/24/2011 08:55 AM, henry wrote:> Greetings,
>
> I am trying to use Reference Class methods in Snowfall, using R 2.12.1
> on Ubuntu Natty. Using then directly seems to work (stanza 2 below), but
> using them indirectly does not (stanza 3 below). I get an "attempt to
> apply non-function" error.
>
> In addition to exporting the instance of the object I created to the
> Snowfall slaves, I also made several attempts to export the method
> itself (e.g. sfExport("instance$method") or some such), with no
success.
> Very strangely, if I run Stanza 2 below, then stanza 3 will work.
>
> I tried emailing the Snowfall author, but have not heard back. Any help
> is greatly appreciated!
>
> Henry Bryant
> Texas A&M University
>
>
>
> library("snowfall")
> library("methods")
>
> # set up a simple reference class
> calculator <- setRefClass("calculator")
> calculator$methods(do_calc = function(x) {print(x*x)})
> my_calc <- calculator$new()
> wrapper <- function(x) {my_calc$do_calc(x)}
>
> # STANZA 2: use snowfall without wrapper -- WORKING
> #sfInit(parallel=TRUE, cpus=2, type="SOCK")
> #sfExport("calculator")
> #sfExport("my_calc")
> #results <- sfLapply(1:10,my_calc$do_calc)
> #sfStop()
> #print(results)
>
> # STANZA 3: use snowfall with wrapper -- NOT WORKING
> # (not working by itself, but does work if the previous stanza is run
> first!!)
> sfInit(parallel=TRUE, cpus=2, type="SOCK")
> sfExport("calculator")
> sfExport("my_calc")
> sfExport("wrapper")
> last_results <- sfLapply(1:10,wrapper)
Hi Henry --
The problem here is that references classes implement their methods in a
kind of 'lazy' way -- my_calc$do_calc doesn't exist until it's
actually
invoked for the first time, so when you
sfExport("my_calc")
you export the object but "do_calc" is defined as NULL
> sfClusterEval(my_calc$do_calc)
[[1]]
NULL
[[2]]
NULL
This seems to be a variant of the problem experienced elsewhere
https://stat.ethz.ch/pipermail/r-devel/2011-June/061260.html
http://stackoverflow.com/questions/7331518/method-initialisation-in-r-reference-classes
The solutions seem mostly ugly, e.g., to evaluate locally
> my_calc$do_calc
and then sfExport, or maybe a bit better as
calculator <- setRefClass("calculator")
calculator$methods(
initialize = function(...) {
callSuper(...)
.self$do_calc # force definition
.self
},
do_calc = function(x) x*x)
I'm unclear about whether there are other aspects of the reference class
implementation that tie the instance to the class definition in a way
that would undermine parallel evaluation.
Also in general it seems better to write truly 'functional programming'
style functions that don't rely on aspects of the environment to work,
e.g.,
wrapper <- function(x, calc) calc$do_calc(x)
sfLapply(1:10, wrapper, my_calc)
Martin
> sfStop()
> cat("\nRESULTS FROM SECOND TRY:\n")
> print(last_results)
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Computational Biology
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109
Location: M1-B861
Telephone: 206 667-2793