All, I am not understanding the scoping used in foreach when it is used inside a function. I keep getting "could not find function" errors for functions that are in memory when I try to use foreach within a function call. I have a simple example below. "testFun" is in memory and works when called by foreach directly, but when I place foreach in a function called "helper" and then call helper, foreach cannot find "testFun". Can anyone provide a fix? I hope its just a simple mistake on my part and not a limitation of foreach.> source(.trPaths[5], echo=TRUE, max.deparse.length=150)> "testFun" <- function(m) {+ out <- m*m + }> out <- foreach(m=1:10, .combine=rbind) %dopar% testFun(m)> "helper" <- function(i) {+ + out <- foreach(m=1:i, .combine=rbind) %dopar% testFun(m) + }> out <- helper(10)Error in testFun(m) : task 1 failed - "could not find function "testFun"">Thanks, Roger *************************************************************** This message is for the named person's use only. It may\...{{dropped:20}}
On Jul 12, 2011, at 5:28 PM, Bos, Roger wrote:> All, > > I am not understanding the scoping used in foreach when it is used > inside a function. I keep getting "could not find function" errors > for > functions that are in memory when I try to use foreach within a > function > call. I have a simple example below. "testFun" is in memory and > works > when called by foreach directly, but when I place foreach in a > function > called "helper" and then call helper, foreach cannot find "testFun". > Can anyone provide a fix? I hope its just a simple mistake on my part > and not a limitation of foreach.I'm wondering if Uwe Ligges' posting in response to what seems like a similar question not to long ago may have the answer. Details from the header on my reader: From: ligges at statistik.tu-dortmund.de Subject: Re: [R] parallel computing with 'foreach' Date: June 30, 2011 1:24:34 PM EDT To: staceyjeanwood at gmail.com -- david.> >> source(.trPaths[5], echo=TRUE, max.deparse.length=150) > >> "testFun" <- function(m) { > + out <- m*m > + } > >> out <- foreach(m=1:10, .combine=rbind) %dopar% testFun(m) > >> "helper" <- function(i) { > + > + out <- foreach(m=1:i, .combine=rbind) %dopar% testFun(m) > + } > >> out <- helper(10) > Error in testFun(m) : task 1 failed - "could not find function > "testFun"" >> > > Thanks, > > Roger > > *************************************************************** > > This message is for the named person's use only. It may\...{{dropped: > 20}} > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD West Hartford, CT
Roger, I think I know why you are having a problem. Your foreach is using %dopar% and judging from your output you have active parallel processing. When running something in paralell, the master is the processor used by the Rgui, and the slaves are in the background. My guess is the the function you made is the memory of the master, but not in the memory for the slaves. You need to pass the function "TestFun" to your slaves. If the slave don't know about it, using foreach with %dopar% will fail since the function doesn't exist on the slave processors. Without knowing what package you used to do your parallel processing I can't tell you how to pass the function you created to your slave processors. Try running the foreach command with %do% in place of %dopar%. If the command works, then it is probably a problem that function isn't in memory on the slave processors. Adrienne On Tue, Jul 12, 2011 at 5:28 PM, Bos, Roger <roger.bos at rothschild.com> wrote:> All, > > I am not understanding the scoping used in foreach when it is used > inside a function. ?I keep getting "could not find function" errors for > functions that are in memory when I try to use foreach within a function > call. ?I have a simple example below. ?"testFun" is in memory and works > when called by foreach directly, but when I place foreach in a function > called "helper" and then call helper, foreach cannot find "testFun". > Can anyone provide a fix? ?I hope its just a simple mistake on my part > and not a limitation of foreach. > >> source(.trPaths[5], echo=TRUE, max.deparse.length=150) > >> "testFun" <- function(m) { > + ? ? out <- m*m > + } > >> out <- foreach(m=1:10, .combine=rbind) %dopar% testFun(m) > >> "helper" <- function(i) { > + > + ? ? out <- foreach(m=1:i, .combine=rbind) %dopar% testFun(m) > + } > >> out <- helper(10) > Error in testFun(m) : task 1 failed - "could not find function > "testFun"" >> > > Thanks, > > Roger > > *************************************************************** > > This message is for the named person's use only. It may\...{{dropped:20}} > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Adrienne Wootten Graduate Research Assistant State Climate Office of North Carolina Department of Marine, Earth and Atmospheric Sciences North Carolina State University