R help - Feb 2011 - Question about foreach (with doSNOW), is that a bug?

Hi all,

Within a foreach loop with doSNOW, we cant call functions which come from
the non-default package. We need to load(require/library) the package once
more within the foreach loop. Anyone knows why would happen like this? Is it
caused by the snow package and something happened when "snow"
parallelize
the job?

Other than load the package once more with in the foreach loop, is there any
other solutions?

Here's an example about what i said:


# install.packages("lmtest")
# install.packages("doSNOW")

require(lmtest)
require(foreach)
require(doSNOW)

rm(list=ls(all=TRUE))

# load the data
data(Mandible)

# run the OLS and compute the SD
# no problem at all
coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))

# Do the same thing twice with foreach
# no problem at all
foreach(i=1:2) %do% {
    coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
}

# Do the same thing twice with foreach and doSNOW
# it fails!! the error msg was (could not find function "coeftest")
cl<-makeCluster(2)
registerDoSNOW(cl)
foreach(i=1:2) %dopar% {
        coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
}
stopCluster(cl)

# However if i put require(lmtest) inside the foreach loop, it works again!
cl<-makeCluster(2)
registerDoSNOW(cl)
foreach(i=1:2) %dopar% {
        require(lmtest)
        coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
}
stopCluster(cl)




Regards,
Julian

	[[alternative HTML version deleted]]

You'll want to use the .packages option to foreach to ensure that the
appropriate packages are also loaded in the child instances of R. There's an
example of its use in Section 5.1 "Parallel Random Forest" of the
foreach
reference manual:

http://cran.r-project.org/web/packages/foreach/vignettes/foreach.pdf

# David Smith

On Thu, Feb 24, 2011 at 11:57 PM, Julian TszKin Chan <cjulian@bu.edu>
wrote:
> Hi all,
>
> Within a foreach loop with doSNOW, we cant call functions which come from
> the non-default package. We need to load(require/library) the package once
> more within the foreach loop. Anyone knows why would happen like this? Is
> it
> caused by the snow package and something happened when "snow"
parallelize
> the job?
>
> Other than load the package once more with in the foreach loop, is there
> any
> other solutions?
>
> Here's an example about what i said:
>
>
> # install.packages("lmtest")
> # install.packages("doSNOW")
>
> require(lmtest)
> require(foreach)
> require(doSNOW)
>
> rm(list=ls(all=TRUE))
>
> # load the data
> data(Mandible)
>
> # run the OLS and compute the SD
> # no problem at all
> coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
>
> # Do the same thing twice with foreach
> # no problem at all
> foreach(i=1:2) %do% {
>    coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
> }
>
> # Do the same thing twice with foreach and doSNOW
> # it fails!! the error msg was (could not find function
"coeftest")
> cl<-makeCluster(2)
> registerDoSNOW(cl)
> foreach(i=1:2) %dopar% {
>        coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
> }
> stopCluster(cl)
>
> # However if i put require(lmtest) inside the foreach loop, it works again!
> cl<-makeCluster(2)
> registerDoSNOW(cl)
> foreach(i=1:2) %dopar% {
>        require(lmtest)
>        coeftest(lm(length ~ age, data=Mandible, subset=(age <= 28)))
> }
> stopCluster(cl)
>
>
>
>
> Regards,
> Julian
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
David M Smith <david@revolutionanalytics.com>
VP of Marketing, Revolution Analytics  http://blog.revolutionanalytics.com
Tel: +1 (650) 646-9523 (Palo Alto, CA, USA)

	[[alternative HTML version deleted]]