Hello,
I would like to run some code in parallel with each cluster reading/writing to a
different working directory. I've tried the following code without success.
The error I get is: "Error in setwd(x) : cannot change working
directory"
library(parallel)
dirs <- list("out1","out2","out3") # these
directories are located within the current working directory
temp <- 1:3
testF <- function(x) {
setwd(x)
saveRDS(temp,"temp.drs")
}
mclapply(dirs, testF)
Any help would be appreciated!
--David
*********************************************
David R. Schaefer, Ph.D.
Assistant Professor
School of Social and Family Dynamics
Arizona State University
www.public.asu.edu/~schaef/
On Tue, Apr 17, 2012 at 11:06:05AM -0700, David Schaefer wrote:> Hello, > > I would like to run some code in parallel with each cluster reading/writing to a different working directory. I've tried the following code without success. The error I get is: "Error in setwd(x) : cannot change working directory" > > library(parallel) > dirs <- list("out1","out2","out3") # these directories are located within the current working directory > temp <- 1:3 > testF <- function(x) { > setwd(x) > saveRDS(temp,"temp.drs") > } > mclapply(dirs, testF)Hi. Try to include print(getwd()) to testF() function, so that you see, where the script is running. Another option is not to use setwd() and use saveRDS(temp, paste(x, "temp.drs", sep="/")) instead of saveRDS(temp,"temp.drs"). Hope this helps. Petr Savicky.
On 04/17/2012 11:06 AM, David Schaefer wrote:> Hello, > > I would like to run some code in parallel with each cluster reading/writing to a different working directory. I've tried the following code without success. The error I get is: "Error in setwd(x) : cannot change working directory" > > library(parallel) > dirs<- list("out1","out2","out3") # these directories are located within the current working directory > temp<- 1:3 > testF<- function(x) { > setwd(x) > saveRDS(temp,"temp.drs") > } > mclapply(dirs, testF) > > Any help would be appreciated!Hi David, Suppose that jobs 1 and 3 are executed on processor 2. Then after the first iteration the directory is ./out1 and on the second iteration setwd() tries to change to ./out1/out3. Full path names might help. My unasked-for advice would be to put 'more' of the processing in testF, so that it is reasonable to return and then aggregate the results without writing to disk. Martin> > --David > > ********************************************* > David R. Schaefer, Ph.D. > Assistant Professor > School of Social and Family Dynamics > Arizona State University > www.public.asu.edu/~schaef/ > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 17/04/12 20:06, David Schaefer wrote:> Hello, > > I would like to run some code in parallel with each cluster reading/writing to a different > working directory. I've tried the following code without success. The error I get is: "Error > in setwd(x) : cannot change working directory"1) more likely to get a response from the sig-hpc mailing list as here 2) What scheduler is the cluster using? Some use the current working directory, others change the working directory - so check there 3) What I did in a similar situation (cluster using torque and maoi) was to use tempdir to creat the directory in the script - might work for you or not 4) to use print(getwd()) in your script is a good idea - but you could even use echo `pwd` in your submit script to check Many possibilities what is going wrong, many solutions - we need more info Cheers, Rainer> > library(parallel) dirs <- list("out1","out2","out3") # these directories are located within > the current working directory temp <- 1:3 testF <- function(x) { setwd(x) > saveRDS(temp,"temp.drs") } mclapply(dirs, testF) > > Any help would be appreciated! > > --David > > ********************************************* David R. Schaefer, Ph.D. Assistant Professor > School of Social and Family Dynamics Arizona State University www.public.asu.edu/~schaef/ > > ______________________________________________ R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, > reproducible code.- -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer at krugs.de Skype: RMkrug -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+OaZEACgkQoYgNqgF2egpY6wCfaPNnuUIrOiHv+T6G8mnmxgIJ 4NQAn0S+vmTP06jkHVJAayBcoCNYoqav =RqJW -----END PGP SIGNATURE-----
Hi Rainier, Thanks for your suggestions. I should have been more specific, I am using multiple cores on a Mac Pro running Snow Leopard. I can see where that makes a difference. --David On 4/18/12 12:13 AM, "Rainer M Krug" <r.m.krug@gmail.com> wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 17/04/12 20:06, David Schaefer wrote:> Hello, > > I would like to run some code in parallel with each cluster reading/writing to a different > working directory. I''ve tried the following code without success. The error I get is: "Error > in setwd(x) : cannot change working directory"1) more likely to get a response from the sig-hpc mailing list as here 2) What scheduler is the cluster using? Some use the current working directory, others change the working directory - so check there 3) What I did in a similar situation (cluster using torque and maoi) was to use tempdir to creat the directory in the script - might work for you or not 4) to use print(getwd()) in your script is a good idea - but you could even use echo `pwd` in your submit script to check Many possibilities what is going wrong, many solutions - we need more info Cheers, Rainer> > library(parallel) dirs <- list("out1","out2","out3") # these directories are located within > the current working directory temp <- 1:3 testF <- function(x) { setwd(x) > saveRDS(temp,"temp.drs") } mclapply(dirs, testF) > > Any help would be appreciated! > > --David > > ********************************************* David R. Schaefer, Ph.D. Assistant Professor > School of Social and Family Dynamics Arizona State University www.public.asu.edu/~schaef/ > > ______________________________________________ R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, > reproducible code.- -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer@krugs.de Skype: RMkrug -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+OaZEACgkQoYgNqgF2egpY6wCfaPNnuUIrOiHv+T6G8mnmxgIJ 4NQAn0S+vmTP06jkHVJAayBcoCNYoqav =RqJW -----END PGP SIGNATURE----- ********************************************* David R. Schaefer, Ph.D. Assistant Professor School of Social and Family Dynamics Arizona State University www.public.asu.edu/~schaef/ ********************************************* [[alternative HTML version deleted]]