Hi all, Is it possible to use the 'renice' option together with parallel clusters of type 'PSOCK'? The help page for parallel::makeCluster is not specific about which options are supported on which types and I am getting the following message when passing renice = 19 :> cl <- parallel::makeCluster(2, renice = 19)nice: ?+19?: No such file or directory Kind regards, Andreas
Looks like a bug to me due to wrong assumptions about 'nice' arguments, but could be because a "non-standard" 'nice' is used. If we do:> trace(system, tracer = quote(print(command)))Tracing function "system" in package "base" we see that the system call used is:> cl <- parallel::makePSOCKcluster(2L, renice = 19)Tracing system(cmd, wait = FALSE) on entry [1] "nice +19 '/usr/lib/R/bin/Rscript' --default-packages=datasets,utils,grDevices,graphics,stats,methods -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE" nice: ?+19?: No such file or directory ^C The code that prepends that 'nice +19' is in parallel:::newPSOCKnode: if (!is.na(renice) && renice) cmd <- sprintf("nice +%d %s", as.integer(renice), cmd) I don't know where that originates from and on what platform it was tests/validated. On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have 'nice' from "GNU coreutils" and they all complain about using '+', e.g. $ nice +19 date nice: +19: No such file or directory but '-' works: $ nice -19 date Sun Dec 3 20:01:31 PST 2017 Neither 'nice --help' nor 'man help' mention the use of a +n option. WORKAROUND: As a workaround, you can use: cl <- future::makeClusterPSOCK(2L, rscript = c("nice", "--adjustment=10", file.path(R.home("bin"), "Rscript"))) which is backward compatible with parallel::makePSOCKcluster() but provides you with more detailed control. Try adding verbose = TRUE to see what the exact call looks like. /Henrik On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha <andreas.leha at med.uni-goettingen.de> wrote:> Hi all, > > Is it possible to use the 'renice' option together with parallel > clusters of type 'PSOCK'? The help page for parallel::makeCluster is > not specific about which options are supported on which types and I am > getting the following message when passing renice = 19 : > >> cl <- parallel::makeCluster(2, renice = 19) > nice: ?+19?: No such file or directory > > Kind regards, > Andreas > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Hi Henrik, Thanks for the detailed in fast reply! My guess would be that the confusion comes from the different use of nice and renice. The workraund you provided work fine! Thanks a lot. Best, Andreas Henrik Bengtsson <henrik.bengtsson at gmail.com> writes:> Looks like a bug to me due to wrong assumptions about 'nice' > arguments, but could be because a "non-standard" 'nice' is used. If > we do: > >> trace(system, tracer = quote(print(command))) > Tracing function "system" in package "base" > > we see that the system call used is: > >> cl <- parallel::makePSOCKcluster(2L, renice = 19) > Tracing system(cmd, wait = FALSE) on entry > [1] "nice +19 '/usr/lib/R/bin/Rscript' > --default-packages=datasets,utils,grDevices,graphics,stats,methods -e > 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11146 OUT=/dev/null > TIMEOUT=2592000 XDR=TRUE" > nice: ?+19?: No such file or directory > ^C > > The code that prepends that 'nice +19' is in parallel:::newPSOCKnode: > > if (!is.na(renice) && renice) > cmd <- sprintf("nice +%d %s", as.integer(renice), cmd) > > I don't know where that originates from and on what platform it was > tests/validated. On Ubuntu 16.04, CentOS 6.6, and CentOS 7.4, I have > 'nice' from "GNU coreutils" and they all complain about using '+', > e.g. > > $ nice +19 date > nice: +19: No such file or directory > > but '-' works: > > $ nice -19 date > Sun Dec 3 20:01:31 PST 2017 > > Neither 'nice --help' nor 'man help' mention the use of a +n option. > > > WORKAROUND: As a workaround, you can use: > > cl <- future::makeClusterPSOCK(2L, rscript = c("nice", > "--adjustment=10", file.path(R.home("bin"), "Rscript"))) > > which is backward compatible with parallel::makePSOCKcluster() but > provides you with more detailed control. Try adding verbose = TRUE to > see what the exact call looks like. > > /Henrik > > > On Sun, Dec 3, 2017 at 7:35 PM, Andreas Leha > <andreas.leha at med.uni-goettingen.de> wrote: >> Hi all, >> >> Is it possible to use the 'renice' option together with parallel >> clusters of type 'PSOCK'? The help page for parallel::makeCluster is >> not specific about which options are supported on which types and I am >> getting the following message when passing renice = 19 : >> >>> cl <- parallel::makeCluster(2, renice = 19) >> nice: ?+19?: No such file or directory >> >> Kind regards, >> Andreas >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Reasonably Related Threads
- PSOCK cluster and renice
- PSOCK cluster and renice
- SUGGESTION: Proposal to mitigate problem with stray processes left behind by parallel::makeCluster()
- parallel:::newPSOCKnode(): background worker fails immediately if socket on master is not set up in time (BUG?)
- parallel:::newPSOCKnode(): background worker fails immediately if socket on master is not set up in time (BUG?)