Tevlin, Dylan
2014-Sep-19 14:49 UTC
[R] package Parallel - accessing remote cores on Windows using plink (PuTTY)
Hello all. My system information: R 2.14.1 x64 Windows 7 I'm attempting to access cores on other machines in my LAN. As setup, I can ping all the machines successfully and it's a private office network where I have access to all computers, so there really shouldn't be any problems arising from the network itself. All machines I am connecting to have identical system information. I issue the following command to make a connection to the remote machine and access one of its cores. cl<-makePSOCKcluster(c(remoteHostIP), rshcmd="plink", outfile="") The parallel package, to make remote connections, issues a system command (found in the file snowSOCK.R, function newPSOCKnode of the source) of the following form on the remote machine ssh -l user otherhost '/usr/lib/R/bin/Rscript' -e 'parallel:::.slaveRSOCK()' MASTER=myhost PORT=10187 OUT=/dev/null TIMEOUT=2592000 METHODS=TRUE XDR=TRUE Where the specific command (ssh) can be specified by the rshcmd parameter of the command makePSOCKcluster. Given that a command such as ssh must be present on the machine, I installed PuTTY on both the master and host, and added the pertinent path to the system PATH variable so that plink is always accessible from the command line. I'm not set on PuTTY, it's just free and I've used it before. Here are my two cases. If I issue my above command, the process hangs indefinitely. If I set the manual flag equal to TRUE, as in the following cl<-makePSOCKcluster(c(remoteHostIP), rshcmd="plink", outfile="", manual=TRUE) I'm able to make the connection to the remote host just fine. This indicates to me that there is something wrong with the system command that is being issued with manual=FALSE (the default). Another problem with this is that it is seemingly very hard to test, as the hanging process gives me no information on what is actually occurring. Has anyone had any success with this? Are there any ideas on steps I could take to fix or debug this issue? Regards, Dylan Dylan Tevlin (480) 419-3611 [[alternative HTML version deleted]]
Jeff Newmiller
2014-Sep-19 15:40 UTC
[R] package Parallel - accessing remote cores on Windows using plink (PuTTY)
While I hope someone with first hand experience or time to try it out pipes up,
I noticed something wrong in your description. The ssh command is executed on
the local machine, not the remote machine. That command must succeed on the
local machine in order to start Rscript on the remote machine. Since the remote
machine is a windows box, the given path to Rscript on the remote machine
won't be usable unless the sshd running there is part of cygwin to simulate
a standard Unix directory structure. So you need to change the path in the ssh
arguments or make sure the remote machine understands the given path.
I don't see you describing how the remote machine is supposed to receive the
ssh connection request, but you say that is working.
The plink command is supposed to be a replacement for ssh... but I don't
know if R is compatible with plink.. someone else will have to chime in. If it
is, renaming it would seem logical.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live
Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.
On September 19, 2014 7:49:36 AM PDT, "Tevlin, Dylan" <Dylan.Tevlin
at kochind.com> wrote:>Hello all.
>
>My system information:
>
>R 2.14.1 x64
>Windows 7
>
>I'm attempting to access cores on other machines in my LAN. As setup,
>I can ping all the machines successfully and it's a private office
>network where I have access to all computers, so there really shouldn't
>be any problems arising from the network itself.
>
>All machines I am connecting to have identical system information.
>
>I issue the following command to make a connection to the remote
>machine and access one of its cores.
>
>cl<-makePSOCKcluster(c(remoteHostIP), rshcmd="plink",
outfile="")
>
>The parallel package, to make remote connections, issues a system
>command (found in the file snowSOCK.R, function newPSOCKnode of the
>source) of the following form on the remote machine
>
>ssh -l user otherhost '/usr/lib/R/bin/Rscript' -e
>'parallel:::.slaveRSOCK()' MASTER=myhost PORT=10187 OUT=/dev/null
>TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
>
>Where the specific command (ssh) can be specified by the rshcmd
>parameter of the command makePSOCKcluster. Given that a command such
>as ssh must be present on the machine, I installed PuTTY on both the
>master and host, and added the pertinent path to the system PATH
>variable so that plink is always accessible from the command line.
>I'm not set on PuTTY, it's just free and I've used it before.
>
>Here are my two cases. If I issue my above command, the process hangs
>indefinitely. If I set the manual flag equal to TRUE, as in the
>following
>
>cl<-makePSOCKcluster(c(remoteHostIP), rshcmd="plink",
outfile="",
>manual=TRUE)
>
>I'm able to make the connection to the remote host just fine. This
>indicates to me that there is something wrong with the system command
>that is being issued with manual=FALSE (the default).
>
>Another problem with this is that it is seemingly very hard to test, as
>the hanging process gives me no information on what is actually
>occurring. Has anyone had any success with this? Are there any ideas
>on steps I could take to fix or debug this issue?
>
>Regards,
>
>Dylan
>
>
>
>Dylan Tevlin
>(480) 419-3611
>
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>R-help at r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.