R help - Apr 2004 - socket clusters on snow dies easily

hello,

  I'm using R 1.8.1 with the lastest snow package on FreeBSD 4.9.
  However, when I try to using socket clusters, it's very unstable.
  Sometimes it dies half way when I run parSapply(), sometimes
  it dies when cluster connection is idle.

  I create a socket cluster by following cmd

  cl = makeCluster("foo", type = "SOCK",
outfile="/tmp/rafanlog");

  Then, I just idle in R, and tail -f outfile, I got following:

  [... some R msgs ...]

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for a HTML browser interface to help.
Type 'q()' to quit R.

>
> #library(serialize)
> library(snow)
>
> slaveLoop(makeSOCKmaster())
Error in unserialize(node$con) : error reading from connection
Execution halted


  after makeCluster() returns, the outfile hangs around the empty lines
  above, about 1 minute later, all msg shows up, also it dies.

  This is reproducible on my machine. Also, one of my friends also 
  got this result on Linux machines w/ R 1.8.x.

  Is there any one also encounter this problem?
  Any comments or suggestion are welcome :)

Regards,
Rong-En Fan

If you absolutely have to use the socket approach you will probably
have to track down the particular reason for this failure in your
environment by looking at what is going on under the hood and checking
the pieces.  The master sets up a server socket, the slave noted
connect to that socket, and for some reason the first read on that
connection seems to be failing.  It could be a timing issues--you may
need to adjust socket timeouts, it could be a permission or firewall
issue; hard to tell without experimenting.

You are probably better off using the PVM version if at all possible.
Overall it more solid than the socket version and likely to have fewer
issues.

Best,

luke

On Fri, 9 Apr 2004, Rong-En Fan wrote:
> hello,
> 
>   I'm using R 1.8.1 with the lastest snow package on FreeBSD 4.9.
>   However, when I try to using socket clusters, it's very unstable.
>   Sometimes it dies half way when I run parSapply(), sometimes
>   it dies when cluster connection is idle.
> 
>   I create a socket cluster by following cmd
> 
>   cl = makeCluster("foo", type = "SOCK",
outfile="/tmp/rafanlog");
> 
>   Then, I just idle in R, and tail -f outfile, I got following:
> 
>   [... some R msgs ...]
> 
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for a HTML browser interface to help.
> Type 'q()' to quit R.
> 
> 
> >
> > #library(serialize)
> > library(snow)
> >
> > slaveLoop(makeSOCKmaster())
> Error in unserialize(node$con) : error reading from connection
> Execution halted
> 
> 
>   after makeCluster() returns, the outfile hangs around the empty lines
>   above, about 1 minute later, all msg shows up, also it dies.
> 
>   This is reproducible on my machine. Also, one of my friends also 
>   got this result on Linux machines w/ R 1.8.x.
> 
>   Is there any one also encounter this problem?
>   Any comments or suggestion are welcome :)
> 
> Regards,
> Rong-En Fan
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
> 
-- 
Luke Tierney
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
   Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

I don't think the socket stuff is as well tested as PVM and MPI.
We've had a few problems with PVM via rpvm in the past, but I've not
had problems recently (maybe this message will shake them out) and
LAM-MPI via Rmpi seems a bit more stable (though perhaps less
tested). 

best,
-tony


Rong-En Fan <rafan at infor.org> writes:
> hello,
>
>   I'm using R 1.8.1 with the lastest snow package on FreeBSD 4.9.
>   However, when I try to using socket clusters, it's very unstable.
>   Sometimes it dies half way when I run parSapply(), sometimes
>   it dies when cluster connection is idle.
>
>   I create a socket cluster by following cmd
>
>   cl = makeCluster("foo", type = "SOCK",
outfile="/tmp/rafanlog");
>
>   Then, I just idle in R, and tail -f outfile, I got following:
>
>   [... some R msgs ...]
>
> Type 'demo()' for some demos, 'help()' for on-line help, or
> 'help.start()' for a HTML browser interface to help.
> Type 'q()' to quit R.
>
>
>>
>> #library(serialize)
>> library(snow)
>>
>> slaveLoop(makeSOCKmaster())
> Error in unserialize(node$con) : error reading from connection
> Execution halted
>
>
>   after makeCluster() returns, the outfile hangs around the empty lines
>   above, about 1 minute later, all msg shows up, also it dies.
>
>   This is reproducible on my machine. Also, one of my friends also 
>   got this result on Linux machines w/ R 1.8.x.
>
>   Is there any one also encounter this problem?
>   Any comments or suggestion are welcome :)
>
> Regards,
> Rong-En Fan
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>
-- 
rossini at u.washington.edu            http://www.analytics.washington.edu/ 
Biomedical and Health Informatics   University of Washington
Biostatistics, SCHARP/HVTN          Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC  (M/W): 206-667-7025 FAX=206-667-4812 | use Email

CONFIDENTIALITY NOTICE: This e-mail message and any attachme...{{dropped}}