R help - Apr 2006 - Parallel computing with the snow package: external file I/O possible?

Hello,

After getting help to solve part of my problem and some delay on my
part, I am posting a more refined version to see if someone can help me
further.  I am trying to autocalibrate a model in my subject area using
the snow and rgenoud packages.  I want to use the key function "fn"
that
is called by genoud() to finalize input, run the model executable, and
compute the objective function on the model output.  My problem is that
only in the master node do all of these steps happen.  In the slave node
of a cluster of two, the input file gets created but my system() call to
execute the model doesn't happen.  I don't understand why the model
doesn't get run in the slave mode, but I noticed something described
below that I thought might be related. 

It is hard for me to figure out a short and simple example of my
genoud() problem for posting here, so let me start with some code that
uses clusterCall().  As I understand the snow package, each execution of
the function "fun" from clusterCall() (or of the function fn() from
genoud()) should be independent.  However, in the code below, the
directory created by each node has the same random number in its name.
I was expecting the contents of fun() or fn() to be independent from all
other executions of the same function.  What am I missing here?

#  Begin code
library(snow)
setDefaultClusterOptions(outfile="/tmp/cluster1")
setDefaultClusterOptions(master="moab")
cl <- makeCluster(c("moab", "escalante"),
type="SOCK")

# Define base pathname for output from my.test()
base.dir <- "~"

# Define a function that is called by clusterCall()
my.test <- function(base.dir) {
  this.host <- as.character(system("hostname", intern=T))
  this.rnd <- sample(1:1e6, 1)
  test.file <- paste(sep="", base.dir, this.host, "_",
this.rnd)
  file.create(test.file)
}  # end my.test()

clusterCall(cl, my.test, base.dir)
stopCluster(cl)
#  End code 

For example, the files "moab_65835" and "escalante_65835"
are created.

Regards,
Scott Waichler
Pacific Northwest National Laboratory
scott.waichler at pnl.gov

"Waichler, Scott R" <Scott.Waichler at pnl.gov> writes:
> genoud()) should be independent.  However, in the code below, the
> directory created by each node has the same random number in its name.
This is probably because 'random' numbers are generated starting from
a 'seed', the seed is determined (by default) from the system time,
and the system time on the two nodes is identical. The same seed is
being used, hence the same random number sequence.

See http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html

Martin
> I was expecting the contents of fun() or fn() to be independent from all
> other executions of the same function.  What am I missing here?
>
> #  Begin code
> library(snow)
> setDefaultClusterOptions(outfile="/tmp/cluster1")
> setDefaultClusterOptions(master="moab")
> cl <- makeCluster(c("moab", "escalante"),
type="SOCK")
>
> # Define base pathname for output from my.test()
> base.dir <- "~"
>
> # Define a function that is called by clusterCall()
> my.test <- function(base.dir) {
>   this.host <- as.character(system("hostname", intern=T))
>   this.rnd <- sample(1:1e6, 1)
>   test.file <- paste(sep="", base.dir, this.host,
"_", this.rnd)
>   file.create(test.file)
> }  # end my.test()
>
> clusterCall(cl, my.test, base.dir)
> stopCluster(cl)
> #  End code 
>
> For example, the files "moab_65835" and
"escalante_65835" are created.
>
> Regards,
> Scott Waichler
> Pacific Northwest National Laboratory
> scott.waichler at pnl.gov