R help - Mar 2015 - Use BatchJobs package to submit Rmpi job

Hi,
BatchJobs package is popular to submit R batch jobs on various job scheduler.
Now we are facing the question is to submit a Rmpi jobs. So we want to know how
we can leverage BatchJobs to do so. After a bit investigation/enhancement, we
tested below approach on OpenLava job scheduler. we want to get some inputs from
the group to see how this makes sense to you. thanks,

1. R script that submit a Rmpi job using BatchJobs:
library(BatchJobs)
conf = BatchJobs:::getBatchJobsConf()
conf$cluster.functions =
makeClusterFunctionsOpenLava("/home/clusteradmin/rmpi.tmpl")     
<-- specify job template file that includes "mpirun" to start R
job.
reg = makeRegistry(id = "RMPIJobsExample")
f = function(x) {                                                <-- RMPI
algorithm function. This is the normal Rmpi R script.
                library("Rmpi")
                ...
                mpi.spawn.Rslaves(nslaves=slaveno)
                                                                ...
                mpi.remote.exec(paste("I
am",mpi.comm.rank(),"of",mpi.comm.size()))
                ...
}
batchMap(reg, f, 1)                                       <-- specify 1 Rmpi
job
submitJobs(reg, np=2)                                  <-- specify number of
processors to use
showStatus(reg)
waitForJobs(reg)
                                            showStatus(reg)

2. the job template: rmpi.tmpl
                                            #BSUB -q mpi_job_queue
                                            /opt/openlava/bin/openmpi-mpirun -np
1 R CMD BATCH "<%= rscript %>"


Please let us know your comments,

Thanks,
Feng Li
Teraproc Inc.


	[[alternative HTML version deleted]]