search for: clusterapplylb

...R-Users, we are using a linux-cluster with RMPI and the snow package. We would like to do a double boostrap. We have a general function that implements the first boostrap (the outer) and we are wondering if we can include another bootstrap (the inner) in the same general function including another clusterApplyLB. For example: general function = function(...) { clusterApplyLB(cl, fun,...) } where "fun" is the function for the inner boostrap outer bootstrap = clusterApplyLB(cl,general function,...) Does R accept this kind of expression? Thank you very much in advance, Michela and Marco

I have this function and want to run it parallel with different sets of data. Using SNOW and clusterApplyLB. system.time(out <- mclapply(cData, plotGraph)) #each cData contains 100X6000 doubles system.time(out <- mclapply(cData2, plotGraph)) system.time(out <- mclapply(cData3, plotGraph)) system.time(out <- mclapply(cData4, plotGraph)) system.time(out <- mclapply(cData5, plotGraph)) syste...

...problem and I wonder if anyone else has seen this behavior. I am running R 2.11.1 with SNOW 0.3-3 on a Dell cluster running CentOS 5.5. I create my cluster using: cluster<- makeCluster(nodes,type="SOCK",port=10191) # nodes is a vector of compute nodes I then wrap a loop around clusterApplyLB to evaluate my function multiple times, with different parameters, without recreating the cluster every time. The following code segment shows what I am trying to do: for (j in loopstart:loopend) { call.m=list( step1=T, dat=x.m[,c(1:7, j)] # x.m is data from a csv f...

...kers seem to be active, whereas the remaining 56 seem to run idle. I tried to find out in more detail what parLapplyLB() actually does and I was astonished to see that, in the same way as parLapply(), it splits the input list X into as many parts as there are nodes in the cluster and then calls clusterApplyLB() on these chunks. By the way, parLapply() does exactly the same, but then calls clusterApply(). I may miss a point, but this means to me, since there are as many chunks as nodes in the cluster, that actually no load balancing takes place, right? My list of data sets are actually ordered by siz...

...e. However, if I source the script the function works. For example, in a method for parallelization of analysis on each chromosome simultaneously I am receiving error at the following position of the code: # this profile the information chromosome wise and results in "clRes" > clRes = clusterApplyLB(cl, Chrs, profilingChr, GenoRanges = GenoRanges, Sample = Sample) > class(clRes) [1] "list" > clRes [[1]] S4 Object of class GElist; 14 features start : 14350 69251 84185 116264 153924 end : 18349 73250 88184 120263 157923 chr : chr20 chr20 chr20 chr20 chr20 strand : * * * *...

...31/why-does-parlapplylb-not-actually-balance-load [2]: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 ## The Call Chain First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** 2. **splitList:** clusterApply.R:157 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** 4. **dynamicClusterApply:** clusterApply.R:39 ## splitList We used both our whiteboard and an R session to manually *run* a few examples. We were using lists of 100 elements and 5 wo...

...hogTest$X) e.test.lin <- sqrt(sum((results.lin-hogTest$Y)^2)/length(hogTest$Y)) return(e.test.lin) } } cl<- makeCluster(10, type="SOCK" ) clusterEvalQ(cl,library(e1071)) clusterExport(cl,c("data.X","data.Y","NR","cost1")) RMSEP<-clusterApplyLB(cl,cost1,sv.lin) stopCluster(cl) -- View this message in context: http://old.nabble.com/SVM-Param-Tuning-with-using-SNOW-package-tp26399401p26399401.html Sent from the R help mailing list archive at Nabble.com.

...s://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 >> >> >> ## The Call Chain >> >> First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: >> >> 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** >> 2. **splitList:** clusterApply.R:157 >> 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** >> 4. **dynamicClusterApply:** clusterApply.R:39 >> >> >> ## splitList >> >> We used both our whiteboard and an R session to manual...

...ance-load > [2]: https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16792 > > > ## The Call Chain > > First, we traced the relevant R function calls through the code, beginning with `parLapplyLB`: > > 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then **clusterApplyLB** > 2. **splitList:** clusterApply.R:157 > 3. **clusterApplyLB:** clusterApply.R:87, calls **dynamicClusterApply** > 4. **dynamicClusterApply:** clusterApply.R:39 > > > ## splitList > > We used both our whiteboard and an R session to manually *run* a few examples. We w...

...t;>>> >>>> >>>> ## The Call Chain >>>> >>>> First, we traced the relevant R function calls through the code, >> beginning with `parLapplyLB`: >>>> 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then >> **clusterApplyLB** >>>> 2. **splitList:** clusterApply.R:157 >>>> 3. **clusterApplyLB:** clusterApply.R:87, calls >> **dynamicClusterApply** >>>> 4. **dynamicClusterApply:** clusterApply.R:39 >>>> >>>> >>>> ## splitList >>>&gt...

I'm trying to use snow in my dual-core (hopefully later this is going to run in a cluster). So, at this moment I create a cluster using SOCK connection (MPI in the future). However when I try to use clusterApplyLB I got "Error in socketSelect(socklist) : not a socket connection". Any ideas ? Do you know if that is going to be an isuue too when I swith from SOCK to MPI ? Sample code is attached. Thanks. -------------- next part -------------- An embedded and charset-unspecified text was scrubbe...

Looking at parLapplyLB, one sees that it takes in X and then passes splitList(X, length(cl)) to clusterApplyLB, which then calls dynamicClusterApply. Thus while dynamicClusterApply does handle tasks in a load-balancing fashion, sending out individual tasks as previous tasks complete, parLapplyLB preempts that by splitting up the tasks in advance into as many groups of tasks as there are cluster processes....

...w_bug.cgi?id=16792 >>> >>> >>> ## The Call Chain >>> >>> First, we traced the relevant R function calls through the code, >beginning with `parLapplyLB`: >>> >>> 1. **parLapplyLB:** clusterApply.R:177, calls **splitList**, then >**clusterApplyLB** >>> 2. **splitList:** clusterApply.R:157 >>> 3. **clusterApplyLB:** clusterApply.R:87, calls >**dynamicClusterApply** >>> 4. **dynamicClusterApply:** clusterApply.R:39 >>> >>> >>> ## splitList >>> >>> We used both our w...

...gt;> >>>>> ## The Call Chain >>>>> >>>>> First, we traced the relevant R function calls through the code, >>> beginning with `parLapplyLB`: >>>>> 1.? **parLapplyLB:** clusterApply.R:177, calls **splitList**, then >>> **clusterApplyLB** >>>>> 2.? **splitList:** clusterApply.R:157 >>>>> 3.? **clusterApplyLB:** clusterApply.R:87, calls >>> **dynamicClusterApply** >>>>> 4.? **dynamicClusterApply:** clusterApply.R:39 >>>>> >>>>> >>>>>...

...re independent of the others. I start the simulation on one node, create a cluster of several nodes, load my custom package and snow on all of them, use clusterExport(cl, "x1") to export the variable x1(among other variable I need), then I call my simulation on the cluster using clusterApplyLB(cl, 2:S, simClust) where cl is the cluster and S is a constant defined above as 500. Using print statements (since snow, or R for that matter, has next to no ability to debug) I found that the error cropped up in this statement: theta6 = optim(c(0,0,0,0,0,0,.2), loglikelihood, scrore6, met...

...time. Is there any way I can find cl = cor(cdata) utilizing the computers in the clusters(I am using 32 nodes ). I am using the following code to get the cluster.. library(snow) cl <- makeCluster(64) clusterExport(cl, "fakeData") clusterEvalQ(cl, library(boot)) system.time(out2 <- clusterApplyLB(cl, pair, geneCor)) stopCluster(cl) But here the geneCor and pair is calculated as, pair <- combn(1:nrow(fakeData), 2, simplify = F) geneCor <- function(x, gene = fakeData) { cor(t(gene[x[1], ]), t(gene[x[2], ])) } #This calculates for each pairs. But I want cor(data) for a 2D matrix to be...

...more on this below). (c) Found: 'clusterApply' calls 'fun' on the first node with arguments 'seq[[1]]' and 'clusterApply' has no 'seq' arg. There are a few other occurences of mistake (c) in the \details and \value section, for clusterApply() and clusterApplyLB(). Omissions --------- o X: A vector (atomic or list) for ?parLapply? and ?parSapply?, an array for ?parApply?. ==> Nothing is said for the 'X' argument of parLapplyLB() and parSapplyLB(). Inconsistencies --------------- The only function in that man page wi...

...detects processors in all processor groups, not just the group R is running in (impacts particularly systems with more than 64 logical processors). Reported by Arunkumar Srinivasan. * On Windows, socketSelect() would hang with more than 64 sockets, and hence parallel::clusterApplyLB() would hang with more than 64 workers. Reported by Arunkumar Srinivasan. * as(1L, "double") now does coerce (PR#17457). * lm.influence(), influence.measures(), rstudent() etc now work (more) correctly for multivariate models ("mlm"), thanks to (anon...

...detects processors in all processor groups, not just the group R is running in (impacts particularly systems with more than 64 logical processors). Reported by Arunkumar Srinivasan. * On Windows, socketSelect() would hang with more than 64 sockets, and hence parallel::clusterApplyLB() would hang with more than 64 workers. Reported by Arunkumar Srinivasan. * as(1L, "double") now does coerce (PR#17457). * lm.influence(), influence.measures(), rstudent() etc now work (more) correctly for multivariate models ("mlm"), thanks to (anon...

...detects processors in all processor groups, not just the group R is running in (impacts particularly systems with more than 64 logical processors). Reported by Arunkumar Srinivasan. * On Windows, socketSelect() would hang with more than 64 sockets, and hence parallel::clusterApplyLB() would hang with more than 64 workers. Reported by Arunkumar Srinivasan. * as(1L, "double") now does coerce (PR#17457). * lm.influence(), influence.measures(), rstudent() etc now work (more) correctly for multivariate models ("mlm"), thanks to (anon...