similar to: Parallel runs of an external executable with snow in local

Dear R gurus, I have a very embarrassingly parallelizable job that I am trying to speed up with snow on our local cluster. Basically, I am doing ~50,000 t.test for a series of micro-array experiments, one gene at a time. Thus, I can easily spread the load across multiple processors and nodes. So, I have a master list object that tells me what rows to pick up for each genes to do the t.test from

Dear R-user, I am trying to use the function 'parLapply' from the 'snow' package which is supposed to work the same wys as 'lapply' but for a parallelized cluster of computers. The function I am trying to call in parallel is 'dudi.pca' (from the 'ade4' package) which performs principal component analyses. When I call this function on a list of

I am slightly confused by the way the standard output is redirected in a R snow cluster environment. I am using parLapply from the snow package to execute a function on my MPI/LAM cluster. How can I redirect standard output (produced using "cat") from this function back to the terminal where I invoked it? I intend to transmit some status information in advance to the final result of the

Hello, Is the first time I am using SNOW package and I am trying to tune the cost parameter for a linear SVM, where the cost (variable cost1) takes 10 values between 0.5 and 30. I have a large dataset and a pc which is not very powerful, so I need to tune the parameters using both CPUs of the pc. Somehow I cannot manage to do it. It seems that both CPUs are fitting the model for the same values

Hi R-developers In the package Parallel, the function parLapply(cl, x, f) seems to allow transmission of only one parameter (x) to the function f. Hence in order to compute f(x, y) parallelly, I had to define f(x, y) as f(x) and tried to access y within the function, whereas y was defined outside of f(x). Script: library(parallel) f <- function(x) { z <- 2 * x + .GlobalEnv$y # Try to

Hi, The following function use to work, but now it doesn't giving the error "> CallSnow(, 100) Using snow package, asking for 2 nodes 2 slaves are spawned successfully. 0 failed. Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: no applicable method for 'lapply' applied to an object of class "list" ". Where this is the

I'm parallelizing some computation on hierarchical data, and would find it natural to do something like this (where a call to parLapply is embedded in outer call to parLapply): cl <- makeCluster(rep.int('localhost', 5), type='SOCK') clusterExport(cl, 'cl') parLapply(cl, 1:5, function(i) parLapply(cl, 1:5, function(j) i * j)) Snow

I am trying to do post-hoc tests associated with a repeated measures analysis with on factor nested within respondents. The factor (SOI) has 17 levels. The overall testing is working fine, but I can''t seem to get the multiple comparisons to work. The first step is to "stack" the data. Then I used "lme" to specify and test the overall model. Finally

Hello R-helpers: I have data like this: sample replicate height weight age A 1.00 12.0 0.64 6.00 A 2.00 12.2 0.38 6.00 A 3.00 12.4 0.49 6.00 B 1.00 12.7 0.65 4.00 B 2.00 12.8 0.78 5.00 C 1.00 11.9 0.45 6.00 C 2.00 11.84 0.44 2.00 C 3.00 11.43 0.32 3.00 C 4.00 10.24 0.84 4.00 D

Thank you for your answer! I agree with you except for the 3 (Error) example and I realize now I should have started with that in the explanation. >From my point of view parLapply(cl = clu, X = 1:2, fun = fun, c = 1) shouldn't give an error. This could be easily avoided by using all the argument names in the custerApply call of parLapply which means changing, parLapply <-

Dear R-users, I don't know if there is a problem in wtd.quantile (from library "Hmisc"): -------------------------------- x <- c(1,2,3,4,5) w <- c(0.5,0.4,0.3,0.2,0.1) wtd.quantile(x,weights=w) ------------------------------- The output is: 0% 25% 50% 75% 100% 3.00 3.25 3.50 3.75 4.00 The version of R I am using is: 2.1.0 Best,Jing

Hi! I recognized that the argument matching of clusterApply (and therefore parLapply) goes wrong when one of the arguments of the function is called "c". In this case, the argument "c" is used as cluster and the functions give the following error message "Error in checkCluster(cl) : not a valid cluster". Of course, "c" is for many reasons an unfortunate

Hello everybody, I've got a bit of a problem with parLapply that's left me scratching my head today. I've tried this in R 2.11 and the 23 bit Revolution R Enterprise and gotten the same result, OS in question is Windows XP, the package involved is the snow package. I've got a list of 20 rain/no rain (1/0) situations for these two stations i and j, all the items in this list look

On 03/15/2018 05:25 PM, Henrik Bengtsson wrote: > On Thu, Mar 15, 2018 at 3:39 AM, <FlorianSchwendinger at gmx.at> wrote: >> Thank you for your answer! >> I agree with you except for the 3 (Error) example and >> I realize now I should have started with that in the explanation. >> >> From my point of view >> parLapply(cl = clu, X = 1:2, fun = fun, c =

We got some errors and eventually figured out that parallel::clusterExport second argument is "varlist" while in snow::clusterExport it is "list". The user had loaded parallel first, but did something else which inadvertently loaded snow, then clusterExport failed because we had "varlist" and not "list". Are these different on purpose? pj -- Paul E.

In principle, I'd like to be able to do something like this: sge.parLapply(seq(10), function(x) parLapply(seq(x), function(x) x^2)) In practice, however, I have to resort to acrobatics like this: sge.options(sge.remove.files=FALSE) sge.options(sge.qsub.options='-cwd -V') sge.parLapply(seq(10), function(x) { sge.options(sge.save.global=TRUE)

invoking parLapply without a cluster fails to find a previously registered cluster > library(parallel) > setDefaultCluster(makePSOCKcluster(2)) > parLapply(X=1:2, fun=function(...) {}) Error in cut.default(i, breaks) : invalid number of intervals This is because in parLapply length(cl) is determined before defaultCluster(cl) is called. By inspection, this appears to be true of

Dear R users, I have some problems with the parLapply function from the "parallel" package: I use parLapply on a pretty big R object without changing the object within the called function. If I execute parLapply alone, everything works fine. It seems that the object resides only once in the memory. But if I use the same call within another function, the object seems to be multiplied to

While using parallelization R seems to clone all environments (that are normally passed by reference) that are returned from a child process. In particular, consider the following example: library(parallel) env1 <- new.env() envs2 <- lapply(1:4, function(x) env1) cl<-makeCluster(2, type="FORK") envs3 <- parLapply(cl, 1:4, function(x) env1) envs4 <- parLapply(cl, 1:4,

hi, recently i discovered the functionability summary.formula, awesome! from the help page i understand that method=reverse allows to summarize all variables on the right hand side of formula (the help page on line 229 wrongly refers to the left? hand side variables) in categories which are determined by a single left hand side variable. my problem is that the argument fun seems not to be