similar to: Parallelizing cor() for large data-set using Cluster

Hello, Is the first time I am using SNOW package and I am trying to tune the cost parameter for a linear SVM, where the cost (variable cost1) takes 10 values between 0.5 and 30. I have a large dataset and a pc which is not very powerful, so I need to tune the parameters using both CPUs of the pc. Somehow I cannot manage to do it. It seems that both CPUs are fitting the model for the same values

I have this function and want to run it parallel with different sets of data. Using SNOW and clusterApplyLB. system.time(out <- mclapply(cData, plotGraph)) #each cData contains 100X6000 doubles system.time(out <- mclapply(cData2, plotGraph)) system.time(out <- mclapply(cData3, plotGraph)) system.time(out <- mclapply(cData4, plotGraph)) system.time(out <- mclapply(cData5,

Hello and thanks in advance for your time. I've created a simulation on my cluster which uses a custom package developed by me for different functions and also the snow package. Right now I'm using LAM to communicate between nodes and am currently only testing my code on 3 nodes for simplicity, though I plan on expanding to 16 later. My problem is this error: "Error in

What is the best way to produce a barplot from my data? I would like the barplot to show each person with the values stacked val1+val2+val3, so there is one bar for each person When I use barplot(data.matrix(realdata)), it shows one bar for each value instead. To post here, I created an artificical data set, but it works fine. fakedata <- as.data.frame(list(LETTERS[1:3])) colnames(fakedata)

[cross-posted on R-devel and Bioc-devel, since the functions from the parallel package discussed here are mirrored in the BiocGenerics package] Hi, I am currently running a lengthy simulation study (no details necessary) on a large multi-core system. The simulated data sets are stored in a long list and they are unevenly sized (hence, the computation times vary greatly between data sets), so

Dear R-Users, we are using a linux-cluster with RMPI and the snow package. We would like to do a double boostrap. We have a general function that implements the first boostrap (the outer) and we are wondering if we can include another bootstrap (the inner) in the same general function including another clusterApplyLB. For example: general function = function(...) { clusterApplyLB(cl,

We got some errors and eventually figured out that parallel::clusterExport second argument is "varlist" while in snow::clusterExport it is "list". The user had loaded parallel first, but did something else which inadvertently loaded snow, then clusterExport failed because we had "varlist" and not "list". Are these different on purpose? pj -- Paul E.

Greetings, I am having an interesting problem and I wonder if anyone else has seen this behavior. I am running R 2.11.1 with SNOW 0.3-3 on a Dell cluster running CentOS 5.5. I create my cluster using: cluster<- makeCluster(nodes,type="SOCK",port=10191) # nodes is a vector of compute nodes I then wrap a loop around clusterApplyLB to evaluate my function multiple times, with

I am trying to do parallel programming and I tried this library(doSNOW) library(foreach) testfunc<-function(x){ x<-x+1 x } noc<-2 cl <- makeCluster(do.call(rbind,rep(list("localhost"),noc)), type = "SOCK") registerDoSNOW(cl) clusterExport(cl=cl,c("testfunc.r")) testl<-foreach(pp=1:2) %dopar% { testfunc(pp) } And this works but if I try to

Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped data, so for the original there is more data and I show the arithmetic mean with standard deviation. The

Hello, I have a rather large set of data I need to analyze, currently I need to work with a 200000 by 200000 matrix, I'm using the package bigmemory but so far I can only allocate a 66000 by 66000 matrix, when I increase those values I get the following error:   > AdjMat <- big.matrix(nrow=68000,ncol=68000) Cannot allocate memory BigMatrix.cpp line 225 Error in big.matrix(nrow = 68000,

Thanks for the report. Will look into it when I get a chance unless someone else gets there first. A simpler reprex: ## create and serialize a memmory-mapped file object filePath <- "x.dat" con <- file(filePath, "wrb") writeBin(rep(0.0,10),con) close(con) library(simplemmap) x <- mmap(filePath, "double") saveRDS(x, file = "x.Rds") ## in a

Hi all, I am not able to export an ALTREP object when `gctorture` is on in the worker. The package simplemmap can be used to reproduce the problem. See the example below ``` ## Create a temporary file filePath <- tempfile() con <- file(filePath, "wrb") writeBin(rep(0.0,10),con) close(con) library(simplemmap) library(parallel) cl <- makeCluster(1) x <- mmap(filePath,

Hello R-Help, ----------------------------------------------------------------------------------------------------------------------------------------- Issues (there are 2): 1) Possible bug when using lmtest::encomptest() with a linear model created using nlme::lmList() 2) Possible modification to lmtest::encomptest() to fix confusing fail when models provided are, in fact, nested. I have

Dear Henrik, The rationale is just that it is within these extremes and that it is really simple to calculate, without making any assumptions and knowing that it won't be perfect. The extremes A and B you are mentioning are special cases based on assumptions. Case A is based on the assumption that the function has a long runtime or varying runtime, then you are likely to get the best load

R Version : 2.14.1 x64 Running on Windows 7 Connecting to a database on a remote Microsoft SQL Server 2012 The short form of my problem is the following. I have an unordered vectors of names, say: names<-c("A", "B", "A", "C","C") each of which have an id in a table in my db. I need to convert the names to their corresponding ids. I

Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it (attached). A similar fix has also been provided [here][2]. [1]:

Whas missing the R in the command line: "mpirun -n --hostfile /home/hostfile R --no-save -f rtest.R" Hope this helps mario On 13-Jan-11 22:08, Justin Moriarty wrote: > Hi, > Just wanted to share a working example of doSNOW and foreach for an openMPI cluster. The function eddcmp() is just an example and returns some inocuous warnings. The

I found that also; fixed in r79386 in the trunk. Will port to R-patched shortly. Best, luke On Thu, 29 Oct 2020, Martin Morgan wrote: > This > > Index: src/main/altrep.c > =================================================================== > --- src/main/altrep.c (revision 79385) > +++ src/main/altrep.c (working copy) > @@ -275,10 +275,11 @@ > SEXP psym =

Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let us know ;) Best Regards On 02/26/2018 04:01 PM, Tomas Kalibera wrote: > Dear Christian and Henrik, > > thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP