similar to: using SNOW and clusterApplyLB to run jobs parallel

Hi. I have a 925 by 925 correlation matrix corM. I want to identify all variables that have correlation greater than 0.9. Can anyone suggest an "R way" of doing this? Thank you. -- View this message in context: http://www.nabble.com/search-through-a-matrix-tp23153538p23153538.html Sent from the R help mailing list archive at Nabble.com.

Dear R-Users, we are using a linux-cluster with RMPI and the snow package. We would like to do a double boostrap. We have a general function that implements the first boostrap (the outer) and we are wondering if we can include another bootstrap (the inner) in the same general function including another clusterApplyLB. For example: general function = function(...) { clusterApplyLB(cl,

Hi, I am trying to convert a 2D correlation matrix to 3 columns for graphical representation: rdata = replicate(100, rnorm(15)) #construct a 2D matrix c1 = cor(rdata) #outputs a correlation matrix Now I want to convert the 2D c1 to (row#, col#, correlation) 1 1 cor1 1 2 cor2 1 3 cor3 ... 2 1 cor.. Is there a way to do this? The main reason I am doing this is to find a correlation based graph

Greetings, 1) Is there a nice way of extracting the variance estimates from an lme fit? They don't seem to be part of the lme object. 2) In a series of simulations, I am finding that with ML fitting one of my random effect variances is sometimes being estimated as essentially zero with massive CI instead of the finite value it should have, whilst using REML I get the expected value. I guess

Greetings, I am having an interesting problem and I wonder if anyone else has seen this behavior. I am running R 2.11.1 with SNOW 0.3-3 on a Dell cluster running CentOS 5.5. I create my cluster using: cluster<- makeCluster(nodes,type="SOCK",port=10191) # nodes is a vector of compute nodes I then wrap a loop around clusterApplyLB to evaluate my function multiple times, with

[cross-posted on R-devel and Bioc-devel, since the functions from the parallel package discussed here are mirrored in the BiocGenerics package] Hi, I am currently running a lengthy simulation study (no details necessary) on a large multi-core system. The simulated data sets are stored in a long list and they are unevenly sized (hence, the computation times vary greatly between data sets), so

Dear all, Can anybody tell me how to export a 3d figure made with the plot3d function? I'm careless about whether it's still interactive or not in another format, as long I can get it out of R. Thanks! Alejandro Gonz?lez Departamento de Biodiversidad y Conservaci?n Real Jard?n Bot?nico Consejo Superior de Investigaciones Cient?ficas Claudio Moyano, 1 28014 Madrid, Spain Tel +0034

Hi, I am currently building an R package and I am facing a peculiar problem where some of the functions does not work within the package. However, if I source the script the function works. For example, in a method for parallelization of analysis on each chromosome simultaneously I am receiving error at the following position of the code: # this profile the information chromosome wise and

Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it (attached). A similar fix has also been provided [here][2]. [1]:

Hello, Is the first time I am using SNOW package and I am trying to tune the cost parameter for a linear SVM, where the cost (variable cost1) takes 10 values between 0.5 and 30. I have a large dataset and a pc which is not very powerful, so I need to tune the parameters using both CPUs of the pc. Somehow I cannot manage to do it. It seems that both CPUs are fitting the model for the same values

Hi, I'm trying to understand the rationale for your proposed amount of splitting and more precisely why that one is THE one. If I put labels on your example numbers in one of your previous post: nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up the processing in chunks such that all workers are utilized: (A) Each worker, called

Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped data, so for the original there is more data and I show the arithmetic mean with standard deviation. The

Dear Christian and Henrik, thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static scheduling via internal function staticClusterApply, yet with a fixed chunk size; parLapplyLB already

I'm trying to use snow in my dual-core (hopefully later this is going to run in a cluster). So, at this moment I create a cluster using SOCK connection (MPI in the future). However when I try to use clusterApplyLB I got "Error in socketSelect(socklist) : not a socket connection". Any ideas ? Do you know if that is going to be an isuue too when I swith from SOCK to MPI ? Sample code

Looking at parLapplyLB, one sees that it takes in X and then passes splitList(X, length(cl)) to clusterApplyLB, which then calls dynamicClusterApply. Thus while dynamicClusterApply does handle tasks in a load-balancing fashion, sending out individual tasks as previous tasks complete, parLapplyLB preempts that by splitting up the tasks in advance into as many groups of tasks as there are cluster

Dear Henrik, The rationale is just that it is within these extremes and that it is really simple to calculate, without making any assumptions and knowing that it won't be perfect. The extremes A and B you are mentioning are special cases based on assumptions. Case A is based on the assumption that the function has a long runtime or varying runtime, then you are likely to get the best load

Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let us know ;) Best Regards On 02/26/2018 04:01 PM, Tomas Kalibera wrote: > Dear Christian and Henrik, > > thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP

Hello and thanks in advance for your time. I've created a simulation on my cluster which uses a custom package developed by me for different functions and also the snow package. Right now I'm using LAM to communicate between nodes and am currently only testing my code on 3 nodes for simplicity, though I plan on expanding to 16 later. My problem is this error: "Error in

Hi, I am running my code in a cluster at Arizona State University. I have a huge climate data, 66000 X 500 I am not sure if I can find correlation of such a huge data in the cluster. Normally I allocate 20000M and operate on 5 X 20000. Even this is taking lot of time. Is there any way I can find cl = cor(cdata) utilizing the computers in the clusters(I am using 32 nodes ). I am using the

Hi, Here are the issues I found: Typos ----- (a) Found: It a parallel version of ?evalq?, "is" missing. (b) Found: 'parLapplyLB', 'parSapplyLB' are load-balancing versions, intended for use when applying ?FUN? to 'parLapplyLB' has no 'FUN' arg (more on this below). (c) Found: 'clusterApply' calls 'fun' on the first