similar to: Package ff and parallel processing

REvolution Computing has just released three new packages for R to CRAN (under the open-source Apache 2.0 license): foreach, iterators, and doMC. Together, they provide a simple, scalable parallel computing framework for R that lets you take advantage of your multicore or multiprocessor workstation to program loops that run faster than traditional loops in R. The three packages build on each

REvolution Computing has just released three new packages for R to CRAN (under the open-source Apache 2.0 license): foreach, iterators, and doMC. Together, they provide a simple, scalable parallel computing framework for R that lets you take advantage of your multicore or multiprocessor workstation to program loops that run faster than traditional loops in R. The three packages build on each

Hi all, Sometimes I'll find myself "ctrl-c"-ing like a madman to kill some code that's parallelized via foreach/doMC when I realized that I just set my cpu off to do something boneheaded, and it will keep doing that thing for a while. In these situations, since I interrupted its normal execution, foreach/doMC doesn't "clean up" after itself by killing the

dear R experts--- I am experimenting with multicore processing, so far with pretty disappointing results. Here is my simple example: A <- 100000 randvalues <- abs(rnorm(A)) minfn <- function( x, i ) { log(abs(x))+x^3+i/A+randvalues[i] } ?## an arbitrary function ARGV <- commandArgs(trailingOnly=TRUE) if (ARGV[1] == "do-onecore") { ?library(foreach) ?discard <-

Hi, in R 2.12.1, R CMD check hangs when building a vignette that uses a foreach loop with the doMC parallel backend. This does not happen in R 2.13.1, nor if I use doSEQ instead of doMC. All versions of multicore, doMC and foreach are the same on both my R installations. Has anybody encountered a similar issue? Thank you. Renaud ### UNIVERSITY OF CAPE TOWN This e-mail is subject to the

Hi Does anybody know how to create ff objects with data reading from stream objects, such as data reading from PostgreSQL database through RPostgreSQL. For this purpose although we can save the data to a csv file through external tools and then read it through csv readers, but it requires one more data read and write operation, which is of high I/O cost for large datasets. Xiaobo.Gu

Hello dear R community. I just started playing with the snowfall package (a wrapper for the snow package), and found it very convenient. (See also this great website: http://www.imbi.uni-freiburg.de/parallel/ ) I was wondering if it is possible to connect snowfall with the foreach package (since it has some connection to snow). My final goal is to do some simple parallel simulations on my two

On 2009-07-01, David M Smith <david at revolution-computing.com> wrote: > REvolution Computing has just released three new packages for R to > CRAN (under the open-source Apache 2.0 license): foreach, iterators, > and doMC. Together, they provide a simple, scalable parallel computing > framework for R that lets you take advantage of your multicore or > multiprocessor

The CRAN package snow is superseded by the parallel package which is distributed with R since version 2.14.0. Here are the release notes \item There is a new package \pkg{parallel}. It incorporates (slightly revised) copies of packages \CRANpkg{multicore} and \CRANpkg{snow} (excluding MPI, PVM and NWS clusters). Code written to use the higher-level API functions in those packages should

Oh, I completely missed that one. It's very neat as it seems to work both on Windows and Unix. Thanks! Xavier On 10/02/15 10:52, Martyn Plummer wrote: > The CRAN package snow is superseded by the parallel package which is > distributed with R since version 2.14.0. Here are the release notes > > \item There is a new package \pkg{parallel}. > > It incorporates (slightly

dear R experts---I have been experimenting with the foreach package (with doMC) for a while. my first impression is that it is a very easy way to acquire parallel processing capabilities. (thanks, revolution R.) the only two gotchas were about installation (it required an exit and restart), and the precedence order of the foreach (higher than '+', I think), but once I understood this,

People of R(th), I have been ramming my head against this problem, and I wondered if anyone could lend a hand. I want to parallelize a bootstrap of a linear model on my 8-core mac. Below is the process that I want to parallelize (namely, the m2.ph.rlm.boot<-boot(m2.ph,m2.ph.fun, R = nboot) command). This is an extension of the bootstrapping linear models example in Venables and Ripley to

Dear all, Just wondering if someone could help me out converting my code from a for() loop into a foreach() loop or using one of the apply() function. I have a very large dataset and so I'm hoping to make use of a parallel backend to speed up the processing time. I'm having trouble getting selecting three variables in the dataset to use in the foreach() loops. My for() loop code is:

The attached code produces corrupted pdfs (test2.pdf, test4.pdf and test5.pdf). The resulting pdf depends on how many cores are available on the machine. I don't see why there should be any difference between the pdfs (exept for the timestamp). Doing many operations involving mclapply can increase the size of the resulting pdf by ten times! Thank you for checking this. require(multicore)

Hi, all, I have a really big matrix that I want to run k-means on. I tried: >data <- read.big.memory('mydata.csv',type='double',backingfile='mydata.bin',descriptorfile='mydata.desc') I'm using doMC to register multicore. >library(doMC) >registerDoMC(cores=8) >ans<-bigkmeans(data,k) In system monitor, it seems only one thread running R. Is

foreach (or virtually anything you might use for concurrent programming) only really makes sense if the work the "clients" are doing is substantial enough to overwhelm the communication overhead. And there are many ways to accomplish the same task more or less efficiently (for example, doing blocks of tasks in chunks rather than passing each one as an individual job). But more to the

On Tue, Apr 28, 2020 at 9:00 PM Shian Su <su.s at wehi.edu.au> wrote: > > Thanks Simon, > > I will take note of the sensible default for core usage. I?m trying to achieve small scale parallelism, where tasks take 1-5 seconds and make fuller use of consumer hardware. Its not a HPC-worthy computation but even laptops these days come with 4 cores and I don?t see a reason to not make

Hello MasteRs- Because I want to parallelize several calls to the glmulti package, what I'm essentially doing is trying to parallelize different calls to rJava. I'm using plyr functions which use foreach and then doMC which means multicore is my backend for parallelizing. I've tried several approaches to this but have not succeeded, i also find virtually no record of folks trying

Hi, I would like to get suggestion about parallel computing package on a multicore windows workstation. I tried doSMP, but it crashes R a lot. I am wondering if "snow" and "snowfall" can be used on a single workstation (i,e, not cluster). At suggestion would be appreciated, Best, Richard [[alternative HTML version deleted]]

Hi, The read.csv.ffdf function in package ff will create the ff object physical file in the default directories, I am trying to let the files created in the paths users specify, I think the point is to make use of the asffdf_args parameter, I have a test CSV file named D:\rtemp\fftest.csv, the content of the file is as following: col1,col2,col3 1,"amber",2.4 2,"linda",4.5