Displaying 20 results from an estimated 700 matches similar to: "[R-sig-hpc] doMC - compiler - concatenate an expression vector into a single expression?"
2011 Jun 28
1
doMC - compiler - concatenate an expression vector into a single expression?
Hi,
this post is about foreach operators, the compiler package and the last
update of doMC that includes support for the compiler functionality.
I am using a home-made %dopar%-like operator that adds some custom
expression to be executed before the foreach loop expression itself (see
sample code below).
It used to work perfectly with doMC 1.2.1, but with the introduction of
the compiler
2011 Aug 17
1
R cmd check and multicore foreach loop
Hi,
in R 2.12.1, R CMD check hangs when building a vignette that uses a
foreach loop with the doMC parallel backend.
This does not happen in R 2.13.1, nor if I use doSEQ instead of doMC.
All versions of multicore, doMC and foreach are the same on both my R
installations.
Has anybody encountered a similar issue?
Thank you.
Renaud
###
UNIVERSITY OF CAPE TOWN
This e-mail is subject to the
2010 Nov 03
1
Auto-killing processes spawned by foreach::doMC
Hi all,
Sometimes I'll find myself "ctrl-c"-ing like a madman to kill some
code that's parallelized via foreach/doMC when I realized that I just
set my cpu off to do something boneheaded, and it will keep doing that
thing for a while.
In these situations, since I interrupted its normal execution,
foreach/doMC doesn't "clean up" after itself by killing the
2013 Feb 01
0
R code parallelized using plyr and doMC: error message: Error in do.ply(i) : task 1 failed - “could not find function ”getClass“”
Dear list,
I'm just getting started learning how to use remote supercomputers for
execution of parallelized code. I got a lot of initial help from this
<http://stackoverflow.com/questions/14553357/parallelizing-on-a-supercomputer-and-then-combining-the-parallel-results-r>
previous post, as well as one particularly helpful and patient XSEDE guy.
I'm only using one node (for the
2009 Jul 01
0
Parallel programming packages iterators, foreach and doMC released
REvolution Computing has just released three new packages for R to
CRAN (under the open-source Apache 2.0 license): foreach, iterators,
and doMC. Together, they provide a simple, scalable parallel computing
framework for R that lets you take advantage of your multicore or
multiprocessor workstation to program loops that run faster than
traditional loops in R.
The three packages build on each
2009 Jul 01
0
Parallel programming packages iterators, foreach and doMC released
REvolution Computing has just released three new packages for R to
CRAN (under the open-source Apache 2.0 license): foreach, iterators,
and doMC. Together, they provide a simple, scalable parallel computing
framework for R that lets you take advantage of your multicore or
multiprocessor workstation to program loops that run faster than
traditional loops in R.
The three packages build on each
2010 Nov 11
0
logging interim results using foreach/doMC
Dear all,
I am converting a large process to a parallel backhend using doMC and
foreach. Basically, I havea long list of input graph files and each of
them calls soem basic igraph package functions. I am parallelizing the
run, in order to save time. All works fine, and each %dopar% call ends
with a vector of results that at the end got fed into a data frame and
saved as a csv table.
When I
2011 Oct 17
2
Foreach (doMC)
Hello,
I am trying to run a small example with foreach, but I am having some
problems. Here is the code:
*library(doMC)
registerDoMC()
zappa = list()
frank = list()
foreach (i = 1:4) %dopar% {
zappa[[i]] = kmeans (iris[-5],4)
frank[[i]] = warnings()
}*
The code runs without error. However the zappa and frank will be empty
lists.
If I use regular *for *instead, the list will be filled up
2010 Nov 16
2
Debugging segfault in foreach
Hi,
I'm using R-2.12 on a linux 64bit machine.
When I run a chunk of code inside a foreach() %do% { ...} or %dopar%
{...} (with doMC backend) I keep getting a segfault. Running the
*same* code within lapply(something, function(x) ... ) doesn't result
in any segfaults. I'll paste the output below, but I'm not sure it
would be helpful.
I'm more curious how to go about smoking
2011 Jul 04
1
writeLines + foreach/doMC
Hi
I'm processing sequencing data trying to collapsing the locations of each
unique sequence and write the results to a file (as storing that in a table
will require 10GB mem at least)
so I wrote a function that, given a sequence id, provide the needed line to
be stored
library(doMC) # load library
registerDoMC(12) # assign the Number of CPU
2011 Aug 27
3
Exception while using NeweyWest function with doMC
Dear R users,
I am using R right now for a simulation of a model that needs a lot of
memory. Therefore I use the *bigmemory* package and - to make it faster -
the *doMC* package. See my code posted on http://pastebin.com/dFRGdNrG
Now, if I use the foreach loop with the addon %do% (for sequential run) I
have no problems at all - only here and there some singularities in
regressor matrices which
2011 Nov 28
1
Avoid package in build process when not supported on OS
Dear all,
I am currently working on a package which involves some simulation where no current simulation run depends on a previous simulation run.
That is why I decided to parallelize the computation using the doMC package (which exists only for unix-like OS).
I can create a package without any R CMD check and R CMD build errors on my computers (Ubuntu Linux 32bit & 64 bit).
The problem
2023 Mar 13
0
scan(..., skip=1e11): infinite loop; cannot interrupt
With
?if?(!j--)?{
?????R_CheckUserInterrupt();
?????j?=?10000;
?}
as?in?current?R?devel?(r83976),?j goes negative (-1) and interrupt is checked every 10001 instead of 10000. I?prefer
?if?(!--j)?{
?????R_CheckUserInterrupt();
?????j?=?10000;
?}
.
In?current?R?devel?(r83976),?if?EOF?is?reached,?the?outer?loop?keeps?going,?i?keeps?incrementing?until?nskip.
2023 Feb 11
1
scan(..., skip=1e11): infinite loop; cannot interrupt
On Fri, 10 Feb 2023 23:38:55 -0600
Spencer Graves <spencer.graves at prodsyse.com> wrote:
> I have a 4.54 GB file that I'm trying to read in chunks using
> "scan(..., skip=__)". It works as expected for small values of
> "skip" but goes into an infinite loop for "skip=1e11" and similar
> large values of skip: I cannot even interrupt it; I
2011 Feb 11
1
foreach with registerDoMC on R 2.12.0 OSX 10.6 --- errors and warnings
some hints for the search engines.
I just did
install.packages("foreach")
install.packages("doMC")
library(doMC)
registerDoMC()
library(foreach)
> foreach(i = 1:3) %dopar% sqrt(i)
The process has forked and you cannot use this CoreFoundation
functionality safely. You MUST exec().
Break on
2006 May 09
0
is it possible share memory between domains like shmget, shmat??
hi all
am a newbie to xen. was wondering if there is a way to share memory between domains. i need to be able to write to a shared memory from one domain and allow other domain(s) to read from it. for example domA is always the writer and domB or domC are always readers. to be able to do that does hypervisor provide anything like "shmget" for domA and "shmat" for domB
2012 Feb 20
1
bigmemory not really parallel
Hi, all,
I have a really big matrix that I want to run k-means on.
I tried:
>data <-
read.big.memory('mydata.csv',type='double',backingfile='mydata.bin',descriptorfile='mydata.desc')
I'm using doMC to register multicore.
>library(doMC)
>registerDoMC(cores=8)
>ans<-bigkmeans(data,k)
In system monitor, it seems only one thread running R. Is
2015 Feb 09
2
R CMD check: Uses the superseded package: ‘doSNOW’
Dear list,
When I run an R CMD check --as-cran on my package (pROC) I get the
following note:
> Uses the superseded package: ?doSNOW?
The fact that it uses the doSNOW package is correct as I have the
following example in an .Rd file:
> #ifdef windows
> if (require(doSNOW)) {
> registerDoSNOW(cl <- makeCluster(2, type = "SOCK"))
> ci(roc2,
2015 Feb 10
0
R CMD check: Uses the superseded package: ‘doSNOW’
The CRAN package snow is superseded by the parallel package which is
distributed with R since version 2.14.0. Here are the release notes
\item There is a new package \pkg{parallel}.
It incorporates (slightly revised) copies of packages
\CRANpkg{multicore} and \CRANpkg{snow} (excluding MPI, PVM and NWS
clusters). Code written to use the higher-level API functions in
those packages should
2011 Feb 27
0
foreach() package for parallel computing
dear R experts---I have been experimenting with the foreach package
(with doMC) for a while.
my first impression is that it is a very easy way to acquire parallel
processing capabilities. (thanks, revolution R.) the only two
gotchas were about installation (it required an exit and restart), and
the precedence order of the foreach (higher than '+', I think), but
once I understood this,