R help - Oct 2009 - Multicore package: sharing/modifying variable accross processes

Hi,

I want to parallelize some computations when it's possible on multicore 
machines.
Each computation produces a big objects that I don't want to store if 
not necessary: in the end only the object that best fits my data have to 
be returned. In non-parallel mode, a single gloabl object is updated if 
the current computation gets a better result than the best previously found.
My plan was to use package multicore. But there is obviously an issue of 
concurrent access to the global result variable.
Is there a way to implement something like a lock/mutex to ensure make 
the procedure thread safe?
Maybe something already exist to deal with such things?
It looks like package multicore run the different processes in different 
environments with copy-on-change of everything when forking. Anybody has 
experimented working with a shared environment with package multicore?

Thanks.

Renaud,

Package bigmemory can help you with shared-memory matrices, either in RAM or
filebacked.  Mutex support currently exists as part of the package, although
for various reasons will soon be abstracted from the package and provided
via a new package, synchronicity.

bigmemory works beautifully with multicore.  Feel free to email us with
questions, and we appreciate feedback.

Jay



Original message:

Hi,

I want to parallelize some computations when it's possible on multicore
machines.
Each computation produces a big objects that I don't want to store if
not necessary: in the end only the object that best fits my data have to
be returned. In non-parallel mode, a single gloabl object is updated if
the current computation gets a better result than the best previously found.
My plan was to use package multicore. But there is obviously an issue of
concurrent access to the global result variable.
Is there a way to implement something like a lock/mutex to ensure make
the procedure thread safe?
Maybe something already exist to deal with such things?
It looks like package multicore run the different processes in different
environments with copy-on-change of everything when forking. Anybody has
experimented working with a shared environment with package multicore?



-- 
John W. Emerson (Jay)
Associate Professor of Statistics
Department of Statistics
Yale University
http://www.stat.yale.edu/~jay

	[[alternative HTML version deleted]]