R help - Feb 2007 - How to protect two jobs running on the same directory at the same time not to corrupt each other results:

Hi,

I have a large group of jobs, some of them are running on the same 
directory.  All of them in batch mode.
What are the best ways to protect from corrupting the results two or 
more jobs running on the same directory.
One, I would think can be to run each job in a separate directory, 
collect the results and after remove the directories. But I have 
thousands of jobs that will run in parallel and I have placed about 100 
of them in each directory. They all do the same process, but on 
different variables, replications etc.

Is there any other solution better than creating separate directories in 
R? I am thinking if there is any option in R to create a unique id which 
has its own unique .Rdata, although in the same directory?

SAS for example to each batch job it assigns a different ID and a 
separate temp space, and does not mix it with another job running in 
parallel.

Thanks,

Aldi

--

Am Donnerstag, 8. Februar 2007 21:33 schrieb Aldi Kraja:
> Is there any other solution better than creating separate directories in
> R? I am thinking if there is any option in R to create a unique id which
> has its own unique .Rdata, although in the same directory?
You could try using tempfile() in combination with basename(). From the 
helppage on tempfile():
    The names are very likely to be unique among calls to 'tempfile'
     in an R session and across simultaneous R sessions.  The filenames
     are guaranteed not to be currently in use.

So this is my suggestion for the generation of a useful, unique id in your 
working dir:

prefix <- paste("var_", var, sep = "")
while(file.exists(id <- basename(tempfile(pattern = prefix)))) {}
save(result, file = paste(id, ".Rdata", sep = ""))

Hope this helps.

Michel Lang

You can 'save' the objects produced by each BATCH
job in a file whose name relates to the job.  With this
technique, you can run as many BATCH jobs on the same
data as you like.

Once the jobs are done, then you can 'load' the files that
were saved.

Patrick Burns
patrick at burns-stat.com
+44 (0)20 8525 0696
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")

Aldi Kraja wrote:
>Hi,
>
>I have a large group of jobs, some of them are running on the same 
>directory.  All of them in batch mode.
>What are the best ways to protect from corrupting the results two or 
>more jobs running on the same directory.
>One, I would think can be to run each job in a separate directory, 
>collect the results and after remove the directories. But I have 
>thousands of jobs that will run in parallel and I have placed about 100 
>of them in each directory. They all do the same process, but on 
>different variables, replications etc.
>
>Is there any other solution better than creating separate directories in 
>R? I am thinking if there is any option in R to create a unique id which 
>has its own unique .Rdata, although in the same directory?
>
>SAS for example to each batch job it assigns a different ID and a 
>separate temp space, and does not mix it with another job running in 
>parallel.
>
>Thanks,
>
>Aldi
>
>--
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
>
>
>  
>