R devel - Mar 2014 - Varying results of package checks due to random seed

> From: Philippe GROSJEAN <Philippe.GROSJEAN at umons.ac.be>
>
> ... for latest CRAN version, we have successfully installed 4999
> packages among the 5321 CRAN package on our platform. ... It is
> strange that a large portion of R CMD check errors on CRAN occur and
> disappear *without any version update* of a package or any of its
> direct or indirect dependencies! That is, a fraction of errors or
> warnings seem to appear and disappear without any code update. 
Some of these are likely the result of packages running tests using
random number generation without setting the random numbers seed, in
which case the seed is set based on the current time and process id,
with an obvious possibility of results varying from run to run.

In the current development version of pqR (in branch 19-mods, found at
https://github.com/radfordneal/pqR/tree/19-mods), I have implemented a
change so that if the R_SEED environment variable is set, the random
seed is initialized to its value, rather than from the time and
process id.  This was motivated by exactly this problem - I can now
just set R_SEED to something before running all the package checks.

   Radford Neal

Hi, developers

When I run `R CMD INSTALL --clean my_package`, only the .o files and the .so
file are removed, however, they were deleted not because of the clean target
defined in Makevars.
The clean target was indeed not executed.

I have the following simple Marvars to show that clean target is not
functioning.

Makevars
-----------------------------

all: foo $(SHLIB)

clean: 
	rm -f *.o $(SHLIB)
	rm -f b

foo:
	touch b


After I run ``R CMD INSTALL --clean my_package``, the file `b` is still there.

Any thought?

Best,
Randy

On 03/22/2014 01:32 PM, Radford Neal wrote:
>> From: Philippe GROSJEAN <Philippe.GROSJEAN at umons.ac.be>
>>
>> ... for latest CRAN version, we have successfully installed 4999
>> packages among the 5321 CRAN package on our platform. ... It is
>> strange that a large portion of R CMD check errors on CRAN occur and
>> disappear *without any version update* of a package or any of its
>> direct or indirect dependencies! That is, a fraction of errors or
>> warnings seem to appear and disappear without any code update.
>
> Some of these are likely the result of packages running tests using
> random number generation without setting the random numbers seed, in
> which case the seed is set based on the current time and process id,
> with an obvious possibility of results varying from run to run.
>
> In the current development version of pqR (in branch 19-mods, found at
> https://github.com/radfordneal/pqR/tree/19-mods), I have implemented a
> change so that if the R_SEED environment variable is set, the random
> seed is initialized to its value, rather than from the time and
> process id.  This was motivated by exactly this problem - I can now
> just set R_SEED to something before running all the package checks.
Beware, if you are serious about reproducing things, that you really 
need to save information about the uniform and other generators you use, 
such as the normal generator. The defaults do not change often, but have 
in the past, and could in the future if something better comes along. 
There are some small utilities and examples in the package setRNG which 
can help.

Also remember that you need to beware of a side effect of the 
environment variable approach. It is great for reproducing things, as 
you would want to do in package tests, but be careful how you use it in 
functions as it may mess up the randomness if you always set the seed to 
the same starting value.

Paul
>
>     Radford Neal
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>