Hello community/devs, Is there an option to run package tests during R CMD check and not stop on first error? I know that testing frameworks (testhat and others) can do that but asking about just R and base packages. Currently when package check runs test scripts in ./tests directory it will stop after first fail. Do you think it could be optionally available to continue to run tests after failures? Regards, Jan Gorecki [[alternative HTML version deleted]]
Martin Maechler
2016-Nov-03 10:45 UTC
[Rd] Running package tests and not stop on first fail
>>>>> Jan Gorecki <J.Gorecki at wit.edu.pl> >>>>> on Tue, 1 Nov 2016 22:51:28 +0000 writes:> Hello community/devs, Is there an option to run package > tests during R CMD check and not stop on first error? I > know that testing frameworks (testhat and others) can do > that but asking about just R and base packages. Currently > when package check runs test scripts in ./tests directory > it will stop after first fail. Do you think it could be > optionally available to continue to run tests after > failures? Regards, Jan Gorecki I agree that this would be a useful option sometimes. So I would be supportive to get such an option, say, R CMD check --no-stop-on-error <pkg> into R if someone provided (relatively small) patches to the R sources (i.e. subversion repos at https://svn.r-project.org/R/trunk/ ). The relevant source code should basically all be in src/library/tools/R/testing.R Note that this may be complicated, also because "parallel" checking is available in parts, via the TEST_MC_CORES environment variable ((which is currently only quickly documented in the 'R Administration ..' manual)) Martin Maechler ETH Zurich
Hi Martin, Jan, On 11/03/2016 03:45 AM, Martin Maechler wrote:>>>>>> Jan Gorecki <J.Gorecki at wit.edu.pl> >>>>>> on Tue, 1 Nov 2016 22:51:28 +0000 writes: > > > Hello community/devs, Is there an option to run package > > tests during R CMD check and not stop on first error? I > > know that testing frameworks (testhat and others) can do > > that but asking about just R and base packages. Currently > > when package check runs test scripts in ./tests directory > > it will stop after first fail. Do you think it could be > > optionally available to continue to run tests after > > failures? Regards, Jan Gorecki > > I agree that this would be a useful option sometimes. > > So I would be supportive to get such an option, say, > > R CMD check --no-stop-on-error <pkg>A couple of years ago the behavior of 'R CMD check' was changed to continue checking (e.g. the examples) after many types of errors, and to output a summary count of errors at the end if any have occurred. So --no-stop-on-error could easily be interpreted as an option that controls this behavior (and would also suggest that the default has been reverted back to what it was prior to R 3.2.0), rather than an option that specifically controls what should happen while running the tests. Cheers, H.> > into R if someone provided (relatively small) patches to the R > sources (i.e. subversion repos at https://svn.r-project.org/R/trunk/ ). > The relevant source code should basically all be in > src/library/tools/R/testing.R > > Note that this may be complicated, also because "parallel" > checking is available in parts, via the TEST_MC_CORES > environment variable ((which is currently only quickly > documented in the 'R Administration ..' manual)) > > > Martin Maechler > ETH Zurich > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >-- Herv? Pag?s Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpages at fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319