R devel - Sep 2012 - R-devel Digest, Vol 115, Issue 18

> In general, as a package user, I don't want people to be able to
> suppress checks on CRAN.  I want things fixed.
>
> So I am pretty sure there won't ever be a reliable
"CRAN-detector" put
> into R.  It would devalue the brand.
>
> Duncan Murdoch
My problem is that CRAN demands that I suppress a large fraction of my checks,
in order to
fit within time constraints.  This leaves me with 3 choices.

1. Add lines to my code that tries to guess if CRAN is invoker.  A cat and mouse
game per
your desire above.

2. Remove large portions of my test suite.  I consider the survival package to
be one of
the pre-eminent current code sets in the world precisely because of the
extensive
validations, this action would change it to a second class citizen.

3. Add a magic environment variable to my local world, only do the full tests if
it is
present, and make the dumbed down version the default.  Others who want to run
the full
set are then SOL, which I very much don't like.

I agree that CRAN avoidence, other than the time constraint, should be verboten.
But I
don't think that security through obscurity is the answer.  And note that
under scenario
3, which is essentially what is currently being forced on us, I can do such
micshief as
easily as under number 1.

Terry Therneau

On 19/09/2012 10:08 AM, Terry Therneau wrote:
> > In general, as a package user, I don't want people to be able to
> > suppress checks on CRAN.  I want things fixed.
> >
> > So I am pretty sure there won't ever be a reliable
"CRAN-detector" put
> > into R.  It would devalue the brand.
> >
> > Duncan Murdoch
>
> My problem is that CRAN demands that I suppress a large fraction of my
checks, in order to
> fit within time constraints.  This leaves me with 3 choices.
>
> 1. Add lines to my code that tries to guess if CRAN is invoker.  A cat and
mouse game per
> your desire above.
>
> 2. Remove large portions of my test suite.  I consider the survival package
to be one of
> the pre-eminent current code sets in the world precisely because of the
extensive
> validations, this action would change it to a second class citizen.
>
> 3. Add a magic environment variable to my local world, only do the full
tests if it is
> present, and make the dumbed down version the default.  Others who want to
run the full
> set are then SOL, which I very much don't like.
>
> I agree that CRAN avoidence, other than the time constraint, should be
verboten.  But I
> don't think that security through obscurity is the answer.  And note
that under scenario
> 3, which is essentially what is currently being forced on us, I can do such
micshief as
> easily as under number 1.
>
> Terry Therneau
I understand the issue with time constraints on checks, and I think 
there are discussions in progress about that.  My suggestion has been to 
put in place a way for a tester to say that checks need to be run within 
a tight time limit, and CRAN as tester will do that in cases where it 
cares about the timing.

But even with the current (or past, it may have changed already) 
behaviour of tight limits for CRAN testing, you can put in code in your 
package that allows for longer tests in certain conditions. You'll run 
them, and if you advertise them, others will run them too.

Duncan Murdoch

( subject changed from Re: [Rd] R-devel Digest, Vol 115, Issue 18 )

I have the impression from this, and previous discussions on the 
subject, that package developers and CRAN maintainers are talking at 
cross-purposes. Many package maintainers are thinking that they should 
be responsible for choosing which tests are run and which are not run by 
CRAN, whereas CRAN maintainers may want to run all possible tests 
sometimes, or a trimmed down set when time constraints demand this. With 
good reason, CRAN may want to run all possible tests sometimes. There 
are too many packages on CRAN that remain there because they don't have 
any testing or vignettes, and very few examples. Encouraging more of 
that is a bad thing.

If I understand correctly, the --as-cran option was introduced to help 
developers specify options that CRAN uses, so they would find problems 
that CRAN would notice, and correct before submitting. The Rd 
discussions of this have morphed into a discussion of how package 
developers can use --as-cran to control which tests are run by CRAN.

I tend to be more sympathetic with what I call the CRAN maintainer view 
above, even though I am a package developer. I think packages should 
have extensive testing and that all the tests should go in the source 
package on CRAN, so the testing is available for CRAN and everyone else. 
(Although, it is sometimes not clear if CRAN maintainers like me doing 
this, because they are torn between time demands and maintaining quality 
- that is part of the confusion.)

The question becomes: how does information get passed along to indicate 
things that may take a long time to run. The discussion so far has 
focused on developers setting, or using, some flags to indicate tests 
and examples that take a long time. Another option would be to have the 
check/build process generate a file with information about the time it 
took to run tests, vignettes, and examples, probably with some 
information about the speed of the machine it was run on. Then CRAN and 
anyone else that wants to run tests can take this information into 
consideration.

Paul

On 12-09-19 10:08 AM, Terry Therneau wrote:
>> In general, as a package user, I don't want people to be able to
>> suppress checks on CRAN.  I want things fixed.
>>
>> So I am pretty sure there won't ever be a reliable
"CRAN-detector" put
>> into R.  It would devalue the brand.
>>
>> Duncan Murdoch
>
> My problem is that CRAN demands that I suppress a large fraction of my
> checks, in order to fit within time constraints.  This leaves me with 3
> choices.
>
> 1. Add lines to my code that tries to guess if CRAN is invoker.  A cat
> and mouse game per your desire above.
>
> 2. Remove large portions of my test suite.  I consider the survival
> package to be one of the pre-eminent current code sets in the world
> precisely because of the extensive validations, this action would change
> it to a second class citizen.
>
> 3. Add a magic environment variable to my local world, only do the full
> tests if it is present, and make the dumbed down version the default.
> Others who want to run the full set are then SOL, which I very much
> don't like.
>
> I agree that CRAN avoidence, other than the time constraint, should be
> verboten.  But I don't think that security through obscurity is the
> answer.  And note that under scenario 3, which is essentially what is
> currently being forced on us, I can do such micshief as easily as under
> number 1.
>
> Terry Therneau
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel