R devel - May 2021 - Testing R build when using --without-recommended-packages?

>>>>> Gabriel Becker 
>>>>>     on Tue, 4 May 2021 14:40:22 -0700 writes:
    > Hmm, that's fair enough Ben, I stand corrected.  I will say that
this seems
    > to be a pretty "soft" recommendation, as these things go,
given that it
    > isn't tested for by R CMD check, including with the -as-cran
extensions. In
    > principle, it seems like it could be, similar checks are made in
package
    > code for inappropriate external-package-symbol usage/

    > Either way, though, I suppose I have a number of packages which have
been
    > invisibly non-best-practices compliant for their entire lifetimes (or
at
    > least, the portion of that where they had tests/vignettes...).

    > Best,
    > ~G

    > On Tue, May 4, 2021 at 2:22 PM Ben Bolker <bbolker at gmail.com>
wrote:

    >> Sorry if this has been pointed out already, but some relevant text
    >> from
    >> 
    >>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages
    >> 
    >> > Note that someone wanting to run the examples/tests/vignettes
may not
    >> have a suggested package available (and it may not even be possible
to
    >> install it for that platform). The recommendation used to be to
make
    >> their use conditional via if(require("pkgname")): this is
OK if that
    >> conditioning is done in examples/tests/vignettes, although using
    >> if(requireNamespace("pkgname")) is preferred, if
possible.
    >> 
    >> ...
    >> 
    >> > Some people have assumed that a ?recommended? package in
?Suggests?
    >> can safely be used unconditionally, but this is not so. (R can be
    >> installed without recommended packages, and which packages are
    >> ?recommended? may change.)


Thank you all (Henrik, Gabe, Dirk & Ben) !

I think it would be a good community effort  and worth the time
also of R core to further move into the right direction
as Dirk suggested.

I think we all agree it would be nice if Henrik (and anybody)
could use  'make check' on R's own sources after using
 --without-recommended-packages

Even one more piece of evidence is the   tests/README   file in
the R sources.  It has much more but simply starts with

---------------------------------------------------------------------------
There is a hierarchy of check targets:

     make check

for all builders.  If this works one can be reasonably happy R is working
and do `make install' (or the equivalent).

    make check-devel

for people changing the code: this runs things like the demos and
no-segfault which might be broken by code changes, and checks on the
documentation (effectively R CMD check on each of the base packages).
This needs recommended packages installed.

    make check-all

runs all the checks, those in check-devel plus tests of the recommended
packages.

Note that for complete testing you will need a number of other
......................
......................

---------------------------------------------------------------------------

So, our (R core) own intent has been that   'make check'  should
run w/o rec.packages  but further checking not.

So, yes, please, you are encouraged to send patches against the
R devel trunk  to fix such examples and tests.

Best,
Martin

On 05/05/2021 5:13 a.m., Martin Maechler wrote:
>>>>>> Gabriel Becker
>>>>>>      on Tue, 4 May 2021 14:40:22 -0700 writes:
> 
>      > Hmm, that's fair enough Ben, I stand corrected.  I will say
that this seems
>      > to be a pretty "soft" recommendation, as these things
go, given that it
>      > isn't tested for by R CMD check, including with the -as-cran
extensions. In
>      > principle, it seems like it could be, similar checks are made in
package
>      > code for inappropriate external-package-symbol usage/
> 
>      > Either way, though, I suppose I have a number of packages which
have been
>      > invisibly non-best-practices compliant for their entire lifetimes
(or at
>      > least, the portion of that where they had tests/vignettes...).
> 
>      > Best,
>      > ~G
> 
>      > On Tue, May 4, 2021 at 2:22 PM Ben Bolker <bbolker at
gmail.com> wrote:
> 
>      >> Sorry if this has been pointed out already, but some relevant
text
>      >> from
>      >>
>      >>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages
>      >>
>      >> > Note that someone wanting to run the
examples/tests/vignettes may not
>      >> have a suggested package available (and it may not even be
possible to
>      >> install it for that platform). The recommendation used to be
to make
>      >> their use conditional via if(require("pkgname")):
this is OK if that
>      >> conditioning is done in examples/tests/vignettes, although
using
>      >> if(requireNamespace("pkgname")) is preferred, if
possible.
>      >>
>      >> ...
>      >>
>      >> > Some people have assumed that a ?recommended? package in
?Suggests?
>      >> can safely be used unconditionally, but this is not so. (R
can be
>      >> installed without recommended packages, and which packages
are
>      >> ?recommended? may change.)
> 
> 
> Thank you all (Henrik, Gabe, Dirk & Ben) !
> 
> I think it would be a good community effort  and worth the time
> also of R core to further move into the right direction
> as Dirk suggested.
> 
> I think we all agree it would be nice if Henrik (and anybody)
> could use  'make check' on R's own sources after using
>   --without-recommended-packages
> 
> Even one more piece of evidence is the   tests/README   file in
> the R sources.  It has much more but simply starts with
> 
> ---------------------------------------------------------------------------
> There is a hierarchy of check targets:
> 
>       make check
> 
> for all builders.  If this works one can be reasonably happy R is working
> and do `make install' (or the equivalent).
> 
>      make check-devel
> 
> for people changing the code: this runs things like the demos and
> no-segfault which might be broken by code changes, and checks on the
> documentation (effectively R CMD check on each of the base packages).
> This needs recommended packages installed.
> 
>      make check-all
> 
> runs all the checks, those in check-devel plus tests of the recommended
> packages.
> 
> Note that for complete testing you will need a number of other
> ......................
> ......................
> 
> ---------------------------------------------------------------------------
> 
> So, our (R core) own intent has been that   'make check'  should
> run w/o rec.packages  but further checking not.
> 
> So, yes, please, you are encouraged to send patches against the
> R devel trunk  to fix such examples and tests.
I think it would be useful to issue some kind of warning if tests are 
skipped.  As mentioned earlier, this is impossible in user-contributed 
packages, which can only return OK or ERROR from their tests.

Duncan Murdoch

On Wed, May 5, 2021 at 2:13 AM Martin Maechler
<maechler at stat.math.ethz.ch> wrote:
>
> >>>>> Gabriel Becker
> >>>>>     on Tue, 4 May 2021 14:40:22 -0700 writes:
>
>     > Hmm, that's fair enough Ben, I stand corrected.  I will say
that this seems
>     > to be a pretty "soft" recommendation, as these things
go, given that it
>     > isn't tested for by R CMD check, including with the -as-cran
extensions. In
>     > principle, it seems like it could be, similar checks are made in
package
>     > code for inappropriate external-package-symbol usage/
>
>     > Either way, though, I suppose I have a number of packages which
have been
>     > invisibly non-best-practices compliant for their entire lifetimes
(or at
>     > least, the portion of that where they had tests/vignettes...).
>
>     > Best,
>     > ~G
>
>     > On Tue, May 4, 2021 at 2:22 PM Ben Bolker <bbolker at
gmail.com> wrote:
>
>     >> Sorry if this has been pointed out already, but some relevant
text
>     >> from
>     >>
>     >>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages
>     >>
>     >> > Note that someone wanting to run the
examples/tests/vignettes may not
>     >> have a suggested package available (and it may not even be
possible to
>     >> install it for that platform). The recommendation used to be
to make
>     >> their use conditional via if(require("pkgname")):
this is OK if that
>     >> conditioning is done in examples/tests/vignettes, although
using
>     >> if(requireNamespace("pkgname")) is preferred, if
possible.
>     >>
>     >> ...
>     >>
>     >> > Some people have assumed that a ?recommended? package in
?Suggests?
>     >> can safely be used unconditionally, but this is not so. (R can
be
>     >> installed without recommended packages, and which packages are
>     >> ?recommended? may change.)
>
>
> Thank you all (Henrik, Gabe, Dirk & Ben) !
>
> I think it would be a good community effort  and worth the time
> also of R core to further move into the right direction
> as Dirk suggested.
>
> I think we all agree it would be nice if Henrik (and anybody)
> could use  'make check' on R's own sources after using
>  --without-recommended-packages
>
> Even one more piece of evidence is the   tests/README   file in
> the R sources.  It has much more but simply starts with
>
> ---------------------------------------------------------------------------
> There is a hierarchy of check targets:
>
>      make check
>
> for all builders.  If this works one can be reasonably happy R is working
> and do `make install' (or the equivalent).
>
>     make check-devel
>
> for people changing the code: this runs things like the demos and
> no-segfault which might be broken by code changes, and checks on the
> documentation (effectively R CMD check on each of the base packages).
> This needs recommended packages installed.
>
>     make check-all
>
> runs all the checks, those in check-devel plus tests of the recommended
> packages.
>
> Note that for complete testing you will need a number of other
> ......................
> ......................
>
> ---------------------------------------------------------------------------
>
> So, our (R core) own intent has been that   'make check'  should
> run w/o rec.packages  but further checking not.
>
> So, yes, please, you are encouraged to send patches against the
> R devel trunk  to fix such examples and tests.
Thanks Martin!  Thanks for confirming and for being open to patches.
This encourages me to try to patch what we've got so that 'make
check'
and 'make check-devel' can complete also without 'recommended'
packages.

/Henrik
>
> Best,
> Martin
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel