R devel - May 2021 - Testing R build when using --without-recommended-packages?

Hmm, that's fair enough Ben, I stand corrected.  I will say that this seems
to be a pretty "soft" recommendation, as these things go, given that
it
isn't tested for by R CMD check, including with the -as-cran extensions. In
principle, it seems like it could be, similar checks are made in package
code for inappropriate external-package-symbol usage/

Either way, though, I suppose I have a number of packages which have been
invisibly non-best-practices compliant for their entire lifetimes (or at
least, the portion of that where they had tests/vignettes...).

Best,
~G

On Tue, May 4, 2021 at 2:22 PM Ben Bolker <bbolker at gmail.com> wrote:
>
>    Sorry if this has been pointed out already, but some relevant text
> from
>
>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages
>
>  > Note that someone wanting to run the examples/tests/vignettes may not
> have a suggested package available (and it may not even be possible to
> install it for that platform). The recommendation used to be to make
> their use conditional via if(require("pkgname")): this is OK if
that
> conditioning is done in examples/tests/vignettes, although using
> if(requireNamespace("pkgname")) is preferred, if possible.
>
> ...
>
>  > Some people have assumed that a ?recommended? package in ?Suggests?
> can safely be used unconditionally, but this is not so. (R can be
> installed without recommended packages, and which packages are
> ?recommended? may change.)
>
>
>
> On 5/4/21 5:10 PM, Gabriel Becker wrote:
> > Hi Henrik,
> >
> > A couple of things. Firstly, so far asI have ever heard, it's
valid that
> a
> > package have hard dependencies in its tests for packages listed only
in
> > Suggests.  In fact, that is one of the stated purposes of Suggests. An
> > argument could be made, I suppose, that the base packages should be
under
> > stricter guidelines, but stats isn't violating the letter or
intention of
> > Suggests by doing this.
> >
> >
> > Secondly, I don't have time to dig through the make
files/administration
> > docs, but I do know that R CMD check has --no-stop-on-error, so you
can
> > either separately or as part of make check, use that option for stats
> (and
> > elsewhere as needed?) and just know that the stats tests that depend
on
> > MASS are "false positive" (or, more accurately, missing
value) test
> > results, rather than real positives, and go from there.
> >
> > You could also "patch" the tests as part of your build
process.
> Somewhere I
> > worked had to do that for parts of the internet tests that were unable
to
> > get through the firewall.
> >
> > Best,
> > ~G
> >
> >
> >
> > On Tue, May 4, 2021 at 1:04 PM Henrik Bengtsson <
> henrik.bengtsson at gmail.com>
> > wrote:
> >
> >> Two questions to R Core:
> >>
> >> 1. Is R designed so that 'recommended' packages are
optional, or
> >> should that be considered uncharted territories?
> >>
> >> 2. Can such an R build/installation be validated using existing
check
> >> methods?
> >>
> >>
> >> --
> >>
> >> Dirk, it's not clear to me whether you know for sure, or you
draw
> >> conclusions based your long experience and reading. I think
it's very
> >> important that others don't find this thread later on and read
your
> >> comments as if they're the "truth" (unless they
are).  I haven't
> >> re-read it from start to finish, but there are passages in 'R
> >> Installation and Administration' suggesting you can build and
install
> >> R without 'recommended' packages.  For example,
post-installation,
> >> Section 'Testing an Installation' suggests you can run
(after making
> >> sure `make install-tests`):
> >>
> >> cd tests
> >> ../bin/R CMD make check
> >>
> >> but they fail the same way.  The passage continuous "... and
other
> >> useful targets are test-BasePackages and test-Recommended to run
tests
> >> of the standard and recommended packages (if installed)
respectively."
> >> (*).  So, to me that hints at 'recommended' packages are
optional just
> >> as they're "Priority: recommended".  Further down,
there's also a
> >> mentioning of:
> >>
> >> $ R_LIBS_USER="" R --vanilla
> >>> Sys.setenv(LC_COLLATE = "C", LC_TIME =
"C", LANGUAGE = "en")
> >>> tools::testInstalledPackages(scope = "base")
> >>
> >> which also produces errors when 'recommended' packages are
missing,
> >> e.g. "Failed with error:  'there is no package called
'nlme'".
> >>
> >> (*) BTW, '../bin/R CMD make test-BasePackages' gives
"make: *** No
> >> rule to make target 'test-BasePackages'.  Stop."
> >>
> >> Thanks,
> >>
> >> /Henrik
> >>
> >> On Tue, May 4, 2021 at 12:22 PM Dirk Eddelbuettel <edd at
debian.org>
> wrote:
> >>>
> >>>
> >>> On 4 May 2021 at 11:25, Henrik Bengtsson wrote:
> >>> | FWIW,
> >>> |
> >>> | $ ./configure --help
> >>> | ...
> >>> |   --with-recommended-packages
> >>> |                           use/install recommended R packages
[yes]
> >>>
> >>> Of course. But look at the verb in your Subject: no
optionality _in
> >> testing_ there.
> >>>
> >>> You obviously need to be able to build R itself to then build
the
> >> recommended
> >>> packages you need for testing.
> >>>
> >>> Dirk
> >>>
> >>> --
> >>> https://dirk.eddelbuettel.com | @eddelbuettel | edd at
debian.org
> >>
> >> ______________________________________________
> >> R-devel at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]

>>>>> Gabriel Becker 
>>>>>     on Tue, 4 May 2021 14:40:22 -0700 writes:
    > Hmm, that's fair enough Ben, I stand corrected.  I will say that
this seems
    > to be a pretty "soft" recommendation, as these things go,
given that it
    > isn't tested for by R CMD check, including with the -as-cran
extensions. In
    > principle, it seems like it could be, similar checks are made in
package
    > code for inappropriate external-package-symbol usage/

    > Either way, though, I suppose I have a number of packages which have
been
    > invisibly non-best-practices compliant for their entire lifetimes (or
at
    > least, the portion of that where they had tests/vignettes...).

    > Best,
    > ~G

    > On Tue, May 4, 2021 at 2:22 PM Ben Bolker <bbolker at gmail.com>
wrote:

    >> Sorry if this has been pointed out already, but some relevant text
    >> from
    >> 
    >>
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Suggested-packages
    >> 
    >> > Note that someone wanting to run the examples/tests/vignettes
may not
    >> have a suggested package available (and it may not even be possible
to
    >> install it for that platform). The recommendation used to be to
make
    >> their use conditional via if(require("pkgname")): this is
OK if that
    >> conditioning is done in examples/tests/vignettes, although using
    >> if(requireNamespace("pkgname")) is preferred, if
possible.
    >> 
    >> ...
    >> 
    >> > Some people have assumed that a ?recommended? package in
?Suggests?
    >> can safely be used unconditionally, but this is not so. (R can be
    >> installed without recommended packages, and which packages are
    >> ?recommended? may change.)


Thank you all (Henrik, Gabe, Dirk & Ben) !

I think it would be a good community effort  and worth the time
also of R core to further move into the right direction
as Dirk suggested.

I think we all agree it would be nice if Henrik (and anybody)
could use  'make check' on R's own sources after using
 --without-recommended-packages

Even one more piece of evidence is the   tests/README   file in
the R sources.  It has much more but simply starts with

---------------------------------------------------------------------------
There is a hierarchy of check targets:

     make check

for all builders.  If this works one can be reasonably happy R is working
and do `make install' (or the equivalent).

    make check-devel

for people changing the code: this runs things like the demos and
no-segfault which might be broken by code changes, and checks on the
documentation (effectively R CMD check on each of the base packages).
This needs recommended packages installed.

    make check-all

runs all the checks, those in check-devel plus tests of the recommended
packages.

Note that for complete testing you will need a number of other
......................
......................

---------------------------------------------------------------------------

So, our (R core) own intent has been that   'make check'  should
run w/o rec.packages  but further checking not.

So, yes, please, you are encouraged to send patches against the
R devel trunk  to fix such examples and tests.

Best,
Martin