R devel - Oct 2007 - Regression test failed when building on an "older" Linux cluster

We have an aging Linux Cluster here, running Red Hat 7.3. We have some
business reasons for not upgrading the OS version. I don't recall the exact
hardware (dual Pentium III, 4 Gbyte RAM, 1 GHz clock?), but it was pretty
good in 2001 or so.

We recently tried to build and install R, ver 2.6.0, for this cluster.
It built and apparently ran correctly, but it failed "make check".
Earlier versions of R failed "make check" as well, not always for the
same
reason.

I just located the failure in the tests.
It is in the file: R-2.6.0/tests/reg-tests-1.R and fails in the code:

## related checks on eff.aovlist
example(eff.aovlist) # helmert contrasts
eff1 <- eff.aovlist(fit)
fit <- aov(Yield ~ A * B * C + Error(Block), data = aovdat)
eff2 <- eff.aovlist(fit)
stopifnot(all.equal(eff1, eff2)) # will have rounding-error differences
options(contrasts = old)
## Were different in earlier versions

The failure occurred in the 'stopifnot()' call, but was preceded by a
warning about a lack of documentation in "example(eff.aovlist)". I
looked at
"example" and found that it relies on available documentation to work,
returning this warning when no documentation is found:

    file <- index.search(topic, INDICES, "AnIndex",
"R-ex")
    if (file == "") {
        warning(gettextf("no help file found for '%s'",
topic),
            domain = NA)
        return(invisible())

Well, our old Linux cluster doesn't support building R documentation; why
would it when we are very limited on disk space for each node? No one can
log in there; it is all run via LSF. The lack of documentation makes
"example()" fail to return anything and the "reg-tests-1.R"
requires that a
value for 'fit' be returned in the global environment for comparison. In
short, in the absence of documentation, this regression test must fail. (I
am assuming that my interpretation of the failure is correct; please feel
free to let me know that I am mis-interpreting the cause of failure!).

If I comment out the above code, all the other tests run correctly. Of
course, normally we would build on another, better equipped host, and then
copy R to each host. That is not possible in this case as the glibc is so
old that it is not compatible with the calls generated on newer hosts.

I am surprised that this problem with "example()" doesn't cause
other
failures in the "make check". It makes me think that the use of
"example()"
here is unusual (or that I am misinterpreting the failure). If that is the
case, then I suggest removing the call to "example" and replacing it
with
the code from within that example. I believe that running R on a compute
cluster without documentation building tools is not an unreasonable use
case. (If that is so, why haven't others stumbled onto this problem before
me? I don't know.) Making the regression tests work in the absence of
documentation, along with the rest of R seems reasonable.

We are now using R on the compute cluster. This email's purpose is to
suggest that R may at times be built in an environment without documentation
tools and it would be nice if the regression tests still worked there. I
hope the developers will consider this use case.

Regards,
Mark Dalphin

PS Having written this email, I am consumed by guilt; I haven't properly and
recently checked the "R admin manual" that I should be able to build R
without documentation. Checking for "essential programs for building and
running R" doesn't seem to show that building documentation is
required. I
am relieved to report that building documentation, despite being under
"Essential Programs" section is described in a manner suggesting that
documentation is optional.
http://cran.r-project.org/doc/manuals/R-admin.html#Essential-and-useful-othe
r-programs-under-Unix

----------------------
Mark Dalphin
Dept Comp Biol, M/S AW2/D3262
Amgen, Inc.
1201 Amgen Court W
Seattle, WA 98119
Phone: +1-206-265-7951

	[[alternative HTML version deleted]]

We don't offer the option of building R without documentation, so I 
presume you do not have Perl installed.

Without the examples, there is no check of large swathes of code, so 'make 
check' should not succeed (it would give a false sense of security to pass 
most tests by omitting them).  I've altered it so it will always fail 
without Perl.

I don't buy the disc space argument: you don't have to _install_ the 
documentation on each node, and 'make check' runs on the build and not
the
installed copy.  We don't offer the possibility of installing a cut-down 
version, but surely someone with such a specialized need will be able to 
script that?  And if disc space is tight, why have a copy on each node? 
(We don't on our clusters, where space is not tight: it makes maintenance 
of packages so much easier to have a single shared copy.)

On Thu, 18 Oct 2007, Dalphin, Mark wrote:
> We have an aging Linux Cluster here, running Red Hat 7.3. We have some
> business reasons for not upgrading the OS version. I don't recall the
exact
> hardware (dual Pentium III, 4 Gbyte RAM, 1 GHz clock?), but it was pretty
> good in 2001 or so.
>
> We recently tried to build and install R, ver 2.6.0, for this cluster.
> It built and apparently ran correctly, but it failed "make
check".
> Earlier versions of R failed "make check" as well, not always for
the same
> reason.
>
> I just located the failure in the tests.
> It is in the file: R-2.6.0/tests/reg-tests-1.R and fails in the code:
>
> ## related checks on eff.aovlist
> example(eff.aovlist) # helmert contrasts
> eff1 <- eff.aovlist(fit)
> fit <- aov(Yield ~ A * B * C + Error(Block), data = aovdat)
> eff2 <- eff.aovlist(fit)
> stopifnot(all.equal(eff1, eff2)) # will have rounding-error differences
> options(contrasts = old)
> ## Were different in earlier versions
>
> The failure occurred in the 'stopifnot()' call, but was preceded by
a
> warning about a lack of documentation in "example(eff.aovlist)".
I looked at
> "example" and found that it relies on available documentation to
work,
> returning this warning when no documentation is found:
>
>    file <- index.search(topic, INDICES, "AnIndex",
"R-ex")
>    if (file == "") {
>        warning(gettextf("no help file found for '%s'",
topic),
>            domain = NA)
>        return(invisible())
>
> Well, our old Linux cluster doesn't support building R documentation;
why
> would it when we are very limited on disk space for each node? No one can
> log in there; it is all run via LSF. The lack of documentation makes
> "example()" fail to return anything and the
"reg-tests-1.R" requires that a
> value for 'fit' be returned in the global environment for
comparison. In
> short, in the absence of documentation, this regression test must fail. (I
> am assuming that my interpretation of the failure is correct; please feel
> free to let me know that I am mis-interpreting the cause of failure!).
>
> If I comment out the above code, all the other tests run correctly. Of
> course, normally we would build on another, better equipped host, and then
> copy R to each host. That is not possible in this case as the glibc is so
> old that it is not compatible with the calls generated on newer hosts.
>
> I am surprised that this problem with "example()" doesn't
cause other
> failures in the "make check". It makes me think that the use of
"example()"
> here is unusual (or that I am misinterpreting the failure). If that is the
> case, then I suggest removing the call to "example" and replacing
it with
> the code from within that example. I believe that running R on a compute
> cluster without documentation building tools is not an unreasonable use
> case. (If that is so, why haven't others stumbled onto this problem
before
> me? I don't know.) Making the regression tests work in the absence of
> documentation, along with the rest of R seems reasonable.
>
> We are now using R on the compute cluster. This email's purpose is to
> suggest that R may at times be built in an environment without
documentation
> tools and it would be nice if the regression tests still worked there. I
> hope the developers will consider this use case.
>
> Regards,
> Mark Dalphin
>
> PS Having written this email, I am consumed by guilt; I haven't
properly and
> recently checked the "R admin manual" that I should be able to
build R
> without documentation. Checking for "essential programs for building
and
> running R" doesn't seem to show that building documentation is
required. I
> am relieved to report that building documentation, despite being under
> "Essential Programs" section is described in a manner suggesting
that
> documentation is optional.
>
http://cran.r-project.org/doc/manuals/R-admin.html#Essential-and-useful-othe
> r-programs-under-Unix
>
> ----------------------
> Mark Dalphin
> Dept Comp Biol, M/S AW2/D3262
> Amgen, Inc.
> 1201 Amgen Court W
> Seattle, WA 98119
> Phone: +1-206-265-7951
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595