R help - Sep 2013 - Should I wrap more package examples in \dontrun{} ?

I have a package with more than 100 datasets, each of which has an
\examples{} section.  On the plus side, these example test the "R
ecosystem" to make sure that everything is working (both my package and
others' packages).  On the down side, changes in this ecosystem have caused
repeated NOTEs and WARNINGs from CRAN.

As a commentary on one recent R-help discussion, none of the code breakages
have been caused by use of ":::".  All of the problems have been
caused by
(1) Changes in "stable" packages and (2) changing CRAN requirements
(3)
changes in "beta" packages. In roughly that order.

In the interest of long-term package stability I'm thinking about wrapping
more of the examples in my package in \dontrun{}.  Especially the parts
that depend on other packages.

I'm interested to know how other package developers approach this problem.

-- 
Kevin Wright

	[[alternative HTML version deleted]]

On 03/09/2013 10:49 AM, Kevin Wright wrote:
> I have a package with more than 100 datasets, each of which has an
> \examples{} section.  On the plus side, these example test the "R
> ecosystem" to make sure that everything is working (both my package
and
> others' packages).  On the down side, changes in this ecosystem have
caused
> repeated NOTEs and WARNINGs from CRAN.
>
> As a commentary on one recent R-help discussion, none of the code breakages
> have been caused by use of ":::".  All of the problems have been
caused by
> (1) Changes in "stable" packages and (2) changing CRAN
requirements (3)
> changes in "beta" packages. In roughly that order.
>
> In the interest of long-term package stability I'm thinking about
wrapping
> more of the examples in my package in \dontrun{}.  Especially the parts
> that depend on other packages.
>
> I'm interested to know how other package developers approach this
problem.
>
As a user of your package, I would find it irritating if example(foo) 
didn't run anything.   It would be more irritating (and would indicate 
sloppiness on your part) if the examples failed when I cut and pasted 
them.  These both suggest leaving the examples running.

As the author of your package, it sounds as though you find it quite 
irritating when other authors break your code.

Isn't the right solution to this to work with the other package authors 
to come up with code that is unlikely to break?  If that's not possible, 
then maybe don't use those packages that cause you trouble.

Duncan Murdoch

On Tue, Sep 3, 2013 at 10:49 AM, Kevin Wright <kw.stat at gmail.com>
wrote:
> I have a package with more than 100 datasets, each of which has an
> \examples{} section.  On the plus side, these example test the "R
> ecosystem" to make sure that everything is working (both my package
and
> others' packages).  On the down side, changes in this ecosystem have
caused
> repeated NOTEs and WARNINGs from CRAN.
>
> As a commentary on one recent R-help discussion, none of the code breakages
> have been caused by use of ":::".  All of the problems have been
caused by
> (1) Changes in "stable" packages and (2) changing CRAN
requirements (3)
> changes in "beta" packages. In roughly that order.
>
> In the interest of long-term package stability I'm thinking about
wrapping
> more of the examples in my package in \dontrun{}.  Especially the parts
> that depend on other packages.
>
You can avoid checks with any of these:
1. wrap your example in:   if (interactive()) { ... }
2. turn your example into a demo
3. turn your example into a unit test using RUnit, svUnit or other
unit testing package

-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

To give a specific example, the simple code for my test suite is given at the
bottom of
this message.  A simpler (simple-minded maybe) approach than creating a new
packge for
testing.  I now run this on the survival package every time that I submit a new
version to
CRAN.  It takes a while, since there are over 200 dependencies.  It creates a
file
"progress" containing each package name as it is run folllowed by
either "Ok" or "Failed"
along with a directory "tests" containing the results.  Almost every
run generates 1-3 hits.
   I have not automated this further because many runs also lead to exceptions,
often
packages that won't load because I don't have some ancillary piece of
software installed
that they depend on.  (I can't seem to get JAVA set up sufficient to satisfy
everyone, for
example, and have very low motivation to work harder at the task.)  And a small
number
have made it to the bad actors "I give up don't even bother to
test" list.

Note that any package I want to fully test was installed on this local machine
using
     install.packages("xxx", dependencies=TRUE,
INSTALL_opts="--install-tests")
where "xxx" is the name of the package.

Terry T.


On 09/04/2013 05:00 AM, r-help-request at r-project.org
wrote:
> n 03/09/2013 1:53 PM, Hadley Wickham wrote:
>>> >  >  As a user of your package, I would find it irritating
if example(foo) didn't
>>> >  >  run anything.   It would be more irritating (and would
indicate sloppiness
>>> >  >  on your part) if the examples failed when I cut and
pasted them.  These both
>>> >  >  suggest leaving the examples running.
>>> >  >
>>> >  >  As the author of your package, it sounds as though you
find it quite
>>> >  >  irritating when other authors break your code.
>>> >  >
>>> >  >  Isn't the right solution to this to work with the
other package authors to
>>> >  >  come up with code that is unlikely to break?  If
that's not possible, then
>>> >  >  maybe don't use those packages that cause you
trouble.
>> >
>> >  It was my understanding that package authors are responsible for
not
>> >  breaking other CRAN packages without warning.  For example,
before I
>> >  release a new version of plyr or ggplot2, I run R CMD check on
every
>> >  package that depends on my package. I then let the maintainers
know if
>> >  something is broken - sometimes it's because I introduced a
bug, and
>> >  other times it's because I'm enforcing a stricter check
than I did
>> >  previously
> It sounds as though you're doing the right thing.   Can you describe
how
> you determine the set of packages to check, and how you do your checks?
> It would be great if we could convince everyone to follow those steps.
>
> Duncan Murdoch
tmt% cat checkdeps.R
require("tools")

# First set a repository to look at
#chooseCRANmirror()     # do it graphically
#chooseBioCmirror()
options(repos=c(CRAN="http://streaming.stat.iastate.edu/CRAN/",
                 BioC="http://bioconductor.org/packages/2.11/bioc/"))

# This function is provided by Uwe Wigges
reverse <-
function(packages, which = c("Depends", "Imports",
"LinkingTo"),
          recursive = FALSE)
{
     description <- sprintf("%s/web/packages/packages.rds",
                            getOption("repos")["CRAN"])
     con <- if(substring(description, 1L, 7L) == "file://")
         file(description, "rb")
     else
         url(description, "rb")
     on.exit(close(con))
     db <- readRDS(gzcon(con))
     rownames(db) <- NULL

     rdepends <- package_dependencies(packages, db, which,
                                      recursive = recursive,
                                      reverse = TRUE)
     rdepends <- sort(unique(unlist(rdepends)))
     pos <- match(rdepends, db[, "Package"], nomatch = 0L)

     db[pos, c("Package", "Version",
"Maintainer")]
}

survdep <- reverse("survival")[,1]

# I don't want to check coxme (since I maintain a more up to date
# local copy), and there are a few known bad actors
avoid <- c("coxme", "STAR", "compareGroups")
survdep <- survdep[is.na(match(survdep, avoid))]

# Some packages may have failed to install, don't test those
inplace <- installed.packages()[,"Package"]  #ones we already have
missed <-  is.na(match(survdep, inplace))
if (any(missed)) {
     message("Unable to load packages ",
             paste(survdep[missed], collapse=", "), "\n")
     survdep <- survdep[!missed]
}

# Do the long list of tests
unlink("progress")
unlink("tests", recursive=TRUE)
system("mkdir tests")
pfile <- file("progress", open="write")
for (testpkg in survdep) {
     z <- testInstalledPackage(testpkg, outDir="tests")
     cat(testpkg, c("Ok", "Failed")[z+1], "\n",
file=pfile)
}