R devel - Dec 2013 - Strategies for keeping autogenerated .Rd files out of a Git tree

Hi

Quite a few R packages are now available on GitHub long before they 
appear on CRAN, installation is simple thanks to 
devtools::install_github(). However, it seems to be common practice to 
keep the .Rd files (and NAMESPACE and the Collate section in the 
DESCRIPTION) in the Git tree, and to manually update it, even if they 
are autogenerated from the R code by roxygen2. This requires extra work 
for each update of the documentation and also binds package development 
to a specific version of roxygen2 (because otherwise lots of bogus 
changes can be added by roxygenizing with a different version).

What options are there to generate the .Rd files during build/install? 
In https://github.com/hadley/devtools/issues/43 the issue has been 
discussed, perhaps it can be summarized as follows:

- The devtools package is not the right place to implement 
roxygenize-before-build
- A continuous integration service would be better for that, but 
currently there's nothing that would be easy to use
- Roxygenizing via src/Makefile could work but requires further 
investigation and an installation of Rtools/xcode on Windows/OS X

Especially the last point looks interesting to me, but since this is not 
widely used there must be pitfalls I'm not aware of. The general idea 
would be:

- Place code that builds/updates the .Rd and NAMESPACE files into 
src/Makefile
- Users installing the package from source will require infrastructure 
(Rtools/make)
- For binary packages, the .Rd files are already generated and added to 
the .tar.gz during R CMD build before they are submitted to 
CRAN/WinBuilder, and they are also generated (in theory) by R CMD build 
--binary

I'd like to hear your opinion on that. I have also found a thread on 
package development workflow 
(https://stat.ethz.ch/pipermail/r-devel/2011-September/061955.html) but 
there was nothing on un-versioning .Rd files.


Cheers

Kirill

Hi,

this is maybe mostly a personal preference, but I prefer not to put
generated files in the vc repository. Changes in the generated files,
especially if there is many of them, pollute the diffs and make them
less useful.

If you really want to be able to install the package directly from
github, one solution is to
1. create another repository, that contains the complete generated
package, so that install_github() can install it.
2. set up a CI service, that can download the package from github,
build the package or the generated files (check the package, while it
is at it), and then push the build stuff back to github.
3. set up a hook on github, that invokes the CI after each commit.

I have used this setup in various projects with jenkins-ci and it
works well. Diffs are clean, the package is checked and built
frequently, and people can download it without having to install the
tools that generate the generated files.

The only downside is that you need to install a CI, so you need a
"server" for that. Maybe you can do this with travis-ci, maybe not, I
am not familiar with it that much.

Best,
Gabor

On Wed, Dec 11, 2013 at 7:39 PM, Kirill M?ller
<kirill.mueller at ivt.baug.ethz.ch> wrote:
> Hi
>
> Quite a few R packages are now available on GitHub long before they appear
> on CRAN, installation is simple thanks to devtools::install_github().
> However, it seems to be common practice to keep the .Rd files (and
NAMESPACE
> and the Collate section in the DESCRIPTION) in the Git tree, and to
manually
> update it, even if they are autogenerated from the R code by roxygen2. This
> requires extra work for each update of the documentation and also binds
> package development to a specific version of roxygen2 (because otherwise
> lots of bogus changes can be added by roxygenizing with a different
> version).
>
> What options are there to generate the .Rd files during build/install? In
> https://github.com/hadley/devtools/issues/43 the issue has been discussed,
> perhaps it can be summarized as follows:
>
> - The devtools package is not the right place to implement
> roxygenize-before-build
> - A continuous integration service would be better for that, but currently
> there's nothing that would be easy to use
> - Roxygenizing via src/Makefile could work but requires further
> investigation and an installation of Rtools/xcode on Windows/OS X
>
> Especially the last point looks interesting to me, but since this is not
> widely used there must be pitfalls I'm not aware of. The general idea
would
> be:
>
> - Place code that builds/updates the .Rd and NAMESPACE files into
> src/Makefile
> - Users installing the package from source will require infrastructure
> (Rtools/make)
> - For binary packages, the .Rd files are already generated and added to the
> .tar.gz during R CMD build before they are submitted to CRAN/WinBuilder,
and
> they are also generated (in theory) by R CMD build --binary
>
> I'd like to hear your opinion on that. I have also found a thread on
package
> development workflow
> (https://stat.ethz.ch/pipermail/r-devel/2011-September/061955.html) but
> there was nothing on un-versioning .Rd files.
>
>
> Cheers
>
> Kirill
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

On 12/11/2013 4:39 PM, Kirill M?ller wrote:
> Hi
>
> Quite a few R packages are now available on GitHub long before they
> appear on CRAN, installation is simple thanks to
> devtools::install_github(). However, it seems to be common practice to
> keep the .Rd files (and NAMESPACE and the Collate section in the
> DESCRIPTION) in the Git tree, and to manually update it, even if they
> are autogenerated from the R code by roxygen2. This requires extra work
> for each update of the documentation and also binds package development
> to a specific version of roxygen2 (because otherwise lots of bogus
> changes can be added by roxygenizing with a different version).
>
> What options are there to generate the .Rd files during build/install?
> In https://github.com/hadley/devtools/issues/43 the issue has been
> discussed, perhaps it can be summarized as follows:
>
> - The devtools package is not the right place to implement
> roxygenize-before-build
> - A continuous integration service would be better for that, but
> currently there's nothing that would be easy to use
> - Roxygenizing via src/Makefile could work but requires further
> investigation and an installation of Rtools/xcode on Windows/OS X
>
> Especially the last point looks interesting to me, but since this is not
> widely used there must be pitfalls I'm not aware of. The general idea
> would be:
>
> - Place code that builds/updates the .Rd and NAMESPACE files into
> src/Makefile
> - Users installing the package from source will require infrastructure
> (Rtools/make)
> - For binary packages, the .Rd files are already generated and added to
> the .tar.gz during R CMD build before they are submitted to
> CRAN/WinBuilder, and they are also generated (in theory) by R CMD build
> --binary
>
> I'd like to hear your opinion on that. I have also found a thread on
> package development workflow
> (https://stat.ethz.ch/pipermail/r-devel/2011-September/061955.html) but
> there was nothing on un-versioning .Rd files.
One downside I can see with this third approach is that by making the 
package documentation generation part of the build process, you must 
then make the package depend/require roxygen (or whatever tools you are 
using to generate documentation). This dependence, though, is just to 
build the package, not to actually use the package. And by pushing this 
dependency onto the end users of the package, you have transferred the 
problem you mentioned ("... and also binds package development to a 
specific version of roxygen2 ...") to the many end users rather than the 
few developers.
> Cheers
>
> Kirill
>

-- 
Brian S. Diggs, PhD
Senior Research Associate, Department of Surgery
Oregon Health & Science University