R devel - Nov 2004 - Building Packages on Windows using .Rbuildignore (PR#7379)

On Thu, 18 Nov 2004 00:38:47 +0000 (UTC), Gabor Grothendieck
<ggrothendieck@myway.com> wrote :
>DIFFERENCE BETWEEN USING .RBUILDIGNORE AND NOT
>
>The reason that the processing is different according to whether one
>uses .Rbuildignore or not is that 
>  R CMD build 
>takes the .Rbuildignore file into account but 
>  R CMD install
>  R CMD check
>  R CMD build --binary 
>do not take .Rbuildignore into account.  
Okay, now I understand.  I think I'd call the last of those a bug, and
it would seem to me that the install and check scripts should also
respect this directive.   I've now copied this to the r-bugs list.
(This was reported for Windows; I don't know if it applies to other
platforms as well.)

Just for clarification:  I can see you would use this when you have
S-PLUS code in the same directory as R code, and you don't want to
include that in an R build.  Are there other files that must be
excluded?

Duncan Murdoch

murdoch@stats.uwo.ca wrote:
> On Thu, 18 Nov 2004 00:38:47 +0000 (UTC), Gabor Grothendieck
> <ggrothendieck@myway.com> wrote :
> 
> 
>>DIFFERENCE BETWEEN USING .RBUILDIGNORE AND NOT
>>
>>The reason that the processing is different according to whether one
>>uses .Rbuildignore or not is that 
>> R CMD build 
>>takes the .Rbuildignore file into account but 
>> R CMD install
>> R CMD check
>> R CMD build --binary 
>>do not take .Rbuildignore into account.  
> 
> 
> Okay, now I understand.  I think I'd call the last of those a bug, and
> it would seem to me that the install and check scripts should also
> respect this directive.   I've now copied this to the r-bugs list.
> (This was reported for Windows; I don't know if it applies to other
> platforms as well.)
> 
> Just for clarification:  I can see you would use this when you have
> S-PLUS code in the same directory as R code, and you don't want to
> include that in an R build.  Are there other files that must be
> excluded?
I'm not sure if you meant this as a Windows question. On Unix/Linux:
CVS for sure, and if you run things in you source directly and 
occassionaly get core dumps then it is pretty import to ignore them or 
you get very large packages. (Yes it would be better to clean up.) I 
also ignore several files that get generate compiling src code, or in 
some cases files that generate things that are included (For example, I 
generate man from a mansrc directory.) My  .Rbuildignore is

^Makefile$
(^|.*/)core$
(^|.*/)run$
(^|.*/)zot$
CVS
(^|.*/)/CVS$
(^|.*/)/(^|.*/)/CVS$
TMP$
FLAGS$
mansrc$
guidesrc$
zot$
run$
inst/doc/(^|.*)\.tex$
inst/doc/(^|.*)\.R$
inst/doc/(^|.*)\.log$
inst/doc/(^|.*)\.stdout$
exec/(^|.*)\.pro
exec/getpadi
exec/putpadi
exec/objectinfo
exec/padi_simple_svc
exec/getpwuid
exec/x11arima
exec/(^|.*)server.log.(^|.*)
data/create

Paul Gilbert
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-devel@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

>>>>> "Duncan" == Duncan Murdoch
<murdoch@stats.uwo.ca>
>>>>>     on Thu, 18 Nov 2004 16:36:03 +0100 (CET) writes:
    Duncan> On Thu, 18 Nov 2004 00:38:47 +0000 (UTC), Gabor
    Duncan> Grothendieck <ggrothendieck@myway.com> wrote :

    >> DIFFERENCE BETWEEN USING .RBUILDIGNORE AND NOT
    >> 
    >> The reason that the processing is different according to
    >> whether one uses .Rbuildignore or not is that R CMD build
    >> takes the .Rbuildignore file into account but R CMD
    >> install R CMD check R CMD build --binary do not take
    >> .Rbuildignore into account.

    Duncan> Okay, now I understand.  I think I'd call the last
    Duncan> of those a bug, and it would seem to me that the
    Duncan> install and check scripts should also respect this
    Duncan> directive.  I've now copied this to the r-bugs list.

    Duncan> (This was reported for Windows; I don't know if it
    Duncan> applies to other platforms as well.)

Yes it does (*), but I think it is NOT a bug but a feature,
at least for "check" and "install" (*) 
and very much desired in some cases :

For instance, the package developer may want more
regression tests in <Pkg>/tests/ :

1) Have extra *.Rout.save files that are architecture
   dependent and hence not for general distribution of the
   package, but important for the package developer in order to
   assure that functionality doesn't change when the package is
   extended, reorganized, ....

2) Have more  tests/*.R  files  that take a long time to run.
   Time that the package developer wants to spend, but doesn't
   dare to put on the daily CRAN or Bioconductor test suites.

3) similarly for vignettes

4) similar issues for experimental  R/*.R files  or man/*.Rd
   files for these.

One I thing that would be quite useful and would even solve
Gabor's problem: 
The introduction of a new command line switch, say "--build-ignore",
to the commands 'R CMD check' and 'R CMD install'

--------

(*) I do agree that  "R CMD build --binary" probably really should
    follow the ".Rbuildignore" file ``directives'' if it
doesn't
    currently.

Martin

<murdoch <at> stats.uwo.ca> writes:

: 
: On Thu, 18 Nov 2004 00:38:47 +0000 (UTC), Gabor Grothendieck
: <ggrothendieck <at> myway.com> wrote :
: 
: >DIFFERENCE BETWEEN USING .RBUILDIGNORE AND NOT
: >
: >The reason that the processing is different according to whether one
: >uses .Rbuildignore or not is that 
: >  R CMD build 
: >takes the .Rbuildignore file into account but 
: >  R CMD install
: >  R CMD check
: >  R CMD build --binary 
: >do not take .Rbuildignore into account.  
: 
: Okay, now I understand.  I think I'd call the last of those a bug, and
: it would seem to me that the install and check scripts should also
: respect this directive.   I've now copied this to the r-bugs list.
: (This was reported for Windows; I don't know if it applies to other
: platforms as well.)
: 
: Just for clarification:  I can see you would use this when you have
: S-PLUS code in the same directory as R code, and you don't want to
: include that in an R build.  Are there other files that must be
: excluded?

My primary directory is mirrored by svn and I put everything
in there that I want other developers to be able to see,
even if its not ready or not intended for inclusion in the
build.  

The .Rbuildignore is important to distinguish what is to be
built and what is just there to be shared.

Some instances where this useful are:
- have abc.R but don't want it in build until abc.Rd is available
- have abc.Rd but don't want it in build until abc.R is written
- abc.R is available but too buggy yet it needs to be shared with
  others so they know about it
- interface code which interfaces with package X and will 
  ultimately go into package X, not this one, but needs to be
  around in the meantime
- code which depends on a package that we don't want to invoke
  for now and will probably be replaced anyways
- certain notes to be shared among the developers

Also it would really be nice to be able to turn the
.Rbuildignore on and off using a command line switch on 
R CMD ... as someone on the thread has already suggested.  It
can be useful to check and install the ignored stuff to
ensure its ready before actually removing it from
.Rbuildignore.  It might even be nice to be able to specify
alternate .Rbuildignore files although I am not sure whether
that is going too far or not.

What I am doing in the meantime is that I have a Windows
batch script that can be used like this:

   myscript install mypkg
   myscript check mypkg
   myscript binary mypkg

where each of these first does a build creating a .tar.gz
file and then detars it into another directory and invokes
install, check or build --binary on the built copy of the
source in that directory.

If I want to do an install, check or build --binary without
respecting the .Rbuildignore then I just use the original
R CMD ... commands instead of my script.

On Thu, 18 Nov 2004 19:32:25 +0100, Martin Maechler
<maechler@stat.math.ethz.ch> wrote:
>>>>>> "Duncan" == Duncan Murdoch
<murdoch@stats.uwo.ca>
>>>>>>     on Thu, 18 Nov 2004 16:36:03 +0100 (CET) writes:
>
>    Duncan> On Thu, 18 Nov 2004 00:38:47 +0000 (UTC), Gabor
>    Duncan> Grothendieck <ggrothendieck@myway.com> wrote :
>
>    >> DIFFERENCE BETWEEN USING .RBUILDIGNORE AND NOT
>    >> 
>    >> The reason that the processing is different according to
>    >> whether one uses .Rbuildignore or not is that R CMD build
>    >> takes the .Rbuildignore file into account but R CMD
>    >> install R CMD check R CMD build --binary do not take
>    >> .Rbuildignore into account.
>
>    Duncan> Okay, now I understand.  I think I'd call the last
>    Duncan> of those a bug, and it would seem to me that the
>    Duncan> install and check scripts should also respect this
>    Duncan> directive.  I've now copied this to the r-bugs list.
>
>    Duncan> (This was reported for Windows; I don't know if it
>    Duncan> applies to other platforms as well.)
>
>Yes it does (*), but I think it is NOT a bug but a feature,
>at least for "check" and "install" (*) 
>and very much desired in some cases :
>
>For instance, the package developer may want more
>regression tests in <Pkg>/tests/ :
>
>1) Have extra *.Rout.save files that are architecture
>   dependent and hence not for general distribution of the
>   package, but important for the package developer in order to
>   assure that functionality doesn't change when the package is
>   extended, reorganized, ....
>
>2) Have more  tests/*.R  files  that take a long time to run.
>   Time that the package developer wants to spend, but doesn't
>   dare to put on the daily CRAN or Bioconductor test suites.
>
>3) similarly for vignettes
>
>4) similar issues for experimental  R/*.R files  or man/*.Rd
>   files for these.
>
>One I thing that would be quite useful and would even solve
>Gabor's problem: 
>The introduction of a new command line switch, say
"--build-ignore",
>to the commands 'R CMD check' and 'R CMD install'
Shouldn't that option be "--ignore-build-ignore"? :-)

More seriously:  I suspect that an on/off switch for the .Rbuildignore
file wouldn't be sufficiently flexible: the same package author might
want different things at different times, e.g. building a binary for
Windows would include different files than a binary for another
platform.  We could put together some sort of system of conditionals
within the .Rbuildignore file, but I think it would be better to just
advise such package writers to maintain one or more .Rbuildignore.in
files, and manually (or by makefile) copy the appropriate one in place
first.

Duncan Murdoch
>
>--------
>
>(*) I do agree that  "R CMD build --binary" probably really should
>    follow the ".Rbuildignore" file ``directives'' if it
doesn't
>    currently.

On Thu, 18 Nov 2004 16:36:03 +0100 (CET), murdoch@stats.uwo.ca wrote :
>>The reason that the processing is different according to whether one
>>uses .Rbuildignore or not is that 
>>  R CMD build 
>>takes the .Rbuildignore file into account but 
>>  R CMD install
>>  R CMD check
>>  R CMD build --binary 
>>do not take .Rbuildignore into account.  
>
>Okay, now I understand.  I think I'd call the last of those a bug, and
>it would seem to me that the install and check scripts should also
>respect this directive.   I've now copied this to the r-bugs list.
>(This was reported for Windows; I don't know if it applies to other
>platforms as well.)
I've taken a look at the source now, and it appears that the intention
is that R CMD check should respect the .Rbuildignore file, but (on my
system) it's looking for it in the wrong place:  it looks in the
parent of the package directory, not in the main package directory.
That is, if I have a package in subdirectory test, then 

Rcmd build test

looks for test/.Rbuildignore as it should, but 

Rcmd check test

looks for .Rbuildignore.  The problem is in these lines of code:

    my $exclude_file = ".Rbuildignore";
    ## This is a bit tricky for bundles where the build ignore pattern
    ## file is in the top-level bundle dir.
    $exclude_file = &file_path(dirname($pkgdir), $exclude_file);

In my test, $pkgdir was "d:/stuff/R/test", and $exclude_file ended up
as "d:/stuff/R/.Rbuildignore".

Could someone on Unix confirm this behaviour?  I think the fix should
be to change the last line to 

    $exclude_file = &file_path($pkgdir, $exclude_file);

but I don't want to mess things up on other platforms.

Even with this change, Rcmd check is still going to install the files
it's supposed to ignore, because it uses Rcmd INSTALL, and there's no
.Rbuildignore support there.

Duncan Murdoch