R devel - Aug 2024 - Apply .Rbuildignore before copying files in R CMD build

On 2024-08-28 5:59 p.m., Alexey Sergushichev wrote:
> Hi,
> 
> Is there any reason why .Rbuildignore is not used before copying package
> files in R CMD build?
> 
> For some of the packages I develop I have rather large directories with
> miscellaneous files for testing and other purposes. They are in my
> .Rbuildignore (and .gitignore) file, but that doesn't prevent R CMD
build
> from trying to copy them on the build process. Having them copied either
> breaks the build completely because /tmp directory gets out of space, or
> just slows it down a lot. So I wonder if there is a specific reason for
> this behavior and whether it could be change or controlled by some
> parameter.
> 
> There is some discussion in the context of pkgbuild package:
> https://github.com/r-lib/pkgbuild/issues/59 It provides a hackish
> workaround for that, which also does not work on Windows.
I think the reason is simplicity.  The build process can add, delete or 
modify files.  You wouldn't want that to happen on the original source 
files, so R copies the files to a temporary location to run things.

If it applied .Rbuildignore first, then important files for the build 
might not be available, and the build could fail.

Having an R package that needs so much data that you can't fit two 
copies of it on your disk is a really unusual situation.  I think it 
will have to be up to you to fix it (by increasing your temp space, or 
decreasing the size of some of those files, or something else).

Duncan Murdoch

> I think the reason is simplicity.  The build process can add, delete or
modify files.  You wouldn't want that to happen on the original source
files, so R copies the files to a temporary location to run things.
> If it applied .Rbuildignore first, then important files for the build
might not be available, and the build could fail.

Yeah, I guess that makes sense. Thanks, I didn't realize that.
> Having an R package that needs so much data that you can't fit two
copies of it on your disk is a really unusual situation.  I think it
will have to be up to you to fix it (by increasing your temp space, or
decreasing the size of some of those files, or something else).

These files are not required, but help to run and test some things. But I
guess I have to deal with that.

--
Alexey


On Wed, Aug 28, 2024 at 5:11?PM Duncan Murdoch <murdoch.duncan at
gmail.com>
wrote:
> On 2024-08-28 5:59 p.m., Alexey Sergushichev wrote:
> > Hi,
> >
> > Is there any reason why .Rbuildignore is not used before copying
package
> > files in R CMD build?
> >
> > For some of the packages I develop I have rather large directories
with
> > miscellaneous files for testing and other purposes. They are in my
> > .Rbuildignore (and .gitignore) file, but that doesn't prevent R
CMD build
> > from trying to copy them on the build process. Having them copied
either
> > breaks the build completely because /tmp directory gets out of space,
or
> > just slows it down a lot. So I wonder if there is a specific reason
for
> > this behavior and whether it could be change or controlled by some
> > parameter.
> >
> > There is some discussion in the context of pkgbuild package:
> > https://github.com/r-lib/pkgbuild/issues/59 It provides a hackish
> > workaround for that, which also does not work on Windows.
>
> I think the reason is simplicity.  The build process can add, delete or
> modify files.  You wouldn't want that to happen on the original source
> files, so R copies the files to a temporary location to run things.
>
> If it applied .Rbuildignore first, then important files for the build
> might not be available, and the build could fail.
>
> Having an R package that needs so much data that you can't fit two
> copies of it on your disk is a really unusual situation.  I think it
> will have to be up to you to fix it (by increasing your temp space, or
> decreasing the size of some of those files, or something else).
>
> Duncan Murdoch
>
>
	[[alternative HTML version deleted]]

On Thu, Aug 29, 2024 at 12:12?AM Duncan Murdoch
<murdoch.duncan at gmail.com> wrote:
[...]
> I think the reason is simplicity.  The build process can add, delete or
> modify files.  You wouldn't want that to happen on the original source
> files, so R copies the files to a temporary location to run things.
>
> If it applied .Rbuildignore first, then important files for the build
> might not be available, and the build could fail.
AFAICT the ignored files are deleted right after the copy, so they
are not present during the build process. (But FIXME.)