R devel - Apr 2014 - R 3.1.0: 'R CMD Sweave' deletes non tex files created upon batch mode exit

On 14/04/2014, 10:23 AM, Marc Schwartz wrote:
> Hi Martin,
>
> Thanks for your confirmation on this.
>
> I normally do not use R CMD Sweave, as I too run under ESS in normal day to
day operations. This finding was a quirk of having a particular Rnw document
that I occasionally run using R CMD Sweave and I had done so over the weekend,
realizing this behavior.
This sounds like an argument for dropping R CMD Sweave, rather than 
fixing it.  The bug was introduced in July, 2013, and nobody noticed it 
because so few people use that feature, and apparently nobody who does 
use it bothers to test pre-release versions.

Duncan Murdoch
> Thanks again.
>
> Regards,
>
> Marc
>
>
> On Apr 14, 2014, at 7:28 AM, Martin Maechler <maechler at
stat.math.ethz.ch> wrote:
>
>>>>>>> Marc Schwartz <marc_schwartz at me.com>
>>>>>>>     on Sun, 13 Apr 2014 10:22:55 -0500 writes:
>>
>> [on the R-SIG-Mac  mailing list] :	
>>
>>> Hi all,
>>> With R version 3.1.0 on OSX, using either the Snow Leopard or the
Mavericks binary installation on a Mac with fully updated Mavericks, there has
been a change in behavior since 3.0.3.
>>
>> I've just written to R-core about this:
>>
>>     R CMD Sweave
>>
>> has *serious* problems also in the very simple case when it
>> should produce figures: They are not available after the
>> completion of R CMD Sweave,
>> confirming your  'deletes non tex files created upon batch mode
exit'
>>
>> So this is not related to OSX only, but also a big problem
>> at least on other *nix descendent platforms, such as Linux.
>>
>> In short, failure (no graphic produce in the example below) by
>>
>>    R CMD Sweave foo.Rnw
>>
>> but all is fine with using Sweave(), as you Marc, noted, and hence with
>>
>>    Rscript -e 'Sweave("foo.Rnw")'
>>
>> ... and to answer your question:
>> No this was not intended and is probably one of the bigger /
>> most embarrassing bugs in a newly released version of R
>> in my view:
>>
>> Basically  'R CMD Sweave' is partly broken in R 3.1.0.
>> Yes, this should never have happened.
>>
>> I had to partially revert the R 3.1.0 installation here (our
>> statistics dept), by making R 3.0.3 the default 'R'
>> for now, as we are relying on 'R CMD Sweave ..' in many places.
>>
>> Personally I've never noticed the problem, as I seem to always
>> Sweave from ESS (Emacs speaks statistics) which calls  Sweave()
>> in R, and that works fine, also in  R 3.1.0, as you've already
>> noted
>>
>> Martin Maechler, ETH Zurich
>>
>> %------------------------------------------------------------
>> \documentclass[12pt]{article}
>> \usepackage{Sweave}
>> \begin{document}  Just a simple graphic
>>
>> <<qqnorm, fig=TRUE>>>> qqnorm(rnorm(20))
>> @
>>
>> and that's all, folks!
>> \end{document}
>> %------------------------------------------------------------
>>
>>> I have a master .Rnw file which runs a series of outputs from
multiple R code files, each called in BATCH mode using system() from within the
master .Rnw file. The output of the R code files go to separate text files in
order to catch some of the function call output that would not otherwise be
included in the resultant .tex file due to output redirection.
>>
>>> Those text files are then included in the resultant .tex file
using, for example:
>>
>>> \lstinputlisting[caption={}]{test.out}
>>
>>> directives which are included in the .Rnw source file.
>>
>>> A simple example to replicate the observed behaviors.
>>
>>> The test.Rnw file content:
>>
>>> %% R CMD Sweave test.Rnw
>>> <<results=tex,echo=false>>>>> system("R
CMD BATCH test.R test.out")
>>> @
>>
>>
>>> The test.R file content:
>>
>>> options(echo = FALSE)
>>> options(useFancyQuotes = FALSE)
>>> installed.packages()
>>
>>
>>> On version 3.0.3, the file test.out is created, along with
test.tex. test.out contains the output of installed.packages(). I did not
include the aforementioned listing directive in test.Rnw here for simplicity.
>>
>>> On version 3.1.0, the file test.out is created, but when the R CMD
Sweave command exits and returns to the CLI in the console, test.out is deleted,
presumably as part of a post R batch session clean up process. The file test.tex
is retained.
>>
>>> I uninstalled 3.1.0 and reinstalled 3.0.3 and observed the prior
behavior. I then tried clean installs of both the Snow Leopard and Mavericks
3.1.0 binaries, with the new behavior observed in both cases.
>>
>>> In reading the NEWS file for 3.1.0, there are multiple references
to Sweave, but there is nothing explicit about this new behavior.
>>
>>> I should note that when the .Rnw file is run from within an R 3.1.0
interactive session using:
>>
>>> Sweave("test.Rnw")
>>
>>> the test.out file is created and not deleted upon the function
exit, or when exiting the R session back to the console.
>>
>>> Thus, this new behavior seems to be limited to running Sweave from
the CLI using R CMD. It is not clear to me if this new behavior is by design or
perhaps an unintended consequence of changes elsewhere in Sweave processing or
in the handling of R in BATCH mode.
>>
>>> When watching the folder where the file output activity takes
place, I note that a file .build.timestamp is created and then deleted, which
leads me to believe that the new functions fileSnapshot() and changedFiles() are
being used in the course of processing R in BATCH mode. If that is correct and
there is a clean up step that is occurring upon BATCH mode exit, which deletes
all files and folders other than .tex files, is it possible that this process is
being overly aggressive? Or if not, is there a way to not have these files
deleted or have a protected folder where these files can be retained?
>>
>>> Thanks for any insights.
>>
>>> Regards,
>>
>>> Marc Schwartz
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

On 15 Apr 2014, at 13:14 , Duncan Murdoch <murdoch.duncan at gmail.com>
wrote:
> On 14/04/2014, 10:23 AM, Marc Schwartz wrote:
>> Hi Martin,
>> 
>> Thanks for your confirmation on this.
>> 
>> I normally do not use R CMD Sweave, as I too run under ESS in normal
day to day operations. This finding was a quirk of having a particular Rnw
document that I occasionally run using R CMD Sweave and I had done so over the
weekend, realizing this behavior.
> 
> This sounds like an argument for dropping R CMD Sweave, rather than fixing
it.  The bug was introduced in July, 2013, and nobody noticed it because so few
people use that feature, and apparently nobody who does use it bothers to test
pre-release versions.
> 
I'd say that that cat is out of the bag. There are probably umpteen
documents around suggesting "R CMD Sweave". As people use Sweave only
sporadically, it could take years before the old usage got stamped out. And
anyways, the command format is the obvious way to generate documents in scripts
and makefiles, isn't it?

-pd

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com

On Tue, Apr 15, 2014 at 7:14 AM, Duncan Murdoch
<murdoch.duncan@gmail.com>wrote:
> On 14/04/2014, 10:23 AM, Marc Schwartz wrote:
>
>> Hi Martin,
>>
>> Thanks for your confirmation on this.
>>
>> I normally do not use R CMD Sweave, as I too run under ESS in normal
day
>> to day operations. This finding was a quirk of having a particular Rnw
>> document that I occasionally run using R CMD Sweave and I had done so
over
>> the weekend, realizing this behavior.
>>
>
> This sounds like an argument for dropping R CMD Sweave, rather than fixing
> it.  The bug was introduced in July, 2013, and nobody noticed it because so
> few people use that feature, and apparently nobody who does use it bothers
> to test pre-release versions.
>
Or they just fix it for themselves, and do not report it. At least I know
of one such case....

Gabor

	[[alternative HTML version deleted]]