murdoch@stats.uwo.ca
2005-Jul-08 13:26 UTC
[Rd] Sweave resource leak: leftover temp files (PR#7998)
Harold, I've taken a closer look at your example and I'd call this an
Sweave bug. It creates tempfiles each time you run it, and doesn't
delete them at the end. For example:
> list.files(tempdir())
character(0)
> testfile <- system.file("Sweave",
"Sweave-test-1.Rnw", package = "utils")
> Sweave(testfile, out="junk.tex")
Writing to file junk.tex
Processing code chunks ...
1 : print term verbatim
2 : term hide
3 : echo print term verbatim
4 : term verbatim
5 : echo term verbatim
6 : echo term verbatim eps pdf
7 : echo term verbatim eps pdf
You can now run LaTeX on 'junk.tex'
> list.files(tempdir())
[1] "Rf10523" "Rf13872" "Rf17129"
"Rf2055" "Rf2203" "Rf2403" "Rf27095"
[8] "Rf2892" "Rf31415" "Rf5290"
"Rf6251" "Rf6482" "Rf7055" "Rf724"
> Sweave(testfile, out="C:/temp/junk.tex")
Writing to file C:/temp/junk.tex
Processing code chunks ...
1 : print term verbatim
2 : term hide
3 : echo print term verbatim
4 : term verbatim
5 : echo term verbatim
6 : echo term verbatim eps pdf
7 : echo term verbatim eps pdf
You can now run LaTeX on 'C:/temp/junk.tex'
> list.files(tempdir())
[1] "Rf10523" "Rf12679" "Rf1311"
"Rf13484" "Rf13872" "Rf17129" "Rf17288"
[8] "Rf2055" "Rf21774" "Rf2203"
"Rf23417" "Rf2403" "Rf27095" "Rf2892"
[15] "Rf29444" "Rf31128" "Rf31415"
"Rf32520" "Rf3338" "Rf5290" "Rf5551"
[22] "Rf6251" "Rf6482" "Rf7055"
"Rf724" "Rf7543" "Rf758" "Rf7673"
> unlink(list.files(tempdir(),full=T))
> list.files(tempdir())
character(0)
Harold: a workaround for this would be to wrap your Sweave call in
something like this:
keep <- list.files(tempdir(), full=TRUE) # keep previous temp files
... Call Sweave here ...
temps <- list.files(tempdir(), full=TRUE)
unlink(temps[!(temps %in% keep)]) # delete the newly created ones
ripley@stats.ox.ac.uk
2005-Jul-12 10:54 UTC
[Rd] Sweave resource leak: leftover temp files (PR#7998)
This is actually a Windows bug. Those files are unlink()ed, and it seems Windows is not respecting that (not an unknown phenomenon). I have tried a few workarounds, and am about to commit one that seems to work. No files are left over on a decent operating system, e.g. Solaris or FC3 Linux. On Fri, 8 Jul 2005 murdoch at stats.uwo.ca wrote:> Harold, I've taken a closer look at your example and I'd call this an > Sweave bug. It creates tempfiles each time you run it, and doesn't > delete them at the end. For example: > > > list.files(tempdir()) > character(0) > > testfile <- system.file("Sweave", "Sweave-test-1.Rnw", package = "utils") > > Sweave(testfile, out="junk.tex") > Writing to file junk.tex > Processing code chunks ... > 1 : print term verbatim > 2 : term hide > 3 : echo print term verbatim > 4 : term verbatim > 5 : echo term verbatim > 6 : echo term verbatim eps pdf > 7 : echo term verbatim eps pdf > > You can now run LaTeX on 'junk.tex' > > list.files(tempdir()) > [1] "Rf10523" "Rf13872" "Rf17129" "Rf2055" "Rf2203" "Rf2403" "Rf27095" > [8] "Rf2892" "Rf31415" "Rf5290" "Rf6251" "Rf6482" "Rf7055" "Rf724" > > Sweave(testfile, out="C:/temp/junk.tex") > Writing to file C:/temp/junk.tex > Processing code chunks ... > 1 : print term verbatim > 2 : term hide > 3 : echo print term verbatim > 4 : term verbatim > 5 : echo term verbatim > 6 : echo term verbatim eps pdf > 7 : echo term verbatim eps pdf > > You can now run LaTeX on 'C:/temp/junk.tex' > > list.files(tempdir()) > [1] "Rf10523" "Rf12679" "Rf1311" "Rf13484" "Rf13872" "Rf17129" "Rf17288" > [8] "Rf2055" "Rf21774" "Rf2203" "Rf23417" "Rf2403" "Rf27095" "Rf2892" > [15] "Rf29444" "Rf31128" "Rf31415" "Rf32520" "Rf3338" "Rf5290" "Rf5551" > [22] "Rf6251" "Rf6482" "Rf7055" "Rf724" "Rf7543" "Rf758" "Rf7673" > > unlink(list.files(tempdir(),full=T)) > > list.files(tempdir()) > character(0) > > Harold: a workaround for this would be to wrap your Sweave call in > something like this: > > keep <- list.files(tempdir(), full=TRUE) # keep previous temp files > > ... Call Sweave here ... > > temps <- list.files(tempdir(), full=TRUE) > unlink(temps[!(temps %in% keep)]) # delete the newly created ones > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
murdoch@stats.uwo.ca
2005-Jul-12 12:32 UTC
[Rd] Sweave resource leak: leftover temp files (PR#7998)
Prof Brian Ripley wrote:> This is actually a Windows bug. Those files are unlink()ed, and it seems > Windows is not respecting that (not an unknown phenomenon). I have tried > a few workarounds, and am about to commit one that seems to work.I guess you mean the C unlink, since I don't see the R unlink being called. Generally deletes fail on Windows when files are locked, e.g. open for reading or writing. I haven't seen Windows file deletes fail in other circumstances, so I suspect this was an R, MinGW or MSVCRT bug rather than a Windows bug. Duncan Murdoch> > No files are left over on a decent operating system, e.g. Solaris or FC3 > Linux. > > On Fri, 8 Jul 2005 murdoch at stats.uwo.ca wrote: > > >>Harold, I've taken a closer look at your example and I'd call this an >>Sweave bug. It creates tempfiles each time you run it, and doesn't >>delete them at the end. For example: >> >> >>>list.files(tempdir()) >> >>character(0) >> >>>testfile <- system.file("Sweave", "Sweave-test-1.Rnw", package = "utils") >>>Sweave(testfile, out="junk.tex") >> >>Writing to file junk.tex >>Processing code chunks ... >> 1 : print term verbatim >> 2 : term hide >> 3 : echo print term verbatim >> 4 : term verbatim >> 5 : echo term verbatim >> 6 : echo term verbatim eps pdf >> 7 : echo term verbatim eps pdf >> >>You can now run LaTeX on 'junk.tex' >> >>>list.files(tempdir()) >> >> [1] "Rf10523" "Rf13872" "Rf17129" "Rf2055" "Rf2203" "Rf2403" "Rf27095" >> [8] "Rf2892" "Rf31415" "Rf5290" "Rf6251" "Rf6482" "Rf7055" "Rf724" >> >>>Sweave(testfile, out="C:/temp/junk.tex") >> >>Writing to file C:/temp/junk.tex >>Processing code chunks ... >> 1 : print term verbatim >> 2 : term hide >> 3 : echo print term verbatim >> 4 : term verbatim >> 5 : echo term verbatim >> 6 : echo term verbatim eps pdf >> 7 : echo term verbatim eps pdf >> >>You can now run LaTeX on 'C:/temp/junk.tex' >> >>>list.files(tempdir()) >> >> [1] "Rf10523" "Rf12679" "Rf1311" "Rf13484" "Rf13872" "Rf17129" "Rf17288" >> [8] "Rf2055" "Rf21774" "Rf2203" "Rf23417" "Rf2403" "Rf27095" "Rf2892" >>[15] "Rf29444" "Rf31128" "Rf31415" "Rf32520" "Rf3338" "Rf5290" "Rf5551" >>[22] "Rf6251" "Rf6482" "Rf7055" "Rf724" "Rf7543" "Rf758" "Rf7673" >> >>>unlink(list.files(tempdir(),full=T)) >>>list.files(tempdir()) >> >>character(0) >> >>Harold: a workaround for this would be to wrap your Sweave call in >>something like this: >> >>keep <- list.files(tempdir(), full=TRUE) # keep previous temp files >> >>... Call Sweave here ... >> >>temps <- list.files(tempdir(), full=TRUE) >>unlink(temps[!(temps %in% keep)]) # delete the newly created ones >> >>______________________________________________ >>R-devel at r-project.org mailing list >>https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > >
ripley@stats.ox.ac.uk
2005-Jul-12 12:58 UTC
[Rd] Sweave resource leak: leftover temp files (PR#7998)
On Tue, 12 Jul 2005, Duncan Murdoch wrote:> Prof Brian Ripley wrote: >> This is actually a Windows bug. Those files are unlink()ed, and it seems >> Windows is not respecting that (not an unknown phenomenon). I have tried a >> few workarounds, and am about to commit one that seems to work. > > I guess you mean the C unlink, since I don't see the R unlink being called.Yes, I do, but the R unlink calls the C unlink and so has the same problem.> Generally deletes fail on Windows when files are locked, e.g. open for > reading or writing. I haven't seen Windows file deletes fail in other > circumstances, so I suspect this was an R, MinGW or MSVCRT bug rather than a > Windows bug.But unlink() is required by POSIX to work when files are open, so it is a MSVCRT `feature' (and that is part of Windows since Win95 OSR2). Indeed, there is nothing in the MSDN documentation on unlink() that says anything about not working if files are `locked'. If people use the same name and arg sequence of a standard library call and partially implement it without saying so or mentioning the restrictions I believe it is perfectly reasonable to call that a bug.> > Duncan Murdoch >> >> No files are left over on a decent operating system, e.g. Solaris or FC3 >> Linux. >> >> On Fri, 8 Jul 2005 murdoch at stats.uwo.ca wrote: >> >> >>> Harold, I've taken a closer look at your example and I'd call this an >>> Sweave bug. It creates tempfiles each time you run it, and doesn't >>> delete them at the end. For example: >>> >>> >>>> list.files(tempdir()) >>> >>> character(0) >>> >>>> testfile <- system.file("Sweave", "Sweave-test-1.Rnw", package = "utils") >>>> Sweave(testfile, out="junk.tex") >>> >>> Writing to file junk.tex >>> Processing code chunks ... >>> 1 : print term verbatim >>> 2 : term hide >>> 3 : echo print term verbatim >>> 4 : term verbatim >>> 5 : echo term verbatim >>> 6 : echo term verbatim eps pdf >>> 7 : echo term verbatim eps pdf >>> >>> You can now run LaTeX on 'junk.tex' >>> >>>> list.files(tempdir()) >>> >>> [1] "Rf10523" "Rf13872" "Rf17129" "Rf2055" "Rf2203" "Rf2403" "Rf27095" >>> [8] "Rf2892" "Rf31415" "Rf5290" "Rf6251" "Rf6482" "Rf7055" "Rf724" >>> >>>> Sweave(testfile, out="C:/temp/junk.tex") >>> >>> Writing to file C:/temp/junk.tex >>> Processing code chunks ... >>> 1 : print term verbatim >>> 2 : term hide >>> 3 : echo print term verbatim >>> 4 : term verbatim >>> 5 : echo term verbatim >>> 6 : echo term verbatim eps pdf >>> 7 : echo term verbatim eps pdf >>> >>> You can now run LaTeX on 'C:/temp/junk.tex' >>> >>>> list.files(tempdir()) >>> >>> [1] "Rf10523" "Rf12679" "Rf1311" "Rf13484" "Rf13872" "Rf17129" "Rf17288" >>> [8] "Rf2055" "Rf21774" "Rf2203" "Rf23417" "Rf2403" "Rf27095" "Rf2892" >>> [15] "Rf29444" "Rf31128" "Rf31415" "Rf32520" "Rf3338" "Rf5290" "Rf5551" >>> [22] "Rf6251" "Rf6482" "Rf7055" "Rf724" "Rf7543" "Rf758" "Rf7673" >>> >>>> unlink(list.files(tempdir(),full=T)) >>>> list.files(tempdir()) >>> >>> character(0) >>> >>> Harold: a workaround for this would be to wrap your Sweave call in >>> something like this: >>> >>> keep <- list.files(tempdir(), full=TRUE) # keep previous temp files >>> >>> ... Call Sweave here ... >>> >>> temps <- list.files(tempdir(), full=TRUE) >>> unlink(temps[!(temps %in% keep)]) # delete the newly created ones >>> >>> ______________________________________________ >>> R-devel at r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> >>> >> >> > >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
murdoch@stats.uwo.ca
2005-Jul-12 13:41 UTC
[Rd] Sweave resource leak: leftover temp files (PR#7998)
On 7/12/2005 8:58 AM, Prof Brian Ripley wrote:> On Tue, 12 Jul 2005, Duncan Murdoch wrote: > >> Prof Brian Ripley wrote: >>> This is actually a Windows bug. Those files are unlink()ed, and it seems >>> Windows is not respecting that (not an unknown phenomenon). I have tried a >>> few workarounds, and am about to commit one that seems to work. >> >> I guess you mean the C unlink, since I don't see the R unlink being called. > > Yes, I do, but the R unlink calls the C unlink and so has the same > problem. > >> Generally deletes fail on Windows when files are locked, e.g. open for >> reading or writing. I haven't seen Windows file deletes fail in other >> circumstances, so I suspect this was an R, MinGW or MSVCRT bug rather than a >> Windows bug. > > But unlink() is required by POSIX to work when files are open, so it is a > MSVCRT `feature' (and that is part of Windows since Win95 OSR2). Indeed, > there is nothing in the MSDN documentation on unlink() that says anything > about not working if files are `locked'. > > If people use the same name and arg sequence of a standard library call > and partially implement it without saying so or mentioning the > restrictions I believe it is perfectly reasonable to call that a bug.This is documented in the API call DeleteFile. I wouldn't call MSVCRT part of Windows, but I guess this is the same thing as saying that Linux is the kernel, not the whole collection of things you get in a Linux distribution. Windows provides a different mechanism for this, the FILE_FLAG_DELETE_ON_CLOSE flag to CreateFile(). Your patch has solved the problem, so I can't see that it would be worthwhile using this, but one way to implement the POSIX behaviour would be to open a file with this flag, then immediately close it. As long as all other handles to the file are eventually closed the file should be deleted. Duncan> >> >> Duncan Murdoch >>> >>> No files are left over on a decent operating system, e.g. Solaris or FC3 >>> Linux. >>> >>> On Fri, 8 Jul 2005 murdoch at stats.uwo.ca wrote: >>> >>> >>>> Harold, I've taken a closer look at your example and I'd call this an >>>> Sweave bug. It creates tempfiles each time you run it, and doesn't >>>> delete them at the end. For example: >>>> >>>> >>>>> list.files(tempdir()) >>>> >>>> character(0) >>>> >>>>> testfile <- system.file("Sweave", "Sweave-test-1.Rnw", package = "utils") >>>>> Sweave(testfile, out="junk.tex") >>>> >>>> Writing to file junk.tex >>>> Processing code chunks ... >>>> 1 : print term verbatim >>>> 2 : term hide >>>> 3 : echo print term verbatim >>>> 4 : term verbatim >>>> 5 : echo term verbatim >>>> 6 : echo term verbatim eps pdf >>>> 7 : echo term verbatim eps pdf >>>> >>>> You can now run LaTeX on 'junk.tex' >>>> >>>>> list.files(tempdir()) >>>> >>>> [1] "Rf10523" "Rf13872" "Rf17129" "Rf2055" "Rf2203" "Rf2403" "Rf27095" >>>> [8] "Rf2892" "Rf31415" "Rf5290" "Rf6251" "Rf6482" "Rf7055" "Rf724" >>>> >>>>> Sweave(testfile, out="C:/temp/junk.tex") >>>> >>>> Writing to file C:/temp/junk.tex >>>> Processing code chunks ... >>>> 1 : print term verbatim >>>> 2 : term hide >>>> 3 : echo print term verbatim >>>> 4 : term verbatim >>>> 5 : echo term verbatim >>>> 6 : echo term verbatim eps pdf >>>> 7 : echo term verbatim eps pdf >>>> >>>> You can now run LaTeX on 'C:/temp/junk.tex' >>>> >>>>> list.files(tempdir()) >>>> >>>> [1] "Rf10523" "Rf12679" "Rf1311" "Rf13484" "Rf13872" "Rf17129" "Rf17288" >>>> [8] "Rf2055" "Rf21774" "Rf2203" "Rf23417" "Rf2403" "Rf27095" "Rf2892" >>>> [15] "Rf29444" "Rf31128" "Rf31415" "Rf32520" "Rf3338" "Rf5290" "Rf5551" >>>> [22] "Rf6251" "Rf6482" "Rf7055" "Rf724" "Rf7543" "Rf758" "Rf7673" >>>> >>>>> unlink(list.files(tempdir(),full=T)) >>>>> list.files(tempdir()) >>>> >>>> character(0) >>>> >>>> Harold: a workaround for this would be to wrap your Sweave call in >>>> something like this: >>>> >>>> keep <- list.files(tempdir(), full=TRUE) # keep previous temp files >>>> >>>> ... Call Sweave here ... >>>> >>>> temps <- list.files(tempdir(), full=TRUE) >>>> unlink(temps[!(temps %in% keep)]) # delete the newly created ones >>>> >>>> ______________________________________________ >>>> R-devel at r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-devel >>>> >>>> >>> >>> >> >> >