Dear List: I am encountering an error that I can't resolve. I'm looping through rows of a dataframe to generate individual tex files using Sweave. At random points along the way, I encounter the following error Error in file() : cannot find unused tempfile name At which point Sweave halts. There isn't a logical pattern that I can identify in terms of why the program stops at certain points. It just seems to be random as far as I can tell. I've searched the archives and of course Sweave FAQs but haven't found much that sheds light on what this error indicates and what I should do to resolve it. There are approximately 20,000 rows, meaning that about 20,000 tex files are created. If I sample 5,000 or even 10,000 and run the program, I do not encounter an error. It only occurs when I run the program on the full dataframe and even then the error is not occuring at the same point. That is, the row at which the program halts varies each time. Does anyone have any thoughts on this problem? -Harold [[alternative HTML version deleted]]
Hi Harold, I know nothing about Sweave (though it certainly sounds like a great idea!). Does Sweave hold these files open simultaneously - many operating systems wouldn't be able to cope with 20,000 simultaneously open files. Could you be running out of disk space? or inodes - if you're on a unix type system? can the directory hold that many files? can the disk hold that many files? Many filesystems are formatted with an assumption about the average file size. Perhaps if you told us what type of operating system and type(s) of filesystem(s)? cheers! Sean On 7/6/05, Doran, Harold <HDoran at air.org> wrote:> Dear List: > > I am encountering an error that I can't resolve. I'm looping through > rows of a dataframe to generate individual tex files using Sweave. At > random points along the way, I encounter the following error > > Error in file() : cannot find unused tempfile name > > At which point Sweave halts. There isn't a logical pattern that I can > identify in terms of why the program stops at certain points. It just > seems to be random as far as I can tell. I've searched the archives and > of course Sweave FAQs but haven't found much that sheds light on what > this error indicates and what I should do to resolve it. > > There are approximately 20,000 rows, meaning that about 20,000 tex files > are created. If I sample 5,000 or even 10,000 and run the program, I do > not encounter an error. It only occurs when I run the program on the > full dataframe and even then the error is not occuring at the same > point. That is, the row at which the program halts varies each time. > > Does anyone have any thoughts on this problem? > > -Harold > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html >
Doran, Harold wrote:> Dear List: > > I am encountering an error that I can't resolve. I'm looping through > rows of a dataframe to generate individual tex files using Sweave. At > random points along the way, I encounter the following error > > Error in file() : cannot find unused tempfile nameWhich version of R is this? I think during one of the latest releases tempfile() name generation has been imporved, because R did not tried hard enough to find new (random) filenames for tempfiles in older releases of R. Uwe Ligges> At which point Sweave halts. There isn't a logical pattern that I can > identify in terms of why the program stops at certain points. It just > seems to be random as far as I can tell. I've searched the archives and > of course Sweave FAQs but haven't found much that sheds light on what > this error indicates and what I should do to resolve it. > > There are approximately 20,000 rows, meaning that about 20,000 tex files > are created. If I sample 5,000 or even 10,000 and run the program, I do > not encounter an error. It only occurs when I run the program on the > full dataframe and even then the error is not occuring at the same > point. That is, the row at which the program halts varies each time. > > Does anyone have any thoughts on this problem? > > -Harold > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Dear Uwe and Sean: Thank you for your reply. I think I did a poor job framing my problem. Here are a few things I should have added. First, I previously stated that the program would halt in random places. This is *not* true. It always halts at the same row, which happens to be row 1038. There isn't a logical reason for stopping in this row as far as I can tell, all data needed to generate the file are present. I am using R version 2.10 for Windows on an XP machine with 2 gb ram. My tex distribution is MikTex (I'm not sure if that matters). Traceback gives the following information> traceback()9: file() 8: driver$runcode(drobj, chunk, chunkopts) 7: Sweave("fam_template.Rnw", output = tmp2) 6: eval.with.vis(expr, envir, enclos) 5: eval.with.vis(ei, envir) 4: source("run_fam.r") 3: eval.with.vis(expr, envir, enclos) 2: eval.with.vis(ei, envir) 1: source(file.choose()) I did tempfile() to identify the location of the files and have cleaned that out repeatedly. I've searched the archives and have seen some discussion regarding and on.exit() command, but I'm not sure I see how it would be utilized here. Here is the code for the looping file that subsets the dataframe. I wonder if I need to add something here that would close a connection or generate a random tempfile name to avoid this problem in the loop. list <- unique(wide$stuid) master = "master.tex" for (i in list){ tmp1 <- subset(wide, stuid==i) tmp2 <- paste(i, "tex", sep=".") Sweave("fam_template.Rnw", output=tmp2) file.append("fam_master.tex", tmp2) } Thanks for your time on this, Harold -----Original Message----- From: Uwe Ligges [mailto:ligges at statistik.uni-dortmund.de] Sent: Thursday, July 07, 2005 4:52 AM To: Doran, Harold Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Tempfile error Doran, Harold wrote:> Dear List: > > I am encountering an error that I can't resolve. I'm looping through > rows of a dataframe to generate individual tex files using Sweave. At > random points along the way, I encounter the following error > > Error in file() : cannot find unused tempfile nameWhich version of R is this? I think during one of the latest releases tempfile() name generation has been imporved, because R did not tried hard enough to find new (random) filenames for tempfiles in older releases of R. Uwe Ligges> At which point Sweave halts. There isn't a logical pattern that I can > identify in terms of why the program stops at certain points. It just > seems to be random as far as I can tell. I've searched the archives > and of course Sweave FAQs but haven't found much that sheds light on > what this error indicates and what I should do to resolve it. > > There are approximately 20,000 rows, meaning that about 20,000 tex > files are created. If I sample 5,000 or even 10,000 and run the > program, I do not encounter an error. It only occurs when I run the > program on the full dataframe and even then the error is not occuring > at the same point. That is, the row at which the program halts varieseach time.> > Does anyone have any thoughts on this problem? > > -Harold > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html
We just explored the issue a little further here and here is what we have found. If we look in the temp directory we noticed that the last temp file name is rf32767. This number coincides with 2^16 with a 16 bit signed integer. Is there a way to modify the temp file settings to use an unsigned or 32 bit integer for temp file names? We changed the location where temp files are stored. We also disabled the realtime virus protection, but do not think this has anything to do with the problem. -----Original Message----- From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Doran, Harold Sent: Thursday, July 07, 2005 10:46 AM To: Uwe Ligges; Sean O'Riordain Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Tempfile error Dear Uwe and Sean: Thank you for your reply. I think I did a poor job framing my problem. Here are a few things I should have added. First, I previously stated that the program would halt in random places. This is *not* true. It always halts at the same row, which happens to be row 1038. There isn't a logical reason for stopping in this row as far as I can tell, all data needed to generate the file are present. I am using R version 2.10 for Windows on an XP machine with 2 gb ram. My tex distribution is MikTex (I'm not sure if that matters). Traceback gives the following information> traceback()9: file() 8: driver$runcode(drobj, chunk, chunkopts) 7: Sweave("fam_template.Rnw", output = tmp2) 6: eval.with.vis(expr, envir, enclos) 5: eval.with.vis(ei, envir) 4: source("run_fam.r") 3: eval.with.vis(expr, envir, enclos) 2: eval.with.vis(ei, envir) 1: source(file.choose()) I did tempfile() to identify the location of the files and have cleaned that out repeatedly. I've searched the archives and have seen some discussion regarding and on.exit() command, but I'm not sure I see how it would be utilized here. Here is the code for the looping file that subsets the dataframe. I wonder if I need to add something here that would close a connection or generate a random tempfile name to avoid this problem in the loop. list <- unique(wide$stuid) master = "master.tex" for (i in list){ tmp1 <- subset(wide, stuid==i) tmp2 <- paste(i, "tex", sep=".") Sweave("fam_template.Rnw", output=tmp2) file.append("fam_master.tex", tmp2) } Thanks for your time on this, Harold -----Original Message----- From: Uwe Ligges [mailto:ligges at statistik.uni-dortmund.de] Sent: Thursday, July 07, 2005 4:52 AM To: Doran, Harold Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Tempfile error Doran, Harold wrote:> Dear List: > > I am encountering an error that I can't resolve. I'm looping through > rows of a dataframe to generate individual tex files using Sweave. At > random points along the way, I encounter the following error > > Error in file() : cannot find unused tempfile nameWhich version of R is this? I think during one of the latest releases tempfile() name generation has been imporved, because R did not tried hard enough to find new (random) filenames for tempfiles in older releases of R. Uwe Ligges> At which point Sweave halts. There isn't a logical pattern that I can > identify in terms of why the program stops at certain points. It just > seems to be random as far as I can tell. I've searched the archives > and of course Sweave FAQs but haven't found much that sheds light on > what this error indicates and what I should do to resolve it. > > There are approximately 20,000 rows, meaning that about 20,000 tex > files are created. If I sample 5,000 or even 10,000 and run the > program, I do not encounter an error. It only occurs when I run the > program on the full dataframe and even then the error is not occuring > at the same point. That is, the row at which the program halts varieseach time.> > Does anyone have any thoughts on this problem? > > -Harold > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html