This seriously looks like a bug in load(): the gzcon() magic in the code
leaves the connection open. On Linux x86_64 I can do:
file.name <- "/tmp/foo.RData"
data <- runif(10)
save(data, file=file.name)
for ( i in 1:4000 ) { con <- gzfile(file.name, "rb");
load(file=con);
close(con); }
## No problem!
for ( i in 1:4000 ) { con <- file(file.name, "rb"); load(file=con);
close(con); }
# Error in load(file = con) : cannot open the connection
# In addition: Warning messages:
# 1: In gzcon(file) : cannot open file '/tmp/foo.RData': Too many open
files
# 2: In load(file = con) :
# cannot open file '/tmp/foo.RData': Too many open files
print(i)
# [1] 1021
showConnections()
# description class mode text isopen can read can write
M&R: Use the gzfile() workaround above.
R-helpers: (a) is this a bug? (b) where does one report bugs in core R
anyhow?
Allan
On 16/07/09 21:55, Marilyn & Rich Short wrote:> Hello,
>
> I'm having a problem in R. I'm getting an error message that reads,
> "Too many open files". I'm opening files and closing them
(and
> unlinking them), but when I go through that process 509 times, the
> program halts and I get this error message: "cannot open the
> connection" with warning messages: "Too many open files".
I've been
> working on this problem for a couple of weeks and have gleaned a bit
> of info from different internet threads, but no solutions yet.
>
> I'm using Windows XP, SP3 and R 2.9.1.
> Here is my session info:
>
>> sessionInfo()
> R version 2.9.1 (2009-06-26)
> i386-pc-mingw32
>
> locale:
> LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
> States.1252;LC_MONETARY=English_United
> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> (The problem also occurs on my Vista machine as well.)
>
> There is some talk on the internet that some windows systems have a
> limit of 512 files that can be open at one time. Even though I'm
> closing my files each time, something is keeping track of how many
> times I've opened and closed a file in a session. I've talked to
> Microsoft and run a test program in Visual Studio C#, and, at the
> moment, it looks like the problem does not lie in the Microsoft arena.
> The C# program performed a similar task 10,000 times without a
> problem. I'm not totally convinced, but the current evidence says to
> look elsewhere.
>
> I've attached a script that will induce the problem. However, be
> warned that, if you use it, you will have to get out of R after you
> run it. R will no longer be able to access files such as help or
> sessionInfo(). This can be solved by getting out of the R GUI and back
> in. Also, I'm using my pathname, "MyPathnameA", so you'll
probably
> want to edit it with your own.
>
> R E-mails from as far back as 2006 ask for help on the issue. There
> have been several suggestions, but no confirmed solution that I can
> find. You will see my attempt at these suggestions in the script [
> rm(outMIA);rm(junk); and gc(); after close(outMIA); and
> unlink(FileNameMIA);]. For me, this becomes important because it
> limits the total number if iterations in nested do-loops. This is
> where I ran into the problem. The program below is a distillation from
> the original program.
>
> Any suggestion would be greatly appreciated.
>
> I'm a retired engineer and am picking up R to use on a genetic
> algorithm I want to play with.
>
> Thanks for your attention,
> Rich Short
>
> # This script induces an error: "cannot open the connection" with
>
> # warnings: "Too many open files".
>
> ?
>
> # ---------------- Create a junk file and "save" it for use
further
> into the program ---
>
> junk <- 1
>
> MyPathnameA <- "C:\\Documents and Settings\\All
> Users\\Documents\\SIREPO\\DBFS-0150"
>
> connectionX <- paste(MyPathnameA,"junk",sep="\\")
>
> outJunk <- file(connectionX, open="wb")
>
> save(junk, file=outJunk)
>
> close(outJunk)
>
>
>
>
>
>
> # The next two lines are a repeat from above. They will be useful if
> you want to run
>
> # this script again after "junk" has been loaded.
>
> MyPathnameA <- "C:\\Documents and Settings\\All
> Users\\Documents\\SIREPO\\DBFS-0150"
>
> connectionX <- paste(MyPathnameA,"junk",sep="\\")
>
> FileNameMIA <- connectionX
>
> for(i in 1:4000){
>
> # ----------------- Load "junk"
---------------------------------
>
> outMIA <- file(FileNameMIA, open="rb")
>
> load(file=outMIA) # load a file
>
> # ----------------- close "junk"; unlink; remove;
----------------
>
> close(outMIA) # close the file
>
> unlink(FileNameMIA) # Unlink
>
> rm(outMIA) # This should be unnecessary. I
> tried it just to be sure the retention of the variable was
>
> rm(junk) # not causing the connection
> to be kept open.
>
> cat(" i = ",i,sep=" ") # Show what
iteration we are on.
>
> gc() # Garbage collection. This
> should be unnecessary. Another failed attempt at a work-around.
>
> zzz <- showConnections(all=FALSE)
>
> cat(" zzz = ",zzz,"\n",sep=" ") # This
screen print shows that
> there are no open files
>
> }
>
> ?
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.