R help - Nov 2010 - file reading problem unique to windows. Err info: Error in file(file, ifelse(append, "a", "w")). cannot open the connection

Dear List

I asked this question before, got some tips but can't get it solved.
it is clear now that this problem only occurs when run on windows (I
tested it on windows XP) nothing wrong at all when run on unix.
unfortunately, sometimes I have to run it on windows,
I appreciate any suggestion on how to circumvent this problem when run
on windows.
below is the problem description I submitted before.

#############################################################

I am running a loop downloading  web pages and save the html to a
temporary file (use download.file() )
 then read (using readLines)  it in for processing;
finally write useful info from each processed page to a unique file

the problem is once the loop runs up to somewhere near  5000, it will
throw out an err like below and won't go further.

----------------------------------------------------------------
Error in file(file, ifelse(append, "a", "w")) :
 cannot open the connection
-----------------------------------------------------------------

In the meantime, a request for new connection won't be successful, for
example, a request for the help page of "file" will trigger err below

-----------------------------------------------------------------------
 ?file
Error in gzfile(file, "rb") : cannot open the connection
In addition: Warning message:
In gzfile(file, "rb") :
 cannot open compressed file
'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable
reason 'Too many open files'
-----------------------------------------------------------------------

I am not sure if the problem is too many connections not closed. since
I close the file connection after each readLines.
checking with showConnections(all=T) does not show excessive
connections and closeAllConnections() does not help.

Can any one help me on this?


Any answer highly appreciated.

yong

We don't have any of the information asked for in the posting guide, 
such as your version of R, reproducible example ....

But please try R-patched, since this might be

     ? download.file() could leave the destination file open if the URL
       was not able to be opened.  (PR#14414)

(If you had followed the posting guide you would have tried R-patched 
before posting ....)


On Wed, 24 Nov 2010, Yong Wang wrote:
> Dear List
>
> I asked this question before, got some tips but can't get it solved.
Where?  You didn't give a reference, and it would have helped the 
helpers.
> it is clear now that this problem only occurs when run on windows (I
> tested it on windows XP) nothing wrong at all when run on unix.
> unfortunately, sometimes I have to run it on windows,
> I appreciate any suggestion on how to circumvent this problem when run
> on windows.
> below is the problem description I submitted before.
>
> #############################################################
>
> I am running a loop downloading  web pages and save the html to a
> temporary file (use download.file() )
> then read (using readLines)  it in for processing;
> finally write useful info from each processed page to a unique file
>
> the problem is once the loop runs up to somewhere near  5000, it will
> throw out an err like below and won't go further.
>
> ----------------------------------------------------------------
> Error in file(file, ifelse(append, "a", "w")) :
> cannot open the connection
> -----------------------------------------------------------------
>
> In the meantime, a request for new connection won't be successful, for
> example, a request for the help page of "file" will trigger err
below
>
> -----------------------------------------------------------------------
> ?file
> Error in gzfile(file, "rb") : cannot open the connection
> In addition: Warning message:
> In gzfile(file, "rb") :
> cannot open compressed file
> 'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable
> reason 'Too many open files'
> -----------------------------------------------------------------------
>
> I am not sure if the problem is too many connections not closed. since
> I close the file connection after each readLines.
> checking with showConnections(all=T) does not show excessive
> connections and closeAllConnections() does not help.
>
> Can any one help me on this?
>
>
> Any answer highly appreciated.
>
> yong
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595