R help - Nov 2005 - TryCatch() with read.csv("http://...")

Hi, folks!

I'm trying to pull in data using read.csv("my URL goes here"), and
it really
works fantastically. Amazing to pull in live data right off the internet,
into RAM, and get busy...

however...

occasionally there is a server problem, or the data are not up  yet, and
instead of pushing through a nice CSV file, the server sends a 404 "Not
Found" page...

Since the read is inside a "FOR" loop, (pulling in and analyzing
different
data each time) I've been trying to use TryCatch() to pull in the file if
it's OK, but to jump out to the next iteration of the loop otherwise (using
"next"). I don't want to exit the script, I just want to skip this
iteration
and move on. This seems like it should be obvious, but I've read through the
? files and tried everything my smooth little brain can come up with... no
luck. I still end up jumping out of the entire script if I hit an error
page.

Does anyone have some example syntax that I might examine or try out?

I'm feeling pretty thick here...

--
---------------------------------------
David L. Van Brunt, Ph.D.
mailto:dlvanbrunt@gmail.com

	[[alternative HTML version deleted]]

How are you trying to break the loop?
The next statement should move to the next in the sequence (from the
Introduction To R manual)

On Wed, 2005-23-11 at 11:57 -0500, David L. Van Brunt, Ph.D.
wrote:
> Hi, folks!
> 
> I'm trying to pull in data using read.csv("my URL goes
here"), and it really
> works fantastically. Amazing to pull in live data right off the internet,
> into RAM, and get busy...
> 
> however...
> 
> occasionally there is a server problem, or the data are not up  yet, and
> instead of pushing through a nice CSV file, the server sends a 404
"Not
> Found" page...
> 
> Since the read is inside a "FOR" loop, (pulling in and analyzing
different
> data each time) I've been trying to use TryCatch() to pull in the file
if
> it's OK, but to jump out to the next iteration of the loop otherwise
(using
> "next"). I don't want to exit the script, I just want to skip
this iteration
> and move on. This seems like it should be obvious, but I've read
through the
> ? files and tried everything my smooth little brain can come up with... no
> luck. I still end up jumping out of the entire script if I hit an error
> page.
> 
> Does anyone have some example syntax that I might examine or try out?
> 
> I'm feeling pretty thick here...
> 
> --
> ---------------------------------------
> David L. Van Brunt, Ph.D.
> mailto:dlvanbrunt at gmail.com
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>

Hi David,

On 23 Nov 2005, dlvanbrunt at gmail.com wrote:
> Since the read is inside a "FOR" loop, (pulling in and analyzing
> different data each time) I've been trying to use TryCatch() to pull
> in the file if it's OK, but to jump out to the next iteration of the
> loop otherwise (using "next"). 
I think this is a common approach.  Here's an example using try (not
tryCatch):

doRead <- function() {
    x <- runif(1)
    if (x < 0.2)
      stop("failure in doRead")
    x
}

iters <- 20
vals <- NULL
for (i in seq(length=iters)) {
    val <- try(doRead(), silent=TRUE)
    if (inherits(val, "try-error"))
      next
    else
      vals <- c(vals, val)
}


length(vals)
[1] 16   ## there were 4 errors

> This seems like it should be obvious, but I've read through the ?
> files and tried everything my smooth little brain can come up
> with... no luck. I still end up jumping out of the entire script if
> I hit an error page.
tryCatch and the associated tools are many things (very powerful, very
cool), but I think they are not obvious :-)

I'm not sure if you can use tryCatch directly... probably so, but it
isn't obvious to me ;-)  However, here is an example using
withCallingHandlers and withRestarts.

vals <- NULL
iters <- 20
withCallingHandlers({
    for (i in seq(length=iters)) {
        val <- withRestarts(doRead(),
                            skipError=function() return(NULL))
        if (!is.null(val))
          vals <- c(vals, val)
    }},
                    error=function(e) invokeRestart("skipError"))


What's happening here as I understand it:

1. Establish a restart with

      val <- withRestarts(doRead(),
                          skipError=function() return(NULL))

   This says, "mark this spot with a recovery function called
   'skipError'".  This code does not handle the error.  But marks a
   spot where recovery occur.

2. Establish a handler for errors.  This is the withCallingHandlers
   call.  When an error occurs the function specified by the error arg
   gets called.  In this case, we invoke the restart function created
   above.  The restart function is evaluated "in the right place".

Why this is good: it decouples the handling of errors from the
recovery from errors.  With try(), you have to catch and handle the
error all at once.  With the withCallingHandlers/withRestarts
approach, you can use the same inner code (the same restarts) and
build wrappers that do different things.

HTH,

+ seth


PS: I found the following chapter to be helpful:
http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html