Hi - Many organizations now make their data available as XML via a REST web service architecture. Is there any R package or facility to access this type of data directly (eg, to make the HTTP GET request and have the downloaded data put into an R data frame)? I used several R search sites to look for an answer, but came up with very little. Any help would be appreciated. Thanks very much. Gary Lewis
On Mon, Sep 28, 2009 at 10:01 AM, Gary Lewis <gary.m.lewis@gmail.com> wrote:> Hi - Many organizations now make their data available as XML via a > REST web service architecture. Is there any R package or facility to > access this type of data directly (eg, to make the HTTP GET request > and have the downloaded data put into an R data frame)? >The url() function together with the XML package should let you do this -- Rajarshi Guha NIH Chemical Genomics Center [[alternative HTML version deleted]]
On Mon, Sep 28, 2009 at 3:01 PM, Gary Lewis <gary.m.lewis at gmail.com> wrote:> Hi - Many organizations now make their data available as XML via a > REST web service architecture. Is there any R package or facility to > access this type of data directly (eg, to make the HTTP GET request > and have the downloaded data put into an R data ?frame)? > > I used several R search sites to look for an answer, but came up with > very little. Any help would be appreciated.You can (in many places) replace a filename with url(address) and R functions such as read.table will work. If the response is XML then getting the response into a data frame will be so dependent on the format then it's unlikely there's a generic solution. Specific solutions exist - for example my geonames package that queries the geonames server for geographic information: http://r-forge.r-project.org/projects/geonames/ Note that instead of getting XML and having to parse that, it gets the JSON representation and use rjson to decode it - which saves a bit of weight since XML parsing is a bit heavy. I suspect that if SOAP and WSDL interfaces were more widely used then some of the R Omegahat projects might be more useful: http://www.omegahat.org/download/R/packages/ but it seems that informal service definitions via simple REST interfaces and XML responses are winning. Barry
Gary, Echoing Barry's suggestion of Omegahat packages, take a look at RCurl - http://www.omegahat.org/RCurl/ It has a function, getForm, which passes GET form parameters as a list to curl. I used it and the XML package in writing RLastFM (http://cran.r-project.org/web/packages/RLastFM/index.html) which implements this for last.fm. With a few XPath queries (in the XML) package, you can format the data into a data frame. Greg Gary Lewis wrote:> Hi - Many organizations now make their data available as XML via a > REST web service architecture. Is there any R package or facility to > access this type of data directly (eg, to make the HTTP GET request > and have the downloaded data put into an R data frame)? > > I used several R search sites to look for an answer, but came up with > very little. Any help would be appreciated. > > Thanks very much. > > Gary Lewis > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Greg Hirson ghirson at ucdavis.edu Graduate Student Agricultural and Environmental Chemistry 1106 Robert Mondavi Institute North One Shields Avenue Davis, CA 95616
On Mon, Sep 28, 2009 at 9:01 AM, Gary Lewis <gary.m.lewis at gmail.com> wrote:> Hi - Many organizations now make their data available as XML via a > REST web service architecture. Is there any R package or facility to > access this type of data directly (eg, to make the HTTP GET request > and have the downloaded data put into an R data ?frame)? > > I used several R search sites to look for an answer, but came up with > very little. Any help would be appreciated.See also http://github.com/hadley/crantastic/blob/cran-indexing/curr.R for the beginnings of some REST code for R. Hadley -- http://had.co.nz/
Thanks to all of you who replied. You've provided some great leads. I really appreciate it. Gary On Mon, Sep 28, 2009 at 10:01 AM, Gary Lewis <gary.m.lewis at gmail.com> wrote:> Hi - Many organizations now make their data available as XML via a > REST web service architecture. Is there any R package or facility to > access this type of data directly (eg, to make the HTTP GET request > and have the downloaded data put into an R data ?frame)? > > I used several R search sites to look for an answer, but came up with > very little. Any help would be appreciated. > > Thanks very much. > > Gary Lewis >