similar to: Scraping a web page.

Hello, I am trying to access Microsoft Live Search Using SOAP through R. In R I am using the RCurl packages to make the calls. I have the following situation that looks crazy and cannot figure out how to solve it: #SOAP Request library(RCurl) h = basicTextGatherer() body='<?xml version="1.0" encoding="ISO-8859-15"?> <SOAP-ENV:Envelope

Hi, I am working with the RCurl package and I am using the curlPerform function for an soap-query. The problem is that the code is usually working well, but sometimes the connection gets lost. So I wrote a while-loop to repeat the query if anything might happened so that the same query runs again, but if the query-faults it takes a very long time for the repetition. My question is if there

Hello, I am having trouble getting the curlPerform function to authenticate using the .netrc file. From the documentation I've read it certainly seems as though this function should be able to authenticate via the .netrc file. The example I am using here comes from the "R as a Web Client- the RCurl package" paper and demonstrates using the .netrc file to access the

Hi everybody, I think that I am missing something fundamental in how strings are passed from a postForm() call in R to the curl or libcurl functions underneath. For example, I can do the following using curl from the command line: $ curl -d "Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people"

Hi, Has anyone successfully used RCurl for posting data to a password-protected site? I have tired using option netrc=1 with both postForm and curlPerform (with postfields option) but can't authenticate. I would happily provide more details if some one has had some experience with this. Thanks very much. Valerie

Hi List, I am trying to leverage my knowledge of R in trying to use it for tasks that may not make R the best choice for these tasks. I wish to automate a web scraping task, which requires a multi-step procedure: 1) log in to a website 2) Go to a particular page 3) From the drop down menu, click on a particular link 4) From the tabulated data presented, choose relevant information based on a

Dear R-Help, >From reading the help file, it is my understanding the the download.file() function does not support HTTPS connections. So therefore, understandably, the follow produces an error: ### R Code > url <- "https://stat.ethz.ch/pipermail/r-help/2008-October/thread.html" > destfile <- "//PFO-SBS001/Redirected/tonyb/Desktop/R_web_test/tmp.txt" >

Hello, I don't know if the problem originates from R-devel 3.2 or Rcurl itself. I post this message to the R-devel list and to the author of RCurl (duncan at r-project.org). > library("RCurl") Le chargement a n?cessit? le package : bitops > print(sessionInfo()) R Under development (unstable) (2015-02-03 r67717) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under:

Martin, Well, thanks for jumping in! We need all the help we can get ;) I changed the execute bit as you suggested and recompiled, no luck, still the same error message. Below is the output you wanted me to look at, its a bit beyond me so I include both a brief grep summary and then the whole enchilada. I do note that my output is different from yours, but I'm not sure how to interpret. I

Martin, Well, thanks for jumping in! We need all the help we can get ;) I changed the execute bit as you suggested and recompiled, no luck, still the same error message. Below is the output you wanted me to look at, its a bit beyond me so I include both a brief grep summary and then the whole enchilada. I do note that my output is different from yours, but I'm not sure how to interpret. I

Hi, I'm trying to download some data from the web and am running into problems with 'embedded null' characters. These seem to indicate to R that it should stop processing the page so I'd like to remove them. I've been looking around and can't seem to identify exactly what the character is and consequently how to remove it. # THE CODE WORKS ON THIS PAGE library(RCurl)

I hope the subject says it all. I want to be able to use an lm object and the associated coefficients to create function that can produce "expected" "y" values given inputs. Thanks, KW -- [[alternative HTML version deleted]]

Folks, I have a data frame with columns 200401, 200402, ..., 201207, 201208. These represent years/months. What would be the best way to sum these columns by year? What about by quarter? Thanks for your time, KW -- [[alternative HTML version deleted]]

Folks, Sorry for butting in here. I ran the code from John Kane below and it worked fine. I did however get a deprecation message suggesting the use of ggpairs from the GGally package to make this chart. Unfortunately I haven't found the correct incantation to get the diagonal to display the density plots using the "diag" parameter. Any suggestions? Just trying to learn,

Hi, I am starting coding in r and one of the things that i want to do is to scrape some data from the web. The problem that I am having is that I cannot get passed the disclaimer page (which produces a session cookie). I have been able to collect some ideas and combine them in the code below but I dont get passed the disclaimer page. I am trying to agree the disclaimer with the postForm and write

Dear all, I am having a problem downloading webpages through R when i run it in the DOS window under Windows Vista Basic. I have downloaded the batchfiles from http://code.google.com/p/batchfiles/ and have successfully set the PATH. I open up 'Command Prompt' in Vista and type (after the C:\...> stuff): ### START ### C:\Users\Karen>Rscript -e "library(RCurl);

A port of R to iOS? Thanks for your time, KW -- [[alternative HTML version deleted]]

Hi All, I am running into what appears to be character size limit in a JSON string when trying retrieve data from either `curlPerform()` or `getURL()`. Here is non-reproducible code [1], but it should shed some light on the problem. # Note that .base.url is the basic url for the API, q is a query, user # is specified, etc. session = getCurlHandle() curl.opts <- list(userpwd

I am using RCurl's curlPerform command to send an XML string to an HTTP server running on the localhost. The command is something like this: reader <- basicTextGatherer() curlPerform(url="http://127.0.0.1/", httpheader=c('Content-Type' = "text/xml; charset=utf-8"), postfields=toString.XMLNode(xmlRoot(xdoc)), writefunction=reader$update,

Dear Rexperts, I am using R to query google. I am getting different results (in size) for manual queries and queries sent through "getForm" of RCurl. It seems that RCurl limits the size of the text retrieved (the maximum I could get is around 32 k bits). Any idea how to get around this ? Thanks in advance [[alternative HTML version deleted]]