similar to: how to access Internet from within R?

Hi RUsers, Suppose I want to see the data on the website url <- "http://www.nseindia.com/content/indices/ind_histvalues.htm" for the index "S&P CNX NIFTY" for dates "FromDate"="01-11-2010","ToDate"="02-11-2010" then read the html table from the page using readHTMLtable() I am using this code webpage <-

Hi, I am starting coding in r and one of the things that i want to do is to scrape some data from the web. The problem that I am having is that I cannot get passed the disclaimer page (which produces a session cookie). I have been able to collect some ideas and combine them in the code below but I dont get passed the disclaimer page. I am trying to agree the disclaimer with the postForm and write

Hi RUsers, Suppose I want to see the data on the website url <- "http://www.nseindia.com/content/indices/ind_histvalues.htm" for the index "S&P CNX NIFTY" for dates "FromDate"="01-11-2010","ToDate"="02-11-2010" then read the html table from the page using readHTMLtable() I am using this code webpage <-

Dear colleagues, Could I get some assistance using postForm() to scrape the business names and addresses at this website: http://www.brantford.ca/business/LocalBusinessCommunity/Pages/BusinessDirectorySearch.aspx I've read through (http://www.omegahat.org/RCurl/RCurlJSS.pdf) and scoured the web for tutorials, but I can't crack it. I'm aware that this is probably a pretty basic

Hi all, I am trying to download some tab separated data from the internet. The data is not available directly at the URL that could be known apriori. There is an intermediate form where start and end dates have to be given to get to the required page. For example, I want to download data for a station 03015795. The form for this station is at:

I am having trouble with the postForm function in RCurl. I want to send a the command DELETE https://somewebsite.com.json but I can't seem to find it. I could try: postForm(url, _method="DELETE", .opts = list(username:password) ) but I get the error: Error: unexpected input in " postForm(url4, _" this error seems to be due to the underscore "_"

Hi, Has anyone successfully used RCurl for posting data to a password-protected site? I have tired using option netrc=1 with both postForm and curlPerform (with postfields option) but can't authenticate. I would happily provide more details if some one has had some experience with this. Thanks very much. Valerie

According to [1] and [2], using RCurl to post a form with basic authentication is done using the postForm method. I'm trying to post generated interpolation data from R onto an HTTP form. The call I'm using is page <- postForm('http://our.server.com/dbInt/new', opts = curlOptions=(userpwd="test:test", verbose=T), profileid = "-1", value="1.801",

Hi everybody, I think that I am missing something fundamental in how strings are passed from a postForm() call in R to the curl or libcurl functions underneath. For example, I can do the following using curl from the command line: $ curl -d "Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people"

Dear R-Help, I am looking into using the Open Calais web service (http:// sws.clearforest.com/calaisViewer/) for text mining purposes. I would like to use R to post text into one of the forms on their website. In package RCurl, there is a function called postForm(). This sounds like it would do the job. Unfortunately the URL used in the example is no longer valid (i have emailed the maintainer

Hi, I am trying to submit a form to the EXPASY protscale server ( http://www.expasy.ch/tools/protscale.html). I am using the RCurl package and the postForm function available in it. I have extracted the variables for the form from the HTML source page. According to the syntax of postForm, I just need to mention the url and assign values to the input mentioned in the HTML code. The code that I am

Hi, I am trying to submit a form to the EXPASY protscale server ( http://www.expasy.ch/tools/protscale.html). I am using the RCurl package and the postForm function available in it. I have extracted the variables for the form from the HTML source page. According to the syntax of postForm, I just need to mention the url and assign values to the input mentioned in the HTML code. The code that I am

I am trying to write a program that uses R and takes a pdb file, and converts it to a pqr file. This task is simple generally, using the website, http://pdb2pqr-1.wustl.edu/pdb2pqr/. How do you use R to input a pdb file (that is on hand) into the upload pdb file input, and run the website and give the return file to be a pqr file. Thanks for your help. -- View this message in context:

Hello. I know that it's usually possible to write cookies to a cookie file by removing the curl handle and doing a gc() call. I can do this with getURL(), but I just can't obtain the same results with postForm(). If I use: curlHandle <- getCurlHandle(cookiefile=FILE, cookiejar=FILE) and then do: getURL(http://example.com/script.cgi, curl=curlHandle) rm(curlHandle) gc() it's

Hi, I am trying to use a CGI service (Pubchem PUG) via RCurl and am running into a problem where the data must be supplied via POST - but I don't know the keyword for the argument. The data to be sent is an XML fragment. I can do this via the command line using curl: I save the XML string to a file called query.xml and then do curl -d @query.xml

Hi, I'm trying to download some data from the web and am running into problems with 'embedded null' characters. These seem to indicate to R that it should stop processing the page so I'd like to remove them. I've been looking around and can't seem to identify exactly what the character is and consequently how to remove it. # THE CODE WORKS ON THIS PAGE library(RCurl)

Dear R-help, There seems to be a web page I am unable to download using RCurl. I don't understand why it won't download: > library(RCurl) > my.url <- "http://www.nytimes.com/2009/01/07/technology/business-computing/07program.html?_r=2" > getURL(my.url) [1] "" Other web pages are ok to download but this is the first time I have been unable to download a

Dear List, Sharepoint sites are used more and more often by researchers as a platform to share all the information in a joint research project. Typically, they can be accessed only by a Hypertext Transfer Protocol Secure (HTTPS) URL. We are investigating whether it is possible to read and write data to Microsoft Sharepoint sites using R. We have managed to open a read-only connection to read

To get USDCAD rates from Bank of Canada, we first go url <- "http://banqueducanada.ca/en/rates/exchange-avg.html" select 12 months for Rates for the past and click "Get Rates" button. Then the page moves to address <- "http://banqueducanada.ca/cgi-bin/famecgi_fdps" and the rates show in the html page. htmlParse() can read the html document but

Hi, I have to search in an online db for registered manufacturers of raw materials. Can I use R for the following: I have a list with monograph numbers eg: l <- c(198, 731,355) Now I want to make a dataframe, containing the monograph number and the information listed under COS: Certificate holder, certificate number, Status, Type Is this possible with R? kind regards Bart -- View this