search for: geturl

Dear list, I'm trying to download a text file directly from the internet using the RCurl package and the command getURL. Duncan Lang graciously helped me solve the first step in this problem using the following command: ################# txtfile <- getURL('ftp://ftp.wcc.nrcs.usda.gov/data/snow/snow_course/table/history/idaho/13e19.txt', ftp.use.epsv = FALSE) ################# This brings the text fil...

...evious issue, however if I use icecast on Windows XP it works fine, but not from Fedora Core 1. I've made sure to check my config files multiple times and looked at the error logs but I still can't figure out whats going on. This is a snip of the error log: [2004-04-08 20:46:13] DBUG geturl/curl_print_header_result SID -> () [2004-04-08 20:46:13] DBUG geturl/curl_print_header_result Message -> () [2004-04-08 20:46:13] DBUG geturl/curl_print_header_result Touch Freq -> (0) [2004-04-08 20:46:13] DBUG geturl/curl_print_header_result Response -> (0) [2004-04-08 20:46:13]...

...the icecast server, it seems to perform very badly while trying to contact the YP server. After a couple of minutes, the streaming begins to work, but it starts spawning processes at the rate of a couple a minute. the following also starts appearing in the server log: [2003-04-08 10:26:28] DBUG geturl/curl_print_header_result SID -> () [2003-04-08 10:26:28] DBUG geturl/curl_print_header_result Message -> () [2003-04-08 10:26:28] DBUG geturl/curl_print_header_result Touch Freq -> (0) [2003-04-08 10:26:28] DBUG geturl/curl_print_header_result Response -> (0) [2003-04-08 10:26:28] E...

Hello. I know that it's usually possible to write cookies to a cookie file by removing the curl handle and doing a gc() call. I can do this with getURL(), but I just can't obtain the same results with postForm(). If I use: curlHandle <- getCurlHandle(cookiefile=FILE, cookiejar=FILE) and then do: getURL(http://example.com/script.cgi, curl=curlHandle) rm(curlHandle) gc() it's OK, the cookie is there. But, if I do (same handle; the pa...

Hello dear R-help mailing list. Looks like the same issue in Russian: library(RCurl) library(XML) u = " http://www.cian.ru/cat.php?deal_type=2&obl_id=1&room1=1" a = getURL(u) a # Here - the Russian is fine. a2 <- htmlParse(a) a2 # Here it is a mess... None of these seem to fix it: htmlParse(a, encoding = "windows-1251") htmlParse(a, encoding = "CP1251") htmlParse(a, encoding = "cp1251") htmlParse(a, encoding = "iso885...

...a problem with removing streams from the YP server. The main source on our server is on a machine that's used for development work. This means that it has to be rebooted from time to time. When this was done, messages like this began appearing in the server log: [2003-06-15 18:49:45] DBUG geturl/curl_print_header_result Message -> (Server already exists) [2003-06-15 18:49:45] EROR yp/yp_submit_url Got a NAK from yp_add(Server already exists) (http://www.oddsock.org/cgi-bin/yp-cgi) It seems these have persisted all week. There are nearly 12 thousand lines like this in the 111 MB logfi...

...a problem with removing streams from the YP server. The main source on our server is on a machine that's used for development work. This means that it has to be rebooted from time to time. When this was done, messages like this began appearing in the server log: [2003-06-15 18:49:45] DBUG geturl/curl_print_header_result Message -> (Server already exists) [2003-06-15 18:49:45] EROR yp/yp_submit_url Got a NAK from yp_add(Server already exists) (http://www.oddsock.org/cgi-bin/yp-cgi) It seems these have persisted all week. There are nearly 12 thousand lines like this in the 111 MB logfi...

...streams from the YP server. > >The main source on our server is on a machine that's used for development >work. This means that it has to be rebooted from time to time. When this >was done, messages like this began appearing in the server log: > >[2003-06-15 18:49:45] DBUG geturl/curl_print_header_result Message -> >(Server already exists) >[2003-06-15 18:49:45] EROR yp/yp_submit_url Got a NAK from yp_add(Server >already exists) (http://www.oddsock.org/cgi-bin/yp-cgi) > >It seems these have persisted all week. There are nearly 12 thousand lines >like...

I'm having a problem getting my stream published on dir.xiph.org using icecast2/ices2-beta4 The output in my log is: [snip] DBUG geturl/curl_print_header_result SID -> () DBUG geturl/curl_print_header_result Message -> () DBUG geturl/curl_print_header_result Touch Freq -> (0) DBUG geturl/curl_print_header_result Response -> (0) EROR yp/yp_submit_url Got a NAK from yp_add(Unknown) (http://www.oddsock.org/cgi-bin/yp-cgi)...

...up the files I access are at times inconsistent with regards to time posted, if they post at all, etc.... Here's some of the code I use: library(RCurl) url1 = paste("ftp://user:password at a.great.website.com/", "file", num1, ".csv", sep = "") data1 = getURL(url1) write(data1, file = paste("inMyFolder", num1, ".csv", sep = "")) Sometimes this process works perfectly, and sometimes I get an error message like this attached to "data1 = getURL(url1)": Error in curlPerform(curl = curl, .opts = opts, .encoding = ....

Dear R-help, There seems to be a web page I am unable to download using RCurl. I don't understand why it won't download: > library(RCurl) > my.url <- "http://www.nytimes.com/2009/01/07/technology/business-computing/07program.html?_r=2" > getURL(my.url) [1] "" Other web pages are ok to download but this is the first time I have been unable to download a web page using the very nice RCurl package. While i can download the webpage using the RDCOMClient, i would like to understand why it doesn't work as above please? >...

can you provide a dump of the conversation between your icecast server and the YP ? Use something like tcpdump or ethereal...In the dump, look for the URL call to yp-cgi and post what is being sent... oddsock <p>What I saw of the output of tcpdump didn't show any mention of the yp url. The communications seem to run only between the client and the server and nowhere else. This is the

Dear all, I have a problem with the command 'getURL' from the RCurl package, which I have been using to obtain a ftp directory list from the MOD16 (ET, DSI) products, and then to download them. (part of the script by Tomislav Hengl, spatial-analyst). Instead of the list of files (from ftp), I am getting the complete html code. Anyone knows why...

...esulting HTML code. Now I'd like to download all the references automatically. For this, I have tried to use RCurl, but I can't seem to get it working: I always get error code "403 Forbidden" from the web server. Initially I tried to do this without using cookies: library(RCurl) getURL(" http://scholar.google.fi/scholar.enw?q=info:U6Gfb4QPVFMJ:scholar.google.com/&output=citation&hl=fi&oe=ASCII&ct=citation&cd=0 ") or getURLContent(" http://scholar.google.fi/scholar.enw?q=info:U6Gfb4QPVFMJ:scholar.google.com/&output=citation&hl=fi&oe...

Hi all, Sorry for the rather uninformative subject, but the error I get is not very informative either. When using the XML and RCurl package to retrieve the content of an html page, htmlTreeParse fails, printing out the beginning of the HTML: Error in htmlTreeParse(getURL(url)) : File <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="de" lang="de"> <head> <...

...and nowhere else. This is the dump I got: If I may jump in here since my previous thread/problem went unsolved ;-) I've got nothing for a dump either. 'snort -dev dst dir.xiph.org' gives nothing on the firewall or the host. Just the neverending stream of [2004-04-12 18:37:26] DBUG geturl/curl_print_header_result SID -> () [2004-04-12 18:37:26] DBUG geturl/curl_print_header_result Message -> () [2004-04-12 18:37:26] DBUG geturl/curl_print_header_result Touch Freq -> (0) [2004-04-12 18:37:26] DBUG geturl/curl_print_header_result Response -> (0) [2004-04-12 18:37:26]...

Anyone recognized yet that the yp functions do not work toogether with chroot enabled? I'm running icecast2 and it successfully announces to yp servers if i don't have the chroot enabled. Then i do nothing else than enabling chroot and the log says: [2004-09-14 00:07:59] DBUG geturl/curl_print_header_result SID -> () [2004-09-14 00:07:59] DBUG geturl/curl_print_header_result Message -> () [2004-09-14 00:07:59] DBUG geturl/curl_print_header_result Touch Freq -> (0) [2004-09-14 00:07:59] DBUG geturl/curl_print_header_result Response -> (0) [2004-09-14 00:07:59] E...

month <- "2010-August" list <- "r-help" ##list <- "r-sig-ecology" ##list <- "r-sig-mixed-models" ## month <- "2010q3" n <- 50 baseurl <- "https://stat.ethz.ch/pipermail/" library(RCurl) z <- getURL(paste(baseurl,list,"/",month,"/author.html",sep="")) zz <- strsplit(z,"<LI>")[[1]] namefun <- function(x) { gsub("\\n","",gsub("^.+<I>","",gsub("</I>.+$","",x))) } cnames &...

Hi, I have problems downloading complete folders via ftp with R. Single files work fine. I tried Rcurl, but it does not work. This is my code: url = "ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/Derived_Products/3B42_V6/Daily/2009/" filenames = getURL(url, ftp.use.epsv = FALSE, ftplistonly = TRUE, crlf = TRUE) filenames = paste(url, strsplit(filenames, "\r*\n")[[1]], sep = "") no problems until here, I get a list with the files I want. But then, when I use: download.file(filenames, destfile="") I don't know how...

Hello, I am trying to use R to download a bunch of .csv files such as: http://biocache.ala.org.au/ws/occurrences/download?q=Banksia+ericifolia I have tried the following and neither work: a<- getURL(" http://biocache.ala.org.au/ws/occurrences/download?q=Banksia+ericifolia") Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) : embedded nul in string: and a<-httpPOST(" http://biocache.ala.org.au/ws/occurrences/download?q=Banksia+ericifolia") Error:...