R-helpers, When I use cURL in the Terminal: curl --cookie-jar cookie.txt --url "http://corpusdelespanol.org/x.asp" --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0" --location --include a cookie file "cookie.txt" is saved to my working directory. However, when I try what I think is the equivalent command R with RCurl: ch <- getCurlHandle(followlocation = T, header = T, useragent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0") getURL(url = "http://www.corpusdelespanol.org/x.asp", cookiejar = "cookie.txt", curl = ch) no cookie file is saved. What am I missing to reproduce in RCurl what I'm successfully doing in the Terminal? Thank you for your time and help. Earl Brown ----- Earl K. Brown, PhD Assistant Professor of Spanish Linguistics Advisor, TEFL MA Program Department of Modern Languages Kansas State University
Hi Earl, Have you read the libCurl documentation for CURLOPT_COOKIEJAR? : Pass a file name as char *, zero terminated. This will make libcurl write all internally known cookies to the specified file when curl_easy_cleanup(3)is called. If no cookies are known, no file will be created. Specify "-" to instead have the cookies written to stdout. Using this option also enables cookies for this session, so if you for example follow a location it will make matching cookies get sent accordingly. If the cookie jar file can't be created or written to (when the curl_easy_cleanup(3) is called), libcurl will not and cannot report an error for this. UsingCURLOPT_VERBOSE or CURLOPT_DEBUGFUNCTION will get a warning to display, but that is the only visible feedback you get about this possibly lethal situation. So it may be that Rcurl doesn't call curl_easy_cleanup, no cookies are known (seems unlikely), or curl can't create the file (so try adding the verbose config) Hadley On Sun, Aug 25, 2013 at 1:01 AM, Earl Brown <ekbrown at k-state.edu> wrote:> R-helpers, > > When I use cURL in the Terminal: > > curl --cookie-jar cookie.txt --url "http://corpusdelespanol.org/x.asp" --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0" --location --include > > a cookie file "cookie.txt" is saved to my working directory. However, when I try what I think is the equivalent command R with RCurl: > > ch <- getCurlHandle(followlocation = T, header = T, useragent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0") > getURL(url = "http://www.corpusdelespanol.org/x.asp", cookiejar = "cookie.txt", curl = ch) > > no cookie file is saved. > > What am I missing to reproduce in RCurl what I'm successfully doing in the Terminal? > > Thank you for your time and help. Earl Brown > > ----- > Earl K. Brown, PhD > Assistant Professor of Spanish Linguistics > Advisor, TEFL MA Program > Department of Modern Languages > Kansas State University > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Chief Scientist, RStudio http://had.co.nz/
Hi Earl The cookies will only be written to the file specified by the cookiejar option when the curl handle is garbage collected. If you use rm(ch) gc() the cookie.txt file should be created. This is the way libcurl behaves rather than something RCurl introduces. If you don't explicitly specify a curl handle in a request, the cookiejar option works as on expects because the implicit curl handle is destroyed at the end of the call and often garbage collection occurs. D. On 8/24/13 11:01 PM, Earl Brown wrote:> R-helpers, > > When I use cURL in the Terminal: > > curl --cookie-jar cookie.txt --url "http://corpusdelespanol.org/x.asp" --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0" --location --include > > a cookie file "cookie.txt" is saved to my working directory. However, when I try what I think is the equivalent command R with RCurl: > > ch <- getCurlHandle(followlocation = T, header = T, useragent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20100101 Firefox/23.0") > getURL(url = "http://www.corpusdelespanol.org/x.asp", cookiejar = "cookie.txt", curl = ch) > > no cookie file is saved. > > What am I missing to reproduce in RCurl what I'm successfully doing in the Terminal? > > Thank you for your time and help. Earl Brown > > ----- > Earl K. Brown, PhD > Assistant Professor of Spanish Linguistics > Advisor, TEFL MA Program > Department of Modern Languages > Kansas State University > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >