b.rowlingson@lancaster.ac.uk
2003-May-26 18:48 UTC
[Rd] internet.R test hangs if http proxy needed (PR#3108)
Full_Name: Barry Rowlingson Version: 1.7.0 OS: RH8/Mandrake 9.1 Submission from: (NULL) (194.80.32.8) internet.R in the tests directory hangs during 'make check' if an http proxy is needed for http access from the machine. This happens in the httpget function defined therein. It tries to make a direct socket connection to port 80 of a remote machine, and if local network policy does not permit that, then the test can hang. A fix? Perhaps skip that test if Sys.getenv("http_proxy") is defined. Or rewrite httpget to use the proxy if defined. Barry
Prof Brian Ripley
2003-May-27 09:45 UTC
[Rd] internet.R test hangs if http proxy needed (PR#3108)
This is intended to be a test of the socket functions. It should fail within the time-out period, not hang. On Mon, 26 May 2003 b.rowlingson@lancaster.ac.uk wrote:> Full_Name: Barry Rowlingson > Version: 1.7.0 > OS: RH8/Mandrake 9.1 > Submission from: (NULL) (194.80.32.8) > > > internet.R in the tests directory hangs during 'make check' if an http proxy is > needed for http access from the machine. This happens in the httpget function > defined therein. > > It tries to make a direct socket connection to port 80 of a remote machine, and > if local network policy does not permit that, then the test can hang.Why does it hang? Isn't that the bug that needs fixing? Do the other tests fail gracefully if no proxy is set? (They all do the couple of places I have tried them in very restricted setups.)> A fix? Perhaps skip that test if Sys.getenv("http_proxy") is defined. Or rewrite > httpget to use the proxy if defined.There seem to be two possibilities: a) make.socket is failing and hanging or b) the local setup is listening on the port and not sending replies, or replies that cannot be parsed. We need help, as to reproduce this needs some particular local setup and there have been no reports from anyone during the beta-test period for 1.7.0, nor from previous versions. (That makes me suspect it is rare local setup.) Can you please debug this further. -- Brian D. Ripley, ripley@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Barry Rowlingson
2003-May-27 14:23 UTC
[Rd] internet.R test hangs if http proxy needed (PR#3108)
Prof Brian Ripley wrote:> We need help, as to reproduce this needs some particular local setup and > there have been no reports from anyone during the beta-test period for > 1.7.0, nor from previous versions. (That makes me suspect it is rare > local setup.) Can you please debug this further. >It appears to be partly a bug in my impatience:>system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat")) Error in make.socket(host, port = port) : Socket not established Timing stopped at: 0 0 189 0 0 The function times out after 189 seconds, whereas my impatience timeout is 3 minutes. I will upgrade my impatience to the latest version so that it is compatible with everyone elses. At first I thought it was taking 3 * options()$timeout to produce the timeout in make.socket. A quick investigation shows that to be purely coincidental with the initial value of options()$timeout:> options(timeout=2) >system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat")) making socket Error in make.socket(host, port = port) : Socket not established Timing stopped at: 0 0 189.04 0 0 The 189 second timeout appears to be part of our particular local setup (port 80 is blocked off-site). options()$timeout seems to have no effect on connections that fail due to non-use of our proxies. Is this a bug? The doc claims: timeout: integer. The timeout for Internet operations, in seconds. Default 60 seconds. which is a bit vague. If I do something like: read.table("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat") without setting my proxy I get 189 seconds, regardless of options()$timeout yet it is undoubtedly an internet operation. Back to the original problem: the httpget function in tests/internet.R can be modified to obey an http_proxy setting with a fairly simple adjustment near the start: hp <- Sys.getenv("http_proxy") if(hp != ""){ ## we need to extract host and port from the proxy: ## split on / or : thusly: ## [http]:[]/[]/[hostname]:[port] ## 1 2 3 4 5 hpbits <- strsplit(hp,"[/:]")[[1]] host <- hpbits[4] port <- hpbits[5] ## ask the proxy for the full URL with all its http:// glory: rurl <- url }else{ ## no proxy, connect direct and ask for the URL relative to root: host <- urlel[3] rurl <- paste(c("", urlel[-(1:3)]), collapse = "/") } I still have some qualms about tests that rely on things external to the test site, but without including a socket server program (which would need its own set of tests, I suppose) I cant see any way of testing socket functionality without dribbling on the internet at large. Barry