thr3ads.net - R help - [R] Problem with using read.csv with web address [Aug 2008]

If this information is useful, please help other people find it:
Share via:

Aiste Aistike

2008-Aug-12 21:03 UTC

[R] Problem with using read.csv with web address

Hello,

I have a problem with using read.csv(). I want to read a table from the ONS
neighborhood statistics website which has an address:

http://www.neighbourhood.statistics.gov.uk/dissemination/LeadTableView.do?a=3&b=5940246&c=Stoke&d=14&e=7&f=73&g=401878&i=1001x1003x1004x1005&l=145&o=1&m=0&r=1&s=1211997884568&enc=1&downloadTable=Download+this+table

I was trying the following:

#the parameter FamilyId I will need to change to get other datasets
url1 <-  paste("
http://www.neighbourhood.statistics.gov.uk/dissemination/LeadTableView.do?a=3&b=5940246&c=Stoke&d=14&e=7&f=73&g=401878&i=1001x1003x1004x1005&l=",FamilyId,"&o=1&m=0&r=1&s=1211997884568&enc=1&downloadTable=Download+this+table",
sep="")
x <- read.csv(url1, header = FALSE, stringsAsFactors = FALSE)

I was expecting x to be like this:
# because that's the structure I get if I download the file to my computer
and then try to use read.csv()
1
Housing Stock (UV53)
2

3

4
All Household Spaces
5
Occupied Household Spaces
6
Unoccupied Household Spaces
7
Unoccupied Household Spaces: Second residence / holiday accommodation
8
Unoccupied Household Spaces: Vacant household space
9
Housing Stock (UV53), Apr01
10
Housing Stock (UV53), Apr01
11
Housing Stock (UV53)
12 This material is Crown Copyright. Users are granted permission to
reproduce Crown Copyright material provided that a Click-Use Licence has
been obtained from HMSO. The Click-Use Licence can be obtained from
http://www.clickanduse.hmso.gov.uk. When reproducing this material, the
source should be acknowledged.
                    V2                             V3    V4
V5                V6         V7       V8
1

2                                                           Stoke
Plymouth South West  England
3                                                            Ward Unitary
Authority     Region  Country
4                Count                     Households Apr01  5784
106583    2185966 21262825
5                Count                     Households Apr01  5472
102540    2085984 20451427
6                Count                     Households Apr01
312              4043      99982   811398
7                Count                     Households Apr01
21               336      38381   135202
8                Count                     Households Apr01
291              3707      61601   676196
9          LastUpdated               18 November
2004
10              Source Office for National
Statistics
11 National
Statistics

12


However, I get 107 lines of text. Here is a bit of it:

1
<!DOCTYPE HTML PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd><html<http://www.w3.org/TR/html4/strict.dtd%3E%3Chtml>>2
<head>
3

4
<meta http-equiv=Content-Type content=text/html; charset=iso-8859-1>
5
<meta name=robots content=no follow>
6


<..............>

105
</noscript>
106
</body>
107
</html>

What am I doing wrong? Should this work?

Thank you in advance.


Aiste

	[[alternative HTML version deleted]]

Barry Rowlingson

2008-Aug-12 22:19 UTC

head link

[R] Problem with using read.csv with web address

2008/8/12 Aiste Aistike <aiste.aistike at
gmail.com>:> Hello,
>
> I have a problem with using read.csv(). I want to read a table from the ONS
> neighborhood statistics website which has an address:
>
>
http://www.neighbourhood.statistics.gov.uk/dissemination/LeadTableView.do?a=3&b=5940246&c=Stoke&d=14&e=7&f=73&g=401878&i=1001x1003x1004x1005&l=145&o=1&m=0&r=1&s=1211997884568&enc=1&downloadTable=Download+this+table
>
> I was trying the following:
>
> #the parameter FamilyId I will need to change to get other datasets
> url1 <-  paste("
>
http://www.neighbourhood.statistics.gov.uk/dissemination/LeadTableView.do?a=3&b=5940246&c=Stoke&d=14&e=7&f=73&g=401878&i=1001x1003x1004x1005&l=",FamilyId,"&o=1&m=0&r=1&s=1211997884568&enc=1&downloadTable=Download+this+table",
> sep="")
> x <- read.csv(url1, header = FALSE, stringsAsFactors = FALSE)
 It looks like that URL doesn't actually return the CSV. The page you
get back has a 'refresh' meta-tag in it which sends your web browser
(firefox/IE) to another location. R doesn't care about this - because
it makes no attempt to make sense of the returned page - it just
returns what it got with the web request.

 There's also bits of JavaScript and other cruft in there. I can't
find a direct URL that gets the CSV straightaway which is what you
need. There's session id's and so on in the URL which might be
confusing things. I'll keep nosing around because this is a wonderful
example of how *not* to do web design.

What a mess!

Barry

R help - Aug 2008 - Problem with using read.csv with web address

[R] Problem with using read.csv with web address

[R] Problem with using read.csv with web address

Possibly Parallel Threads