R help - May 2018 - help with json data from the web into data frame in R

I?ve been tinkering and discovered that the link I need to read json data from
is ?https? and there is a certificate warning that I have to click through from
a browser. That might be my issue. Is there any way in the json package to tell
it to ignore self-signed cert errors in a url?

-Rich

From: Eric Berger [mailto:ericjberger at gmail.com]
Sent: Tuesday, May 08, 2018 11:31 AM
To: Evans, Richard K. (GRC-H000)
Cc: r-help at r-project.org
Subject: Re: [R] help with json data from the web into data frame in R

Hi Rich,
Take a look at the function fromJSON found in the rjson package.
Note that the Usage in the help page: ?fromJSON
names the second argument 'file' but if you look at the description the
argument can be a URL.

HTH,
Eric


On Tue, May 8, 2018 at 6:16 PM, Evans, Richard K. (GRC-H000) <richard.k.evans
at nasa.gov<mailto:richard.k.evans at nasa.gov>> wrote:
Hello

I am able to construct a url that points to some data online in the JSON format.
See an example at [0].

I would like to work with this data as a dataframe in R.

I know that there is a package for handling json data [1] but it assumes the
data is in a local file but It is not clear to me how to request the data from
the web in an R script and get the json data converted into a data frame in R.

Can anyone provide a basic example or some guidance please?

-Rich (revansx)

[0]
https://www.semantic-mediawiki.org/w/api.php?action=ask&query=[[Category:City]]&format=json<https://www.semantic-mediawiki.org/w/api.php?action=ask&query=%5b%5bCategory:City%5d%5d&format=json>
[1] https://www.tutorialspoint.com/r/r_json_files.htm

______________________________________________
R-help at r-project.org<mailto:R-help at r-project.org> mailing list -- To
UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


	[[alternative HTML version deleted]]

> On May 8, 2018, at 8:36 AM, Evans, Richard K. (GRC-H000)
<richard.k.evans at nasa.gov> wrote:
> 
> I?ve been tinkering and discovered that the link I need to read json data
from is ?https? and there is a certificate warning that I have to click through
from a browser. That might be my issue. Is there any way in the json package to
tell it to ignore self-signed cert errors in a url?
I didn't have that issue when using the link you offered:

library(jsonlite)
myJSON <- fromJSON(
url("https://www.semantic-mediawiki.org/w/api.php?action=ask&query=%5B%5BCategory:City%5D%5D&format=json")
)

# results in a complex list (not trivially reducible to a dataframe:

str(myJSON)
List of 1
 $ query:List of 5
  ..$ printrequests:'data.frame':	1 obs. of  5 variables:
  .. ..$ label : chr ""
  .. ..$ key   : chr ""
  .. ..$ redi  : chr ""
  .. ..$ typeid: chr "_wpg"
  .. ..$ mode  : int 2
  ..$ results      :List of 39
  .. ..$ File:2166320938 5cfc9ec72a z.jpg                      :List of 6
  .. .. ..$ printouts   : list()
  .. .. ..$ fulltext    : chr "File:2166320938 5cfc9ec72a z.jpg"
  .. .. ..$ fullurl     : chr
"https://www.semantic-mediawiki.org/wiki/File:2166320938_5cfc9ec72a_z.jpg"
#-----trimmed-----------

David
> 
> -Rich
> 
> From: Eric Berger [mailto:ericjberger at gmail.com]
> Sent: Tuesday, May 08, 2018 11:31 AM
> To: Evans, Richard K. (GRC-H000)
> Cc: r-help at r-project.org
> Subject: Re: [R] help with json data from the web into data frame in R
> 
> Hi Rich,
> Take a look at the function fromJSON found in the rjson package.
> Note that the Usage in the help page: ?fromJSON
> names the second argument 'file' but if you look at the description
the argument can be a URL.
> 
> HTH,
> Eric
> 
> 
> On Tue, May 8, 2018 at 6:16 PM, Evans, Richard K. (GRC-H000)
<richard.k.evans at nasa.gov<mailto:richard.k.evans at nasa.gov>>
wrote:
> Hello
> 
> I am able to construct a url that points to some data online in the JSON
format.  See an example at [0].
> 
> I would like to work with this data as a dataframe in R.
> 
> I know that there is a package for handling json data [1] but it assumes
the data is in a local file but It is not clear to me how to request the data
from the web in an R script and get the json data converted into a data frame in
R.
> 
> Can anyone provide a basic example or some guidance please?
> 
> -Rich (revansx)
> 
> [0]
https://www.semantic-mediawiki.org/w/api.php?action=ask&query=[[Category:City]]&format=json<https://www.semantic-mediawiki.org/w/api.php?action=ask&query=%5b%5bCategory:City%5d%5d&format=json>
> [1] https://www.tutorialspoint.com/r/r_json_files.htm
> 
> ______________________________________________
> R-help at r-project.org<mailto:R-help at r-project.org> mailing list
-- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.' 
-Gehm's Corollary to Clarke's Third Law

Right. I'm trying to access a server within my organization which has a cert
error that I cannot fix.

The example link I provided was to a site on the web that does not have the cert
error.

From the linux shell I use the "-k" switch with cURL to ignore cert
errors.. is there an equivalent in the R world?

-Rich


-----Original Message-----
From: David Winsemius [mailto:dwinsemius at comcast.net] 
Sent: Tuesday, May 08, 2018 11:48 AM
To: Evans, Richard K. (GRC-H000)
Cc: Eric Berger; r-help at r-project.org
Subject: Re: [R] help with json data from the web into data frame in R

> On May 8, 2018, at 8:36 AM, Evans, Richard K. (GRC-H000)
<richard.k.evans at nasa.gov> wrote:
> 
> I?ve been tinkering and discovered that the link I need to read json data
from is ?https? and there is a certificate warning that I have to click through
from a browser. That might be my issue. Is there any way in the json package to
tell it to ignore self-signed cert errors in a url?
I didn't have that issue when using the link you offered:

library(jsonlite)
myJSON <- fromJSON(
url("https://www.semantic-mediawiki.org/w/api.php?action=ask&query=%5B%5BCategory:City%5D%5D&format=json")
)

# results in a complex list (not trivially reducible to a dataframe:

str(myJSON)
List of 1
 $ query:List of 5
  ..$ printrequests:'data.frame':	1 obs. of  5 variables:
  .. ..$ label : chr ""
  .. ..$ key   : chr ""
  .. ..$ redi  : chr ""
  .. ..$ typeid: chr "_wpg"
  .. ..$ mode  : int 2
  ..$ results      :List of 39
  .. ..$ File:2166320938 5cfc9ec72a z.jpg                      :List of 6
  .. .. ..$ printouts   : list()
  .. .. ..$ fulltext    : chr "File:2166320938 5cfc9ec72a z.jpg"
  .. .. ..$ fullurl     : chr
"https://www.semantic-mediawiki.org/wiki/File:2166320938_5cfc9ec72a_z.jpg"
#-----trimmed-----------

David
> 
> -Rich
> 
> From: Eric Berger [mailto:ericjberger at gmail.com]
> Sent: Tuesday, May 08, 2018 11:31 AM
> To: Evans, Richard K. (GRC-H000)
> Cc: r-help at r-project.org
> Subject: Re: [R] help with json data from the web into data frame in R
> 
> Hi Rich,
> Take a look at the function fromJSON found in the rjson package.
> Note that the Usage in the help page: ?fromJSON names the second 
> argument 'file' but if you look at the description the argument can
be a URL.
> 
> HTH,
> Eric
> 
> 
> On Tue, May 8, 2018 at 6:16 PM, Evans, Richard K. (GRC-H000)
<richard.k.evans at nasa.gov<mailto:richard.k.evans at nasa.gov>>
wrote:
> Hello
> 
> I am able to construct a url that points to some data online in the JSON
format.  See an example at [0].
> 
> I would like to work with this data as a dataframe in R.
> 
> I know that there is a package for handling json data [1] but it assumes
the data is in a local file but It is not clear to me how to request the data
from the web in an R script and get the json data converted into a data frame in
R.
> 
> Can anyone provide a basic example or some guidance please?
> 
> -Rich (revansx)
> 
> [0] 
> https://www.semantic-mediawiki.org/w/api.php?action=ask&query=[[Catego
>
ry:City]]&format=json<https://www.semantic-mediawiki.org/w/api.php?act
> ion=ask&query=%5b%5bCategory:City%5d%5d&format=json>
> [1] https://www.tutorialspoint.com/r/r_json_files.htm
> 
> ______________________________________________
> R-help at r-project.org<mailto:R-help at r-project.org> mailing list
-- To
> UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see 
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.' 
-Gehm's Corollary to Clarke's Third Law