Thanks. Yes. I did that, it also has a verbose mode so that I could see what it was doing. What I needed was not just escaping but strict escaping. My memory forma number of years back was that I had issues with urlencode from base not being strict. And of course you don't what to encode twice. Thanks, -Roy> On Feb 19, 2020, at 7:08 AM, Ben Tupper <btupper at bigelow.org> wrote: > > Hi, > > Perhaps you could test it out by using httr::GET() with and without > escaping using xml2::url_escape()? > > https://www.rdocumentation.org/packages/xml2/versions/1.2.2/topics/url_escape > > Cheers, > Ben > > On Tue, Feb 18, 2020 at 1:29 PM Roy Mendelssohn - NOAA Federal via > R-help <r-help at r-project.org> wrote: >> >> Hi All: >> >> I hav been trying to go through the code for httr::GET() but it is somewhat beyond what I know. What I am trying to find out is if all urls are automatically percent encoded, or whether the user needs to do that. >> >> Thanks, >> >> -Roy >> >> ********************** >> "The contents of this message do not reflect any position of the U.S. Government or NOAA." >> ********************** >> Roy Mendelssohn >> Supervisory Operations Research Analyst >> NOAA/NMFS >> Environmental Research Division >> Southwest Fisheries Science Center >> ***Note new street address*** >> 110 McAllister Way >> Santa Cruz, CA 95060 >> Phone: (831)-420-3666 >> Fax: (831) 420-3980 >> e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." >> "From those who have been given much, much will be expected" >> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > > > -- > Ben Tupper > Bigelow Laboratory for Ocean Science > West Boothbay Harbor, Maine > http://www.bigelow.org/ > https://eco.bigelow.org********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
curl::curl_escape() ?
https://github.com/jeroen/curl/search?q=curl_escape&unscoped_q=curl_escape
? uses the underlying libcurl curl_easy_escape() which does proper
escaping b/c it's, well, curl.
{httr} uses curl::curl_escape() ?
https://github.com/r-lib/httr/search?q=curl_escape&unscoped_q=curl_escape
The use it's `url-query.r` is the function compose_query().
compose_query() is called from build_url() in url.r. handle_url()
(from handle-url.r) uses build_url().
All the "verbs" use handle_url() ?
https://github.com/r-lib/httr/search?q=handle_url&unscoped_q=handle_url
So {httr} relies on the quintessential standard in URL escaping ?
which is libcurl's ? for all URL machinations.
-boB
On Wed, Feb 19, 2020 at 10:36 AM Roy Mendelssohn - NOAA Federal via
R-help <r-help at r-project.org> wrote:>
> Thanks. Yes. I did that, it also has a verbose mode so that I could see
what it was doing. What I needed was not just escaping but strict escaping. My
memory forma number of years back was that I had issues with urlencode from base
not being strict. And of course you don't what to encode twice.
>
> Thanks,
>
> -Roy
>
>
> > On Feb 19, 2020, at 7:08 AM, Ben Tupper <btupper at bigelow.org>
wrote:
> >
> > Hi,
> >
> > Perhaps you could test it out by using httr::GET() with and without
> > escaping using xml2::url_escape()?
> >
> >
https://www.rdocumentation.org/packages/xml2/versions/1.2.2/topics/url_escape
> >
> > Cheers,
> > Ben
> >
> > On Tue, Feb 18, 2020 at 1:29 PM Roy Mendelssohn - NOAA Federal via
> > R-help <r-help at r-project.org> wrote:
> >>
> >> Hi All:
> >>
> >> I hav been trying to go through the code for httr::GET() but it is
somewhat beyond what I know. What I am trying to find out is if all urls are
automatically percent encoded, or whether the user needs to do that.
> >>
> >> Thanks,
> >>
> >> -Roy
> >>
> >> **********************
> >> "The contents of this message do not reflect any position of
the U.S. Government or NOAA."
> >> **********************
> >> Roy Mendelssohn
> >> Supervisory Operations Research Analyst
> >> NOAA/NMFS
> >> Environmental Research Division
> >> Southwest Fisheries Science Center
> >> ***Note new street address***
> >> 110 McAllister Way
> >> Santa Cruz, CA 95060
> >> Phone: (831)-420-3666
> >> Fax: (831) 420-3980
> >> e-mail: Roy.Mendelssohn at noaa.gov www:
https://www.pfeg.noaa.gov/
> >>
> >> "Old age and treachery will overcome youth and skill."
> >> "From those who have been given much, much will be
expected"
> >> "the arc of the moral universe is long, but it bends toward
justice" -MLK Jr.
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> >
> >
> > --
> > Ben Tupper
> > Bigelow Laboratory for Ocean Science
> > West Boothbay Harbor, Maine
> > http://www.bigelow.org/
> > https://eco.bigelow.org
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward
justice" -MLK Jr.
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
Thanks. Yes that is what I found out. We are switching our web service to require strict encoding (a lot of services got caught when Apache Tomcat made a similar switch) and are trying to see if it that will break any of our R packages before we make the switch, As always, appreciate all the people who took time to respond. -Roy> On Feb 22, 2020, at 5:23 AM, Bob Rudis <bob at rud.is> wrote: > > curl::curl_escape() ? > https://github.com/jeroen/curl/search?q=curl_escape&unscoped_q=curl_escape > ? uses the underlying libcurl curl_easy_escape() which does proper > escaping b/c it's, well, curl. > > {httr} uses curl::curl_escape() ? > https://github.com/r-lib/httr/search?q=curl_escape&unscoped_q=curl_escape > > The use it's `url-query.r` is the function compose_query(). > > compose_query() is called from build_url() in url.r. handle_url() > (from handle-url.r) uses build_url(). > > All the "verbs" use handle_url() ? > https://github.com/r-lib/httr/search?q=handle_url&unscoped_q=handle_url > > So {httr} relies on the quintessential standard in URL escaping ? > which is libcurl's ? for all URL machinations. > > -boB > > On Wed, Feb 19, 2020 at 10:36 AM Roy Mendelssohn - NOAA Federal via > R-help <r-help at r-project.org> wrote: >> >> Thanks. Yes. I did that, it also has a verbose mode so that I could see what it was doing. What I needed was not just escaping but strict escaping. My memory forma number of years back was that I had issues with urlencode from base not being strict. And of course you don't what to encode twice. >> >> Thanks, >> >> -Roy >> >> >>> On Feb 19, 2020, at 7:08 AM, Ben Tupper <btupper at bigelow.org> wrote: >>> >>> Hi, >>> >>> Perhaps you could test it out by using httr::GET() with and without >>> escaping using xml2::url_escape()? >>> >>> https://www.rdocumentation.org/packages/xml2/versions/1.2.2/topics/url_escape >>> >>> Cheers, >>> Ben >>> >>> On Tue, Feb 18, 2020 at 1:29 PM Roy Mendelssohn - NOAA Federal via >>> R-help <r-help at r-project.org> wrote: >>>> >>>> Hi All: >>>> >>>> I hav been trying to go through the code for httr::GET() but it is somewhat beyond what I know. What I am trying to find out is if all urls are automatically percent encoded, or whether the user needs to do that. >>>> >>>> Thanks, >>>> >>>> -Roy >>>> >>>> ********************** >>>> "The contents of this message do not reflect any position of the U.S. Government or NOAA." >>>> ********************** >>>> Roy Mendelssohn >>>> Supervisory Operations Research Analyst >>>> NOAA/NMFS >>>> Environmental Research Division >>>> Southwest Fisheries Science Center >>>> ***Note new street address*** >>>> 110 McAllister Way >>>> Santa Cruz, CA 95060 >>>> Phone: (831)-420-3666 >>>> Fax: (831) 420-3980 >>>> e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ >>>> >>>> "Old age and treachery will overcome youth and skill." >>>> "From those who have been given much, much will be expected" >>>> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. >>>> >>>> ______________________________________________ >>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >>>> and provide commented, minimal, self-contained, reproducible code. >>> >>> >>> >>> -- >>> Ben Tupper >>> Bigelow Laboratory for Ocean Science >>> West Boothbay Harbor, Maine >>> http://www.bigelow.org/ >>> https://eco.bigelow.org >> >> ********************** >> "The contents of this message do not reflect any position of the U.S. Government or NOAA." >> ********************** >> Roy Mendelssohn >> Supervisory Operations Research Analyst >> NOAA/NMFS >> Environmental Research Division >> Southwest Fisheries Science Center >> ***Note new street address*** >> 110 McAllister Way >> Santa Cruz, CA 95060 >> Phone: (831)-420-3666 >> Fax: (831) 420-3980 >> e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." >> "From those who have been given much, much will be expected" >> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code.********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn at noaa.gov www: https://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.