Puppet users - Nov 2013 - Issuing web request (REST) from puppet manifest

Hi,

I''m looking for a resource / type that will allow me to issue web
requests
from a manifest file. I''ve heard about the web_request module , but
seems
quite dificult to use (in fact I do not succeed in issuing a POST request 
with a JSON body doc).

Any ideas about that?

Thanks.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/91813adc-b4c7-4606-bc20-9ad18a0087ee%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Generically, this sounds like you''re trying to retrieve some external
information and hiera is still a better option. In hiera you would have your
backend fetching data from $someplace, wherever that place is (ldap, yaml,
etc.). Maybe even write yourself a new backend.

More specifically, discussing the underlying functionality you are attempting to
implement would help us figure out how to help you.

On Wed, Nov 13, 2013 at 08:03:35AM -0800, ulrich igor ngouagna kouete
wrote:
>    Hi,
>    I''m looking for a resource / type that will allow me to issue
web requests
>    from a manifest file. I''ve heard about the web_request module ,
but seems
>    quite dificult to use (in fact I do not succeed in issuing a POST
request
>    with a JSON body doc).
>    Any ideas about that?
>    Thanks.
> 
>    --
>    You received this message because you are subscribed to the Google
Groups
>    "Puppet Users" group.
>    To unsubscribe from this group and stop receiving emails from it, send
an
>    email to puppet-users+unsubscribe@googlegroups.com.
>    To view this discussion on the web visit
>   
[1]https://groups.google.com/d/msgid/puppet-users/91813adc-b4c7-4606-bc20-9ad18a0087ee%40googlegroups.com.
>    For more options, visit [2]https://groups.google.com/groups/opt_out.
> 
> References
> 
>    Visible links
>    1.
https://groups.google.com/d/msgid/puppet-users/91813adc-b4c7-4606-bc20-9ad18a0087ee%40googlegroups.com
>    2. https://groups.google.com/groups/opt_out
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/20131113174238.GA25208%40iniquitous.heresiarch.ca.
For more options, visit https://groups.google.com/groups/opt_out.

On Wednesday, November 13, 2013 10:03:35 AM UTC-6, ulrich igor ngouagna 
kouete wrote:
>
> Hi,
>
> I''m looking for a resource / type that will allow me to issue web
requests
> from a manifest file. I''ve heard about the web_request module ,
but seems
> quite dificult to use (in fact I do not succeed in issuing a POST request 
> with a JSON body doc).
>
> Any ideas about that?
>
>
Custom resources types are not well suited to the job you describe.  If the 
objective is to retrieve data from a web service into a Puppet variable (or 
similar), then you probably want a function.  The general-purpose 
generate() function could be applied to the task, perhaps in combination 
with wget.  Alternatively, you could look into writing a custom function 
for the purpose -- it''s not that hard if you have a passing familiarity
with Ruby.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/45e09f18-7e9a-4134-896f-7ef37a643e1f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,
thanks for reacting.

To describe more what I want to do:

I have a web API and I want to make some REST requests to it, and verify 
the response satus or body to make sure the request succeeded. With
"curl"
it gives this:

exec{"add a cluster ${clusterName}":
          command    => "curl -u 
${$::clouderapuppet::cm::server::params::cmUsername}:${$::clouderapuppet::cm::server::params::cmUsername}
-X POST -H \"Content-Type:application/json\" 
''http://localhost:7180/api/v4/clusters'' -d 
''{\"items\":[{\"name\":\"${clusterName}\",
\"version\":\"${clusterVersion}\"}]}''"
      }

See how it is long and annoying? (hahah ;) ). And imagine I have to POST a 
huge JSON doc, the code will be unreadable!

Do you tell me I can do that with hiera or generate() function?

I''ll explore those paths now, but feel free to continue the discussion
:)

thanks a lot.

Le mercredi 13 novembre 2013 17:03:35 UTC+1, ulrich igor ngouagna kouete a 
écrit :
>
> Hi,
>
> I''m looking for a resource / type that will allow me to issue web
requests
> from a manifest file. I''ve heard about the web_request module ,
but seems
> quite dificult to use (in fact I do not succeed in issuing a POST request 
> with a JSON body doc).
>
> Any ideas about that?
>
> Thanks.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/a09bca0f-6b45-4875-bed4-bb8c4d3fd293%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, November 14, 2013 3:06:05 AM UTC-6, ulrich igor ngouagna 
kouete wrote:
>
> Hi,
> thanks for reacting.
>
> To describe more what I want to do:
>
> I have a web API and I want to make some REST requests to it, and verify 
> the response satus or body to make sure the request succeeded. With
"curl"
> it gives this:
>
> exec{"add a cluster ${clusterName}":
>           command    => "curl -u 
>
${$::clouderapuppet::cm::server::params::cmUsername}:${$::clouderapuppet::cm::server::params::cmUsername}
> -X POST -H \"Content-Type:application/json\" ''
> http://localhost:7180/api/v4/clusters'' -d 
>
''{\"items\":[{\"name\":\"${clusterName}\",
> \"version\":\"${clusterVersion}\"}]}''"
>       }
>
> See how it is long and annoying? (hahah ;) ). And imagine I have to POST a 
> huge JSON doc, the code will be unreadable!
>
> Do you tell me I can do that with hiera or generate() function?
>
> I''ll explore those paths now, but feel free to continue the
discussion :)
>

On master/agent setups there is an exceedingly important distinction 
between doing this sort of thing (or anything, really) via a resource and 
doing it via a function: resources are applied on the client, after catalog 
compilation is complete, whereas functions are evaluated on the master, 
during catalog compilation.  Therefore, if you want to collect data to 
inform catalog compilation or catalog contents then it must be done via a 
function (or earlier, via a fact), but if you want to manage or test some 
aspect of the target node then you must use a resource, or perhaps a fact.

When you described your problem as wanting to issue a web request "from a 
manifest file", I supposed you had the former case, as manifests are a 
detail of catalog compilation, not directly relevant to clients.  Inasmuch 
as your example directs its web request to localhost, however, it appears 
that you may instead be trying to test details of the configuration of the 
target node.  If that''s so then a function (including hiera) will not
do
the job you require.

If you want to know about the behavior of web requests prior to application 
of any resources, and especially if you want to use that information to 
influence the contents of the node''s catalog, then you need to write a 
custom fact.  Alternatively, if you want to issue a web request in the 
middle of catalog *application*, and possibly to condition whether certain 
other declared resources are applied on the result of the request, then you 
need a resource.

I have no specific experience with the mmorsi-web_request module, and its 
description on the Forge doesn''t provide much detail.  It sounds like
it is
supposed to support behavior similar to what you may be after, however.  
Certainly, it claims to be able to issue POST requests, though I see no 
particular evidence that it supports arbitrary request bodies or allows 
arbitrary headers to be provided.  If it does not have such facilities then 
you could consider submitting a feature request or else modifying the 
module yourself to add the feature you need.

On the other hand, an Exec is certainly capable of doing the job in 
combination with curl, wget, or similar on the target system.  You appear 
to have that figured out already.  Your concern about the complexity of the 
Puppet declaration you must use in that case is well-founded, but inasmuch 
as in that regard you seem more worried about the content body than about 
anything else, you will have that issue even with mmorsi-web_request or any 
similar module.  I would suggest handling it by generating the content body 
via an external ERB template, using Puppet''s built-in template()
function.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/b66ce8ac-ade6-4e99-9d51-a9305a979d47%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I haven''t coded this sort of thing but I know of one example. We use a
module where I work to configure our Opsview monitoring software. It sends
requests to the Opsview REST API, parses the responses, and sends more
requests to update anything that needs it.

If you want to look at it:

https://forge.puppetlabs.com/cparedes/puppet_opsview


Rich




On Thu, Nov 14, 2013 at 1:06 AM, ulrich igor ngouagna kouete <
igor.ngouagna@gmail.com> wrote:
> Hi,
> thanks for reacting.
>
> To describe more what I want to do:
>
> I have a web API and I want to make some REST requests to it, and verify
> the response satus or body to make sure the request succeeded. With
"curl"
> it gives this:
>
> exec{"add a cluster ${clusterName}":
>           command    => "curl -u
>
${$::clouderapuppet::cm::server::params::cmUsername}:${$::clouderapuppet::cm::server::params::cmUsername}
> -X POST -H \"Content-Type:application/json\" ''
> http://localhost:7180/api/v4/clusters'' -d
>
''{\"items\":[{\"name\":\"${clusterName}\",
> \"version\":\"${clusterVersion}\"}]}''"
>       }
>
> See how it is long and annoying? (hahah ;) ). And imagine I have to POST a
> huge JSON doc, the code will be unreadable!
>
> Do you tell me I can do that with hiera or generate() function?
>
> I''ll explore those paths now, but feel free to continue the
discussion :)
>
> thanks a lot.
>
> Le mercredi 13 novembre 2013 17:03:35 UTC+1, ulrich igor ngouagna kouete a
> écrit :
>
>> Hi,
>>
>> I''m looking for a resource / type that will allow me to issue
web
>> requests from a manifest file. I''ve heard about the
web_request module ,
>> but seems quite dificult to use (in fact I do not succeed in issuing a
POST
>> request with a JSON body doc).
>>
>> Any ideas about that?
>>
>> Thanks.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/puppet-users/a09bca0f-6b45-4875-bed4-bb8c4d3fd293%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAPGcbtBsnJ4922LnR4SXkzG6iezxf5yMYfnLJCQy_nJ4uyu9dg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.