I''m wondering what people are doing systems provisioning with, ie the process that gets puppet installed onto a system, running for the first time, and also the handling of certificate signing and so forth. I don''t see this topic discussed much. The mc-provision tools at https://github.com/ripienaar/mcollective-server-provisioner don''t seem to be actively developed anymore, or at least I wasn''t able to find enough documentation to be able to effectively make use of it. Doug -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
----- Original Message -----
| I''m wondering what people are doing systems provisioning with, ie the
| process that gets puppet installed onto a system, running for the
| first time, and also the handling of certificate signing and so
| forth.
| I don''t see this topic discussed much.
|
| The mc-provision tools at
| https://github.com/ripienaar/mcollective-server-provisioner don''t
| seem
| to be actively developed anymore, or at least I wasn''t able to find
| enough documentation to be able to effectively make use of it.
|
| Doug
We have some custom written tools that look after management of our systems
since we''re integrating with a very large Active Directory in an
environment that requires us to pre-stage machines into certain locations. This
tool generates a puppet manifest that manages the generation and removal of
puppet keys from a central mirror server that all of our clients fetch as part
of our post configuration.
Essentially, the hosts ensure that they have a proper FQDN, then they fetch the
puppet certificates from the mirror server and proceed to run puppet and
configure themselves.
You can also use other tools like The Foreman to provision machines.
MCollective is certainly actively developed but perhaps that module is not.
--
James A. Peltier
Manager, IT Services - Research Computing Group
Simon Fraser University - Burnaby Campus
Phone : 778-782-6573
Fax : 778-782-3045
E-Mail : jpeltier@sfu.ca
Website : http://www.sfu.ca/itservices
http://blogs.sfu.ca/people/jpeltier
Success is to be measured not so much by the position that one has reached
in life but as by the obstacles they have overcome. - Booker T. Washington
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
We use Cobbler for our Centos/RHEL kickstarts (around 150+ nodes, mainly VMs) and a very simple .ks that adds puppetd. on first boot it starts up and we manually sign the CSR on the puppet master. Historically, we didn''t want tight integration between provisioning and CM as we were new to both and if one turned out to be a lemon we didn''t want to have to abandon both. We only provision a few nodes a day max. so that works ok. Lately I''ve been thinking of replacing Cobbler with The Foreman, which has better Puppet integration. (our team are happy with editing nodes.pp and Hiera and have never really needed anything like advanced storeconfig etc. but we build a lot of servers for external teams and being able to expose config options etc. is the real driver for Foreman). On 16 September 2012 07:21, Douglas Garstang <doug.garstang@gmail.com> wrote:> I''m wondering what people are doing systems provisioning with, ie the > process that gets puppet installed onto a system, running for the > first time, and also the handling of certificate signing and so forth. > I don''t see this topic discussed much. > > The mc-provision tools at > https://github.com/ripienaar/mcollective-server-provisioner don''t seem > to be actively developed anymore, or at least I wasn''t able to find > enough documentation to be able to effectively make use of it. > > Doug > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
We''re doing a simple combination of PXE boot, kickstart, and internal repos, which then hands off to puppet. We also use some rvc for vmware provisioning, which is mostly what happens in our environment. Not many people appear to be talking about razor<https://github.com/puppetlabs/Razor/wiki>. That, plus puppetdb, seems to offer a way to better handle bare metal provisioning and inventory. Since we''re getting into hadoop for a couple projects which will use a collection of bare metal, it looks compelling, but we haven''t used it yet. -Domenick On Sunday, September 16, 2012 2:22:03 AM UTC-4, Douglas wrote:> > I''m wondering what people are doing systems provisioning with, ie the > process that gets puppet installed onto a system, running for the > first time, and also the handling of certificate signing and so forth. > I don''t see this topic discussed much. > > The mc-provision tools at > https://github.com/ripienaar/mcollective-server-provisioner don''t seem > to be actively developed anymore, or at least I wasn''t able to find > enough documentation to be able to effectively make use of it. > > Doug >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/_VZVxlVbKaUJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 09/16/2012 11:31 AM, Dick Davies wrote:> We use Cobbler for our Centos/RHEL kickstarts (around 150+ nodes, mainly VMs) > and a very simple .ks that adds puppetd. on first boot it starts up > and we manually sign the CSR on the puppet master.+1 except that we allow autosigning from certain subnets.> Historically, we didn''t want tight integration between provisioning and CM as we > were new to both and if one turned out to be a lemon we didn''t want to have to > abandon both.We started with both together at the same time :)> We only provision a few nodes a day max. so that works ok. Lately I''ve been > thinking of replacing Cobbler with The Foreman, which has better > Puppet integration.What does foreman provide better than cobbler? -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi There, I manage a relatively large RHEL environment, we handle provisioning as follows: - PXE + Kickstart to bootstrap and install the base OS + Puppet client onto the platform, be it VMWare or bare metal - Kickstart post scripts put a basic puppet configuration file in place on the host, and a number of the values for things such as environment and puppetmaster come from Foreman''s Macro''s, this allows values in the ENC to flow into the kickstart files before your first puppet run. We then run in the %post section of the kickstart file the following: - A Puppet run that bootstraps the puppet client using tags ie, --tags puppet::client - A full puppet run via puppet agent -tov which applys the SOE to the platform That provides on first boot a fully configured RHEL server that includes all our additional software and customisations in about 3-5 minutes (not including POST) In regards to certs, we have a relatively open autosign.conf on our build networks, so we can provision servers , physical or virtual quite quickly by just hitting F12 for a network boot. I am sure there are some cleaner/more secure things we can do provisioning wise, however these have been slightly hindered by the RHN Satellite server i''ve been slowly pulling out of the environment at the same time, as it had the potential to break things if i wasnt careful. ENC wise, I can''t recommend Foreman enough, version 1.x is just brilliant, you can see the macros it can provide here: http://theforeman.org/projects/foreman/wiki/TemplateWriting Hope this helps, K On Sunday, September 16, 2012 7:22:03 AM UTC+1, Douglas wrote:> > I''m wondering what people are doing systems provisioning with, ie the > process that gets puppet installed onto a system, running for the > first time, and also the handling of certificate signing and so forth. > I don''t see this topic discussed much. > > The mc-provision tools at > https://github.com/ripienaar/mcollective-server-provisioner don''t seem > to be actively developed anymore, or at least I wasn''t able to find > enough documentation to be able to effectively make use of it. > > Doug >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/NrKmbHHiaq8J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I probably should have been clearer with my question. I was more interested in how people are managing certificates? Even if you use autosign, you still need to clean certificates manually. Doug. On Mon, Sep 17, 2012 at 6:25 AM, Keiran Sweet <keiran@gmail.com> wrote:> Hi There, > I manage a relatively large RHEL environment, we handle provisioning as > follows: > > - PXE + Kickstart to bootstrap and install the base OS + Puppet client onto > the platform, be it VMWare or bare metal > - Kickstart post scripts put a basic puppet configuration file in place on > the host, and a number of the values for things such as environment and > puppetmaster come from Foreman''s Macro''s, this allows values in the ENC to > flow into the kickstart files before your first puppet run. > > We then run in the %post section of the kickstart file the following: > - A Puppet run that bootstraps the puppet client using tags ie, --tags > puppet::client > - A full puppet run via puppet agent -tov which applys the SOE to the > platform > > That provides on first boot a fully configured RHEL server that includes all > our additional software and customisations in about 3-5 minutes (not > including POST) > > In regards to certs, we have a relatively open autosign.conf on our build > networks, so we can provision servers , physical or virtual quite quickly by > just hitting F12 for a network boot. I am sure there are some cleaner/more > secure things we can do provisioning wise, however these have been slightly > hindered by the RHN Satellite server i''ve been slowly pulling out of the > environment at the same time, as it had the potential to break things if i > wasnt careful. > > ENC wise, I can''t recommend Foreman enough, version 1.x is just brilliant, > you can see the macros it can provide here: > http://theforeman.org/projects/foreman/wiki/TemplateWriting > > Hope this helps, > > K > > > > > > > > > > On Sunday, September 16, 2012 7:22:03 AM UTC+1, Douglas wrote: >> >> I''m wondering what people are doing systems provisioning with, ie the >> process that gets puppet installed onto a system, running for the >> first time, and also the handling of certificate signing and so forth. >> I don''t see this topic discussed much. >> >> The mc-provision tools at >> https://github.com/ripienaar/mcollective-server-provisioner don''t seem >> to be actively developed anymore, or at least I wasn''t able to find >> enough documentation to be able to effectively make use of it. >> >> Doug > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/NrKmbHHiaq8J. > > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en.-- Regards, Douglas Garstang http://www.linkedin.com/in/garstang Email: doug.garstang@gmail.com Cell: +1-805-340-5627 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Similar setup here: pxe boot, etc. but we keep a pxe booted sparse base image available to clone. And use a script to to do the clone so that we can provision more quickly (saves us the pxe boot time). The script also interacts with puppet dashboard to set the machine role. We can provision a new machine in less than 4 minutes. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ssItzXfne8oJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
If you want the least amount of headache at the cost of security, here
is a sanitised extract from my kickstarts:
#LB: attempt to revoke and delete the certificate for this hostname, this should
stop us having
#to manually clean off every cert.
curl -k -X PUT -H "Content-Type: text/pson" --data
''{"desired_state":"revoked"}''
https://puppet:8140/production/certificate_status/$HOSTNAME
curl -k -X DELETE -H "Accept: pson"
https://puppet:8140/production/certificate_status/$HOSTNAME
#LB: run Puppet, our hostname should be set correctly by now
puppet agent --test --pluginsync --report --environment testing
You will need this in auth.conf on your master:
#allow hosts to delete their own certificates
#path /certificate_status/([^/]+)$
path ~ /certificate_status/([^/]+)$
auth any
allow $1
Hope that helps,
-Luke
On 17/09/12 19:16, Douglas Garstang wrote:> I probably should have been clearer with my question. I was more
> interested in how people are managing certificates? Even if you use
> autosign, you still need to clean certificates manually.
>
> Doug.
>
> On Mon, Sep 17, 2012 at 6:25 AM, Keiran Sweet <keiran@gmail.com>
wrote:
>> Hi There,
>> I manage a relatively large RHEL environment, we handle provisioning as
>> follows:
>>
>> - PXE + Kickstart to bootstrap and install the base OS + Puppet client
onto
>> the platform, be it VMWare or bare metal
>> - Kickstart post scripts put a basic puppet configuration file in place
on
>> the host, and a number of the values for things such as environment and
>> puppetmaster come from Foreman''s Macro''s, this allows
values in the ENC to
>> flow into the kickstart files before your first puppet run.
>>
>> We then run in the %post section of the kickstart file the following:
>> - A Puppet run that bootstraps the puppet client using tags ie, --tags
>> puppet::client
>> - A full puppet run via puppet agent -tov which applys the SOE to the
>> platform
>>
>> That provides on first boot a fully configured RHEL server that
includes all
>> our additional software and customisations in about 3-5 minutes (not
>> including POST)
>>
>> In regards to certs, we have a relatively open autosign.conf on our
build
>> networks, so we can provision servers , physical or virtual quite
quickly by
>> just hitting F12 for a network boot. I am sure there are some
cleaner/more
>> secure things we can do provisioning wise, however these have been
slightly
>> hindered by the RHN Satellite server i''ve been slowly pulling
out of the
>> environment at the same time, as it had the potential to break things
if i
>> wasnt careful.
>>
>> ENC wise, I can''t recommend Foreman enough, version 1.x is
just brilliant,
>> you can see the macros it can provide here:
>> http://theforeman.org/projects/foreman/wiki/TemplateWriting
>>
>> Hope this helps,
>>
>> K
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sunday, September 16, 2012 7:22:03 AM UTC+1, Douglas wrote:
>>> I''m wondering what people are doing systems provisioning
with, ie the
>>> process that gets puppet installed onto a system, running for the
>>> first time, and also the handling of certificate signing and so
forth.
>>> I don''t see this topic discussed much.
>>>
>>> The mc-provision tools at
>>> https://github.com/ripienaar/mcollective-server-provisioner
don''t seem
>>> to be actively developed anymore, or at least I wasn''t
able to find
>>> enough documentation to be able to effectively make use of it.
>>>
>>> Doug
>> --
>> You received this message because you are subscribed to the Google
Groups
>> "Puppet Users" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/puppet-users/-/NrKmbHHiaq8J.
>>
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/puppet-users?hl=en.
>
>
--
Luke Bigum
Senior Systems Engineer
Information Systems
Ph: +44 (0) 20 3192 2520
luke.bigum@lmax.com | http://www.lmax.com
LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN
FX and CFDs are leveraged products that can result in losses exceeding
your deposit. They are not suitable for everyone so please ensure you
fully understand the risks involved. The information in this email is not
directed at residents of the United States of America or any other
jurisdiction where trading in CFDs and/or FX is restricted or prohibited
by local laws or regulations.
The information in this email and any attachment is confidential and is
intended only for the named recipient(s). The email may not be disclosed
or used by any person other than the addressee, nor may it be copied in
any way. If you are not the intended recipient please notify the sender
immediately and delete any copies of this message. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.
LMAX operates a multilateral trading facility. Authorised and regulated
by the Financial Services Authority (firm registration number 509778) and
is registered in England and Wales (number 06505809).
Our registered address is Yellow Building, 1A Nicholas Road, London, W11
4AN.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
On Sep 17, 2012, at 2:16 PM, Douglas Garstang wrote:> I probably should have been clearer with my question. I was more > interested in how people are managing certificates? Even if you use > autosign, you still need to clean certificates manually. > > Doug.Doug, We autosign certs for hosts in our datacenter (based on the subdomain wildcard *.domain.org) and manually sign certs for desktops. All our datacenterhosts are set to try network booting first, so it''s easy to redeploy any of them at any time. I wrote a re/deployment script that automates all the necessary deployment steps for linux hosts in our datacenter: 1. It lists the available PXE configs and links the host''s address to the one you select. 2. It reboots the host using ipmi. 3. It schedules the removal of the PXE link (so the host doesn''t stay in a permanent install cycle.) 4. It cleans the host''s existing puppet cert. I have this script up on github, if you want to see how I''m doing it. It pushes the limits of bash sanity, but it works well. https://github.com/pmbuko/misc-scripts/blob/master/deployserv.sh -- Peter Bukowinski -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
----- Original Message -----> From: "Douglas Garstang" <doug.garstang@gmail.com> > To: puppet-users@googlegroups.com > Sent: Monday, September 17, 2012 7:16:02 PM > Subject: Re: [Puppet Users] Re: Systems Provisioning > > I probably should have been clearer with my question. I was more > interested in how people are managing certificates? Even if you use > autosign, you still need to clean certificates manually.so maybe the question is what would your desired behavior be? what happens with certificates really is a site specific question influenced by policy, deployment, frequency of rebuilds, acceptable time taken to do rebuilds etc. As you''ve seen from the mco provisioner you''ve been playing with its reasonably feasible to implement any behavior you might like with a bit of code. If you were to note a n step process that you want machines to go through on first boot then translating that into a agent + something to drive the agent really isn''t that hard. The mco provisioner captures what I needed at the time and also fed back a bunch of requirements into the mcollective development which would make re-writing it a lot easier today. It''s no doubt too complex for your needs or not a good fit - though it does allow you to customize exactly what it does in 2 ways: - you can provide your own agent that implements different logic for each action. The actions are effectively named API end points but you can decide what the logic is in each case. You did not like the bootstrap environment, no biggie, just rewrite that action to do what you want - you can enable and disable any of the steps to meet your needs so sticking with the bootstrap thing as an example only, you can configure it to just skip that step entirely. stages and chaining did not exist when this provisioner was written so it had a complicated extra step to achieve the same. This worked fine, I had a single provisioner running provisioning both physical machines and cloud instances - complete with different logic in the cloud from that in the datacenter by just deploying different agents leaving the provisioner to just call the agent in the desired order. My setup was quite complex as I had many masters in many locations and had to make a lot of decisions upfront before I could even know what master would be responsible for a specific node. I can think of many other ways to approach this problem though. The question about what to do with certs is purely site specific, its clear from the answers in this thread that for many auto signing is acceptable, others disagree. You have to decide what your desired behavior is based on local policy etc and you''re almost certainly going to have to write some code to match up puppet, kickstart and all the other moving parts with that. Puppet, MCollective and other tools just exist to make it easier for you to reach your goals with a little bit of glue in between - be it ENCs, scripts, cronjobs or a more complex ever present observer like the mco provisioner. It''s all just building blocks that u need to combine into a whole. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 16 September 2012 22:25, Jakov Sosic <jsosic@srce.hr> wrote:> On 09/16/2012 11:31 AM, Dick Davies wrote:>> We only provision a few nodes a day max. so that works ok. Lately I''ve >> been >> thinking of replacing Cobbler with The Foreman, which has better >> Puppet integration. > > > What does foreman provide better than cobbler?Don''t know it that well yet, but the bits that are appealing to me are the dashboard functionality (for puppet reporting) and the ''Smart Proxy'' concept to drive external services as part of provisioning. That takes care of the puppet signing issue but in our case we have bespoke system that runs our DHCP/DNS servers. Cobbler can manage BIND/isc-dhpcd, but we have to click through an external webapp to reserve the IP and FQDN up front. A small REST app in front of that looks like it should be able to allow automation with a Foreman setup. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, A have a question related to this thread. I''d like to ask, is the Foreman smart proxy amenable to being configured using Puppet, to the extent of being able to define the systems that are PXE booted within Puppet? Or is there some other existing means of configuring PXE booted systems from Puppet? Cheers, N -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
We have one script that is called to bootstrap and provision new VMs. There''s another one for destroying. These scripts are also responsible for logging into the puppet master (via ssh), polling for the cert to be created (puppet cert list) and then issuing the sign/clean commands. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/79DXTdWQjqkJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 09/19/2012 01:59 PM, Nick wrote:> Hi, > > A have a question related to this thread. I''d like to ask, is the Foreman smart > proxy amenable to being configured using Puppet, to the extent of being able to > define the systems that are PXE booted within Puppet? Or is there some other > existing means of configuring PXE booted systems from Puppet? > > Cheers,I don''t know about foreman, but I have written cobbler puppet module with 4 custom types (distro, profile, repo, system), so we manage cobbler entirely through puppet. It''s more readable and easier to edit properties. For example, this is how a typical system looks like: cobblersystem { ''typical-node'': ensure => present, profile => ''CentOS-6.3-x86_64'', interface => { ''eth0'' => { mac => ''00:AA:BB:CC:DD:EE'', ip_address => ''192.168.1.99'', subnet => ''255.255.255.0'', static => ''1'' }, }, netboot => false, gateway => ''192.168.1.1'', require => Service[$cobbler::service_name], } -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Is this module posted somewhere public ? Looks interesting enough to try out. On Oct 13, 2012, at 8:52 AM, Jakov Sosic wrote:> On 09/19/2012 01:59 PM, Nick wrote: >> Hi, >> >> A have a question related to this thread. I''d like to ask, is the Foreman smart >> proxy amenable to being configured using Puppet, to the extent of being able to >> define the systems that are PXE booted within Puppet? Or is there some other >> existing means of configuring PXE booted systems from Puppet? >> >> Cheers, > > I don''t know about foreman, but I have written cobbler puppet module with 4 custom types (distro, profile, repo, system), so we manage cobbler entirely through puppet. It''s more readable and easier to edit properties. For example, this is how a typical system looks like: > > > cobblersystem { ''typical-node'': > ensure => present, > profile => ''CentOS-6.3-x86_64'', > interface => { ''eth0'' => { mac => ''00:AA:BB:CC:DD:EE'', ip_address => ''192.168.1.99'', subnet => ''255.255.255.0'', static => ''1'' }, }, > netboot => false, > gateway => ''192.168.1.1'', > require => Service[$cobbler::service_name], > } > > > -- > Jakov Sosic > www.srce.unizg.hr > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/13/2012 05:55 PM, Dan White wrote:> Is this module posted somewhere public ? > Looks interesting enough to try out.It''s not posted yet but it will be soon. I''m currently rewriting some providers from CLI cobbler to XMLRPC calls, and I''m not yet fully satisfied with integration with puppetlabs/apache module. I had to modify apache module to make it work... Also I have some minor issues, like first run adds system without interfaces to cobbler, and subsequent run add interfaces. So you have to run it twice to properly add the system. Maybe someone can point out is there a way to fix this, because I''ve implemented interfaces as property and not param, so provider has separate methods for checking/modifying current state, and I don''t quite get it why it doesn''t run seamlessly in the first run Also, I don''t have documentation ready yet :) Although if you want I can send you current snapshot and help with ongoing issues that occur. And ofcourse you don''t have to use the whole module, you can use just custom types if you wish. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sat, Oct 13, 2012 at 10:50:05PM +0200, Jakov Sosic wrote:> On 10/13/2012 05:55 PM, Dan White wrote: > > Is this module posted somewhere public ? > > Looks interesting enough to try out. > > It''s not posted yet but it will be soon. I''m currently rewriting some > providers from CLI cobbler to XMLRPC calls, and I''m not yet fully > satisfied with integration with puppetlabs/apache module. I had to > modify apache module to make it work... > > > Also I have some minor issues, like first run adds system without > interfaces to cobbler, and subsequent run add interfaces. So you have to > run it twice to properly add the system. > > Maybe someone can point out is there a way to fix this, because I''ve > implemented interfaces as property and not param, so provider has > separate methods for checking/modifying current state, and I don''t quite > get it why it doesn''t run seamlessly in the first run >If puppet has to sync ensure it will not sync any other property. That means if your type defines "ensurable" or you have defined an ensure property manually and your system is not yet present, your create method is called and puppet expects the create method to create your system with interfaces. Otherwise you will see the described behaviour: 1) First run: Puppet finds out ensure is out of sync (is absent, should be present) and calls create 2) Second run: Puppet finds out interfaces is out of sync and and calls interfaces= (or whatever method you have defined for that) (see lib/puppet/transaction/resource_harness.rb#perform_changes) -Stefan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/13/2012 11:17 PM, Stefan Schulte wrote:> If puppet has to sync ensure it will not sync any other property. That > means if your type defines "ensurable" or you have defined an ensure > property manually and your system is not yet present, your create method > is called and puppet expects the create method to create your system with > interfaces.My understanding of the problem was along those lines too. Now I have confirmation...> Otherwise you will see the described behaviour: > > 1) First run: Puppet finds out ensure is out of sync (is absent, should > be present) and calls create > 2) Second run: Puppet finds out interfaces is out of sync and and calls > interfaces= (or whatever method you have defined for that)I am already detecting in my create method if ''system'' is added to ''cobbler'' (if it''s present on the machine agent is running on), and if it is, I choose to edit it rather then to try to create it again. So, can I just call method "interface=", if I detect I have to create the ''system'', or is there any way to solve this issue? Problem is that cobbler is used for provisioning (PXE+installation), so when you add new ''system'' to cobbler, you are going to try to install the machine immediately afterwards (in 99% of cases), so two puppet runs in this case really stand in the way of usability of the whole idea of puppetizing Cobbler setup. So I would love to circumvent it somehow if possible. Thank you -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sat, Oct 13, 2012 at 11:52:49PM +0200, Jakov Sosic wrote:> On 10/13/2012 11:17 PM, Stefan Schulte wrote: > > If puppet has to sync ensure it will not sync any other property. That > > means if your type defines "ensurable" or you have defined an ensure > > property manually and your system is not yet present, your create method > > is called and puppet expects the create method to create your system with > > interfaces. > > My understanding of the problem was along those lines too. Now I have > confirmation... > > > > Otherwise you will see the described behaviour: > > > > 1) First run: Puppet finds out ensure is out of sync (is absent, should > > be present) and calls create > > 2) Second run: Puppet finds out interfaces is out of sync and and calls > > interfaces= (or whatever method you have defined for that) > > > I am already detecting in my create method if ''system'' is added to > ''cobbler'' (if it''s present on the machine agent is running on), and if > it is, I choose to edit it rather then to try to create it again.If you use "ensurable", puppet will expect the provider to have an exists? method and if that returns true your create method will not be called so you do not have to check the existance in the create method again.> > So, can I just call method "interface=", if I detect I have to create > the ''system'', or is there any way to solve this issue?If adding interfaces to a new host is exactly the same as changing interfaces of an already existing host you can call interface = resource[:interface] if resoure[:interface] Is your provider dealing with files or does it execute commands? When dealing with files I find it is often easier to do all the work in the flush method (which is only called if it is implemented by the provider) and the other set methods are only updating the @property_hash hash. So if you want to share your provider code I am always interested ;-) -Stefan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/14/2012 03:01 PM, Stefan Schulte wrote:> If you use "ensurable", puppet will expect the provider to have an > exists? method and if that returns true your create method will not be > called so you do not have to check the existance in the create method > again.But then I would have to change all params to properties and add appropriate methods to provider, but that would both slow it down and complicate it more... I don''t know if it''s worth it just for the sake of log message stating "property changed from A to B" instead of "cobblersystem created" on every param change.> If adding interfaces to a new host is exactly the same as changing > interfaces of an already existing host you can call > > interface = resource[:interface] if resoure[:interface]I''ll try that.> Is your provider dealing with files or does it execute commands? When > dealing with files I find it is often easier to do all the work in the > flush method (which is only called if it is implemented by the provider) > and the other set methods are only updating the @property_hash hash. > > So if you want to share your provider code I am always interested ;-)Offcourse, I have nothing to hide. Here''s the current code: http://pastebin.com/f7GFU2qp -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/14/2012 04:20 PM, Jakov Sosic wrote:> On 10/14/2012 03:01 PM, Stefan Schulte wrote: >> If you use "ensurable", puppet will expect the provider to have an >> exists? method and if that returns true your create method will not be >> called so you do not have to check the existance in the create method >> again. > > But then I would have to change all params to properties and add > appropriate methods to provider, but that would both slow it down and > complicate it more... I don''t know if it''s worth it just for the sake of > log message stating "property changed from A to B" instead of > "cobblersystem created" on every param change. > > >> If adding interfaces to a new host is exactly the same as changing >> interfaces of an already existing host you can call >> >> interface = resource[:interface] if resoure[:interface] > > I''ll try that. >First of all I renamed "interface" to "interfaces", and then I''ve found solution for adding the property in first run, here is what code looks like: # add interfaces unless self.interfaces == @resource.should(:interfaces) self.interfaces = @resource.should(:interfaces) end It''s documented here: http://projects.puppetlabs.com/projects/puppet/wiki/Development_Complete_Resource_Example I forgot about documentation :) So I guess I could be rewriting all of my params to properties now, hm... -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sun, Oct 14, 2012 at 04:20:09PM +0200, Jakov Sosic wrote:> On 10/14/2012 03:01 PM, Stefan Schulte wrote: > > If you use "ensurable", puppet will expect the provider to have an > > exists? method and if that returns true your create method will not be > > called so you do not have to check the existance in the create method > > again. > > But then I would have to change all params to properties and add > appropriate methods to provider, but that would both slow it down and > complicate it more... I don''t know if it''s worth it just for the sake of > log message stating "property changed from A to B" instead of > "cobblersystem created" on every param change. >Is it possible to run a command to get all the desired information about every systems at once? This way you can implement a prefetch pattern. Basically you create provider instances for each system at once and write the current values in the @property_hash hash. Then your get methods just return the cached value which scales pretty well.> > > If adding interfaces to a new host is exactly the same as changing > > interfaces of an already existing host you can call > > > > interface = resource[:interface] if resoure[:interface] > > I''ll try that. > > > > Is your provider dealing with files or does it execute commands? When > > dealing with files I find it is often easier to do all the work in the > > flush method (which is only called if it is implemented by the provider) > > and the other set methods are only updating the @property_hash hash. > > > > So if you want to share your provider code I am always interested ;-) > > Offcourse, I have nothing to hide. Here''s the current code: > > http://pastebin.com/f7GFU2qp > > > > -- > Jakov Sosic > www.srce.unizg.hr > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/15/2012 09:13 AM, Stefan Schulte wrote:> Is it possible to run a command to get all the desired information about > every systems at once? This way you can implement a prefetch pattern. > Basically you create provider instances for each system at once and > write the current values in the @property_hash hash. Then your get methods > just return the cached value which scales pretty well.Yes it is possible to get all systems at once. It''s what the XMLRPC call does: xmlrpcresult = cobblerserver.call("get_systems") I already do that, but I select only one system, examine the hash, and if I notice differences from current puppet settings, approach to running CLI commands editing that system. Your idea would require top to bottom rewrite of provider, and I don''t have currently time for it... also, what are the benefits? Currently I am planning to change params to properties (every param that is not needed for creation of system can be managed as property - it''s more native way as I figured it out now). -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Oct 15, 2012 at 01:09:09PM +0200, Jakov Sosic wrote:> On 10/15/2012 09:13 AM, Stefan Schulte wrote: > > > Is it possible to run a command to get all the desired information about > > every systems at once? This way you can implement a prefetch pattern. > > Basically you create provider instances for each system at once and > > write the current values in the @property_hash hash. Then your get methods > > just return the cached value which scales pretty well. > > > Yes it is possible to get all systems at once. It''s what the XMLRPC call > does: > > xmlrpcresult = cobblerserver.call("get_systems") > > I already do that, but I select only one system, examine the hash, and > if I notice differences from current puppet settings, approach to > running CLI commands editing that system. > > Your idea would require top to bottom rewrite of provider, and I don''t > have currently time for it... also, what are the benefits? > > Currently I am planning to change params to properties (every param that > is not needed for creation of system can be managed as property - it''s > more native way as I figured it out now). >The fact that your exists? method does not really answer the question if a resource is present or absent is a bit strange. And inside the create method you are basically reimplementing properties with parameters. If something has to be checked for correctness it should be a property. Otherwise it is a parameter. Like the service resource: enable is a property because it can be out of sync. hasstatus is a parameter because it cannot be out of sync but only changes the behaviour of the provider Your main concern against properties if I got you correctly was about speed because puppet would run one query for each property. One way around that is to implement a query method that will query all properties at once and store them in a hash (@property_hash). Every get-method now check if @property_hash[:some_property] does already exist and return that value if it does or run the query method that would populate the @property_hash hash. Another speed improvement is to implement an `instances` and `prefetch` method. That has the benefit that puppet does "react" on such methods if they are implemented: * you are able to run "puppet resource cobblersystem" on the command line to get the current configuration of all systems (that depends on an instances classmethod) * you can use the resources type to purge unmanaged systems resources { ''cobblersystem'': purge => true } * the prefetch method is automatically called by puppet if implemented to create provider instances * your get methods become trivial -Stefan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/15/2012 07:23 PM, Stefan Schulte wrote:> The fact that your exists? method does not really answer the question if > a resource is present or absent is a bit strange. And inside the create > method you are basically reimplementing properties with parameters. If > something has to be checked for correctness it should be a property. > Otherwise it is a parameter. Like the service resource: enable is a > property because it can be out of sync. hasstatus is a parameter because > it cannot be out of sync but only changes the behaviour of the providerOK, I''ve figured that out through this conversation... Now this is somewhat fixed code: http://pastebin.com/q0TBX4KB I''ve moved some params to properties.> Your main concern against properties if I got you correctly was about > speed because puppet would run one query for each property. One way around > that is to implement a query method that will query all properties at once > and store them in a hash (@property_hash). Every get-method now check > if @property_hash[:some_property] does already exist and return that > value if it does or run the query method that would populate the > @property_hash hash.That sounds interesting, and more important it seems to me that complete rewrite is not necessary in this case. Do you have some examples of this idea?> Another speed improvement is to implement an `instances` and `prefetch` > method. That has the benefit that puppet does "react" on such methods > if they are implemented: > > * you are able to run "puppet resource cobblersystem" on the command > line to get the current configuration of all systems (that depends on > an instances classmethod) > * you can use the resources type to purge unmanaged systems > > resources { ''cobblersystem'': > purge => true > } > * the prefetch method is automatically called by puppet if implemented > to create provider instances > * your get methods become trivialWow, sounds very interesting. I would plea for possible examples :) -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Oct 15, 2012 at 11:18 AM, Jakov Sosic <jsosic@srce.hr> wrote:> On 10/15/2012 07:23 PM, Stefan Schulte wrote: > >> The fact that your exists? method does not really answer the question if >> a resource is present or absent is a bit strange. And inside the create >> method you are basically reimplementing properties with parameters. If >> something has to be checked for correctness it should be a property. >> Otherwise it is a parameter. Like the service resource: enable is a >> property because it can be out of sync. hasstatus is a parameter because >> it cannot be out of sync but only changes the behaviour of the provider > > OK, I''ve figured that out through this conversation... > > Now this is somewhat fixed code: > > http://pastebin.com/q0TBX4KB > > I''ve moved some params to properties. > > >> Your main concern against properties if I got you correctly was about >> speed because puppet would run one query for each property. One way around >> that is to implement a query method that will query all properties at once >> and store them in a hash (@property_hash). Every get-method now check >> if @property_hash[:some_property] does already exist and return that >> value if it does or run the query method that would populate the >> @property_hash hash. > > That sounds interesting, and more important it seems to me that complete > rewrite is not necessary in this case. Do you have some examples of this > idea? > > >> Another speed improvement is to implement an `instances` and `prefetch` >> method. That has the benefit that puppet does "react" on such methods >> if they are implemented: >> >> * you are able to run "puppet resource cobblersystem" on the command >> line to get the current configuration of all systems (that depends on >> an instances classmethod) >> * you can use the resources type to purge unmanaged systems >> >> resources { ''cobblersystem'': >> purge => true >> } >> * the prefetch method is automatically called by puppet if implemented >> to create provider instances >> * your get methods become trivial > > Wow, sounds very interesting. > > I would plea for possible examples :)You can look for other examples in puppet source code or puppet modules with custom providers. Simply search for self.instances. Here''s the rough draft about self.instances from the provider chapter for Puppet Types and Provider. The first two chapters are available as early preview for O''Reilly Safari users. We are looking forward to getting some feedback as we flush out more content. Also we are looking for a technical reviewer if anyone is interested. Thanks, Nan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Oct 15, 2012 at 08:18:36PM +0200, Jakov Sosic wrote:> On 10/15/2012 07:23 PM, Stefan Schulte wrote: > > > The fact that your exists? method does not really answer the question if > > a resource is present or absent is a bit strange. And inside the create > > method you are basically reimplementing properties with parameters. If > > something has to be checked for correctness it should be a property. > > Otherwise it is a parameter. Like the service resource: enable is a > > property because it can be out of sync. hasstatus is a parameter because > > it cannot be out of sync but only changes the behaviour of the provider > > OK, I''ve figured that out through this conversation... > > Now this is somewhat fixed code: > > http://pastebin.com/q0TBX4KB > > I''ve moved some params to properties. > > > > Your main concern against properties if I got you correctly was about > > speed because puppet would run one query for each property. One way around > > that is to implement a query method that will query all properties at once > > and store them in a hash (@property_hash). Every get-method now check > > if @property_hash[:some_property] does already exist and return that > > value if it does or run the query method that would populate the > > @property_hash hash. > > That sounds interesting, and more important it seems to me that complete > rewrite is not necessary in this case. Do you have some examples of this > idea? > > > > Another speed improvement is to implement an `instances` and `prefetch` > > method. That has the benefit that puppet does "react" on such methods > > if they are implemented: > > > > * you are able to run "puppet resource cobblersystem" on the command > > line to get the current configuration of all systems (that depends on > > an instances classmethod) > > * you can use the resources type to purge unmanaged systems > > > > resources { ''cobblersystem'': > > purge => true > > } > > * the prefetch method is automatically called by puppet if implemented > > to create provider instances > > * your get methods become trivial > > Wow, sounds very interesting. > > I would plea for possible examples :)The instances method is a class method and has to return an array of providers. So this often looks like this def self.instances systems = [] my_fancy_command.each_line do |line| somehow_split_line_into_different_fields_like_name_and_interfaces systems << new( :name => name, :interfaces => interfaces, :ensure => :present ) end systems end One important thing: If you create a new provider instance you can pass a hash (like I did in new(:name => name, :interfaces => interface)) and this hash is stored in the member variable @property_hash of that new provider. An example of a simple instances method: https://github.com/stschulte/puppet-rpmkey/blob/master/lib/puppet/provider/rpmkey/rpm.rb The rpmkey type can make sure that a certain gpg key is imported into rpm. To get the currently installed keys the provider runs rpm -q gpg-key This command can either return with a non zero exit code (no packages found) in case we have zero keys or it will print one line per key. For each line a provider instance is added to the array that is finally returned. prefetch: The prefetch method is called by puppet for each providerclass that implements such a method (see lib/puppet/transaction.rb#prefetch). The prefetch method is called with a hash of every resource that is defined in the user''s manifest (=every resource puppet should manage). The hash will have the form resource[:name] as a key and resource as the value. What the prefetch method can do now is create provider instances and bind the provider instances to resources. A common prefetch method that is also shown in the rpm provider for rpmkey: def self.prefetch(resources) instances.each do |prov| if resource = resources[prov.name] resource.provider = prov end end end The prefetch method first calls instances that will return a list of every key that is currently present. Then I check if that key is also managed by puppet. If the lookup succeeds (the key is indeed managed by puppet), I''ll bind the provider to the resource. At this point the provider instance already has @property_hash[:ensure] set, so when puppet later handles the different rpmkey resources and asks exists? I can simply return the cached value. def exists? get(:ensure) != :absent end Note: get(:ensure) is implemented in lib/provider.rb as def get(param) @property_hash[param.intern] || :absent end A more complex provider that has to manage multiple properties: https://github.com/stschulte/puppetlabs-solaris/blob/feature/master/projects/lib/puppet/provider/project/projadd.rb The instances method is more complicated here but it basically does the same thing: return an array of provider instances. You''ll notice that I do not implement any method to get the current values of the properties. The methodcall "mk_resource_methods" already creates these for me. "mk_resource_methods" will create a get method for every property that is essentially (if your property is named interfaces) def interfaces @property_hash[:interfaces] || :absent end I hope that helps. An example of a query method can be found in the package provider in puppet core. The package provider is a bit special here because it does not trust its prefetched values. This is because one resource can change the state of another resource: package { "mod_ssl": ensure => installed } package { "httpd": ensure => installed } The ensure property of "httpd" can change in the middle of a puppetrun (if mod_ssl is installed first httpd will be installed as a dependency) so the prefetched value of :absent can be obsolete when puppet actually handles the resource Package["httpd"]. For you cobblersystem type I do not think that one resource may influence another resource right? -Stefan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/15/2012 11:44 PM, Stefan Schulte wrote:> The instances method is a class method and has to return an array of > providers. So this often looks like this > > def self.instances > systems = [] > my_fancy_command.each_line do |line| > somehow_split_line_into_different_fields_like_name_and_interfaces > systems << new( > :name => name, > :interfaces => interfaces, > :ensure => :present > ) > end > systems > endHuh, so I have to build whole resources in self.instances, with all the possible properties?> This command can either return with a non zero exit code (no packages > found) in case we have zero keys or it will print one line per key. > For each line a provider instance is added to the array that is finally > returned.Got it from the upper code.> I hope that helps. An example of a query method can be found in the > package provider in puppet core. The package provider is a bit special > here because it does not trust its prefetched values. This is because > one resource can change the state of another resource: > > package { "mod_ssl": ensure => installed } > package { "httpd": ensure => installed } > > The ensure property of "httpd" can change in the middle of a puppetrun (if > mod_ssl is installed first httpd will be installed as a dependency) > so the prefetched value of :absent can be obsolete when puppet actually > handles the resource Package["httpd"]. For you cobblersystem type I do > not think that one resource may influence another resource right?Yeah, you are right, one cobblersystem instance cannot influence creation or modification of other instance. But now that you mention, I do have one more question in this regard. Is it possible for some property to create auto-require with some other resource? Except from cobblersystem I do have 3 another custom types (cobblerprofile, cobblerdistro and cobblerrepo). So, for example: cobblerprofile { ''foo'': ensure => present, } cobblersystem { ''bar'': ensure => present, profile => ''foo'', require => Cobblerprofile[''foo''], } Is there a way to skip the last require, and force it through profile property in the type/provider code? Thank you so much for the great explanation. I will save it and try to code it first on my simpler types like cobblerprofile and if I succeed will jump into fire and rewrite cobblersystem. ty. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Oct 15, 2012 at 3:04 PM, Jakov Sosic <jsosic@srce.hr> wrote:> On 10/15/2012 11:44 PM, Stefan Schulte wrote: > >> The instances method is a class method and has to return an array of >> providers. So this often looks like this >> >> def self.instances >> systems = [] >> my_fancy_command.each_line do |line| >> somehow_split_line_into_different_fields_like_name_and_interfaces >> systems << new( >> :name => name, >> :interfaces => interfaces, >> :ensure => :present >> ) >> end >> systems >> end > > Huh, so I have to build whole resources in self.instances, with all the > possible properties? > > >> This command can either return with a non zero exit code (no packages >> found) in case we have zero keys or it will print one line per key. >> For each line a provider instance is added to the array that is finally >> returned. > > Got it from the upper code. > > >> I hope that helps. An example of a query method can be found in the >> package provider in puppet core. The package provider is a bit special >> here because it does not trust its prefetched values. This is because >> one resource can change the state of another resource: >> >> package { "mod_ssl": ensure => installed } >> package { "httpd": ensure => installed } >> >> The ensure property of "httpd" can change in the middle of a puppetrun (if >> mod_ssl is installed first httpd will be installed as a dependency) >> so the prefetched value of :absent can be obsolete when puppet actually >> handles the resource Package["httpd"]. For you cobblersystem type I do >> not think that one resource may influence another resource right? > > Yeah, you are right, one cobblersystem instance cannot influence > creation or modification of other instance. > > But now that you mention, I do have one more question in this regard. Is > it possible for some property to create auto-require with some other > resource? Except from cobblersystem I do have 3 another custom types > (cobblerprofile, cobblerdistro and cobblerrepo). So, for example: > > cobblerprofile { ''foo'': > ensure => present, > } > cobblersystem { ''bar'': > ensure => present, > profile => ''foo'', > require => Cobblerprofile[''foo''], > } > > Is there a way to skip the last require, and force it through profile > property in the type/provider code?Since you already specify the profile, in your cobblersystem resource type add autorequire. autorequire(:cobblerprofile) do self[:profile] end Thanks, Nan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 10/15/2012 07:23 PM, Stefan Schulte wrote:> Another speed improvement is to implement an `instances` and `prefetch` > method. That has the benefit that puppet does "react" on such methods > if they are implemented: > > * you are able to run "puppet resource cobblersystem" on the command > line to get the current configuration of all systems (that depends on > an instances classmethod) > * you can use the resources type to purge unmanaged systems > > resources { ''cobblersystem'': > purge => true > } > * the prefetch method is automatically called by puppet if implemented > to create provider instances > * your get methods become trivialThank both you and Nan Liu. As the holidays approached I finally got a free day to work on this :) I''ve rewritten 3 out of 4 my custom providers to self.instances schema. I will test this and report back performance improvements on puppet agents run with lots of resources of custom type I wrote, after I rewrite the final provider. And code looks much cleaner too! :) Once more a big thank you both :) -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 12/24/2012 01:59 AM, Jakov Sosic wrote:> Thank both you and Nan Liu. > > As the holidays approached I finally got a free day to work on this :) > I''ve rewritten 3 out of 4 my custom providers to self.instances schema. > > I will test this and report back performance improvements on puppet > agents run with lots of resources of custom type I wrote, after I > rewrite the final provider. > > And code looks much cleaner too! :) Once more a big thank you both :)OK, I''ve finally published Cobbler module for Puppet :) http://forge.puppetlabs.com/jsosic https://bitbucket.org/jsosic/puppet-cobbler And here is the most complex provider I have so far: https://bitbucket.org/jsosic/puppet-cobbler/src/da3cdd684314360dcc5c633f374310f59f87ec42/lib/puppet/provider/cobblersystem/system.rb?at=default And now about performance increase after rewriting my providers to use XMLRPC one-call-per provider (self.instances) instead of few cmdline calls for every resources. These are the agent run times: Site1 (22 x cobblersystem) - before: 68.1s - after: 9.7s Site2 (19 x cobblersystem) - before: 54.0s - after: 10.0s Site3: (11 x cobblersystem) - before: 40.9s - after: 9.7s Site4: (22 x cobblersystem) - before: 80.1s - after: 10.6s So I guess this changes move it from O(n) to O(1) :) -- Jakov Sosic www.srce.unizg.hr -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.