Hello, I encounter issues regarding puppet "self update" that I''m sure are not uncommon: 1. When puppet version updates it doesn''t restart to run the rest of the manifest with the new version. 2. When a new provider is installed (or extra configuration is done to enable an existing provider), puppet still won''t make it available. example for (1): Our vagrant (http://vagrantup.com/) dev base boxes still come with Puppet 2.6.3 while the manifest depends on Puppet 2.7 features. I can manually upgrade puppet manually (and that''s what I do on dev), but when the time to deploy to production comes it practically means that I have to manually upgrade Puppet to 2.7 on all hosts before I push the change. example for (2): We use daemontools (http://cr.yp.to/daemontools.html) to monitor some of our servers. Puppet comes with a "daemontools" Service type provider but since the daemontools package isn''t installed yet the provider is made unavailable for the rest of the run. A second puppet run, assuming it managed to install daemontools in the first one before it failed on other things, succeeds in picking up the daemontools as a provider. another example for (2): I use opsview (https://github.com/dpeters/ puppet-opsview) to monitor hosts. When the type is loaded by puppet it looks for a file called /etc/puppet/opsview.conf to find URL/username/ password of the opsview REST server. This file doesn''t exist (or can be out of date) in the first run causing the Type to fail. My suggestion - Define a new "preload" stage - resources which can affect puppet''s own execution (e.g. types, config files, puppet version updates) can be placed in that stage and eventually a specific function call (or a special built-in trigger inside the Puppet code?) will cause Puppet to drop everything at the end of this stage and start from fresh. The mechanism has to be careful to limit potential inifinite loops - e.g. if nothing was changed during the stage then don''t re-run, or if the stage keeps changing things and puppet re-starts more than a set limit then fail. What do you think about this? Is there another solution/work-around/ standard-best-practice I''m not aware of? Thanks. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, Mar 21, 2012 at 1:11 PM, Amos Shapira <amos.shapira@gmail.com>wrote:> Hello, > > I encounter issues regarding puppet "self update" that I''m sure are > not uncommon: > 1. When puppet version updates it doesn''t restart to run the rest of > the manifest with the new version. > 2. When a new provider is installed (or extra configuration is done to > enable an existing provider), puppet still won''t make it available. > > example for (1): Our vagrant (http://vagrantup.com/) dev base boxes > still come with Puppet 2.6.3 while the manifest depends on Puppet 2.7 > features. I can manually upgrade puppet manually (and that''s what I do > on dev), but when the time to deploy to production comes it > practically means that I have to manually upgrade Puppet to 2.7 on all > hosts before I push the change. > > example for (2): We use daemontools (http://cr.yp.to/daemontools.html) > to monitor some of our servers. Puppet comes with a "daemontools" > Service type provider but since the daemontools package isn''t > installed yet the provider is made unavailable for the rest of the > run. A second puppet run, assuming it managed to install daemontools > in the first one before it failed on other things, succeeds in picking > up the daemontools as a provider. > > another example for (2): I use opsview (https://github.com/dpeters/ > puppet-opsview) to monitor hosts. When the type is loaded by puppet it > looks for a file called /etc/puppet/opsview.conf to find URL/username/ > password of the opsview REST server. This file doesn''t exist (or can > be out of date) in the first run causing the Type to fail. > >Amos, have you seen this feature --> http://projects.puppetlabs.com/issues/6907> My suggestion - > > Define a new "preload" stage - resources which can affect puppet''s own > execution (e.g. types, config files, puppet version updates) can be > placed in that stage and eventually a specific function call (or a > special built-in trigger inside the Puppet code?) will cause Puppet to > drop everything at the end of this stage and start from fresh. > > The mechanism has to be careful to limit potential inifinite loops - > e.g. if nothing was changed during the stage then don''t re-run, or if > the stage keeps changing things and puppet re-starts more than a set > limit then fail. > > What do you think about this? Is there another solution/work-around/ > standard-best-practice I''m not aware of? > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- Gary Larizza Professional Services Engineer Puppet Labs -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks for the pointer, Gary. It seems to address the issue of missing providers, even better than my suggestion. However, I''m not sure if/how it addresses other situations, e.g.: 1. Provider exists but has to be updated. 2. Puppet version update. 3. Configuration files - missing or have to be updated. 4. Other types of configurations which have to take place, e.g. add a network link (e.g. VPN, proxy), or make any other arbitrary change which can affect the rest of the puppet run (or future puppet runs). What I''m thinking about is a generic way to say "OK, now I''m finished doing all sorts of tweaks to the system, please start puppet and take these tweaks into consideration". Is that possible? Cheers, --Amos On Wednesday, March 21, 2012 1:51:48 PM UTC+11, Gary Larizza wrote:> > > > On Wed, Mar 21, 2012 at 1:11 PM, Amos Shapira <amos.shapira@gmail.com>wrote: > >> Hello, >> >> I encounter issues regarding puppet "self update" that I''m sure are >> not uncommon: >> 1. When puppet version updates it doesn''t restart to run the rest of >> the manifest with the new version. >> 2. When a new provider is installed (or extra configuration is done to >> enable an existing provider), puppet still won''t make it available. >> >> example for (1): Our vagrant (http://vagrantup.com/) dev base boxes >> still come with Puppet 2.6.3 while the manifest depends on Puppet 2.7 >> features. I can manually upgrade puppet manually (and that''s what I do >> on dev), but when the time to deploy to production comes it >> practically means that I have to manually upgrade Puppet to 2.7 on all >> hosts before I push the change. >> >> example for (2): We use daemontools (http://cr.yp.to/daemontools.html<http://cr.yp.to/daemontools.html> >> ) >> to monitor some of our servers. Puppet comes with a "daemontools" >> Service type provider but since the daemontools package isn''t >> installed yet the provider is made unavailable for the rest of the >> run. A second puppet run, assuming it managed to install daemontools >> in the first one before it failed on other things, succeeds in picking >> up the daemontools as a provider. >> >> another example for (2): I use opsview (https://github.com/dpeters/ >> puppet-opsview <https://github.com/dpeters/puppet-opsview>) to monitor >> hosts. When the type is loaded by puppet it >> looks for a file called /etc/puppet/opsview.conf to find URL/username/ >> password of the opsview REST server. This file doesn''t exist (or can >> be out of date) in the first run causing the Type to fail. >> >> > Amos, have you seen this feature --> > http://projects.puppetlabs.com/issues/6907<http://projects.puppetlabs.com/issues/6907> > > > > >> My suggestion - >> >> Define a new "preload" stage - resources which can affect puppet''s own >> execution (e.g. types, config files, puppet version updates) can be >> placed in that stage and eventually a specific function call (or a >> special built-in trigger inside the Puppet code?) will cause Puppet to >> drop everything at the end of this stage and start from fresh. >> >> The mechanism has to be careful to limit potential inifinite loops - >> e.g. if nothing was changed during the stage then don''t re-run, or if >> the stage keeps changing things and puppet re-starts more than a set >> limit then fail. >> >> What do you think about this? Is there another solution/work-around/ >> standard-best-practice I''m not aware of? >> >> Thanks. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en> >> . >> >> > > > -- > > Gary Larizza > Professional Services Engineer > Puppet Labs > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/XUeixQ5zq7sJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Amos, One of the least hacky ways I''ve found around the chicken and egg scenario is to use Custom facts to determine what state Agent is in and then use or don''t use your providers accordingly. For example I have a fact for determining if ruby-libvirt is installed and Puppet has set up the necessary networking so I know I can use the ''virt'' custom provider to create VMs. Unfortunately it requires multiple passes, but removes the need for manual intervention and intermediate errors from the logs. -Luke On 23/03/12 06:35, Amos Shapira wrote:> Thanks for the pointer, Gary. > > It seems to address the issue of missing providers, even better than > my suggestion. > > However, I''m not sure if/how it addresses other situations, e.g.: > 1. Provider exists but has to be updated. > 2. Puppet version update. > 3. Configuration files - missing or have to be updated. > 4. Other types of configurations which have to take place, e.g. add a > network link (e.g. VPN, proxy), or make any other arbitrary change > which can affect the rest of the puppet run (or future puppet runs). > > What I''m thinking about is a generic way to say "OK, now I''m finished > doing all sorts of tweaks to the system, please start puppet and take > these tweaks into consideration". > > Is that possible? > > Cheers, > > --Amos > > On Wednesday, March 21, 2012 1:51:48 PM UTC+11, Gary Larizza wrote: > > > > On Wed, Mar 21, 2012 at 1:11 PM, Amos Shapira > <amos.shapira@gmail.com <mailto:amos.shapira@gmail.com>> wrote: > > Hello, > > I encounter issues regarding puppet "self update" that I''m > sure are > not uncommon: > 1. When puppet version updates it doesn''t restart to run the > rest of > the manifest with the new version. > 2. When a new provider is installed (or extra configuration is > done to > enable an existing provider), puppet still won''t make it > available. > > example for (1): Our vagrant (http://vagrantup.com/) dev base > boxes > still come with Puppet 2.6.3 while the manifest depends on > Puppet 2.7 > features. I can manually upgrade puppet manually (and that''s > what I do > on dev), but when the time to deploy to production comes it > practically means that I have to manually upgrade Puppet to > 2.7 on all > hosts before I push the change. > > example for (2): We use daemontools > (http://cr.yp.to/daemontools. html > <http://cr.yp.to/daemontools.html>) > to monitor some of our servers. Puppet comes with a "daemontools" > Service type provider but since the daemontools package isn''t > installed yet the provider is made unavailable for the rest of the > run. A second puppet run, assuming it managed to install > daemontools > in the first one before it failed on other things, succeeds in > picking > up the daemontools as a provider. > > another example for (2): I use opsview > (https://github.com/dpeters/ > puppet-opsview <https://github.com/dpeters/puppet-opsview>) to > monitor hosts. When the type is loaded by puppet it > looks for a file called /etc/puppet/opsview.conf to find > URL/username/ > password of the opsview REST server. This file doesn''t exist > (or can > be out of date) in the first run causing the Type to fail. > > > Amos, have you seen this feature --> http://projects. > puppetlabs.com/issues/6907 > <http://projects.puppetlabs.com/issues/6907> > > > My suggestion - > > Define a new "preload" stage - resources which can affect > puppet''s own > execution (e.g. types, config files, puppet version updates) > can be > placed in that stage and eventually a specific function call (or a > special built-in trigger inside the Puppet code?) will cause > Puppet to > drop everything at the end of this stage and start from fresh. > > The mechanism has to be careful to limit potential inifinite > loops - > e.g. if nothing was changed during the stage then don''t > re-run, or if > the stage keeps changing things and puppet re-starts more than > a set > limit then fail. > > What do you think about this? Is there another > solution/work-around/ > standard-best-practice I''m not aware of? > > Thanks. > > -- > You received this message because you are subscribed to the > Google Groups "Puppet Users" group. > To post to this group, send email to > puppet-users@googlegroups.com > <mailto:puppet-users@googlegroups.com>. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@ googlegroups.com > <mailto:puppet-users%2Bunsubscribe@googlegroups.com>. > For more options, visit this group at > http://groups.google.com/ group/puppet-users?hl=en > <http://groups.google.com/group/puppet-users?hl=en>. > > > > > -- > > Gary Larizza > Professional Services Engineer > Puppet Labs > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/XUeixQ5zq7sJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en.-- Luke Bigum Information Systems Ph: +44 (0) 20 3192 2520 luke.bigum@lmax.com | http://www.lmax.com LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks Luke (and sorry for late reply). This sort of addresses another question I haven''t got around to investigate - the question is whether Puppet has functions/facts to tell me things like whether a particular provider is available or not. I guess it can be somehow scripted. But as you say - your hack won''t avoid re-runs, it might allow cleaner messages, though. Cheers, --Amos -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/gkjGeEs-Bq8J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Amos - its been a long long long time mate :-).> example for (1): Our vagrant (http://vagrantup.com/) dev base boxes > still come with Puppet 2.6.3 while the manifest depends on Puppet 2.7 > features. I can manually upgrade puppet manually (and that''s what I do > on dev), but when the time to deploy to production comes it > practically means that I have to manually upgrade Puppet to 2.7 on all > hosts before I push the change.You know - I''ve generally just used mcollective with the puppetral plugin to do the Puppet agent RPM/deb upgrade in the case where the client is the wrong version, and can now no longer interact with my master: http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/AgentPackage https://github.com/puppetlabs/mcollective-plugins/tree/master/agent/package At the end of the day, it uses the Puppet resource abstraction layer anyway behind the scenes, only difference is that it is making the change directly without the Puppet master interaction. But I can see why you would want Puppet to just deal with this on its own.> 1. Provider exists but has to be updated.I could be mistaken about what I''m about to say, as I haven''t experimented with this new behaviour all that much. I wouldn''t take my word for it - so experiment and see :-). Surely the lazy evaluate solves this, I could be wrong - but if you build up a relationship between the action that does the update - and the provider that users that update then the update should occur first. Then when the provider is evaluated lazily - all is well - it has a new updated command/library/whatever? So: package { "daemontools": ensure => "1.2.3", } Service { require => Package["daemontools"] } Is a rough way of doing it, but the default _is_ of course generic here, and run stages may be a better solution - as its easier to have your ''require''s changed/overridden later on. (because defaults don''t stack with overrides).> 2. Puppet version update.Updating Puppet can be tricky, from an agent perspective if you run it as a daemon there was a problem when updating to the recent Facter 1.6.6 for example - as parts of Facter are loaded/evaluated dynamically each (ie. the facts) - while others are loaded and cached and never re-evaluated. This created a problem with I believe the ec2 fact from memory - which started to use a library in facter/util/ec2 (yes my fault). In that edge case you need to probably force a restart of an agent after the package update really. Its crap I agree ... and I''m all ears for a better solution/policy generally. From the master perspective - you can run into similar scenarios as well. Generally speaking, the ''puppet managing puppet'' case is always hard mate. I always spend the most amount of time helping clients ''manage'' their puppetmaster with Puppet. And their are lots of strategies here :-). What are you hitting that is problematic at the moment? Or what in particular do you see?> 3. Configuration files - missing or have to be updated.Again, build up the dependencies using relationships in Puppet - as per 1. Should work in a similar manner (from the description of the new feature that is).> 4. Other types of configurations which have to take place, e.g. add a > network link (e.g. VPN, proxy),Same again.> or make any other arbitrary change which can > affect the rest of the puppet run (or future puppet runs).You might be getting into the ''manage the puppet case'', again - anything specific you expect?> What I''m thinking about is a generic way to say "OK, now I''m finished doing > all sorts of tweaks to the system, please start puppet and take these tweaks > into consideration". > > Is that possible?To be absolutely honest with you - up until now, this has been done with pre-puppet stuff. That is - done with kickstart, jumpstart or whatever you use to build the system. Of course, this then becomes non-idempotent which makes Puppet less useful overall. Dude - try the lazy evaluate with the edge cases you''ve been seeing and see how far you get - and lets discuss anything that still won''t work. We''re really interested in solving this kind of chicken & egg problems for good if we can. ken. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I just did a 2.6.11 -> 2.7.12 migration with Puppet updating Puppet. The only real hitch I ran into was my puppet masters would update themselves but somehow Apache/Passenger didn''t get restarted, so I had to do that by hand. Since I didn''t start putting any 2.7 features into my manifests until *after* the 2.7 upgrade, the clients basically just upgraded themselves. And while I do have puppet running as a daemon, it''s with no-schedule, and the "real" run happens out of cron. I think I''m actually going to kill off the daemon as it offers no benefit to me (and frequently wedged when I let it schedule updates). Doing it in cron allows me to meet externally imposed scheduling restrictions (no changes while production is running). On Thu, Mar 29, 2012 at 10:42 PM, Ken Barber <ken@puppetlabs.com> wrote:> Hi Amos - its been a long long long time mate :-). > > > example for (1): Our vagrant (http://vagrantup.com/) dev base boxes > > still come with Puppet 2.6.3 while the manifest depends on Puppet 2.7 > > features. I can manually upgrade puppet manually (and that''s what I do > > on dev), but when the time to deploy to production comes it > > practically means that I have to manually upgrade Puppet to 2.7 on all > > hosts before I push the change. > > You know - I''ve generally just used mcollective with the puppetral > plugin to do the Puppet agent RPM/deb upgrade in the case where the > client is the wrong version, and can now no longer interact with my > master: > > > http://projects.puppetlabs.com/projects/mcollective-plugins/wiki/AgentPackage > https://github.com/puppetlabs/mcollective-plugins/tree/master/agent/package > > At the end of the day, it uses the Puppet resource abstraction layer > anyway behind the scenes, only difference is that it is making the > change directly without the Puppet master interaction. But I can see > why you would want Puppet to just deal with this on its own. > > > 1. Provider exists but has to be updated. > > I could be mistaken about what I''m about to say, as I haven''t > experimented with this new behaviour all that much. I wouldn''t take my > word for it - so experiment and see :-). > > Surely the lazy evaluate solves this, I could be wrong - but if you > build up a relationship between the action that does the update - and > the provider that users that update then the update should occur > first. Then when the provider is evaluated lazily - all is well - it > has a new updated command/library/whatever? > > So: > > package { "daemontools": > ensure => "1.2.3", > } > Service { > require => Package["daemontools"] > } > > Is a rough way of doing it, but the default _is_ of course generic > here, and run stages may be a better solution - as its easier to have > your ''require''s changed/overridden later on. (because defaults don''t > stack with overrides). > > > 2. Puppet version update. > > Updating Puppet can be tricky, from an agent perspective if you run it > as a daemon there was a problem when updating to the recent Facter > 1.6.6 for example - as parts of Facter are loaded/evaluated > dynamically each (ie. the facts) - while others are loaded and cached > and never re-evaluated. This created a problem with I believe the ec2 > fact from memory - which started to use a library in facter/util/ec2 > (yes my fault). In that edge case you need to probably force a restart > of an agent after the package update really. Its crap I agree ... and > I''m all ears for a better solution/policy generally. > > From the master perspective - you can run into similar scenarios as > well. Generally speaking, the ''puppet managing puppet'' case is always > hard mate. I always spend the most amount of time helping clients > ''manage'' their puppetmaster with Puppet. And their are lots of > strategies here :-). > > What are you hitting that is problematic at the moment? Or what in > particular do you see? > > > 3. Configuration files - missing or have to be updated. > > Again, build up the dependencies using relationships in Puppet - as > per 1. Should work in a similar manner (from the description of the > new feature that is). > > > 4. Other types of configurations which have to take place, e.g. add a > > network link (e.g. VPN, proxy), > > Same again. > > > or make any other arbitrary change which can > > affect the rest of the puppet run (or future puppet runs). > > You might be getting into the ''manage the puppet case'', again - > anything specific you expect? > > > What I''m thinking about is a generic way to say "OK, now I''m finished > doing > > all sorts of tweaks to the system, please start puppet and take these > tweaks > > into consideration". > > > > Is that possible? > > To be absolutely honest with you - up until now, this has been done > with pre-puppet stuff. That is - done with kickstart, jumpstart or > whatever you use to build the system. Of course, this then becomes > non-idempotent which makes Puppet less useful overall. > > Dude - try the lazy evaluate with the edge cases you''ve been seeing > and see how far you get - and lets discuss anything that still won''t > work. We''re really interested in solving this kind of chicken & egg > problems for good if we can. > > ken. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks Ken, I think I may have been confused with the pre-2.7.8 puppet still being installed before this upgrade was pushed out. Once the later version was installed the manifest seemed to work right. For instance the first opsview run on a server will complain about the missing config file but still work after all since it re-reads the configuration file when it''s actually used, whereas it used to just fail on 2.6.x. I still ended up using polysh to do the initial "yum upgrade puppet" on all servers but hopefully that won''t have to be repeated too much. For puppet upgrade itself - a possible, untested, work-around could be to Exec[] something which will restart the puppet agent if puppet was upgraded. Risky and ugly but might save someone from having to do this manually many servers. For now I guess you can ignore my request. Cheers, --Amos -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/FbkwosHxJ0EJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.