Hi all.
For some background, we''re essentially using puppet in much the same
way you would use the ssh-and-a-for-loop method. I know it''s well,
fundamentally missing the point, but the thing is, there are a few
constraints that we''re currently otherwise unable to satisfy.
Essentailly, we have a large number of nigh upon homogeneous servers
organised into clusters of about 10 machines each. These are entirely
customer-facing, so we need to be able to ensure that the cluster
overall doesn''t go down when we''re doing software deployments.
So, to
do this, we follow a process rather like this:
Disable server inload balancer -> Take server out of monitoring -> run
puppet -> Put server back in monitoring -> Enable server inLoad
balancer
Combined with smoketesting of the server once it''s been upgraded, and
ensuring that it comes back up in monitoring; and ensuring that only
N/M servers are out of service at any one time. In fact, this is more
or less what Luke himself suggested when I met him.
The problem therefore, is that currently, puppet can only say "Make X
look like Y", not "While doing A make X look like Y and otherwise
Z".
So we could say something like:
loadbalancer { $ipaddress: ensure => enabled }
semaphore { $cluster_name: ensure => none }
with-temporary-state {
ensuring => {
LoadBalancer[$ipaddress] { ensure => disabled }
Semaphore[$cluster_name] { ensure => aquired }
}
package { wibble: ensure => "1.0.1" }
}
So here, puppet would know that when entering the
with-temporary-state block it needs to ensure that ensure on the
loadbalancer resource is set to enabled and the semaphore resource is
aquired, and that at all other times, they are set to enabled / none
respectively. Also, if the changing of the state fails, then we''d need
to skip the block (as puppet does with failed dependencies).
I suppose what I''m asking, is firstly, is anyone else doing anything
like this themselves? I imagine someone must be. If so, are they using
wrapper scripts (as we do) or some other mechanism?
Secondly, if my company were to sponsor development of such a feature
(ie: let me develop it at work), what would we have to do to get it
into the mainline release of puppet? I fully imagine that it''ll need a
bit of discussion first, as of course, it''s a bit of a shift in how
puppet models resources.
--
Ceri Storey
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ceri Storey wrote:> For some background, we''re essentially using puppet in much the same > way you would use the ssh-and-a-for-loop method. I know it''s well, > fundamentally missing the point, but the thing is, there are a few > constraints that we''re currently otherwise unable to satisfy. > > Essentailly, we have a large number of nigh upon homogeneous servers > organised into clusters of about 10 machines each. These are entirely > customer-facing, so we need to be able to ensure that the cluster > overall doesn''t go down when we''re doing software deployments. So, to > do this, we follow a process rather like this: > > Disable server inload balancer -> Take server out of monitoring -> run > puppet -> Put server back in monitoring -> Enable server inLoad > balancer > > Combined with smoketesting of the server once it''s been upgraded, and > ensuring that it comes back up in monitoring; and ensuring that only > N/M servers are out of service at any one time. In fact, this is more > or less what Luke himself suggested when I met him. > > The problem therefore, is that currently, puppet can only say "Make X > look like Y", not "While doing A make X look like Y and otherwise Z". > So we could say something like: > > loadbalancer { $ipaddress: ensure => enabled } > semaphore { $cluster_name: ensure => none } > > with-temporary-state { > ensuring => { > LoadBalancer[$ipaddress] { ensure => disabled } > Semaphore[$cluster_name] { ensure => aquired } > } > package { wibble: ensure => "1.0.1" } > } > > So here, puppet would know that when entering the > with-temporary-state block it needs to ensure that ensure on the > loadbalancer resource is set to enabled and the semaphore resource is > aquired, and that at all other times, they are set to enabled / none > respectively. Also, if the changing of the state fails, then we''d need > to skip the block (as puppet does with failed dependencies). > > I suppose what I''m asking, is firstly, is anyone else doing anything > like this themselves? I imagine someone must be. If so, are they using > wrapper scripts (as we do) or some other mechanism? > > Secondly, if my company were to sponsor development of such a feature > (ie: let me develop it at work), what would we have to do to get it > into the mainline release of puppet? I fully imagine that it''ll need a > bit of discussion first, as of course, it''s a bit of a shift in how > puppet models resources.Ceri I don''t think anyone is working on this at this time. Some prior discussions have been held on/around this concept - check the list archives for some of the keywords you''ve used. I''ve cc''ed this to the -dev list where this sort of discussion is best held. Regards James Turnbull - -- Author of: * Pulling Strings with Puppet (http://www.amazon.com/gp/product/1590599780/) * Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) * Hardening Linux (http://www.amazon.com/gp/product/1590594444/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJIrxk9hTGvAxC30ARAh0NAJ9nG2N8Xw05HCb+wpBzwgx+bqWd1gCfTKs5 FJMW53n6KMEo6b79DcGRfgk=L7V5 -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ceri Storey wrote:> The problem therefore, is that currently, puppet can only say "Make X > look like Y", not "While doing A make X look like Y and otherwise Z". > So we could say something like: > > loadbalancer { $ipaddress: ensure => enabled } > semaphore { $cluster_name: ensure => none } > > with-temporary-state { > ensuring => { > LoadBalancer[$ipaddress] { ensure => disabled } > Semaphore[$cluster_name] { ensure => aquired } > } > package { wibble: ensure => "1.0.1" } > } > > So here, puppet would know that when entering the > with-temporary-state block it needs to ensure that ensure on the > loadbalancer resource is set to enabled and the semaphore resource is > aquired, and that at all other times, they are set to enabled / none > respectively. Also, if the changing of the state fails, then we''d need > to skip the block (as puppet does with failed dependencies).Actually, Puppet can be coaxed into doing something like that already, and we do it on a couple of our compute clusters. We run Puppet during the installation of our compute nodes, in the %post section of kickstart (CentOS). Since we don''t want to start all the daemons and other things until the node is fully installed and configured, and has booted properly, we need to make Puppet do different things depending on whether it is run during kickstart or later. A not very well-documented feature of facter, is that you can set environment variables of the form FACTER_FOO, and the fact ''foo'' will be created: $ FACTER_FOO=gazonk.del facter [...] foo => gazonk.del In your manifests you can then check the value of $foo and do different things. When we run Puppet from kickstart, we set the environment variable FACTER_PUP_RUNDAEMONS to "false", and our manifests look like this: # We set $running to stopped when Puppet is run during kickstart, and # use $running in every service definition when we would normally have # specified ''ensure => running'', so services aren''t started while the # machine is installing. $running = $pup_rundaemons ? { "false" => stopped, default => running, } class torque-compute-node { service { "pbs_mom": enable => true, ensure => $running, hasstatus => true; } } The ''if'' and ''case'' statements are of course also helpful for this. You could create some custom fact instead of setting environment variables, if you want to trigger on something else on the machine. /Thomas Bellman --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
(This is somewhat awkwardly on both lists; is everyone ok moving it to -dev?) On Nov 18, 2008, at 6:48 AM, Ceri Storey wrote:> [snipped description of state-based change control] > I suppose what I''m asking, is firstly, is anyone else doing anything > like this themselves? I imagine someone must be. If so, are they using > wrapper scripts (as we do) or some other mechanism?As others have said, yes, this is a common problem. You might look at ControlTier as a way to solve some of these problems more easily, rather than writing your own scripts to do the work. However...> > Secondly, if my company were to sponsor development of such a feature > (ie: let me develop it at work), what would we have to do to get it > into the mainline release of puppet? I fully imagine that it''ll need a > bit of discussion first, as of course, it''s a bit of a shift in how > puppet models resources.I''m keenly interested in getting some of these features added to Puppet. I''ve got some ideas for it, but I haven''t attempted to implement them because I wanted to wait until I had clear use cases with a couple of customers. If this is something you would be able to add, I would gladly work with you to finalize the design and then get it committed to the core. This is definitely on the roadmap of things I want to do. It might actually dovetail nicely with the not-quite-infinitely-long discussion about syntax changes that has been happening on the -dev list. I''ll try to kick it off today by emailing my initial (early) ideas to the list. -- Levy''s Law: The truth is always more interesting than your preconception of what it might be. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---