Hi All, I''ve been trying to address the issue of when (once in a while) you need to push something to a client now, cant wait 30 minutes, or its a one time action. The first path I went though was though was to consider something like puppetrun, and as we don''t run puppetd as a daemon (running it from cron due to large memory consumption while idle), I went forward and implemented a basic daemon which uses puppet certificates (no need for ssh or anything else) infrastructure to kick in puppetd --onetime. nevertheless, I realized that this would not solve all of my problems, many times, one needs to perform an action once, a good example could be package updates, or maybe you have an application "downtime" where you want to restart a service, or just reboot the server. another good reason against it, is when you have a fairly large amount of nodes, calling all of your clients to recompile their configuration in a short time frame, is usually leading to a puppetmaster which is unable to perform, using tags for this special onetime puppetd run can help with file requests, but it is still something that the puppetmaster usually cant handle. until now, I was using puppet for deployment, configuration, and making sure that everything stays that way, but I''m still using ssh for everything else, I wasn''t too happy about it, so I had a look on func, which also seems very nice, has its own certificate (even due you could reuse the puppet certificates) and a small daemon that you can run commands / modules though, but this requires yet another daemon, another language (in this case its python / shell and not puppet dsl) etc... which leads me to my question: Why Puppet cant have a push directive as well? obviously we already have the infrastructure to connect from the server to the clients (e.g. puppetrun, or my xinetd version), we already have all of the puppet types, so it should be easy to execute on a client (possible today with ralsh or puppet directly) things like: Service (manages restarts of services) Packages (maybe something like ensure => latest for a one time package upgrades) Exec for custom scripts ... and maybe additional facts that could be executed only upon request later on, if you have storeconfig ldap or just pure yaml node files, you can add grouping and other nice things.... Cheers, Ohad --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ohad Levy wrote:> puppetrun, and as we don''t run puppetd as a daemon (running it from cron > due to large memory consumption while idle), I went forward andHere is a semi-related thought. Without regard for memory consumption, but rather due to the notion that system-wide, manageable, canonical mechanisms are better than app-specific mechanisms, I would prefer that Puppet did not have a daemon mode, scheduler, sleep-N-minutes mechanism etc., at all; but rather that the Puppet install process set it up in cron. Of course I can set it up that way myself, but what I wish was that Puppet didn''t carry the burden of its own mechanism: no code for it, no command line options, no init.d script, no wiki pages about it, no discussion of it, etc. (I''m sure there is a good reason it made sense at the time, and nonetheless I like Puppet a lot, I continue to be a fan of Puppet!) -- Kyle Cordes http://kylecordes.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ohad, I have been doing some thinking about this as well. I can definitely see situations where I need a "cap shell" type of approach to push a single change or something else, but usually those are for administration needs. Now feel free to correct me if I am wrong, but puppet''s whole goal is to alleviate you from having to perform those mundane tasks. Again as I see it, if you cant wait a few minutes for changes to go out, then well, its obviously an emergency and usually the best route is direct access. My next question is, is there a way you can enable listen on the clients, define a schedule that keeps puppet from running (hack hack hack) and then cron/schedue things on the puppetmaster side to do puppetrun against "groups" of clients you have in a classification tool or something. Anyway, food for thought. Also, whatever happened to that cobbler/puppet related tool you were talking about a few months back? -Jason On Apr 1, 2009, at 6:36 PM, Ohad Levy wrote:> Hi All, > > I''ve been trying to address the issue of when (once in a while) you > need to push something to a client now, cant wait 30 minutes, or its > a one time action. > > The first path I went though was though was to consider something > like puppetrun, and as we don''t run puppetd as a daemon (running it > from cron due to large memory consumption while idle), I went > forward and implemented a basic daemon which uses puppet > certificates (no need for ssh or anything else) infrastructure to > kick in puppetd --onetime. > > nevertheless, I realized that this would not solve all of my > problems, many times, one needs to perform an action once, a good > example could be package updates, or maybe you have an application > "downtime" where you want to restart a service, or just reboot the > server. > another good reason against it, is when you have a fairly large > amount of nodes, calling all of your clients to recompile their > configuration in a short time frame, is usually leading to a > puppetmaster which is unable to perform, using tags for this special > onetime puppetd run can help with file requests, but it is still > something that the puppetmaster usually cant handle. > > until now, I was using puppet for deployment, configuration, and > making sure that everything stays that way, but I''m still using ssh > for everything else, I wasn''t too happy about it, so I had a look on > func, which also seems very nice, has its own certificate (even due > you could reuse the puppet certificates) and a small daemon that you > can run commands / modules though, but this requires yet another > daemon, another language (in this case its python / shell and not > puppet dsl) etc... which leads me to my question: > > Why Puppet cant have a push directive as well? obviously we already > have the infrastructure to connect from the server to the clients > (e.g. puppetrun, or my xinetd version), we already have all of the > puppet types, so it should be easy to execute on a client (possible > today with ralsh or puppet directly) things like: > Service (manages restarts of services) > Packages (maybe something like ensure => latest for a one time > package upgrades) > Exec for custom scripts > ... > and maybe additional facts that could be executed only upon request > > later on, if you have storeconfig ldap or just pure yaml node files, > you can add grouping and other nice things.... > > Cheers, > Ohad > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Thu, Apr 2, 2009 at 10:35 AM, Jason Rojas <jason@nothingbeatsaduck.com>wrote:> > Ohad, I have been doing some thinking about this as well. I can > definitely see situations where I need a "cap shell" type of approach > to push a single change or something else, but usually those are for > administration needs.sure, this are considered "emergency" or good response time to our customers.> > Now feel free to correct me if I am wrong, but puppet''s whole goal is > to alleviate you from having to perform those mundane tasks. Again as > I see it, if you cant wait a few minutes for changes to go out, then > well, its obviously an emergency and usually the best route is direct > access.what happens if you have hundreds of clients that needs to be fixed? we had an issue a few months ago where nscd got crazy due to some bad ldap / tls response, we needed to restarted nscd on all clients....> > > My next question is, is there a way you can enable listen on the > clients, define a schedule that keeps puppet from running (hack hack > hack) and then cron/schedue things on the puppetmaster side to do > puppetrun against "groups" of clients you have in a classification > tool or something. >I''m not really sure I understand your question, I can today use puppetrun (if running puppetd with listen option) or use my own version of mini puppet listener daemon to trigge a puppet run, but that will not solve my previous nscd example (it would make sure that nscd is running, which it is). more than that, if you need to compile configurations for 250 clients in a couple of minutes, with the current puppetmaster implementation, thats almost impossible (e.g. you really need a lot of mongrels / cores).> > > Anyway, food for thought.yep, thats the whole reason for this thread...> > > Also, whatever happened to that cobbler/puppet related tool you were > talking about a few months back?Its a live n kicking, my company still <sigh> discuss which open source license is the most risk free for us...> -Jason > > > On Apr 1, 2009, at 6:36 PM, Ohad Levy wrote: > > > Hi All, > > > > I''ve been trying to address the issue of when (once in a while) you > > need to push something to a client now, cant wait 30 minutes, or its > > a one time action. > > > > The first path I went though was though was to consider something > > like puppetrun, and as we don''t run puppetd as a daemon (running it > > from cron due to large memory consumption while idle), I went > > forward and implemented a basic daemon which uses puppet > > certificates (no need for ssh or anything else) infrastructure to > > kick in puppetd --onetime. > > > > nevertheless, I realized that this would not solve all of my > > problems, many times, one needs to perform an action once, a good > > example could be package updates, or maybe you have an application > > "downtime" where you want to restart a service, or just reboot the > > server. > > another good reason against it, is when you have a fairly large > > amount of nodes, calling all of your clients to recompile their > > configuration in a short time frame, is usually leading to a > > puppetmaster which is unable to perform, using tags for this special > > onetime puppetd run can help with file requests, but it is still > > something that the puppetmaster usually cant handle. > > > > until now, I was using puppet for deployment, configuration, and > > making sure that everything stays that way, but I''m still using ssh > > for everything else, I wasn''t too happy about it, so I had a look on > > func, which also seems very nice, has its own certificate (even due > > you could reuse the puppet certificates) and a small daemon that you > > can run commands / modules though, but this requires yet another > > daemon, another language (in this case its python / shell and not > > puppet dsl) etc... which leads me to my question: > > > > Why Puppet cant have a push directive as well? obviously we already > > have the infrastructure to connect from the server to the clients > > (e.g. puppetrun, or my xinetd version), we already have all of the > > puppet types, so it should be easy to execute on a client (possible > > today with ralsh or puppet directly) things like: > > Service (manages restarts of services) > > Packages (maybe something like ensure => latest for a one time > > package upgrades) > > Exec for custom scripts > > ... > > and maybe additional facts that could be executed only upon request > > > > later on, if you have storeconfig ldap or just pure yaml node files, > > you can add grouping and other nice things.... > > > > Cheers, > > Ohad > > > > > > > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hello there This might not be a solution to your problem (from a philosophical stance anyway), but you could grant your user to run `puppetd -vt` without password via sudoers on all hosts. Then you could `ssh <host> puppetd -vt` from your workstation (or write a wrapper that takes input from a list or whatever). OR if you''re using keybased authentication allow your puppetmaster host access to a set account from where using it''s key it can only run puppetd. Hosts.allow, firewalls etc may need to be configured to the hosts (since so far each is pulling rather than pushing), and then you could for hosts in `puppetca --list --all | grep ^+ | cut -d '' '' -f 2` do ssh $hosts sudo puppetd -vt done but a push architecture is significantly more security vulnerable... on the bright side though, if your central configuration host is compromised, it being able to ssh to hosts is the least of your worries (why attack individual hosts if you have the master key?) cheers chakkerz --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Thu, Apr 2, 2009 at 11:51 AM, chakkerz <chakkerz@gmail.com> wrote:> > > for hosts in `puppetca --list --all | grep ^+ | cut -d '' '' -f 2` > do > ssh $hosts sudo puppetd -vt > donesure, I know / do this, but I though that one of the goals of puppet is to avoid ssh and a for loop.... but seriously, what happens if ssh doesn''t work? ( I mean, usually you need push when something is broken) or you need to deploy something only on a subset of machines, restart a service, or whatever?> but a push architecture is significantly more security vulnerable... > on the bright side though, if your central configuration host is > compromised, it being able to ssh to hosts is the least of your > worries (why attack individual hosts if you have the master key?) >why care about the master key when you can simply change the puppet manifest ? ;)> >Ohad --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ohad Levy wrote:> sure, I know / do this, but I though that one of the goals of puppet > is to avoid ssh and a for loop.... but seriously, what happens if ssh > doesn''t work? ( I mean, usually you need push when something is > broken) > > or you need to deploy something only on a subset of machines, restart > a service, or whatever?dsh -- http://www.netfort.gr.jp/~dancer/software/dsh.html Granted, it only works if ssh does, but it does let you shell to pre-defined groups of systems, arbitrary lists of machines, and optionally run them concurrently. Some sharper person than me might work up a clean way of managing the dsh configuration files with exported resources -- one file containing all machines managed by puppet in /etc/dsh/machines.list and/or /etc/dsh/group/all, another file per group of systems identified by hostgroup ( http://reductivelabs.com/trac/puppet/wiki/Recipes/HostgroupFact ) into /etc/dsh/group/${group}, etc. -- Mike Renfro / R&D Engineer, Center for Manufacturing Research, 931 372-3601 / Tennessee Technological University -- renfro@tntech.edu --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Mike, dsh, clsh, clusterit, func, hope grown scripts etc you name it, I think I know them all, the question remains, why shouldn''t this kind of functionallity cant be achived with Puppet? as far as I see it (and probably I dont see the whole picture) most of the internal infrastructure can support this... am I the only one who wants to have such a feature? I really like to have one "langague/daemon/etc". Cheers, Ohad On Fri, Apr 3, 2009 at 9:56 PM, Mike Renfro <renfro@tntech.edu> wrote:> > Ohad Levy wrote: > > > sure, I know / do this, but I though that one of the goals of puppet > > is to avoid ssh and a for loop.... but seriously, what happens if ssh > > doesn''t work? ( I mean, usually you need push when something is > > broken) > > > > or you need to deploy something only on a subset of machines, restart > > a service, or whatever? > > dsh -- http://www.netfort.gr.jp/~dancer/software/dsh.html<http://www.netfort.gr.jp/%7Edancer/software/dsh.html> > > Granted, it only works if ssh does, but it does let you shell to > pre-defined groups of systems, arbitrary lists of machines, and > optionally run them concurrently. > > Some sharper person than me might work up a clean way of managing the > dsh configuration files with exported resources -- one file containing > all machines managed by puppet in /etc/dsh/machines.list and/or > /etc/dsh/group/all, another file per group of systems identified by > hostgroup ( > http://reductivelabs.com/trac/puppet/wiki/Recipes/HostgroupFact ) into > /etc/dsh/group/${group}, etc. > > -- > Mike Renfro / R&D Engineer, Center for Manufacturing Research, > 931 372-3601 / Tennessee Technological University -- renfro@tntech.edu > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
I love this topic, first though:> dsh, clsh, clusterit, func, hope grown scripts etc you name it, I think I > know them all, the question remains, why shouldn''t this kind of > functionallity cant be achived with Puppet?i don''t know them all but :) i''ll have to check out clsh and clusterit ... i think clsh is the one my technical lead (ab)used and needed me to kill things in a hurry ... anyways though. Also the mainfest was what i meant == the master key :) I''ll go a little deeper than before i think: Push vs pull - push requires each host to be accessible to a central service. The connection from each host will likely need to pass through multiple barriers such as network firewalls, on host firewalls, hosts allow (if tcpwrapped). There are also issues of multiple security zones. One might argue that a centralized configuration management host should be isolated and of the highest security. Alas the powers that be have been known to keep this host(s) on unfirewalled networks with unpatched on host firewalls. Push is undesirable because the high cost to deploy, each firewall, each host need to be configured to allow the connection. Which means any migration (software or IP) may entail serious effort just for access rights. Also push means that even though as Ohad pointed out the manifest is the thing that will / can compromise each host, a pull can be scheduled. Most people probably run hourly cycles. Maybe offsetting production and development hosts (do you want a change to appear quicker or slower on a prod host?) or having tiered solutions with a puppet server for prod and another for dev. Regardless, pull does give the illusion of control because each client needs to ask for the change. So if you are diligent your entire range of hosts won''t go down the drain in one go. (we interpret cf in cfengine as cluster f**k for a reason). Push on the other hand implies each host gets the same change at the same time. And even if you''re tiered you can still do a lot of damage. So i suggest this: given your need to be able affect each host instantaneously, 1) why not have a second instance of puppet (i don''t know if you can run two on the same host though) for urgent things only, and query the different instances at different intervals. 2) have a fact that based on time sets the urgency of your modules, so your regular modules only trip every hour, and your urgent ones straight away. So say have the fact check the time, and if it''s 30 past the hour it does the regular changes, every minute does urgent and every (seventh) day at midnight does the ultra rare? regardless, tune your updates to happen every minute. The minutely urgent modules should run quickly and all the other ones that evaluate your fact as meaning they don''t need to run should be fast, so your urgent change should probably run alone, and after 10 minutes you just change the urgency. I''d probably have a central file that sets the urgency value for each module, so making the change, updating the variable in the central file and a daemon restart later you have 1 minute to pray you didn''t screw up what you''re meaning to fix :). Heck, you could even have a clever cronjob that runs on the puppetmaster that checks the file for urgency variables, and 3 minutes after having been touched it replaces it with the non-emergency one and restarts puppetmaster if needed... But implementing push ... you''re playing with fire mate :) Cheers chakkerz --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
We use something similar. All our servers have the UCE agent installed. So we can initiate a puppetd run via UCE when required for 1. Reporting 2. Urgent updates 3. Standard updates. And UCE is ratified by our security bods. Sure UCE is clunky but you have to use the tools you''ve got. Geoff. On 3 Apr 2009, at 05:42, Ohad Levy <ohadlevy@gmail.com> wrote:> > > On Thu, Apr 2, 2009 at 11:51 AM, chakkerz <chakkerz@gmail.com> wrote: > > > for hosts in `puppetca --list --all | grep ^+ | cut -d '' '' -f 2` > do > ssh $hosts sudo puppetd -vt > done > > sure, I know / do this, but I though that one of the goals of puppet > is to avoid ssh and a for loop.... > but seriously, what happens if ssh doesn''t work? ( I mean, usually > you need push when something is broken) > > or you need to deploy something only on a subset of machines, > restart a service, or whatever? > > > but a push architecture is significantly more security vulnerable... > on the bright side though, if your central configuration host is > compromised, it being able to ssh to hosts is the least of your > worries (why attack individual hosts if you have the master key?) > why care about the master key when you can simply change the puppet > manifest ? ;) > > > Ohad > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ohad, Did you see func ? https://fedorahosted.org/func/ []s On Sat, Apr 4, 2009 at 3:45 AM, Geoff Newell <geoffnewell@gmail.com> wrote:> We use something similar. All our servers have the UCE agent installed. So > we can initiate a puppetd run via UCE when required for > 1. Reporting > 2. Urgent updates > 3. Standard updates. > And UCE is ratified by our security bods. Sure UCE is clunky but you have to > use the tools you''ve got. > > Geoff. > On 3 Apr 2009, at 05:42, Ohad Levy <ohadlevy@gmail.com> wrote: > > > > On Thu, Apr 2, 2009 at 11:51 AM, chakkerz <chakkerz@gmail.com> wrote: >> >> >> for hosts in `puppetca --list --all | grep ^+ | cut -d '' '' -f 2` >> do >> ssh $hosts sudo puppetd -vt >> done > > > sure, I know / do this, but I though that one of the goals of puppet is to > avoid ssh and a for loop.... > but seriously, what happens if ssh doesn''t work? ( I mean, usually you need > push when something is broken) > > or you need to deploy something only on a subset of machines, restart a > service, or whatever? > >> >> but a push architecture is significantly more security vulnerable... >> on the bright side though, if your central configuration host is >> compromised, it being able to ssh to hosts is the least of your >> worries (why attack individual hosts if you have the master key?) > > why care about the master key when you can simply change the puppet manifest > ? ;) >> > > Ohad > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Sat, Apr 4, 2009 at 1:29 PM, chakkerz <chakkerz@gmail.com> wrote:> > I''ll go a little deeper than before i think: > Push vs pull - push requires each host to be accessible to a central > service. The connection from each host will likely need to pass > through multiple barriers such as network firewalls, on host > firewalls, hosts allow (if tcpwrapped). There are also issues of > multiple security zones. One might argue that a centralized > configuration management host should be isolated and of the highest > security. Alas the powers that be have been known to keep this host(s) > on unfirewalled networks with unpatched on host firewalls. >True, but I would still argue that a fairly large amount of nodes does not have this limitation, or that SSL traffic is secure enough to open (which can become a standard in your FW rules).> > Push is undesirable because the high cost to deploy, each firewall, > each host need to be configured to allow the connection. Which means > any migration (software or IP) may entail serious effort just for > access rights. Also push means that even though as Ohad pointed out > the manifest is the thing that will / can compromise each host, a pull > can be scheduled. Most people probably run hourly cycles. Maybe > offsetting production and development hosts (do you want a change to > appear quicker or slower on a prod host?) or having tiered solutions > with a puppet server for prod and another for dev. Regardless, pull > does give the illusion of control because each client needs to ask for > the change. So if you are diligent your entire range of hosts won''t go > down the drain in one go. (we interpret cf in cfengine as cluster f**k > for a reason). >Are you telling you don''t have other means of push already today (ssh??) and at the end of the day, how many people don''t run puppet twice an hour? (really I''m asking!) there is a risk in having a push, but there is a risk in not having it as well.> Push on the other hand implies each host gets the same change at the > same time. And even if you''re tiered you can still do a lot of damage. > > So i suggest this: given your need to be able affect each host > instantaneously, > 1) why not have a second instance of puppet (i don''t know if you can > run two on the same host though) for urgent things only, and query the > different instances at different intervals. >interesting idea... I have to give it some more thought...> > 2) have a fact that based on time sets the urgency of your modules, so > your regular modules only trip every hour, and your urgent ones > straight away. So say have the fact check the time, and if it''s 30 > past the hour it does the regular changes, every minute does urgent > and every (seventh) day at midnight does the ultra rare? >I assume you are trying to address the server load problem, in this case I would guess (without checking too much) that the compile time would be similar to using tags.> > regardless, tune your updates to happen every minute. The minutely > urgent modules should run quickly and all the other ones that evaluate > your fact as meaning they don''t need to run should be fast, so your > urgent change should probably run alone, and after 10 minutes you just > change the urgency. I''d probably have a central file that sets the > urgency value for each module, so making the change, updating the > variable in the central file and a daemon restart later you have 1 > minute to pray you didn''t screw up what you''re meaning to fix :). >> Heck, you could even have a clever cronjob that runs on the > puppetmaster that checks the file for urgency variables, and 3 minutes > after having been touched it replaces it with the non-emergency one > and restarts puppetmaster if needed... > > But implementing push ... you''re playing with fire mate :) >sounds like an ssh loop is still easier ;) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
sure I did, my main point is that it could be integrated withing puppet without using additional tools... On Mon, Apr 6, 2009 at 9:50 PM, Marcus Vechiato <vechiato@gmail.com> wrote:> > Ohad, > > Did you see func ? https://fedorahosted.org/func/ > > []s > > On Sat, Apr 4, 2009 at 3:45 AM, Geoff Newell <geoffnewell@gmail.com> > wrote: > > We use something similar. All our servers have the UCE agent installed. > So > > we can initiate a puppetd run via UCE when required for > > 1. Reporting > > 2. Urgent updates > > 3. Standard updates. > > And UCE is ratified by our security bods. Sure UCE is clunky but you have > to > > use the tools you''ve got. > > > > Geoff. > > On 3 Apr 2009, at 05:42, Ohad Levy <ohadlevy@gmail.com> wrote: > > > > > > > > On Thu, Apr 2, 2009 at 11:51 AM, chakkerz <chakkerz@gmail.com> wrote: > >> > >> > >> for hosts in `puppetca --list --all | grep ^+ | cut -d '' '' -f 2` > >> do > >> ssh $hosts sudo puppetd -vt > >> done > > > > > > sure, I know / do this, but I though that one of the goals of puppet is > to > > avoid ssh and a for loop.... > > but seriously, what happens if ssh doesn''t work? ( I mean, usually you > need > > push when something is broken) > > > > or you need to deploy something only on a subset of machines, restart a > > service, or whatever? > > > >> > >> but a push architecture is significantly more security vulnerable... > >> on the bright side though, if your central configuration host is > >> compromised, it being able to ssh to hosts is the least of your > >> worries (why attack individual hosts if you have the master key?) > > > > why care about the master key when you can simply change the puppet > manifest > > ? ;) > >> > > > > Ohad > > > > > > > > > > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
> > > Why Puppet cant have a push directive as well? obviously we already > have the infrastructure to connect from the server to the clients > (e.g. puppetrun, or my xinetd version), we already have all of the > puppet types, so it should be easy to execute on a client (possible > today with ralsh or puppet directly) things like:What xinetd version? -L -- Larry Ludwig Reductive Labs --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hello again Ohad
I''ve finally found some time to play with this.
I''ll give you the back of the proof of concept version and
i''ll write
some doco for the site when i get a chance.
I''ve done two facts which are probably written in the worst way
possible, but i haven''t done this before :)
[root@tangelo facter]# pwd
/etc/puppet/modules/custom/plugins/facter
[root@tangelo facter]# cat system_time_*
# system_time_hour.rb
Facter.add("system_time_hour") do
time = Time.now
hour = time.hour
setcode do
hour
end
end
# system_time_minute.rb
Facter.add("system_time_minute") do
time = Time.now
minute = time.min
setcode do
minute
end
end
Then my default node class works out what OS, and based on that calls
the class for that OS, either OS class inherits from shared-default
which contains the non OS specific files and that shared one contains
an inheritance (the include was not smiling on me that day) to some
default variables.
My default variables file defines:
class initialisevariables
{
##
## daily denotes an hour while hourly denotes a minute
##
$solaris_daily = "7"
$solaris_hourly = "30"
$linux_daily = "8"
$linux_hourly = "00"
}
which the OS specific class uses to set itself up (this is for Linux):
$change_daily = $linux_daily
$change_hourly = $linux_hourly
So so far so good :)
Now the only module i''ve tested this on is my resolv_conf one which I
added at the start:
class resolv
{
$resolv_conf_urgency = "hourly"
case $resolv_conf_urgency
{
"minutely" : { $process_now = "true" }
"hourly" : { if ($system_time_minute == $change_hourly)
{ $process_now = "true" }}
"daily" : { if ($system_time_hour == $change_daily )
{ $process_now = "true" }}
}
if (($skip_resolv != "true") and ($process_now == "true"))
{
....
I''m not fully happy with this as yet, but in principle it works.
As to your query regarding me not having a current way to push things
out immediately... no i don''t. IF we need to do something like this we
modify our cfengine instance and then hook onto each box that needs
the change urgently and run it by hand. Puppet will hopefully change
things ... especially with this notion i''m putting in place :)
Cheers
chakkerz
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Hi chakkerz, First of all thanks for putting the effort in this. secondly, I''m not sure, but what is the difference between what you just implemented with normal schedule meta parameter? schedule allows you to decide how often a certain resource would be checked/applied, and if I understood your code below, this is what you have implemented. again, if I didn''t miss anything in your implementation, you still have a minimum of time (e.g. by default up to 30 minutes) that it takes to rollout a change, and when we are talking about critical changes this is not an option. I think I''ll enhanced my mini puppet listener script and post it if anyone would be interested... Cheers, Ohad On Wed, Apr 8, 2009 at 6:55 AM, chakkerz <chakkerz@gmail.com> wrote:> > Hello again Ohad > > I''ve finally found some time to play with this. > > I''ll give you the back of the proof of concept version and i''ll write > some doco for the site when i get a chance. > > I''ve done two facts which are probably written in the worst way > possible, but i haven''t done this before :) > > [root@tangelo facter]# pwd > /etc/puppet/modules/custom/plugins/facter > [root@tangelo facter]# cat system_time_* > # system_time_hour.rb > > Facter.add("system_time_hour") do > time = Time.now > hour = time.hour > setcode do > hour > end > end > > # system_time_minute.rb > > Facter.add("system_time_minute") do > time = Time.now > minute = time.min > setcode do > minute > end > end > > Then my default node class works out what OS, and based on that calls > the class for that OS, either OS class inherits from shared-default > which contains the non OS specific files and that shared one contains > an inheritance (the include was not smiling on me that day) to some > default variables. > > My default variables file defines: > class initialisevariables > { > ## > ## daily denotes an hour while hourly denotes a minute > ## > $solaris_daily = "7" > $solaris_hourly = "30" > > $linux_daily = "8" > $linux_hourly = "00" > } > > which the OS specific class uses to set itself up (this is for Linux): > $change_daily = $linux_daily > $change_hourly = $linux_hourly > > So so far so good :) > > Now the only module i''ve tested this on is my resolv_conf one which I > added at the start: > class resolv > { > $resolv_conf_urgency = "hourly" > case $resolv_conf_urgency > { > "minutely" : { $process_now = "true" } > "hourly" : { if ($system_time_minute == $change_hourly) > { $process_now = "true" }} > "daily" : { if ($system_time_hour == $change_daily ) > { $process_now = "true" }} > } > > if (($skip_resolv != "true") and ($process_now == "true")) > { > .... > > I''m not fully happy with this as yet, but in principle it works. > > > As to your query regarding me not having a current way to push things > out immediately... no i don''t. IF we need to do something like this we > modify our cfengine instance and then hook onto each box that needs > the change urgently and run it by hand. Puppet will hopefully change > things ... especially with this notion i''m putting in place :) > > Cheers > chakkerz > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Tue, Apr 7, 2009 at 8:32 PM, Larry Ludwig <larry@reductivelabs.com>wrote:> > > What xinetd version?I wrote a simple daemon which uses puppet certificates and namespaceauth to allow puppetrun on cron based hosts. cheers, Ohad --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Ohad Levy schrieb am 08. Apr 2009 um 05:14:14 CEST:> I think I''ll enhanced my mini puppet listener script and post it if anyone > would be interested...That would be great. Helmut -- ------------------------------------------------------------------------- Helmut Lichtenberg <Helmut.Lichtenberg@fli.bund.de> Tel.: 05034/871-128 Institut für Nutztiergenetik (FLI) 31535 Neustadt Germany ------------------------------------------------------------------------- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hi> I think I''ll enhanced my mini puppet listener script and post it if anyone > would be interested...that would be great. Maybe it''s even something for the contrib directory? cheers pete --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Apr 7, 2009, at 11:15 PM, Ohad Levy wrote:> > > On Tue, Apr 7, 2009 at 8:32 PM, Larry Ludwig > <larry@reductivelabs.com> wrote: > > > What xinetd version? > I wrote a simple daemon which uses puppet certificates and > namespaceauth to allow puppetrun on cron based hosts. >Hi Ohad, Can you post this somewhere? This is something I was looking for and/ or was going to develop. -L> cheers, > Ohad > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
If you log a ticket I am happy to add this into the contrib directory. Regards James Turnbull On 08/04/2009, at 6:00 PM, Larry Ludwig <larry@reductivelabs.com> wrote:> > On Apr 7, 2009, at 11:15 PM, Ohad Levy wrote: > >> >> >> On Tue, Apr 7, 2009 at 8:32 PM, Larry Ludwig >> <larry@reductivelabs.com> wrote: >> >> >> What xinetd version? >> I wrote a simple daemon which uses puppet certificates and >> namespaceauth to allow puppetrun on cron based hosts. >> > > Hi Ohad, > > Can you post this somewhere? This is something I was looking for > and/or was going to develop. > > -L > > >> cheers, >> Ohad >> >> >> >> > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Maybe it''s the purist in me, but I don''t think emergency, one-off changes are what puppet is/was designed for. It''s supposed to be a configuration management tool, and (using your example) firefighting death spiraling daemons is not a puppet task. (in my opinion) There are tools that are already designed for this, most of which have been mentioned. I''d add that pdsh (parallel distributed shell -- https://computing.llnl.gov/linux/pdsh.html) is my tool of choice for massive site-wide fixes like you describe. As an example, I can do the exact job you explained of restarting a service across 400+ nodes in less than 20 seconds. It''s a great tool, written for exactly the situation you describe. There''s something to be said for having the right tool for the job, isn''t there? I understand the want for a one-solution-fits-all approach, which is unfortunately typically fool''s gold. There are existing tools to do what you''re looking for much faster (efficient) than puppet ever could, given it''s current design. My two pennies. --Bill On Apr 1, 6:36 pm, Ohad Levy <ohadl...@gmail.com> wrote:> Hi All, > > I''ve been trying to address the issue of when (once in a while) you need to > push something to a client now, cant wait 30 minutes, or its a one time > action. > > The first path I went though was though was to consider something like > puppetrun, and as we don''t run puppetd as a daemon (running it from cron due > to large memory consumption while idle), I went forward and implemented a > basic daemon which uses puppet certificates (no need for ssh or anything > else) infrastructure to kick in puppetd --onetime. > > nevertheless, I realized that this would not solve all of my problems, many > times, one needs to perform an action once, a good example could be package > updates, or maybe you have an application "downtime" where you want to > restart a service, or just reboot the server. > another good reason against it, is when you have a fairly large amount of > nodes, calling all of your clients to recompile their configuration in a > short time frame, is usually leading to a puppetmaster which is unable to > perform, using tags for this special onetime puppetd run can help with file > requests, but it is still something that the puppetmaster usually cant > handle. > > until now, I was using puppet for deployment, configuration, and making sure > that everything stays that way, but I''m still using ssh for everything else, > I wasn''t too happy about it, so I had a look on func, which also seems very > nice, has its own certificate (even due you could reuse the puppet > certificates) and a small daemon that you can run commands / modules though, > but this requires yet another daemon, another language (in this case its > python / shell and not puppet dsl) etc... which leads me to my question: > > Why Puppet cant have a push directive as well? obviously we already have the > infrastructure to connect from the server to the clients (e.g. puppetrun, or > my xinetd version), we already have all of the puppet types, so it should be > easy to execute on a client (possible today with ralsh or puppet directly) > things like: > Service (manages restarts of services) > Packages (maybe something like ensure => latest for a one time package > upgrades) > Exec for custom scripts > ... > and maybe additional facts that could be executed only upon request > > later on, if you have storeconfig ldap or just pure yaml node files, you can > add grouping and other nice things.... > > Cheers, > Ohad--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Hi Guys, First working version can be found here: http://github.com/ohadlevy/puppet/commit/f18af58687bbde62cfd756adaf14764c6f44e5de I would assume many improvements could be made, patches are welcome. I''ve given up on (x)inetd for now, as it seems that its impossible to implement it purely in ruby (openssl ruby implementation works only with socket, inetd doesn''t provide a socket....) puppetlisen should run on the client you want to trigger a run on, puppetrun is executed from an "admin" server - doesnt have to be your puppetmaster (just valid certificates). permissions are the same like normal puppetrun (e.g. /etc/puppet/namespaceauth.conf). if there is any good feedback, I would be happy to integerate it in the contrib/ext directory of puppet cheers, Ohad On Wed, Apr 8, 2009 at 1:28 PM, Helmut Lichtenberg < Helmut.Lichtenberg@fli.bund.de> wrote:> > Ohad Levy schrieb am 08. Apr 2009 um 05:14:14 CEST: > > I think I''ll enhanced my mini puppet listener script and post it if > anyone > > would be interested... > > That would be great. > > Helmut > > -- > ------------------------------------------------------------------------- > Helmut Lichtenberg <Helmut.Lichtenberg@fli.bund.de> Tel.: 05034/871-128 > Institut für Nutztiergenetik (FLI) 31535 Neustadt Germany > ------------------------------------------------------------------------- > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Apr 1, 2009, at 8:51 PM, Kyle Cordes wrote:> > Ohad Levy wrote: >> puppetrun, and as we don''t run puppetd as a daemon (running it from >> cron >> due to large memory consumption while idle), I went forward and > > Here is a semi-related thought. > > Without regard for memory consumption, but rather due to the notion > that > system-wide, manageable, canonical mechanisms are better than > app-specific mechanisms, I would prefer that Puppet did not have a > daemon mode, scheduler, sleep-N-minutes mechanism etc., at all; but > rather that the Puppet install process set it up in cron. Of course I > can set it up that way myself, but what I wish was that Puppet didn''t > carry the burden of its own mechanism: no code for it, no command line > options, no init.d script, no wiki pages about it, no discussion of > it, etc.The code around being a daemon is a very, very small amount of code, especially given that the server needs to be a daemon anyway. And I have to admit that the reasons for puppetd being a daemon are less valid than I''d hoped - in particular, the ''resource'' API is powerful and potentially really useful, with the right central tools. I do hope we''ll be making them more valid and more useful in the next year or so, but I have lots of hopes and not enough time for them all. In the meantime, it''s easy enough to skip the daemon mode. -- The hypothalamus is one of the most important parts of the brain, involved in many kinds of motivation, among other functions. The hypothalamus controls the "Four F''s": 1. fighting; 2. fleeing; 3. feeding; and 4. mating. -- Psychology professor in neuropsychology intro course --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Luke Kanies wrote:> In the meantime, it''s easy enough to skip the daemon mode.Maybe I''ll do that next time I repackage a new version. (Or maybe, and this is a dream rather than a request, the good folks working on the official Debian packaging will decide that running it in cron via a file in /etc/cron.whatever is more genuinely Debianish, and they''ll set it up that way.) -- Kyle Cordes http://kylecordes.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Thu, Apr 9, 2009 at 11:03 AM, Kyle Cordes <kyle@kylecordes.com> wrote:> Luke Kanies wrote: >> In the meantime, it''s easy enough to skip the daemon mode. > > Maybe I''ll do that next time I repackage a new version. > > (Or maybe, and this is a dream rather than a request, the good folks > working on the official Debian packaging will decide that running it in > cron via a file in /etc/cron.whatever is more genuinely Debianish, and > they''ll set it up that way.)I''d much prefer you cron folks keep configuring it manually :-P I *like* Puppet as a daemon, leave the Debian package alone! (only half kidding) --Paul --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
+1 to Paul''s comments. I like having the daemon and if you really want it the cron option is easy to configure. I''m intending to show both in the 2nd edition of the Puppet book. Regards James Turnbull On 10/04/2009, at 11:43 AM, Paul Lathrop <paul@tertiusfamily.net> wrote:> > On Thu, Apr 9, 2009 at 11:03 AM, Kyle Cordes <kyle@kylecordes.com> > wrote: >> Luke Kanies wrote: >>> In the meantime, it''s easy enough to skip the daemon mode. >> >> Maybe I''ll do that next time I repackage a new version. >> >> (Or maybe, and this is a dream rather than a request, the good folks >> working on the official Debian packaging will decide that running >> it in >> cron via a file in /etc/cron.whatever is more genuinely Debianish, >> and >> they''ll set it up that way.) > > I''d much prefer you cron folks keep configuring it manually :-P I > *like* Puppet as a daemon, leave the Debian package alone! > > (only half kidding) > > --Paul > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---