I know this has come up on the list numerous times before but I thought it would be a good time to see if the state of the art has advanced for this kind of thing. I wanted to know how people are handling higher level deployment of applications - things that have to be done repeatedly but not all the time. An example of this is checking an application out of svn, building it, creating a package and then moving it off to a repo. Or even just building/installing locally for developers. It never seems to fit well into Puppet for me and I end up with crazy complicated manifests to deal with this kind of thing. I recently moved these jobs into Rundeck (www.rundeck.org) which works pretty well but doesn''t really leverage any of the stuff I have within Foreman/Puppet. I''ve seen suggestions to use mcollective but this doesn''t easily integrate our existing scripts (written in many languages) or processes and would require me to force a lot of developers to work differently. I could just have classes that trigger scripts only when some condition is met (like /.buildapp files) or something along those lines but nothing seems elegant. What I''m trying to find out is what other people did to handle this? I want something I can build up over time and slowly migrate legacy apps and processes into without having to do a massive up front development. It should also be relatively simple and not require me to code anything as anyone on the list who knows me can tell you that I am absolutely awful at coding. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
If you want something simple and don''t need a GUI, many folks are using either Capistrano (Ruby) or the very similar Fabric (Python) for deployment. You can populate hostlists via Foreman queries. That said, I am not sure what sort of integration with Puppet/Foreman you are looking for. Cheers, Brian On Tue, Sep 13, 2011 at 3:53 PM, Ashley Penney <apenney@gmail.com> wrote:> I know this has come up on the list numerous times before but I > thought it would be a good time to see if the state of the art has > advanced for this kind of thing. I wanted to know how people are > handling higher level deployment of applications - things that have to > be done repeatedly but not all the time. An example of this is > checking an application out of svn, building it, creating a package > and then moving it off to a repo. Or even just building/installing > locally for developers. > > It never seems to fit well into Puppet for me and I end up with crazy > complicated manifests to deal with this kind of thing. I recently > moved these jobs into Rundeck (www.rundeck.org) which works pretty > well but doesn''t really leverage any of the stuff I have within > Foreman/Puppet. I''ve seen suggestions to use mcollective but this > doesn''t easily integrate our existing scripts (written in many > languages) or processes and would require me to force a lot of > developers to work differently. I could just have classes that > trigger scripts only when some condition is met (like /.buildapp > files) or something along those lines but nothing seems elegant. > > What I''m trying to find out is what other people did to handle this? > I want something I can build up over time and slowly migrate legacy > apps and processes into without having to do a massive up front > development. It should also be relatively simple and not require me > to code anything as anyone on the list who knows me can tell you that > I am absolutely awful at coding. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- <http://aws.amazon.com/solutions/solution-providers/brandorr/> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
The kind of thing I was thinking of was selecting where to run jobs by puppet class, or even requiring that certain classes are assigned to certain nodes (requiring a pre-req class before you actually send over a job to build). In my mind I envisioned extensions to something like Foreman where you can get a list of jobs that are valid for each node by clicking the node (in a job you could apply constraints like only boxes with mysql::server and> 4G of ram).Another example of the kinds of terrible stuff I engineer when left to my own devices is I wrote a job in rundeck that at the end wrote a .pp to /tmp/ and called it so that I could use templates in Puppet to build the configuration file to distribute. Basically a lot of the time I just need a way to trigger one time puppet manifests with a gui of some kind I can give to developers. I could very easily just spit a few scripts onto the boxes and call those for the build, I just want a way to enable/disable the jobs. I can''t think of any other good way to say "do a one time run of project::build_core on the following matching nodes: x, y, z". I am really just using rundeck for the equivalent of that. Other things I would think of using this for is handling deploying a bunch of servers where server 1 has to be fully provisioned before 2 and on 2 at least one service has to be up before 3 can do its thing. It''s something that''s still a hassle to do well within Puppet. On Tue, Sep 13, 2011 at 4:07 PM, Brian Gupta <brian.gupta@brandorr.com>wrote:> If you want something simple and don''t need a GUI, many folks are using > either Capistrano (Ruby) or the very similar Fabric (Python) for deployment. > You can populate hostlists via Foreman queries. That said, I am not sure > what sort of integration with Puppet/Foreman you are looking for. > > Cheers, > Brian > > On Tue, Sep 13, 2011 at 3:53 PM, Ashley Penney <apenney@gmail.com> wrote: > >> I know this has come up on the list numerous times before but I >> thought it would be a good time to see if the state of the art has >> advanced for this kind of thing. I wanted to know how people are >> handling higher level deployment of applications - things that have to >> be done repeatedly but not all the time. An example of this is >> checking an application out of svn, building it, creating a package >> and then moving it off to a repo. Or even just building/installing >> locally for developers. >> >> It never seems to fit well into Puppet for me and I end up with crazy >> complicated manifests to deal with this kind of thing. I recently >> moved these jobs into Rundeck (www.rundeck.org) which works pretty >> well but doesn''t really leverage any of the stuff I have within >> Foreman/Puppet. I''ve seen suggestions to use mcollective but this >> doesn''t easily integrate our existing scripts (written in many >> languages) or processes and would require me to force a lot of >> developers to work differently. I could just have classes that >> trigger scripts only when some condition is met (like /.buildapp >> files) or something along those lines but nothing seems elegant. >> >> What I''m trying to find out is what other people did to handle this? >> I want something I can build up over time and slowly migrate legacy >> apps and processes into without having to do a massive up front >> development. It should also be relatively simple and not require me >> to code anything as anyone on the list who knows me can tell you that >> I am absolutely awful at coding. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> >> > > > -- > <http://aws.amazon.com/solutions/solution-providers/brandorr/> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sep 13, 2:53 pm, Ashley Penney <apen...@gmail.com> wrote:> I know this has come up on the list numerous times before but I > thought it would be a good time to see if the state of the art has > advanced for this kind of thing. I wanted to know how people are > handling higher level deployment of applications - things that have to > be done repeatedly but not all the time. An example of this is > checking an application out of svn, building it, creating a package > and then moving it off to a repo. Or even just building/installing > locally for developers. > > It never seems to fit well into Puppet for me and I end up with crazy > complicated manifests to deal with this kind of thing. I recently > moved these jobs into Rundeck (www.rundeck.org) which works pretty > well but doesn''t really leverage any of the stuff I have within > Foreman/Puppet. I''ve seen suggestions to use mcollective but this > doesn''t easily integrate our existing scripts (written in many > languages) or processes and would require me to force a lot of > developers to work differently. I could just have classes that > trigger scripts only when some condition is met (like /.buildapp > files) or something along those lines but nothing seems elegant. > > What I''m trying to find out is what other people did to handle this? > I want something I can build up over time and slowly migrate legacy > apps and processes into without having to do a massive up front > development. It should also be relatively simple and not require me > to code anything as anyone on the list who knows me can tell you that > I am absolutely awful at coding.Do not mistake Puppet for a script engine or a remote job executor. It is a state management system. Crazy complicated manifests for an uncomplicated area should be a sign to you that you are using the wrong tool for the job, or at minimum that you are using the tool wrongly. For the types of tasks you listed as examples, I would just use an ordinary shell script, run manually -- supposing the steps were complicated enough to warrant automating it in any way. Run them via a remote job executor service (such as Puppetlabs''s own MCollective) if needed. If you insist on doing this sort of thing with Puppet then you need to start out with a high-level view of the problem. You must answer these two questions right off the bat: 1) What are the *states* that (may) need to be achieved _and maintained_? 2) How will clients'' current state be measured and reported? If you have trouble defining the target states, if you can''t (or don''t want to) plan to maintain them once achieved, or if you don''t have a good way to determine the current state, then your task is not well suited to Puppet. If the answers to those questions are complex, then any Puppet manifests that adequately address this area for you must necessarily be complex -- that is the nature of your particular problem. On the other hand, if you have good, reasonably simple answers to those questions then they should lead you to a good, reasonably simple set of manifests. As Brian implied, it may be that Puppet can contribute to your solution even if it is not the primary driver (e.g. by building and deploying config files and similar for other tools). John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Everyone''s right, you don''t want puppet to do application deployments. You want it to setup your application server, but stop there. Case in point, use Puppet to setup Rails/LAMP/Tomcat, but don''t use it to deploy your actual application. Instead use Capistrano, Fabric, Func, etc. If you''re already familiar with Rundeck, be sure to check out http://www.kartar.net/2010/12/puppet-and-rundeck/ - it may do exactly what you want (export Puppet facter info to rundeck) Justin -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Thu, Sep 15, 2011 at 7:04 AM, Justin Ellison <justin@techadvise.com> wrote:> Everyone''s right, you don''t want puppet to do application > deployments. You want it to setup your application server, but stop > there. > > Case in point, use Puppet to setup Rails/LAMP/Tomcat, but don''t use it > to deploy your actual application. Instead use Capistrano, Fabric, > Func, etc. > > If you''re already familiar with Rundeck, be sure to check out > http://www.kartar.net/2010/12/puppet-and-rundeck/ - it may do exactly > what you want (export Puppet facter info to rundeck)I would use mcollective (which can stop/start services, and upgrade packages), wrapped up inside some simple shell scripts to control it''s operation. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thats because puppet isn''t a tool meant to build applications from source. You could do it but its one of those things where it doesn''t make sense. If you need to build from SVN, you might as well look at something like Jenkins that can monitor an SCM and pull out changes and do a bunch of things around it. Then package it up into an RPM, which puppet handles nicely from my experience. run deck might be a good solution for you on your problem in terms of deploying to one system first before the others. If you had the knowledge with writing web applications you could easily use MCollective to do that too and put a nice GUI in front of the job. On Sep 13, 2011, at 6:33 PM, Ashley Penney wrote:> can''t think of any other good way to say "do a one time run of project::build_core on the following matching nodes: x, y, z". I am really just using rundeck for the equivalent of that. Other things I would think of using this for is handling deploying a bunch of servers where server 1 has to be fully provisioned before 2 and on 2 at least one service has to be up before 3 can do its thing. It''s something that''s still a hassle to do well within Puppet.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
You might be interested in Puppi, which is a Puppet module and a bash
command that i''ve written exactly for this reason.
Code: https://github.com/example42/puppi
More info: http://www.example42.com (now terribly slow) or
http://puppetlabs.com/blog/deploying-applications-and-bringing-puppet-information-to-the-cli-with-puppi/
It mixes the possibility of defining inside puppet manifests what you
need to make a deploy with a simple command that is actually used to
launch the deploy (by hand, via cron, via mcollective or triggered by
whatever tool).
The deploy procedure (commands to execute) can be totally customized,
but there are some ready examples to deploy from a Nexus repository,
or deploy directly wars, tarballs, zip archives and so on.
In few words, in order to be able to issue a command like:
puppi deploy supersite
you write Puppet code like this:
puppi::project::war { "supersite":
source => "http://repo.example42.com/deploy/prod/
supersite.war",
deploy_root => "/store/tomcat/myapp/webapps",
report_email => "sysadmins@example42.com",
}
but you can have more complex arguments like:
puppi::project::maven { "supersite":
source => "http://nexus.example42.com/nexus/content/
repositories/releases/it/example42/supersite/",
deploy_root => "/usr/local/tomcat/supersite/webapps",
config_suffix => "cfg",
config_root => "/srv/htdocs/supersite",
document_suffix => "css",
document_root => "/srv/htdocs/supersite",
firewall_src_ip => $site ? {
dr => "192.168.101.1/30",
main => "192.168.1.1/30",
},
backup_retention => "3",
init_script => "tomcat",
report_email => "sysadmins@example42.com",
enable => "true",
}
And, if you need it, there''s the mcollective agent and relevant mc-
puppi command.
Hope it might help,
al
On Sep 13, 9:53 pm, Ashley Penney <apen...@gmail.com>
wrote:> I know this has come up on the list numerous times before but I
> thought it would be a good time to see if the state of the art has
> advanced for this kind of thing. I wanted to know how people are
> handling higher level deployment of applications - things that have to
> be done repeatedly but not all the time. An example of this is
> checking an application out of svn, building it, creating a package
> and then moving it off to a repo. Or even just building/installing
> locally for developers.
>
> It never seems to fit well into Puppet for me and I end up with crazy
> complicated manifests to deal with this kind of thing. I recently
> moved these jobs into Rundeck (www.rundeck.org) which works pretty
> well but doesn''t really leverage any of the stuff I have within
> Foreman/Puppet. I''ve seen suggestions to use mcollective but this
> doesn''t easily integrate our existing scripts (written in many
> languages) or processes and would require me to force a lot of
> developers to work differently. I could just have classes that
> trigger scripts only when some condition is met (like /.buildapp
> files) or something along those lines but nothing seems elegant.
>
> What I''m trying to find out is what other people did to handle
this?
> I want something I can build up over time and slowly migrate legacy
> apps and processes into without having to do a massive up front
> development. It should also be relatively simple and not require me
> to code anything as anyone on the list who knows me can tell you that
> I am absolutely awful at coding.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
This looks fascinating and I''m absolutely going to do some
experimentation
with it this week as a way to do some of the awkward deploys that exist. I
love the idea. As a recent 2.7 upgrader I look forward to seeing the faces
version you talk about. I guess today I''ll finally get mcollective
rolled
out in advance of testing with Puppi. Thanks!
(As for the rest of this thread Volcane convinced me that I was being stupid
and my approach to the problem was wrong and to put the build logic in
Jenkins and keep the deploy logic to package{}.
On Mon, Sep 19, 2011 at 5:01 PM, Alessandro Franceschi <al@lab42.it>
wrote:
> You might be interested in Puppi, which is a Puppet module and a bash
> command that i''ve written exactly for this reason.
> Code: https://github.com/example42/puppi
> More info: http://www.example42.com (now terribly slow) or
>
>
http://puppetlabs.com/blog/deploying-applications-and-bringing-puppet-information-to-the-cli-with-puppi/
> It mixes the possibility of defining inside puppet manifests what you
> need to make a deploy with a simple command that is actually used to
> launch the deploy (by hand, via cron, via mcollective or triggered by
> whatever tool).
> The deploy procedure (commands to execute) can be totally customized,
> but there are some ready examples to deploy from a Nexus repository,
> or deploy directly wars, tarballs, zip archives and so on.
>
> In few words, in order to be able to issue a command like:
> puppi deploy supersite
>
> you write Puppet code like this:
> puppi::project::war { "supersite":
> source => "http://repo.example42.com/deploy/prod/
> supersite.war",
> deploy_root => "/store/tomcat/myapp/webapps",
> report_email => "sysadmins@example42.com",
> }
>
> but you can have more complex arguments like:
> puppi::project::maven { "supersite":
> source => "http://nexus.example42.com/nexus/content/
> repositories/releases/it/example42/supersite/",
> deploy_root => "/usr/local/tomcat/supersite/webapps",
> config_suffix => "cfg",
> config_root => "/srv/htdocs/supersite",
> document_suffix => "css",
> document_root => "/srv/htdocs/supersite",
> firewall_src_ip => $site ? {
> dr => "192.168.101.1/30",
> main => "192.168.1.1/30",
> },
> backup_retention => "3",
> init_script => "tomcat",
> report_email => "sysadmins@example42.com",
> enable => "true",
> }
>
> And, if you need it, there''s the mcollective agent and relevant
mc-
> puppi command.
> Hope it might help,
> al
>
> On Sep 13, 9:53 pm, Ashley Penney <apen...@gmail.com> wrote:
> > I know this has come up on the list numerous times before but I
> > thought it would be a good time to see if the state of the art has
> > advanced for this kind of thing. I wanted to know how people are
> > handling higher level deployment of applications - things that have to
> > be done repeatedly but not all the time. An example of this is
> > checking an application out of svn, building it, creating a package
> > and then moving it off to a repo. Or even just building/installing
> > locally for developers.
> >
> > It never seems to fit well into Puppet for me and I end up with crazy
> > complicated manifests to deal with this kind of thing. I recently
> > moved these jobs into Rundeck (www.rundeck.org) which works pretty
> > well but doesn''t really leverage any of the stuff I have
within
> > Foreman/Puppet. I''ve seen suggestions to use mcollective but
this
> > doesn''t easily integrate our existing scripts (written in
many
> > languages) or processes and would require me to force a lot of
> > developers to work differently. I could just have classes that
> > trigger scripts only when some condition is met (like /.buildapp
> > files) or something along those lines but nothing seems elegant.
> >
> > What I''m trying to find out is what other people did to
handle this?
> > I want something I can build up over time and slowly migrate legacy
> > apps and processes into without having to do a massive up front
> > development. It should also be relatively simple and not require me
> > to code anything as anyone on the list who knows me can tell you that
> > I am absolutely awful at coding.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.