Puppet users - Nov 2012 - How to do release managment integration with puppet?

I am at the beginning of merging the traditional OS/Middleware update 
process with application development release process.  The goal is to be 
able to test a complete versioned OS/Middleware/Application stack as it 
moves through dev/test/qa/prod environments.  The key here is that entire 
stack is labeled under a single version. In other terms, the node and 
everything on it is aligned under a single version. Right now the 
OS/Middleware is managed by puppet.  The application release process is 
not.  I''m hoping some folks would like to comment on the following 
questions I am pondering.

   - How to best version puppet classes and control the classification of 
   nodes based on the version of the node?
   - Are there ENC''s that integrate with SVN/GIT that make this easy?
   - Should application deployment be brought under the control of puppet?
   - If so how do you easily do this without burdening applications 
   developers with having to learn something about or have access to puppet?
   - Is there a puppet API that an existing application release processes 
   can leverage to trigger OS/Middleware updates when an application version 
   is updated?

Looking forward to your comments.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/vAGrAj_5qpUJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 30, 2012, at 10:37 AM, Schofield wrote:
> I am at the beginning of merging the traditional OS/Middleware update
process with application development release process.  The goal is to be able to
test a complete versioned OS/Middleware/Application stack as it moves through
dev/test/qa/prod environments.  The key here is that entire stack is labeled
under a single version. In other terms, the node and everything on it is aligned
under a single version. Right now the OS/Middleware is managed by puppet.  The
application release process is not.  I''m hoping some folks would like
to comment on the following questions I am pondering.
> 	• How to best version puppet classes and control the classification of
nodes based on the version of the node?
> 	• Are there ENC''s that integrate with SVN/GIT that make this
easy?
> 	• Should application deployment be brought under the control of puppet?
> 	• If so how do you easily do this without burdening applications
developers with having to learn something about or have access to puppet?
> 	• Is there a puppet API that an existing application release processes can
leverage to trigger OS/Middleware updates when an application version is
updated?
> Looking forward to your comments.
----
I think the general consensus is that puppet is less than optimal for most
software deployments and there are better tools for the job (for example, we use
capistrano here).

Puppet is more of a configuration management tool and I suspect that even if you
could shoehorn your software deployment needs into something that puppet could
manage, it would probably be slow and confusing.

As for ENC''s and ''developer'' access, I use foreman
(http://theforeman.org) to provide ''other'' user access and I
can contain their rights/access within foreman and yet still, allow them to say
install NGINX (with all of it''s pre-requisite packages) from a web
based GUI.

Again, seriously look at capistrano for handling software deployments and
I''m sure that there''s others that can do the job too.

Craig

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 30 November 2012 23:02, Craig White <craig.white@ttiltd.com>
wrote:
>
> On Nov 30, 2012, at 10:37 AM, Schofield wrote:
>
>> I am at the beginning of merging the traditional OS/Middleware update
process with application development release process.  The goal is to be able to
test a complete versioned OS/Middleware/Application stack as it moves through
dev/test/qa/prod environments.  The key here is that entire stack is labeled
under a single version. In other terms, the node and everything on it is aligned
under a single version. Right now the OS/Middleware is managed by puppet.  The
application release process is not.  I''m hoping some folks would like
to comment on the following questions I am pondering.
>>       • How to best version puppet classes and control the
classification of nodes based on the version of the node?
>>       • Are there ENC''s that integrate with SVN/GIT that make
this easy?
>>       • Should application deployment be brought under the control of
puppet?
>>       • If so how do you easily do this without burdening applications
developers with having to learn something about or have access to puppet?
>>       • Is there a puppet API that an existing application release
processes can leverage to trigger OS/Middleware updates when an application
version is updated?
>> Looking forward to your comments.
> ----
> I think the general consensus is that puppet is less than optimal for most
software deployments and there are better tools for the job (for example, we use
capistrano here).
>
I''m not sure ''general consensus'' is really true, I
think it depends,
in particular on size of stack. Capistrano is incredibly easy to get
started with but has negatives too (like being push based, often
installing/compiling dependencies per machine, not being able to
specify broader version dependencies like on the database server)
> Puppet is more of a configuration management tool and I suspect that even
if you could shoehorn your software deployment needs into something that puppet
could manage, it would probably be slow and confusing.
>
Here''s a question. Would you define a puppet module for Graphite?
Graphite turns out to be a Django base web application. Would you use
Capistrano to deploy Graphite? If your own applications were Django
based would you use Cap? Not quite as simple I feel.

Replying the the original points separately too.

G
> As for ENC''s and ''developer'' access, I use
foreman (http://theforeman.org) to provide ''other'' user access
and I can contain their rights/access within foreman and yet still, allow them
to say install NGINX (with all of it''s pre-requisite packages) from a
web based GUI.
>
> Again, seriously look at capistrano for handling software deployments and
I''m sure that there''s others that can do the job too.
>
> Craig
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>


--
Gareth Rushgrove
@garethr

devopsweekly.com
morethanseven.net
garethrushgrove.com

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 30 November 2012 17:37, Schofield <dbschofield@gmail.com>
wrote:
> I am at the beginning of merging the traditional OS/Middleware update
> process with application development release process.  The goal is to be
> able to test a complete versioned OS/Middleware/Application stack as it
> moves through dev/test/qa/prod environments.  The key here is that entire
> stack is labeled under a single version. In other terms, the node and
> everything on it is aligned under a single version. Right now the
> OS/Middleware is managed by puppet.  The application release process is
not.
> I''m hoping some folks would like to comment on the following
questions I am
> pondering.
>
> How to best version puppet classes and control the classification of nodes
> based on the version of the node?
Although I''ve not done it myself I''ve seen people use Puppet
environments for versioning. You can also use environment variables on
the nodes to help with classification.
> Are there ENC''s that integrate with SVN/GIT that make this easy?
> Should application deployment be brought under the control of puppet?
> If so how do you easily do this without burdening applications developers
> with having to learn something about or have access to puppet?
The vcsrepo module (https://forge.puppetlabs.com/puppetlabs/vcsrepo)
supports defining git/bzr/svn checkouts, but another approach would be
to package your applications up as system packages (debs for
debian/ubuntu, rpms for redhat/centos). This has a number of nice
advantages, including making managing the application version just a
matter of defining a package resource in puppet.
> Is there a puppet API that an existing application release processes can
> leverage to trigger OS/Middleware updates when an application version is
> updated?
So, assuming you do package your applications up as system packages as
described then triggering updates to supporting components could be
done in the package dependencies, or it could be done with Puppet
requires, assuming the dependencies are on the same node. Cross node
dependencies are harder however. Some ideas for that thorny problem:

https://github.com/jamtur01/puppet-noah/
http://www.slideshare.net/lusis/beyond-the-node-arkestration-with-noah

G
>
> Looking forward to your comments.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/vAGrAj_5qpUJ.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.


--
Gareth Rushgrove
@garethr

devopsweekly.com
morethanseven.net
garethrushgrove.com

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Right now we''re deploying via cap and something custom and part of the 
deployment is a config yaml file which puppet parses via facter in order to 
determine how to do the OS/middleware config.  It works fairly well, but it 
feels kludgy to do it this way.

I''ve been wanting to cook something up like what it sounds like you
want
for a while, since I hate push-based models, and I hate using ssh as a 
transport.  This means there is almost nothing that makes me happy around 
right now.

The basic idea is to have a web application that handles deployment, 
maintains a version to revision database(one of our old systems uses SVN 
and does this via tags, which gets, well, bad after a few years), plus 
metadata surrounding the release, like configuration information. 
 Deployment is done by issuing an mcollective command to do a 
pull/clone/checkout/update/whatever from a VCS onto the target server. 
 Config information is transferred into puppet via the master over REST 
from the deployment server using the ruby DSL, with everything stored in 
one big hash.  You would require that module and then use those variables 
in your own modules, with a sanity check to verify the code actually 
updated.


On Friday, November 30, 2012 12:37:11 PM UTC-5, Schofield
wrote:
>
> I am at the beginning of merging the traditional OS/Middleware update 
> process with application development release process.  The goal is to be 
> able to test a complete versioned OS/Middleware/Application stack as it 
> moves through dev/test/qa/prod environments.  The key here is that entire 
> stack is labeled under a single version. In other terms, the node and 
> everything on it is aligned under a single version. Right now the 
> OS/Middleware is managed by puppet.  The application release process is 
> not.  I''m hoping some folks would like to comment on the following
> questions I am pondering.
>
>    - How to best version puppet classes and control the classification of 
>    nodes based on the version of the node?
>    - Are there ENC''s that integrate with SVN/GIT that make this
easy?
>    - Should application deployment be brought under the control of puppet?
>    - If so how do you easily do this without burdening applications 
>    developers with having to learn something about or have access to
puppet?
>    - Is there a puppet API that an existing application release processes 
>    can leverage to trigger OS/Middleware updates when an application
version
>    is updated?
>
> Looking forward to your comments.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/481nwg4Grc0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I run a small startup offering a SAAS application (cloud based, no local 
servers at all).  We have always tried to "do things right" so that we
can
scale - and my view is (happy to be challenged) that our application stack 
should absolutely be managed just as any other part of the configuration.  
We package our software components using "system packages" (deb files
on
ubuntu in our case).  There was a time when we created the debs manually, 
but our software build process does them automatically now using 
git-buildpackage.  the Continuous Integration server gets the latest 
changes from source control, tests them and creates the debs.  deb packages 
handle versions, so if we want we can specify that one part of the app 
depends on a particular version of another part.  As far as possible though 
we try to release the application stack as a whole so we avoid untested 
combinations.  (this is a key recommendation of "the" Continuous
Delivery
book by the thoughtworks guys).

The version numbers of the packages in the different environments is 
managed by puppet, using hiera.  The contents of the config files are also 
managed by puppet (again using hiera).  As far as I can tell this is the 
perfect scenario - I''m really interested to know what problem Noah is 
referring to/ trying to solve (we haven''t found it yet).  If we need
more
capacity, we build another prod app server or db server etc.  The app 
layers support dynamic load balancing - e.g. app servers automatically 
register themselves with the web server pool when they start-up.  The 
web-servers automatically register with the load balancer etc.  

Being open with you, we could probably manage without all of this (for now) 
- we haven''t achieved the scale to need most of it yet.  But having it
like
this means we spend much much less time worrying about environments and 
software releases and more time getting the software right for the users.  
I''d love to hear what problems we''re heading for that this
setup can''t cope
with, so we can change course if necessary.

Regards, James.

On Saturday, December 1, 2012 10:46:06 PM UTC, Nick Cammorato
wrote:
>
> Right now we''re deploying via cap and something custom and part of
the
> deployment is a config yaml file which puppet parses via facter in order to
> determine how to do the OS/middleware config.  It works fairly well, but it
> feels kludgy to do it this way.
>
> I''ve been wanting to cook something up like what it sounds like
you want
> for a while, since I hate push-based models, and I hate using ssh as a 
> transport.  This means there is almost nothing that makes me happy around 
> right now.
>
> The basic idea is to have a web application that handles deployment, 
> maintains a version to revision database(one of our old systems uses SVN 
> and does this via tags, which gets, well, bad after a few years), plus 
> metadata surrounding the release, like configuration information. 
>  Deployment is done by issuing an mcollective command to do a 
> pull/clone/checkout/update/whatever from a VCS onto the target server. 
>  Config information is transferred into puppet via the master over REST 
> from the deployment server using the ruby DSL, with everything stored in 
> one big hash.  You would require that module and then use those variables 
> in your own modules, with a sanity check to verify the code actually 
> updated.
>
>
> On Friday, November 30, 2012 12:37:11 PM UTC-5, Schofield wrote:
>>
>> I am at the beginning of merging the traditional OS/Middleware update 
>> process with application development release process.  The goal is to
be
>> able to test a complete versioned OS/Middleware/Application stack as it
>> moves through dev/test/qa/prod environments.  The key here is that
entire
>> stack is labeled under a single version. In other terms, the node and 
>> everything on it is aligned under a single version. Right now the 
>> OS/Middleware is managed by puppet.  The application release process is
>> not.  I''m hoping some folks would like to comment on the
following
>> questions I am pondering.
>>
>>    - How to best version puppet classes and control the classification 
>>    of nodes based on the version of the node?
>>    - Are there ENC''s that integrate with SVN/GIT that make
this easy?
>>    - Should application deployment be brought under the control of 
>>    puppet?
>>    - If so how do you easily do this without burdening applications 
>>    developers with having to learn something about or have access to
puppet?
>>    - Is there a puppet API that an existing application release 
>>    processes can leverage to trigger OS/Middleware updates when an
application
>>    version is updated?
>>
>> Looking forward to your comments.
>>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/szJxmTB0iyIJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 12/02/2012 02:51 PM, j4m3s wrote:
> I run a small startup offering a SAAS application (cloud based, no local
> servers at all).  We have always tried to "do things right" so
that we
> can scale - and my view is (happy to be challenged) that our application
> stack should absolutely be managed just as any other part of the
> configuration.  We package our software components using "system
> packages" (deb files on ubuntu in our case).  There was a time when we
> created the debs manually, but our software build process does them
> automatically now using git-buildpackage.  the Continuous Integration
> server gets the latest changes from source control, tests them and
> creates the debs.  deb packages handle versions, so if we want we can
> specify that one part of the app depends on a particular version of
> another part.  As far as possible though we try to release the
> application stack as a whole so we avoid untested combinations.  (this
> is a key recommendation of "the" Continuous Delivery book by the
> thoughtworks guys).
>
> The version numbers of the packages in the different environments is
> managed by puppet, using hiera.  The contents of the config files are
> also managed by puppet (again using hiera).  As far as I can tell this
> is the perfect scenario - I''m really interested to know what
problem
> Noah is referring to/ trying to solve (we haven''t found it yet). 
If we
> need more capacity, we build another prod app server or db server etc.
> The app layers support dynamic load balancing - e.g. app servers
> automatically register themselves with the web server pool when they
> start-up.  The web-servers automatically register with the load balancer
> etc.
>
> Being open with you, we could probably manage without all of this (for
> now) - we haven''t achieved the scale to need most of it yet.  But
having
> it like this means we spend much much less time worrying about
> environments and software releases and more time getting the software
> right for the users.  I''d love to hear what problems
we''re heading for
> that this setup can''t cope with, so we can change course if
necessary.
Excellent post James!

That''s the way to do it.



-- 
Jakov Sosic
www.srce.unizg.hr

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks for the great response James.  I think you have done exactly what I 
am looking to do.  I want to ask more about controlling the versions of the 
application packages in hiera.  I have multiple different dev groups who 
each work on different release schedules.  I don''t want to have 
operations/puppet folks involved with every release as application versions 
move through dev/test/prod.  So how would one go about letting dev teams 
control which versions of applications are mapped to which environments?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/nlqpx5hjPX4J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

This is a great thread - it''s really nice to hear others are pondering
similar challenges ;)

Count me in as "Puppet does this well, I''m unclear what the issues
are"

I''m also in the process of pondering this very process. 
Here''s what I''ve
come up with (which seems very similar to James''):

We are a Redhat/Centos shop, and our release artifacts are packaged and
delivered as RPMs. The RPMs are built as part of our Continous Integration
activities, and deployed as versioned artifacts to a Nexus repository.
 Puppet is then used to retrieve the versioned artifacts to create a Yum
repository.   Controlling which version of which artifact is accomplished
by pointing the puppet agent nodes at the Yum repo that contains the
version of the artifacts that are desired to be installed in a particular
environment.  We also release as a "platform" BTW.

I''d also add that using standard packaging (RPMs) forces us to grapple
with
dependencies and coupling (in a good way) between components, and allows a
very clean use of Puppet (via the Yum Provider).

My .02 ;)


On Mon, Dec 3, 2012 at 5:57 AM, Schofield <dbschofield@gmail.com> wrote:
> Thanks for the great response James.  I think you have done exactly what I
> am looking to do.  I want to ask more about controlling the versions of the
> application packages in hiera.  I have multiple different dev groups who
> each work on different release schedules.  I don''t want to have
> operations/puppet folks involved with every release as application versions
> move through dev/test/prod.  So how would one go about letting dev teams
> control which versions of applications are mapped to which environments?
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/nlqpx5hjPX4J.
>
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>


-- 
GVoice: 707.410.0371
LinkedIn: http://www.linkedin.com/in/brendanobra

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

What I am understanding is that people are having success 
building/testing/deploying the OS/Middleware/Application stack by wrapping 
applications in a puppet resource.  Thus bringing all configuration 
management under puppet.  This is good as it is the direction I wanted to 
go.  I would like to shift the direction of the discussion a little bit to 
address how the application versions are updated in puppet.  James 
mentioned updating hiera and Brendan mentioned pointing to a different yum 
repo.  I am sure there are multiple other ways to solve this same problem 
as well.  But what these solutions have in common is that an individual at 
some point in time has to make some change to the puppet system.  Is there 
any tooling or API that can be leveraged to allow these changes to be made 
and control who makes them?   This would be for integrating the update to 
puppet in to our enterprise release management processes.  

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/EsJpWatvCU0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

" But what these solutions have in common is that an individual at some
point in time has to make some change to the puppet system."

Please elaborate - Do you mean that someone has to "tell" Puppet about
what
versions to use?  My ultimate solution will be emitting a manifest containg
the names and versions that go into a "release" as part of our
CI/release
activities that is used to "tell" Puppet what a "platform
release"
consisiste


On Mon, Dec 3, 2012 at 8:25 AM, Schofield <dbschofield@gmail.com> wrote:
> What I am understanding is that people are having success
> building/testing/deploying the OS/Middleware/Application stack by wrapping
> applications in a puppet resource.  Thus bringing all configuration
> management under puppet.  This is good as it is the direction I wanted to
> go.  I would like to shift the direction of the discussion a little bit to
> address how the application versions are updated in puppet.  James
> mentioned updating hiera and Brendan mentioned pointing to a different yum
> repo.  I am sure there are multiple other ways to solve this same problem
> as well.  But what these solutions have in common is that an individual at
> some point in time has to make some change to the puppet system.  Is there
> any tooling or API that can be leveraged to allow these changes to be made
> and control who makes them?   This would be for integrating the update to
> puppet in to our enterprise release management processes.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/EsJpWatvCU0J.
>
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>


-- 
GVoice: 707.410.0371
LinkedIn: http://www.linkedin.com/in/brendanobra

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Monday, December 3, 2012 11:34:53 AM UTC-5, Brendan O''Bra
wrote:
>
> " But what these solutions have in common is that an individual at
some
> point in time has to make some change to the puppet system."
>
> Please elaborate - 
>
Do you mean that someone has to "tell" Puppet about what versions to
use? 
>  My ultimate solution will be emitting a manifest containg the names and 
> versions that go into a "release" as part of our CI/release
activities that
> is used to "tell" Puppet what a "platform release"
consisiste
>
Exactly.  In your case that ''someone'' is the person doing the
CI/release
activities and the update to puppet is done by generating the manifest as 
part of the workflow.   

Evaluating this problem for the first time, there seems to be lots of 
potential solutions for generating the manifest with updated application 
versions or new applications altogether. Best practices for process release 
integration may not be fully established yet and everyone is doing their 
own thing based on what works for them.  




-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/0aIusLOydfgJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Though we have lots of automated testing we still rely on the human touch for
checking UI look and feel, and doing exploratory testing (in the pre production
environment).   Once it has passed this step the human can approve it for
production deployment by updating the version number in hiera. I guess we could
have a button on a web screen or something to make it easier to promote it, but
editing hiera directly is easy enough.  If you have a central change management
system, it could update hiera - but you''ll still want it approved for
promotion by a human won''t you, to take into account live service
stability/ load, etc?

James

----- Reply message -----
From: "Schofield" <dbschofield@gmail.com>
To: <puppet-users@googlegroups.com>
Subject: [Puppet Users] Re: How to do release managment integration with puppet?
Date: Mon, Dec 3, 2012 4:25 pm
What I am understanding is that people are having success
building/testing/deploying the OS/Middleware/Application stack by wrapping
applications in a puppet resource.  Thus bringing all configuration management
under puppet.  This is good as it is the direction I wanted to go.  I would like
to shift the direction of the discussion a little bit to address how the
application versions are updated in puppet.  James mentioned updating hiera and
Brendan mentioned pointing to a different yum repo.  I am sure there are
multiple other ways to solve this same problem as well.  But what these
solutions have in common is that an individual at some point in time has to make
some change to the puppet system.  Is there any tooling or API that can be
leveraged to allow these changes to be made and control who makes them?   This
would be for integrating the update to puppet in to our enterprise release
management processes.




-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/EsJpWatvCU0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.

For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 12/3/2012 5:57 AM, Schofield wrote:
> Thanks for the great response James.  I think you have done exactly what
> I am looking to do.  I want to ask more about controlling the versions
> of the application packages in hiera.  I have multiple different dev
> groups who each work on different release schedules.  I don''t want
to
> have operations/puppet folks involved with every release as application
> versions move through dev/test/prod.  So how would one go about letting
> dev teams control which versions of applications are mapped to which
> environments?
	We have a set of machines that we spin up per event. Use a custom AMI 
and let Puppet add the final 1-2% of config to catch anything that''s 
changed recently. Puppet runs only once and the machines spin down after 
4-10 hours. The custom app code doesn''t change often which seemed like
a
decent candidate for letting the Dev team push versions to Puppet. 
Here''s how it works

	Hiera can''t search two different dir trees for the same backend so I 
set up a second backend, json. Mostly do this so I have final say via 
yaml on versions/config. Probably overkill.

hiera.yaml
---
:hierarchy:
     - %{clientcert}
     - %{environment}/%{role}/%{calling_module}
     - %{environment}/%{role}
     - %{calling_module}
     - %{role}
     - %{environment}
     - common

:backends:
     - yaml
     - json

:yaml:
     :datadir:
''/puppet/etc/environments/%{environment}/hieradata''

:json:
     :datadir: ''/puppet/data/%{environment}/hieradata''

/puppet/data/production/hieradata/mycode.json
{"mycode_ensure":"latest","mycode_version":"somebranch-12-11-27"}

When Dev does a release they update a set of core machines via 
Capistrano and then push the new version into Puppet do the ephemeral 
machines will check out the latest version when they spin up.

	I''ve been meaning to extend the idea as well as move to more of a pull
method for software release, but it hasn''t been pressing.

Ramin

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Monday, December 3, 2012 2:32:15 PM UTC-5, j4m3s
wrote:
>
> Though we have lots of automated testing we still rely on the human touch 
> for checking UI look and feel, and doing exploratory testing (in the pre 
> production environment).   Once it has passed this step the human can 
> approve it for production deployment by updating the version number in 
> hiera. I guess we could have a button on a web screen or something to make 
> it easier to promote it, but editing hiera directly is easy enough.  If you
> have a central change management system, it could update hiera - but
you''ll
> still want it approved for promotion by a human won''t you, to take
into
> account live service stability/ load, etc?
>
Do you use hiera to define not only the versions but also which 
applications should be on which environments?  How do you introduced 
application version 1.0 to dev for the first time?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/5n3FDDQjhlYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Sorry, I have only really given you part of the story haven''t I.   We
have a puppet class hierarchy that we use to specify the various components that
need to go on a particular type of host - e.g app server or web server or db
server. A dev machine includes several roles (machines are identified by a -dev
suffix on the hostname but maybe we could do it based on the environment
variable?).

We use the puppet environment variable to say which environment the machine is
in.   That in turn decides the config variables and package versions.

Dev machines do require additional manual intervention.  The deb packages get
installed as normal (version "latest" or whatever) but the app
component deb installers install themselves plus a symlink pointing to the deb
install location ( e.g. /usr/lib/myapp/current -> /usr/lib/myapp/deb). 
Because the deb installers won''t overwrite files that have been
modified externally, the developers can point the symlink
(/usr/lib/myapp/cuurent) for the relevant component to point to their local
checkout/ compile directory and they can be working on the code while the rest
of their installation keeps up to speed with other package versions and configs.
They need to copy the puppet-created config file for the particular app  into
their working directory and symlink from its normal /etc/myapp location) but
then they can modify it etc as part of the dev process.

I know I''m making a mess of explaining this, sorry - feel free to point
out bits that either aren''t clear or where we could improve our
process!


----- Reply message -----
From: "Schofield" <dbschofield@gmail.com>
To: <puppet-users@googlegroups.com>
Subject: [Puppet Users] Re: How to do release managment integration with puppet?
Date: Mon, Dec 3, 2012 8:18 pm
On Monday, December 3, 2012 2:32:15 PM UTC-5, j4m3s wrote:Though we have lots of
automated testing we still rely on the human touch for checking UI look and
feel, and doing exploratory testing (in the pre production environment).   Once
it has passed this step the human can approve it for production deployment by
updating the version number in hiera. I guess we could have a button on a web
screen or something to make it easier to promote it, but editing hiera directly
is easy enough.  If you have a central change management system, it could update
hiera - but you''ll still want it approved for promotion by a human
won''t you, to take into account live service stability/ load, etc?

Do you use hiera to define not only the versions but also which applications
should be on which environments?  How do you introduced application version 1.0
to dev for the first time?




-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/5n3FDDQjhlYJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.

For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.