Puppet users - Oct 2011 - Better solution for puppet SVN releases

Hi,

we are currently managing our puppet modules with one SVN workspace for each 
admin.
The post commit hook script updates /etc/puppet/ directory and triggers the 
puppet kick of the correct server.

The problem we have with this solution is that sometimes there are many 
small checkins to one change, because the admin forgot to change small 
details in the config file, e.g. forgot to change the access logfile name of 
the vHost, forgot a redirect, misspelling in the comments etc.

What we end up with are many micro checkins, which can be used to tell every 
small mistake the admin has done.

What we want is a solution which lets the admin test his changes on one 
server without checking these changes into the "main" SVN repository.
So that the SVN repository only contains the final releases of the changes.


I have to say that we also manage the dev and QA servers with this 
puppetmaster. Would dividing of these different stages into puppet 
environments help us?
What I really want to know is, do you understand my problem and if you had 
the same problem, how did you solve it? SVN branches? Multiple 
puppetmasters, one to test with a dedicated test SVN repository?

Thanks!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/FLNYwiOI1fwJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I had a similar problem, i use Git, and i though i could use branches and
Git tricks, but it does not mixes well with Puppet environments. What i do
below permits to use many puppet code branches, while having my puppetmaster
serve all different branches at the same time using environments

In my case, my puppetmaster does not checkout/pull from a code repo, but
could, it does not matter. So in my case, i do a rsync --delete  of my
current workingdir from my workstation to the puppetmaster
/etc/puppet/environments/{manifests,modules}. My puppetmaster has
environments configured, so it can have test environments for the work in
progress, and the production environment. The environments permits to have
many copies of my puppet manifests and modules for the staging or testing
puppet clients, while still being able to serve a Production environment to
the prod puppet clients

On my workstation, i commit only when i am sure and everything is tested,
and this completely independant from what is on the puppetmaster. As an
improvement and for better traceability, i might setup my Production puppet
environment as a code repo checkout of my master branch or some tag. I would
not do it for the test environments, because like you, i do not want to have
to commit many small commits to be able to checkout/pull on the puppetmaster

My puppet environments are rsynced like this. I have a script (which i will
probably put on github once i have time, along with all my recipes) which
creates a environments/directory and rsyncs directly into it, and change the
/etc/puppet.conf to set the environment (from an puppet.conf.erb)

    # ls -ld /etc/puppet/environments/*
    lrwxrwxrwx. 1 root puppet   11 Jul 25
14:47*/etc/puppet/environments/production -> /etc/puppet
*
    drwxr-xr-x. 4 root root   4096 Aug  7 07:45
/etc/puppet/environments/alex
    drwxr-xr-x. 4 root root   4096 Sep 26 11:58
/etc/puppet/environments/alextest
    drwxr-xr-x. 4 root root   4096 Aug 19 07:51
/etc/puppet/environments/staging

    # find -type d /etc/puppet/
    /etc/puppet/
    /etc/puppet/modules
    /etc/puppet/modules/java
    /etc/puppet/modules/java/manifests
    /etc/puppet/modules/bluepill
    /etc/puppet/modules/bluepill/files
    /etc/puppet/modules/bluepill/manifests
    /etc/puppet/modules/mongodb
    (...)
    /etc/puppet/manifests
    /etc/puppet/manifests/nodes
    /etc/puppet/manifests/classes
    (...)
    /etc/puppet/environments
    /etc/puppet/environments/staging
    /etc/puppet/environments/staging/modules
    /etc/puppet/environments/staging/modules/java
    /etc/puppet/environments/staging/modules/java/manifests
    /etc/puppet/environments/staging/modules/bluepill
    /etc/puppet/environments/staging/modules/bluepill/files
    /etc/puppet/environments/staging/modules/bluepill/manifests
    /etc/puppet/environments/staging/modules/mongodb
    (...)
    /etc/puppet/environments/staging/manifests
    /etc/puppet/environments/staging/manifests/nodes
    /etc/puppet/environments/staging/manifests/classes
    (...)
    /etc/puppet/environments/alextest
    /etc/puppet/environments/alextest/modules
    /etc/puppet/environments/alextest/modules/java
    /etc/puppet/environments/alextest/modules/java/manifests
    /etc/puppet/environments/alextest/modules/bluepill
    /etc/puppet/environments/alextest/modules/bluepill/files
    /etc/puppet/environments/alextest/modules/bluepill/manifests
    (...)
    /etc/puppet/environments/alextest/manifests
    /etc/puppet/environments/alextest/manifests/nodes
    /etc/puppet/environments/alextest/manifests/classes
    (...)
    /etc/puppet/environments/alex
    /etc/puppet/environments/alex/modules
    /etc/puppet/environments/alex/modules/mongodb
    /etc/puppet/environments/alex/modules/mongodb/manifests
    (...)



2011/10/18 Andreas Paul <a.paul@enbw.com>
> Hi,
>
> we are currently managing our puppet modules with one SVN workspace for
> each admin.
> The post commit hook script updates /etc/puppet/ directory and triggers the
> puppet kick of the correct server.
>
> The problem we have with this solution is that sometimes there are many
> small checkins to one change, because the admin forgot to change small
> details in the config file, e.g. forgot to change the access logfile name
of
> the vHost, forgot a redirect, misspelling in the comments etc.
>
> What we end up with are many micro checkins, which can be used to tell
> every small mistake the admin has done.
>
> What we want is a solution which lets the admin test his changes on one
> server without checking these changes into the "main" SVN
repository.
> So that the SVN repository only contains the final releases of the changes.
>
>
> I have to say that we also manage the dev and QA servers with this
> puppetmaster. Would dividing of these different stages into puppet
> environments help us?
> What I really want to know is, do you understand my problem and if you had
> the same problem, how did you solve it? SVN branches? Multiple
> puppetmasters, one to test with a dedicated test SVN repository?
>
> Thanks!
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/FLNYwiOI1fwJ.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

We use Git here and we tag branches to certain environments. We have a
testing branch which everyone commits in to and what most of our
testing machines are using. We then have other environments like
Staging and Production that have specific tag checked out that (will
eventually) align with our Developers'' Continuous Integration and
Agile release cycle. Our testing environment has lots of micro check-
ins, commits for every little syntax error I make. To test a specific
machine though, I often temporarily change the environment back to
Testing to make sure nothing is broken, then switch it back to it''s
real environment. This is because we don''t have a test machine for
every single system (staff workstations, etc).

The idea we''re aiming towards is that all Puppet changes up to the end
of an iteration will be included in the next release. A tag is "cut"
and checked out into the Staging environment. All those changes get
applied and our own Developer written code is tested alongside any
Puppet changes. If there are any problems we can update the tag, same
as if there was a bug in our code''s release candidate. Then when our
code goes to production, we check out the production Puppet
environment with the same tag (plus any updates).

On Oct 18, 9:52 am, Andreas Paul <a.p...@enbw.com>
wrote:
> Hi,
>
> we are currently managing our puppet modules with one SVN workspace for
each
> admin.
> The post commit hook script updates /etc/puppet/ directory and triggers the
> puppet kick of the correct server.
>
> The problem we have with this solution is that sometimes there are many
> small checkins to one change, because the admin forgot to change small
> details in the config file, e.g. forgot to change the access logfile name
of
> the vHost, forgot a redirect, misspelling in the comments etc.
>
> What we end up with are many micro checkins, which can be used to tell
every
> small mistake the admin has done.
>
> What we want is a solution which lets the admin test his changes on one
> server without checking these changes into the "main" SVN
repository.
> So that the SVN repository only contains the final releases of the changes.
>
> I have to say that we also manage the dev and QA servers with this
> puppetmaster. Would dividing of these different stages into puppet
> environments help us?
> What I really want to know is, do you understand my problem and if you had
> the same problem, how did you solve it? SVN branches? Multiple
> puppetmasters, one to test with a dedicated test SVN repository?
>
> Thanks!
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I use two separate puppetmasters for lab and production. (The lab side has more
of the "dratted commas" style of checkin.)

Testing changes to one particular type of server is where virtualization really
shines. You can spin up a scratch virtual machine with the same manifests as
your target host, snapshot the VM, and then apply your manifest changes. If the
changes break you can revert the VM and fix your manifests. Once things are to
your satisfaction you can commit to production.

For VMWare I like snapshotting at the grub screen on each lab host on first boot
(before the first puppet run), given that reverting to the last snapshot is
higher priority than powering on. That way I have a server where I can
conveniently start "clean" on every boot as long as I remember to
scrub certs from the puppetmaster.

On Tue, Oct 18, 2011 at 01:52:18AM -0700, Andreas Paul
wrote:
>    Hi,
> 
>    we are currently managing our puppet modules with one SVN workspace for
>    each admin.
>    The post commit hook script updates /etc/puppet/ directory and triggers
>    the puppet kick of the correct server.
> 
>    The problem we have with this solution is that sometimes there are many
>    small checkins to one change, because the admin forgot to change small
>    details in the config file, e.g. forgot to change the access logfile
name
>    of the vHost, forgot a redirect, misspelling in the comments etc.
> 
>    What we end up with are many micro checkins, which can be used to tell
>    every small mistake the admin has done.
> 
>    What we want is a solution which lets the admin test his changes on one
>    server without checking these changes into the "main" SVN
repository.
>    So that the SVN repository only contains the final releases of the
>    changes.
> 
>    I have to say that we also manage the dev and QA servers with this
>    puppetmaster. Would dividing of these different stages into puppet
>    environments help us?
>    What I really want to know is, do you understand my problem and if you
had
>    the same problem, how did you solve it? SVN branches? Multiple
>    puppetmasters, one to test with a dedicated test SVN repository?
> 
>    Thanks!
> 
>    --
>    You received this message because you are subscribed to the Google
Groups
>    "Puppet Users" group.
>    To view this discussion on the web visit
>    [1]https://groups.google.com/d/msg/puppet-users/-/FLNYwiOI1fwJ.
>    To post to this group, send email to puppet-users@googlegroups.com.
>    To unsubscribe from this group, send email to
>    puppet-users+unsubscribe@googlegroups.com.
>    For more options, visit this group at
>    http://groups.google.com/group/puppet-users?hl=en.
> 
> References
> 
>    Visible links
>    1. https://groups.google.com/d/msg/puppet-users/-/FLNYwiOI1fwJ
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, Oct 18, 2011 at 01:52:18AM -0700, Andreas Paul
wrote:
> we are currently managing our puppet modules with one SVN workspace for
each
> admin.
> The post commit hook script updates /etc/puppet/ directory and triggers the
> puppet kick of the correct server.
we use git branches. the master branch is _always_ production-ready.
for every issue, we:

* create a topic branch
* hack away
* push the topic branch to origin/$branch_name
* a post-push service hook creates, updates, or deletes
  /opt/puppet/$branch_name

On the puppetmaster, puppet.conf has:

   [main]
       manifestdir    = /opt/puppet/$environment
       manifest       = $manifestdir/site.pp
       modulepath     = $manifestdir/modules:/opt/puppet/3pp

We then test the topic branch via the puppet agent
by running ''puppet agent --environment $branch_name ...''
> The problem we have with this solution is that sometimes there are many 
> small checkins to one change, because the admin forgot to change small 
> details in the config file, e.g. forgot to change the access logfile name
of
> the vHost, forgot a redirect, misspelling in the comments etc.
> 
> What we end up with are many micro checkins, which can be used to tell
every
> small mistake the admin has done.
Our topic branches have lots of these so-called micro-commits,
but that''s ok. Once $branch_name is clean, we:

* squash the micro-commits

   git rebase -i <hash>

   # prepare a linear commit history
   git fetch origin
   git rebase origin/master
   git checkout master
   git merge $branch_name

   # delete local and remote branch
   git branch -d $branch_name
   git push origin :$branch_name

* ship

   git push origin

> What we want is a solution which lets the admin test his changes on one 
> server without checking these changes into the "main" SVN
repository.
> So that the SVN repository only contains the final releases of the changes.
The key is to use rebase to squash the micro-commits in the branch.
I''m not sure if svn has that capability yet, but a quick google
search showed at least one script that sort of simulated a rebase
with svn. Or perhaps you could try git-svn.
> I have to say that we also manage the dev and QA servers with this 
> puppetmaster. Would dividing of these different stages into puppet 
> environments help us?
We manage differences via parameterized modules.
For example:

class foo {
  require foo::params
  package {''fubar'': ensure => $foo::params::fubar_version}
}

class foo::params {
  $fubar_version = $fqdn ? {
    # exceptions
    something.qa.example.com  => ''2.1.0-1.f15'',

    # patterns
    /^.*\.prod\.example\.com$/ => ''1.2.3-4.el5'',
    /^.*\.qa\.example\.com$/   => ''1.2.4-1.el5'',
    /^.*\.dev\.example\.com$/  => latest,
    default => ''1.2.3-4.el5'',
  }
}

The above is not perfect, but it allows us to keep our
node manifests under version control.
> What I really want to know is, do you understand my problem and if you had 
> the same problem, how did you solve it? SVN branches? Multiple 
> puppetmasters, one to test with a dedicated test SVN repository?
We thought about multiple puppetmasters, but it''s more important
to us to maintain a clean version control history in one place.

hth,
-paul

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.