Puppet users - Aug 2012 - Version control design for puppet and data (hiera) repositories

I''m in the process of migrating data out of my standard Puppet
repositories
for modules and manifests, and I''m having second thoughts about my
original
design. I''d appreciate any feedback you can give me.

We currently have two departments managing Puppet: Operations and 
Development. To prevent accidentally breaking all nodes while testing, 
there are two repositories we work with, namely "puppet" (the
production
repo) and "puppet-staging" (the testing and development repo). This
allows
Operations to control access to the production repo (to which Development 
doesn''t have permission to commit changes) and maintain stability.

Normally all nodes run in the "production" environment, grabbing
catalogs
from "puppet", but we can test things at the command line by
specifying the
"staging" environment, which will point them at
"puppet-staging".

When creating repositories for hiera data, I naturally decided to create 
"puppet-data" and "puppet-data-staging" repos, along the
same lines as
above. This means I''ve got 4 repositories to manage, and pushing any 
changes involves a lot of pushing, testing and merging.

How does everyone else handle it? What problems have you run into? I''m
just
getting started with hiera, and my node-specific data is tightly bound to 
modules and manifests, so I''ve got a lot of work ahead of me.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/95EKCsCxm6EJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Good question. I''m wondering about that as well.

My setup works for my small team, but is admittedly far from perfect.
I currently have two Git repo''s, one for my nodes.pp and another for my
modules. Both repo''s have three branches; development, testing and
master.
I''ve created environments on my Puppetmaster, named production,
testing,
dev_blabla, etc. Production is a clone from the master branches, testing is 
a clone from the testing branches, and dev_blabla are any number of clones 
from development branches of developers. Users that need to develop each 
get their own dev_username environment, so they can develop and debug on 
the puppetmaster. They can use the environment=dev_username parameter to 
test on some servers, but are not allowed to do anything in the production 
and testing environments. They can push their commits to a Bitbucket 
remote. Due to Git''s design, anyone with push permissions can push any 
branches, so there''s not much security in the repos. However, only
admins
have access to the testing and production environments. They will fetch 
changes from the Bitbucket remote, review them, test if necessary and merge 
them into the local environments. There''s quite a bit of pushing and 
merging going on, and it''s annoying to commit, push, fetch and merge,
just
to get your changes from one environment to the next. I suppose it does 
help prevent mistakes though.

I hope that story makes some sense. I''d be very interested to see how 
others set up their version control workflow, since I''m looking into
Hiera
as well.

Regards, Martijn

Op donderdag 30 augustus 2012 18:06:07 UTC+2 schreef Stephen Price het 
volgende:
>
> I''m in the process of migrating data out of my standard Puppet 
> repositories for modules and manifests, and I''m having second
thoughts
> about my original design. I''d appreciate any feedback you can give
me.
>
> We currently have two departments managing Puppet: Operations and 
> Development. To prevent accidentally breaking all nodes while testing, 
> there are two repositories we work with, namely "puppet" (the
production
> repo) and "puppet-staging" (the testing and development repo).
This allows
> Operations to control access to the production repo (to which Development 
> doesn''t have permission to commit changes) and maintain stability.
>
> Normally all nodes run in the "production" environment, grabbing
catalogs
> from "puppet", but we can test things at the command line by
specifying the
> "staging" environment, which will point them at
"puppet-staging".
>
> When creating repositories for hiera data, I naturally decided to create 
> "puppet-data" and "puppet-data-staging" repos, along
the same lines as
> above. This means I''ve got 4 repositories to manage, and pushing
any
> changes involves a lot of pushing, testing and merging.
>
> How does everyone else handle it? What problems have you run into?
I''m
> just getting started with hiera, and my node-specific data is tightly bound
> to modules and manifests, so I''ve got a lot of work ahead of me.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/hmN8DPUSP5wJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Just a followup:
I merged my "-staging" repositories in to the main "puppet"
and
"puppet-data" repositories. Instead of managing permissions by
repository,
I''m using Mercurial''s ACL extension to restrict pushes to the
"production"
branch to  specific groups. Makes for a cleaner layout.

On Tuesday, September 4, 2012 7:21:29 AM UTC-7, Martijn
wrote:
>
> Good question. I''m wondering about that as well.
>
> My setup works for my small team, but is admittedly far from perfect.
> I currently have two Git repo''s, one for my nodes.pp and another
for my
> modules. Both repo''s have three branches; development, testing and
master.
> I''ve created environments on my Puppetmaster, named production,
testing,
> dev_blabla, etc. Production is a clone from the master branches, testing is
> a clone from the testing branches, and dev_blabla are any number of clones 
> from development branches of developers. Users that need to develop each 
> get their own dev_username environment, so they can develop and debug on 
> the puppetmaster. They can use the environment=dev_username parameter to 
> test on some servers, but are not allowed to do anything in the production 
> and testing environments. They can push their commits to a Bitbucket 
> remote. Due to Git''s design, anyone with push permissions can push
any
> branches, so there''s not much security in the repos. However, only
admins
> have access to the testing and production environments. They will fetch 
> changes from the Bitbucket remote, review them, test if necessary and merge
> them into the local environments. There''s quite a bit of pushing
and
> merging going on, and it''s annoying to commit, push, fetch and
merge, just
> to get your changes from one environment to the next. I suppose it does 
> help prevent mistakes though.
>
> I hope that story makes some sense. I''d be very interested to see
how
> others set up their version control workflow, since I''m looking
into Hiera
> as well.
>
> Regards, Martijn
>
> Op donderdag 30 augustus 2012 18:06:07 UTC+2 schreef Stephen Price het 
> volgende:
>>
>> I''m in the process of migrating data out of my standard Puppet
>> repositories for modules and manifests, and I''m having second
thoughts
>> about my original design. I''d appreciate any feedback you can
give me.
>>
>> We currently have two departments managing Puppet: Operations and 
>> Development. To prevent accidentally breaking all nodes while testing, 
>> there are two repositories we work with, namely "puppet" (the
production
>> repo) and "puppet-staging" (the testing and development
repo). This allows
>> Operations to control access to the production repo (to which
Development
>> doesn''t have permission to commit changes) and maintain
stability.
>>
>> Normally all nodes run in the "production" environment,
grabbing catalogs
>> from "puppet", but we can test things at the command line by
specifying the
>> "staging" environment, which will point them at
"puppet-staging".
>>
>> When creating repositories for hiera data, I naturally decided to
create
>> "puppet-data" and "puppet-data-staging" repos,
along the same lines as
>> above. This means I''ve got 4 repositories to manage, and
pushing any
>> changes involves a lot of pushing, testing and merging.
>>
>> How does everyone else handle it? What problems have you run into?
I''m
>> just getting started with hiera, and my node-specific data is tightly
bound
>> to modules and manifests, so I''ve got a lot of work ahead of
me.
>>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/U8XQbukyVocJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.