Puppet users - Apr 2013 - ppg: Scheduled rollouts and dashboard with git in decentralized setup

For a "server-less" puppet setup using git for config distribution, I
am drafting out some scaffolding...

Some background in the message I just posted:
https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/A5Ywi1V1OrA

Plan is to have two branches in git: master and production. Commits
will be normally be made to master (which actually acts as the
"queue").

We will have a wrapper, "ppg" for puppetgit -- and avoiding confusion
with PostgreSQL tools.

=Commits and scheduling
Commits can only be made using ppg, enforced through a commit hook.

Commits with ppg can be --immediate, in which case they are committed
to master and prod (in case they are the same).

Alternatively, commits with ppg can be --schedule [timestamp].

On every commit, ppg checks that the "production" branch is a subset
of master, that is, that a `git merge master` will just mean a
fast-forward. If the two branches have diverged, these checks will
force the user to merge back into master to ensure any differences are
resolved and accounted for.

When using --schedule, ppg checks whether an earlier commit is
scheduled for a later time -- and errors out to prevent premature
rollout of changes due to conflicting schedules.

ppg also runs puppet validate over the files being committed.

=Scheduling happens on the Gold server
To implement the scheduled rollouts, ppg tags the commit with a
specially crafted tag. The gold server runs a periodic cron that scans
unmerged changes on master and merges them if the timestamp in the tag
has been reached and the merge is a fast-forward.

=Client side applyOn the client side, `ppg pullapply -- [puppet params]` runs a
git pull
and only invokes puppet apply if git has brought changes to the local
branch (normally production).

ppg pullaply collects the output from puppet and somehow pushes it all
the way back.

=Store-and-forward feedback channel
I am less certain of this part, and input will be specially valuable here.

ppg pullapply will...

 -  apply changes locally, capture stderr/stdout, perhaps more info
that can be negotiated with the puppet client ("facts"?).
 - write state to file(s) in a "puppet-feedback" git repo, commit that
state
 - push to a "feedback" rw repo on the gold server (or on the proxy
server)

ppg on the proxy and gold servers will take care of store-and-forward
until it reaches its destination (a dashboard server). ppg also takes
care of pruning very old data that has already been delivered.

Once the data reaches the dashboard server, it gets fed to the Puppet
Dashboard thingamajig, butterfly-mode is automagically enabled in your
emacs session and you''re so so glad you took the blue pill.

thoughts? comments? bikesheds?



m
--
 martin.langhoff@gmail.com
 -  ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Interesting. It seems nicely thought out, but I stumbled here, reading:

On 04/23/2013 11:22 PM, Martin Langhoff wrote:
> I am less certain of this part, and input will be specially valuable here.
> 
> ppg pullapply will...
> 
>  -  apply changes locally, capture stderr/stdout, perhaps more info
> that can be negotiated with the puppet client ("facts"?).
>  - write state to file(s) in a "puppet-feedback" git repo, commit
that state
>  - push to a "feedback" rw repo on the gold server (or on the
proxy server)
Uhm, what? Why? Why is there a git repository for your transient puppet
reports?

You''re reinventing the wheel I think (although your''s a bit
square-ish ;)

Doesn''t the dashboard usually consume the report as generated by the
agent? Therefor, isn''t what you want a way to transfer that very report
from the agents to the dashboard? I vaguely remember an issue with
masterless not generating reports, but I may misremember this one.
> Once the data reaches the dashboard server, it gets fed to the Puppet
> Dashboard thingamajig, butterfly-mode is automagically enabled in your
> emacs session and you''re so so glad you took the blue pill.
> 
> thoughts? comments? bikesheds?
Yes, actually: I disbelieve puppet runs in Emacs (yet).

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Mon, Apr 29, 2013 at 6:19 PM, Felix Frank
<Felix.Frank@alumni.tu-berlin.de> wrote:
> Interesting. It seems nicely thought out, but I stumbled here, reading:
thanks for reading!
> On 04/23/2013 11:22 PM, Martin Langhoff wrote:
>> I am less certain of this part, and input will be specially valuable
here.
>>
>> ppg pullapply will...
>>
>>  -  apply changes locally, capture stderr/stdout, perhaps more info
>> that can be negotiated with the puppet client ("facts"?).
>>  - write state to file(s) in a "puppet-feedback" git repo,
commit that state
>>  - push to a "feedback" rw repo on the gold server (or on the
proxy server)
>
> Uhm, what? Why? Why is there a git repository for your transient puppet
> reports?
Well, the assumption of this setup is that the server where you''d run
dashboard isn''t necessarily reachable all the time.

For example, during a network outage, or an uplink DoS.

So I need some store-and-forward facility. Using git for this purpose
isn''t the absolute best-fit but limits my tool use, my dependencies. I
could use some other tool (sqlite?) but git handles store-and-forward
setup pretty well (with the normal git push semantics).

Hard to justify added deps and complications unless there''s a great
fit to the proposed alternative...
> You''re reinventing the wheel I think (although your''s a
bit square-ish ;)
But isn''t it cute how it goes thunk! four times per turn?
> Doesn''t the dashboard usually consume the report as generated by
the
> agent? Therefor, isn''t what you want a way to transfer that very
report
> from the agents to the dashboard? I vaguely remember an issue with
> masterless not generating reports, but I may misremember this one.
Correct, masterless won''t generate reports, and that''s part of
what I
am trying to address.


m
--
 martin.langhoff@gmail.com
 -  ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Tue, Apr 23, 2013 at 5:22 PM, Martin Langhoff
<martin.langhoff@gmail.com> wrote:
> We will have a wrapper, "ppg" for puppetgit -- and avoiding
confusion
> with PostgreSQL tools.
After some delays in getting started...

   http://repo.or.cz/w/puppet-git.git/

Still a work in progress, but if I can get two more productive days
in, it''ll be close to complete.

cheers,



m
--
 martin.langhoff@gmail.com
 -  ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.