Puppet users - Nov 2013 - Dashboard running in masterless puppet

Hi All,

I''m running Puppet in masterless mode and trying to make Puppet
dashboard
play nice with it in this non-standard setup.

I''d love to hear how other people are doing this. The process I have in
mind is:

1) Run "puppet apply" from cron on each node
2) Rsync (using --remove-sent-files) the reports from each node''s
/var/lib/puppet/reports dir back to the puppet dashboard server
3) Run rake:import

The issue I''m having is that rake:import will create a new failed task
for
every report that already exists in the database. It does skip them, but I
don''t want to be notified it''s a failed task when it skips
them. I was
thinking of deleting old reports, but given that they get imported by the
delayed task workers, it is hard to know which reports have been processed
to avoid deleting a report that hasn''t been imported yet.

If I can find a way to stop it reporting skipped reports as failed tasks,
then I should be OK.

Any ideas? Anyone else running Puppet dashboard with masterless Puppet?

- GS

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAHpidMBFwOuqiLNNbE_yjSzZeiqYxKH_Xt4UKjkpwh9fM3J4Ow%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

I just came across the dynamic of running Puppet in a masterless mode.  Was 
there a tutorial that you had followed in order to configure it that way?

Thanks!

On Monday, November 4, 2013 12:21:14 AM UTC-5, Gonzalo
wrote:
>
> Hi All,
>
> I''m running Puppet in masterless mode and trying to make Puppet
dashboard
> play nice with it in this non-standard setup.
>
> I''d love to hear how other people are doing this. The process I
have in
> mind is:
>
> 1) Run "puppet apply" from cron on each node
> 2) Rsync (using --remove-sent-files) the reports from each node''s 
> /var/lib/puppet/reports dir back to the puppet dashboard server
> 3) Run rake:import
>
> The issue I''m having is that rake:import will create a new failed
task for
> every report that already exists in the database. It does skip them, but I 
> don''t want to be notified it''s a failed task when it
skips them. I was
> thinking of deleting old reports, but given that they get imported by the 
> delayed task workers, it is hard to know which reports have been processed 
> to avoid deleting a report that hasn''t been imported yet.
>
> If I can find a way to stop it reporting skipped reports as failed tasks, 
> then I should be OK.
>
> Any ideas? Anyone else running Puppet dashboard with masterless Puppet?
>
> - GS
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1ba01e3a-f206-4aca-b7ab-79e924d05d81%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Gonzalo,
> I''m running Puppet in masterless mode and trying to make Puppet
dashboard
> play nice with it in this non-standard setup.
some of the tricks that are useful in this kind of scenario are
encapsulated in puppet-git / ppg, which I''ve written, and use at
Remote Learner, where it is gradually taking over an infra with >2000
VMs.

   http://repo.or.cz/w/puppet-git.git
> I''d love to hear how other people are doing this. The process I
have in mind
> is:
>
> 1) Run "puppet apply" from cron on each node
yep. You can use --detailed-exitcodes and do something smart about
failures. At the risk of sounding like a broken record... ppg has
example code...
> 2) Rsync (using --remove-sent-files) the reports from each node''s
> /var/lib/puppet/reports dir back to the puppet dashboard server
In my case, the chosen transport mechanism is git protocol. ppg pushes
it back to a different git repo (pulls from puppet.git, pushes reports
to reports.git ) . There''s code in ppg to "trim" git history.
> 3) Run rake:import
Instead of that, my code is using curl:

http://repo.or.cz/w/puppet-git.git/blob/670a12233f563d47e32e47f09214590e26451a5a:/ppg-reports-to-dashboard#l38

this is barely tested, and we haven''t put it in prod yet. It mimics
what the puppet clients do.
> The issue I''m having is that rake:import will create a new failed
task
I haven''t got the foggiest idea about using rake:import. Have not dug
into Puppet/Dashboard/Ruby internals too deep yet.

cheers,


m
-- 
 martin.langhoff@gmail.com
 -  ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CACPiFC%2BBgrLz9P7DYh48B8_H_Zr4FqReTdVg77YijZ38fQTp-A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Dan,

there''s quite a bit. Google for ''masterless puppet
howto'' to find a
couple tutorials and discussions that are popular. Add ''git''
to the
search string for more quality options...

The main benefits are

  - scalability
  - pull model -- see http://www.infrastructures.org/bootstrap/pushpull.shtml

The main drawbacks are

 - loss of some dynamic configuration features -- like the
configuration database, what''sitsname?
 - all clients see the whole configuration -- however I would be
personally unwilling to rely too much on this feature of the puppet
server


Myself, I "seeded" my architecture with those tutorials, and wrote the
ppg wrapper/scaffolding/infra I discussed above, which adds some good
bits that Puppet lacks:

 - scheduled rollouts (i.e.: make this effective at 3am)
 - force a specific rollout to happen _now_ (using an ssh loop with an
unprivileged account to touch an inotify  trigger)

Here''s a reasonably good thread
https://groups.google.com/forum/#!topic/puppet-users/7ZpAMrMb2NQ

cheers,



m

On Tue, Nov 5, 2013 at 3:11 PM, Dan Ng <danjng@gmail.com>
wrote:
> I just came across the dynamic of running Puppet in a masterless mode.  Was
> there a tutorial that you had followed in order to configure it that way?
>
> Thanks!
>
>
> On Monday, November 4, 2013 12:21:14 AM UTC-5, Gonzalo wrote:
>>
>> Hi All,
>>
>> I''m running Puppet in masterless mode and trying to make
Puppet dashboard
>> play nice with it in this non-standard setup.
>>
>> I''d love to hear how other people are doing this. The process
I have in
>> mind is:
>>
>> 1) Run "puppet apply" from cron on each node
>> 2) Rsync (using --remove-sent-files) the reports from each
node''s
>> /var/lib/puppet/reports dir back to the puppet dashboard server
>> 3) Run rake:import
>>
>> The issue I''m having is that rake:import will create a new
failed task for
>> every report that already exists in the database. It does skip them,
but I
>> don''t want to be notified it''s a failed task when it
skips them. I was
>> thinking of deleting old reports, but given that they get imported by
the
>> delayed task workers, it is hard to know which reports have been
processed
>> to avoid deleting a report that hasn''t been imported yet.
>>
>> If I can find a way to stop it reporting skipped reports as failed
tasks,
>> then I should be OK.
>>
>> Any ideas? Anyone else running Puppet dashboard with masterless Puppet?
>>
>> - GS
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/puppet-users/1ba01e3a-f206-4aca-b7ab-79e924d05d81%40googlegroups.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.


-- 
 martin.langhoff@gmail.com
 -  ask interesting questions
 - don''t get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CACPiFCLC9jSgiHwJhjae6CttpPpEVGPEXZ3i_-X4-fcajtPmqA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.