Stephen Morton
2014-Jun-16 19:33 UTC
[Puppet Users] Puppet new deployment questions - deployment patterns, sensitivity to network errors, and certificate headaches.
I've got some newbie puppet questions.
My team has a tremendous amount of linux/computer knowledge, but we're new
to Puppet.
We recently started using puppet to manage some 100 servers. Their configs
are all pretty similar with some small changes.
----
History
Prior to Puppet, we already had a management system that involved having
config files under revision control and the config file repo checked out on
every server and the repo config files symlinked into the appropriate place
in the filesystem. Updating the repo would update these files.This was
mostly just great, with the following limitations:
- If the symlink got broken, it didn't work.
- Some files require very specific ownership, or were required not to be
symlinks (e.g. /etc/sudoers. /etc/vsftpd/ files I think)
- Updating a daemon's config file does not mean that the daemon is
restarted. e.g. updating /etc/httpd/conf/httpd.conf does not do a
"service
httpd reload"
- You can't add a new symlink.
- All files must be in revision control to link to. Some
security-sensitive files we want to only be available to some servers and
something like puppet that can send files over the network is a good
solution to this.
----
Puppet to the rescue?
So we've tried a very conservative Puppet implementation. We've left our
existing infrastructure and we just add new rules in Puppet. So far, we
have a single site.pp file and only a dozen or so rules. But already we're
seeing problems.
1. Puppet is good for configuring dynamic stuff that changes. But it
seems silly to have rules for stuff that will be configured just one time
and then will not change. If we set up some files, we don't expect them
to
disappear. In fact if they do disappear we might not want them silently
fixed up we probably want to know what's going on. Doing everything in
puppet results in ever-growing manifests. I don't know of a way to
specify
different manifests, e.g. every 30 minutes I want Puppet to run and request
the lean and mean regular manifest and then once a week I want it to run
the "make sure everything is in the right place" manifest.
2. Puppet seems very sensitive to network glitches. We run puppet from a
cron job and errors were so frequent that we just started sending all
output to /dev/null.
3. Endless certificate issues. It's crazy. So sometimes hosts would get
"dropped"... for unknown reasons their certificates were no longer
accepted. Because we'd already stopped output (see previous bullet point)
we would not know this and the server would be quietly not updated. And
when you get a certificate problem, often simply deleting the cert on the
agent and master won't fix it. Sometimes a restart of the master service
(or more?) is required.
- The solution to this to me is not "you should run puppet dashboard,
then you'd know". This shouldn't be failing in the first
place. If
something is that flaky, I don't want to run it.
(We're running version 3.4.2 on CentOS 6.5, 64-bit.)
---
Questions.
So my questions for the above three issue are I guess as follows
1. Is there a common Puppet pattern to address this? Or am I thinking
about things all wrong.
2. Is there a way to get puppet to be more fault-tolerant, or at least
complain less?
3. Are endless certificate woes the norm? Once an agent has successfully
got its certificates working with the server, is it a known issue that it
should sometimes start to subsequently fail?
Thanks,
Steve
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/08b72832-d18a-4397-9587-a769f0ee2d6e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.