Apologize ahead of time if this has been asked before. Without using external commands, can puppet back-up files and directories to the puppetmaster server? If I delete or change a file, I can back the file, but all I want to do is to take a snap-shot of some configurations. If I can''t do it natively, has anyone created a script that does so using external commands (such as tar, gzip, and scp)? LinuxSneaker
Hi, Michael Manry wrote:> Apologize ahead of time if this has been asked before. Without using > external commands, can puppet back-up files and directories to the > puppetmaster server? If I delete or change a file, I can back the file, > but all I want to do is to take a snap-shot of some configurations. > > If I can''t do it natively, has anyone created a script that does so > using external commands (such as tar, gzip, and scp)?I have a related question. I have Puppet installed on seven hosts and I''m slowly standardizing them as best I can. Is there a "best practice" or a set of tools for normalizing configuration among a number of machines? Currently I''m using a for-loop in shell, scp, sdiff, md5sum, and vim to make things consistent. I''d like to believe there''s a better way. I will contribute coding to a project like this if people are interested since I have to do this anyway... :) -- Bob
On Apr 24, 2007, at 5:35 AM, Michael Manry wrote:> Apologize ahead of time if this has been asked before. Without using > external commands, can puppet back-up files and directories to the > puppetmaster server? If I delete or change a file, I can back the > file, > but all I want to do is to take a snap-shot of some configurations. > > If I can''t do it natively, has anyone created a script that does so > using external commands (such as tar, gzip, and scp)?Hmm, I''d been thinking about this, too. Puppet should automatically send changed files to the central filebucket when it notices changes, but it does not currently do that. I''ll file it as an enhancement request; it should be pretty easy to do. -- Nature and nature''s laws lay hid in night, God said, "Let Newton be," and all was light. It did not last; the devil howling "Ho! Let Einstein be!" restored the status quo. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Apr 24, 2007, at 11:15 AM, Bob Apthorpe wrote:> > I have a related question. I have Puppet installed on seven hosts and > I''m slowly standardizing them as best I can. Is there a "best > practice" > or a set of tools for normalizing configuration among a number of > machines? > > Currently I''m using a for-loop in shell, scp, sdiff, md5sum, and > vim to > make things consistent. I''d like to believe there''s a better way. I > will > contribute coding to a project like this if people are interested > since > I have to do this anyway... :)I haven''t focused as much as I could on this aspect of the tool, but it''s something I''ve been thinking about a lot recently. Mostly, I think people rely on --noop to see what would change with the current configuration. I''m hoping to get better information in place soon, so you can do things like get file diffs directly on the console to see what would change, but I just haven''t had time to work on that. I doubt it''s in any of the packages, but you can also download ''ralsh'' (for Resource Abstraction Layer SHell), which can be used to convert current configuration to Puppet code: luke@phage(0) $ bin/ralsh user luke user { ''luke'': uid => ''501'', gid => ''501'', shell => ''/bin/bash'', home => ''/Users/luke'', comment => ''Luke A. Kanies'' } luke@phage(0) $ This should work with any other type, and if you don''t specify a name then it will list out all instances. Is there anything particular you''re looking for, some process you''re hoping to enable? -- Today at work an ethernet switch decided to take the ''N'' out of NVRAM -- Richard Letts --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Tue, 2007-04-24 at 11:15 -0500, Bob Apthorpe wrote:> Hi, > > Michael Manry wrote: > > Apologize ahead of time if this has been asked before. Without using > > external commands, can puppet back-up files and directories to the > > puppetmaster server? If I delete or change a file, I can back the file, > > but all I want to do is to take a snap-shot of some configurations. > > > > If I can''t do it natively, has anyone created a script that does so > > using external commands (such as tar, gzip, and scp)? > > I have a related question. I have Puppet installed on seven hosts and > I''m slowly standardizing them as best I can. Is there a "best practice" > or a set of tools for normalizing configuration among a number of machines? > > Currently I''m using a for-loop in shell, scp, sdiff, md5sum, and vim to > make things consistent. I''d like to believe there''s a better way. I will > contribute coding to a project like this if people are interested since > I have to do this anyway... :)You can try using cft[1], assuming your scripts always perform all changes. With that, I''d start a cft session on the client, run your config scripts, and finish the cft session. That should give you a record of changes made. Caveat emptor: cft is still pretty rough around the edges; the current release 0.2.0 spews out a lot of unnecessary stuff. I am in the process of remedying that and some other odds and ends and get another release out in the next week or so. David [1] http://cft.et.redhat.com/
Hi, Luke Kanies wrote:> On Apr 24, 2007, at 11:15 AM, Bob Apthorpe wrote: >> I have a related question. I have Puppet installed on seven hosts >> and I''m slowly standardizing them as best I can. Is there a "best >> practice" or a set of tools for normalizing configuration among a >> number of machines? [...] > > I haven''t focused as much as I could on this aspect of the tool, but > it''s something I''ve been thinking about a lot recently. > > Mostly, I think people rely on --noop to see what would change with > the current configuration. I''m hoping to get better information in > place soon, so you can do things like get file diffs directly on the > console to see what would change, but I just haven''t had time to > work on that. > > I doubt it''s in any of the packages, but you can also download > ''ralsh'' (for Resource Abstraction Layer SHell), [...]Cool - I''ll look into that.> Is there anything particular you''re looking for, some process you''re > hoping to enable?Not precisely, but let me give you a little backstory and see if that helps any. I''ve been working on a framework for a series of articles for LoPSA to help people move to a more uniform, scalable method of operations. As part of that, I installed puppet on a few work boxes and started syncing /etc/sudoers. I moved on to similar generic single files (/etc/ntp.conf, /etc/syslog.conf, /etc/aliases), then to single machine-specific files (/etc/apcupsd/apcupsd.conf) Those sorts of files are relatively easy to normalize because they are usually based on the vendor''s originals and changes are relatively small and uniform. apcupsd.conf is a bit trickier since UPS addresses are not accessible from the hosts (not my doing) so I work around that with a few machines hooked to the UPSs with serial cables proxying UPS data to the rest of the machines. The most irritating part has been trying to normalize uids and gids. Before I can make a service-specific class, I need to know what ids are in use where so I can sanely renumber them. To this end I''ve written a bit of perl to cross-reference host and user/group names so it''s easy to spot inconsistent or missing id numbers. I don''t believe the normalization process can be fully automated but I worry that a lot of people will be writing normalization scripts that (ideally) they shouldn''t have to. I''m also working on managing vendor applications - required users & groups, mount points, and eventually software installations, config files, and log cleanup. That is a bit easier than getting existing systems under configuration management because with vendor apps, the problem is usually smaller or more focussed, even if the puppet manifests are more complex. I know this is another case of infinite wants vs finite resources, but I''m thinking in terms of making it easier to get puppet up & running. I''m still trying to get up to speed with both puppet and ruby so there''s a limit to how much I can contribute to core code, but I''m trying to flag issues that occur once in an admin''s experience and are quickly forgotten once we get our systems managed. -- Bob
On Apr 24, 2007, at 2:08 PM, Bob Apthorpe wrote:> [...] > The most irritating part has been trying to normalize uids and gids. > Before I can make a service-specific class, I need to know what ids > are > in use where so I can sanely renumber them. To this end I''ve written a > bit of perl to cross-reference host and user/group names so it''s > easy to > spot inconsistent or missing id numbers. I don''t believe the > normalization process can be fully automated but I worry that a lot of > people will be writing normalization scripts that (ideally) they > shouldn''t have to.Yeah, I''ve thought some about this too. Puppet has the ability to help here internally, but there are no client tools available to do it. For instance, if you start your clients with --listen --serve resource, then you can connect to them with an XMLRPC client and run queries against them. This would allow you to, as an example, collect a given resource or resource type from all of your hosts easily. Right now, you''d have to write that client program, although if you use the existing Puppet code it should be pretty darn easy and I''d be glad to help. The real problem is that it''s not just getting the data that''s the problem -- you still have to manually walk through the 10 different versions of a user and pick the UID to stick with, do that for all of the users, then go to each host and fix the user ID and then chown all files to that new UID. As far as I can tell, getting the UIDs consistent could be automated pretty quickly, and chowning all of the files would be relatively straightforward to implement as a script but it would be massively slow because you''d need to get information on every single file on the filesystem (assuming you''re worried about the user owning files outside of ~), split it out by username, and then chown each file as necessary. It would probably make sense to write these scripts and publish them in some way, but I doubt it would ever make sense to make them a part of Puppet. The querying, however, is already supported and 9/10s, at least, is already done. If you can describe how you''d like to query hosts and what output format you''d like, I can probably kick out a script to make those queries quickly. At least then it would be easy to get the data. I should probably just upgrade ralsh to do this. That shouldn''t be that hard. I''d still need to know what kind of function you''d like -- do you want to just dump the data, do you want to be able to write it, to you want to diff it, do you want it as Ruby objects or Puppet text, do you want to compare many hosts at once, etc..> I know this is another case of infinite wants vs finite resources, but > I''m thinking in terms of making it easier to get puppet up & running. > I''m still trying to get up to speed with both puppet and ruby so > there''s > a limit to how much I can contribute to core code, but I''m trying to > flag issues that occur once in an admin''s experience and are quickly > forgotten once we get our systems managed.Yeah, this is exactly the kind of feedback I''m looking for -- what can we do to make Puppet easier to use. -- The time to repair the roof is when the sun is shining. -- John F. Kennedy --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com