On Aug 27, 2006, at 8:56 AM, Thorsten Sandfuchs wrote:> hio, > I''m trying to build a "master"-directory (general) with some base- > files to > spread, and a host/class-based directory-structure to override the > defaults, > if necessary. If a file only is present in the general-section, it > shouldn''t > harm the setup, if it''s only present in the host/class-section, it > should work > and if present in both, all of them, I would like to use the > "latest" line, in > the .pp-file as the valid one.Puppet is not very good at doing this style of full-directory overlay, and it''s somewhat intentional. Its development is much more focused on managing elements, rather than whole files -- for instance, for apt sources I would specify each source which would then generate the file, rather than specifying the whole file. I have done what I could to make some aspects of file-based management possible, but I don''t think you''ll ever get the behaviour you want out of doing full directory overlays like it looks like you''re trying to do. In fact, Puppet''s parser specifically forbids the duplication you''ve got in your example -- you can''t have two objects named ''/etc'', because that is considered a conflict that Puppet doesn''t know how to resolve. You can specify multiple file sources for individual file, and the first source that exists will be used, so, for example, you could say the following: file { "/etc/apt/sources.list": source => [ ".../host/$hostname/sources.list", ".../general/sources.list" ] } If the host-specific file exists, it will be used; else, the general file will be used. However, that puts you into the somewhat complicated position of having to directly match your file structure to your configuration structure, which I have found to get confusing over time. I''ve also found that trying to have your file source directory mirror the file structure at the destination makes things a bit difficult -- it''s a better idea to have "apps/apt/sources.list" than "etc/apt/ sources.list", since it''s a semantic structure, instead of literal. A much better option is to generate the sources.list file, and then use subclasses to override how you want the file generated. I haven''t had to differentiate my sources.list files before; can you explain how the host file differs from the general file? I''ll see if I can come up with a way to generate it that meets your needs. -- Luke Kanies http://madstop.com | http://reductivelabs.com | 615-594-8199
On Sun, Aug 27, 2006 at 05:56:31PM +0200, Thorsten Sandfuchs wrote:> hio, > I''m trying to build a "master"-directory (general) with some base-files to > spread, and a host/class-based directory-structure to override the defaults, > if necessary. If a file only is present in the general-section, it shouldn''t > harm the setup, if it''s only present in the host/class-section, it should work > and if present in both, all of them, I would like to use the "latest" line, in > the .pp-file as the valid one.It might not be exactly what you want but I wonder if the new %h,%H functionality in the fileserver can be extended to do something close to this. A change to the fileserver to return [export]/file when [export]/hostname/file doesn''t exist will allow you to override files without touching any of the manifests. Kostas
On Aug 27, 2006, at 12:36 PM, Kostas Georgiou wrote:> > It might not be exactly what you want but I wonder if the new %h,%H > functionality > in the fileserver can be extended to do something close to this. > > A change to the fileserver to return [export]/file when [export]/ > hostname/file > doesn''t exist will allow you to override files without touching any > of the manifests.I''ve also been thinking about extending Puppet a little bit along the lines of bcfg2, even though I think it''s going a bit in the wrong direction. Bcfg2 works by creating file generators; the simplest one just copies a whole file, but you can also write custom generators that create a file based on arbitrary mechanisms. It''d be easy to add something like Puppet''s templating support to do something similar here -- you specify a generator, instead of a template. I''m just starting to think about this, though, and like I said I think it''s the wrong direction, because only the server knows how these things can work -- you couldn''t do any introspection on the client, and you can''t see that modeling through the XMLRPC interface. It might, however, make getting the right file to the right host a bit easier, in those cases where you don''t want to go through the effort of modeling. -- Luke Kanies http://madstop.com | http://reductivelabs.com | 615-594-8199
On Aug 27, 2006, at 4:19 PM, Thorsten Sandfuchs wrote:> I couldn''t find a propper way to do this, yet. > > With the "content"-directive, I can''t "add" text, with the "source"- > directive > neither, and I think "just adding" lines, can leed to serious > problems with > some files, which are more complex, than a sources.list. :)Here''s a relatively simple custom definition I wrote for managing mail aliases: define alias(ensure, file = "/etc/postfix/aliases") { $pattern = "''^$name''" case $ensure { absent: { exec { "rm-alias-$name": command => "/usr/bin/sed -i -e ''/^$name/d'' $file", onlyif => "/usr/bin/grep $pattern $file" } } default: { $line = "$name: $ensure" exec { "add-alias-$name": command => "/bin/echo ''$line'' >> $file", unless => "/usr/bin/grep $pattern $file" } exec { "fix-alias-$name": command => "/usr/bin/sed -i -e ''s/^$name:..*\$/ $line/'' $file", unless => "/usr/bin/grep ''^$line\$'' $file", require => exec["add-alias-$name"] } } } } alias { luke: ensure => "luke@madstop.com" } #alias { luke: ensure => absent } You could do something pretty easily with apt sources, although the key would have to be the URL. Otherwise, I agree, Puppet does not yet provide a way to concatenate contents.> Perhaps with a template-construct, but you probably could point me > to "your > way" and I''ll be glad to adopt it. > >> You can specify multiple file sources for individual file, and the >> first source that exists will be used, so, for example, you could say >> the following: > > nice one - didn''t find that in the documentation/examples. Is there > a similar > functionality with a content/classes combination to concatenate some > file-contents?No, but the ''content'' attribute of files probably should concatenate values. If the above code doesn''t work for you, I''d be glad to find a way to get the behaviour you want if you can describe the differences in these files you''re managing. I''ve just added some docs on the source search paths to the reference.>> However, that puts you into the somewhat complicated position of >> having to directly match your file structure to your configuration >> structure, which I have found to get confusing over time. I''ve also >> found that trying to have your file source directory mirror the file >> structure at the destination makes things a bit difficult -- it''s a >> better idea to have "apps/apt/sources.list" than "etc/apt/ >> sources.list", since it''s a semantic structure, instead of literal. > > Now I''m beginning to like your way of arranging the files, it''s more > intuitively.Cool, glad to help.>> A much better option is to generate the sources.list file, and then >> use subclasses to override how you want the file generated. >> I haven''t had to differentiate my sources.list files before; can you >> explain how the host file differs from the general file? I''ll see if > > well apparently this was only an example for a file where it might > be needed, > e.g. if I have a host/class with another distribution (testing/ > unstable) and > so on... > > Another example would be to only add "some" public-keys to a > authorized_keys-file.Yeah, authorized_keys management is relatively high on my list. I expect to have that in the next release of Puppet, so you shouldn''t need to worry about it.> I already though of some scripts, which concatenate the files, I > would place > in directories like: /etc/apt/sources.list.d-puppet and then call a > find/xargs > combination to combine them altogether, but that would be a dirty > hack, I > think.It''s not really that bad, actually, for files that don''t natively support a way to aggregate. However, I think the sources.list file is simple enough that it''ll be easy to manage, even using a custom definition, without any ruby.> And another approach could be using a diff/patch solution to change > only > disired parts of files, but I''m sure you already considered this > and found > some arguments against it.... Year, just found the thread on > cfengine on the > devel-list (2006-May). :)-- Luke Kanies http://madstop.com | http://reductivelabs.com | 615-594-8199