Hello everyone,
I''m currently looking into using puppet to manage my homedir .dot
files,
including .vim/ folder with a bunch of plugins totaling a few tens of files
(almost 100), in order to have a pleasant and productive working experience
on our organization''s 500+ machines.
While this looks like a cool thing to do, I can''t help but wonder how
will
this affect puppet''s performance both on the masters'' and
agents'' sides and
how good puppet is at distributing lots of infrequently updated files.
My common sense tells me that from a performance point of view, I could use
the following methods, first in the list being the most recommended:
file {
content => file(''file''),
}
file {
source => "puppet:///modules/...",
}
file {
content => template(''template.erb''),
}
Personally, I''d go with using file() function for distributing static
files
and source => "puppet:///" to manage directories recursively, which
would
simplify the setup part a lot.
Does anyone have a more in-depth insight into the performance of all these
methods ? Or, for that matter, why I shouldn''t be doing the above as it
blows puppet performance in ways I can''t comprehend ?
Thanks for your feedback,
Mihai
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/psSkZ5Ww2H8J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2012-Oct-11 13:39 UTC
[Puppet Users] Re: performance of source vs template vs file
On Tuesday, October 9, 2012 9:45:42 AM UTC-5, Mihai Stan wrote:> > Hello everyone, > > I''m currently looking into using puppet to manage my homedir .dot files, > including .vim/ folder with a bunch of plugins totaling a few tens of files > (almost 100), in order to have a pleasant and productive working experience > on our organization''s 500+ machines. > > While this looks like a cool thing to do, I can''t help but wonder how will > this affect puppet''s performance both on the masters'' and agents'' sides and > how good puppet is at distributing lots of infrequently updated files. > > My common sense tells me that from a performance point of view, I could > use the following methods, first in the list being the most recommended: > > file { > content => file(''file''), > } > > file { > source => "puppet:///modules/...", > } > > file { > content => template(''template.erb''), > } > > > Personally, I''d go with using file() function for distributing static > files and source => "puppet:///" to manage directories recursively, which > would simplify the setup part a lot. > > > Does anyone have a more in-depth insight into the performance of all these > methods ? Or, for that matter, why I shouldn''t be doing the above as it > blows puppet performance in ways I can''t comprehend ? >I would expect a bigger difference between using ''content'' and using ''source'' than between specifying the content via file() and specifying it via template(). When you use ''content'', the body of the target file is included in the catalog, whereas when you use ''source'', the agent will request file content from the Puppet file server if it needs it. For ~100 small files (<= 1 KB on average, say), my gut tells me that ''content'' will win at least when there are more than a couple of changes, and that it will not lose too badly when there are few or none. Certainly you should not use template() for completely static files, but when you do need it, you need it. Managing directories recursively is a potential problem, but it really comes down more to the total number and combined size of the files you are managing. Each file and directory adds memory and processing cost on both ends, and, separately, Puppet expends computing effort proportional to the total content size to compute hashes of file contents. My expectation would be that for dotfiles, you will see acceptable performance any way around. Nevertheless, as with any performance question, the only way to get a reliable answer is to test it. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wmKO8HfxPvMJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Eric Sorenson
2012-Oct-12 00:10 UTC
[Puppet Users] Re: performance of source vs template vs file
I think all that is right John--one subtlety that led me to use content=>template everywhere instead of mixing it with content=>file for things which didn''t need erb interpolation is that the argument to template() uses module-like syntax and file() needs a full filesystem path, so it doesn''t work very well with environments that have different root paths. (this is https://projects.puppetlabs.com/issues/5158) -=Eric On Thursday, October 11, 2012 6:39:30 AM UTC-7, jcbollinger wrote:> > > > On Tuesday, October 9, 2012 9:45:42 AM UTC-5, Mihai Stan wrote: >> >> Hello everyone, >> >> I''m currently looking into using puppet to manage my homedir .dot files, >> including .vim/ folder with a bunch of plugins totaling a few tens of files >> (almost 100), in order to have a pleasant and productive working experience >> on our organization''s 500+ machines. >> >> While this looks like a cool thing to do, I can''t help but wonder how >> will this affect puppet''s performance both on the masters'' and agents'' >> sides and how good puppet is at distributing lots of infrequently updated >> files. >> >> My common sense tells me that from a performance point of view, I could >> use the following methods, first in the list being the most recommended: >> >> file { >> content => file(''file''), >> } >> >> file { >> source => "puppet:///modules/...", >> } >> >> file { >> content => template(''template.erb''), >> } >> >> >> Personally, I''d go with using file() function for distributing static >> files and source => "puppet:///" to manage directories recursively, which >> would simplify the setup part a lot. >> >> >> Does anyone have a more in-depth insight into the performance of all >> these methods ? Or, for that matter, why I shouldn''t be doing the above as >> it blows puppet performance in ways I can''t comprehend ? >> > > I would expect a bigger difference between using ''content'' and using > ''source'' than between specifying the content via file() and specifying it > via template(). When you use ''content'', the body of the target file is > included in the catalog, whereas when you use ''source'', the agent will > request file content from the Puppet file server if it needs it. > > For ~100 small files (<= 1 KB on average, say), my gut tells me that > ''content'' will win at least when there are more than a couple of changes, > and that it will not lose too badly when there are few or none. > > Certainly you should not use template() for completely static files, but > when you do need it, you need it. > > Managing directories recursively is a potential problem, but it really > comes down more to the total number and combined size of the files you are > managing. Each file and directory adds memory and processing cost on both > ends, and, separately, Puppet expends computing effort proportional to the > total content size to compute hashes of file contents. > > My expectation would be that for dotfiles, you will see acceptable > performance any way around. Nevertheless, as with any performance > question, the only way to get a reliable answer is to test it. > > > John > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/cL9ngDH2YEgJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.