I''m trying to scale up Puppet and am wondering if I can use Puppet in an odd way. I''m looking to use Puppet as an authoritative source for all the configuration done to a cluster. That includes fstab''s, modprobe.conf, etc. This reduces duplication. I''m using SystemImager on a cluster and would like to run Puppet during the install, so when the node reboots, it will have all its config files. I''ve found examples for doing this with a central server, and they seem to work. We are currently storing all of our puppet config data in an SVN repository so we can track changes. My question is this. I can pull a copy of that data into the image, chroot into the image, then... Can I point the puppet client directly at the local copy of the servers data rather then running a full blown copy of the puppet server locally? Thanks, Kevin
On Mar 1, 2007, at 12:12 PM, Kevin Fox wrote:> I''m trying to scale up Puppet and am wondering if I can use Puppet > in an > odd way.Just so everyone knows, I''ve now got Puppet running in Mongrel behind an Apache 2.2 proxy. Now that I''ve got the proof-of-concept done, I just need to solidify the code, get some tests written, and merge the code with the existing stuff to get rid of the code duplication. It''s testable right now and should be "usable" within the next few days. And for the record, it''s *always* ok to use Puppet in odd ways. :)> [...] > My question is this. I can pull a copy of that data into the image, > chroot into the image, then... Can I point the puppet client > directly at > the local copy of the servers data rather then running a full blown > copy > of the puppet server locally?Yep, works just fine. That''s exactly how I bootstrap puppetmasters -- check the code out, run "puppet --use-nodes site.pp", and you''re golden. There are the obvious cavaets, like if you use LDAP nodes then you''ll have to tell puppet how to find the nodes in ldap, and if you use nodes in your config then you''ll have to tell puppet (like my example above), but otherwise it works fine. At this point, there''s no real functional difference between doing this on all of your hosts and compiling on the server. Once the export/collect stuff becomes fast enough to be considered stable, then there''ll be a big difference, though. -- The great aim of education is not knowledge but action. -- Herbert Spencer --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On 3/1/07, Luke Kanies <luke@madstop.com> wrote:> > Just so everyone knows, I''ve now got Puppet running in Mongrel behind > an Apache 2.2 proxy. Now that I''ve got the proof-of-concept done, I > just need to solidify the code, get some tests written, and merge the > code with the existing stuff to get rid of the code duplication. > It''s testable right now and should be "usable" within the next few days. >Why do you need to run it behind an Apache proxy? Mongrel is a web server. It may not be able to run a full web site but it should be able to serve up Puppet. - Ian
On Mar 1, 2007, at 3:27 PM, Ian Burrell wrote:> Why do you need to run it behind an Apache proxy? Mongrel is a web > server. It may not be able to run a full web site but it should be > able to serve up Puppet.Mongrel doesn''t speak SSL and never will (according to its author), and it also still can''t use more than one processor core, since it can only use Ruby''s green threads. -- The Chico, California, City Council enacted a ban on nuclear weapons, setting a $500 fine for anyone detonating one within city limits. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Thu, 2007-03-01 at 15:30 -0600, Luke Kanies wrote:> On Mar 1, 2007, at 3:27 PM, Ian Burrell wrote: > > Why do you need to run it behind an Apache proxy? Mongrel is a web > > server. It may not be able to run a full web site but it should be > > able to serve up Puppet. > > Mongrel doesn''t speak SSL and never will (according to its author),Does that mean that anybody who can get to the port that mongrel is listening on can avoid authentication ? David
On Mar 1, 2007, at 4:49 PM, David Lutterkort wrote:>> >> Mongrel doesn''t speak SSL and never will (according to its author), > > Does that mean that anybody who can get to the port that mongrel is > listening on can avoid authentication ?Essentially, yes. You''ll want to set it up to only listen on localhost, which will be the default if/when it ships. If people really want it, I can go through the extra step of validating the cert in mongrel, once it''s been validated by apache, but you have to trust something somewhere, and that''ll add a good bit of load to the system. -- The ships hung in the sky in much the same way that bricks don''t. -- Douglas Adams --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On 3/1/07, Luke Kanies <luke@madstop.com> wrote:> On Mar 1, 2007, at 3:27 PM, Ian Burrell wrote: > > Why do you need to run it behind an Apache proxy? Mongrel is a web > > server. It may not be able to run a full web site but it should be > > able to serve up Puppet. > > Mongrel doesn''t speak SSL and never will (according to its author), > and it also still can''t use more than one processor core, since it > can only use Ruby''s green threads. >What is the advantage of Mongrel? What is the problem with Webrick? Is it multithreaded? Adding Apache as a proxy means a lot of extra complexity. One of the big advantages of the current server is that the Puppet server is self-hosting including all the certificate handling. - Ian
On Mar 2, 2007, at 1:57 PM, Ian Burrell wrote:>> > > What is the advantage of Mongrel? What is the problem with Webrick? > Is it multithreaded? Adding Apache as a proxy means a lot of extra > complexity. One of the big advantages of the current server is that > the Puppet server is self-hosting including all the certificate > handling.The whole point of this exercise is to figure out if there *is* an advantage to Mongrel. There seems to be something weird happening in Webrick where it sometimes just hangs indefinitely, giving what appears to be much worse than linear performance -- possibly because of lock thrashing. We have not yet reproduced this in testing (Jeff McCune is doing a lot of testing with both servers right now), so if it doesn''t actually happen, they''re basically equivalent, performance- wise, but if it does happen, then there might be a significant advantage. Ruby doesn''t have native threads, so there''s no way a single process is ever going to take advantage of more than one processor. In order to scale better, our only real options are to find ways to reduce the time it takes to handle a request, or to spread across multiple processors, which means multiple processes. Once you go multiple processes, you basically have two choices to make: Do you want to handle your own forking, or put a proxy in front? If you do a proxy, do you want the proxy to do the ssl or do you want the multiple processes to do it? I don''t think I want to write my own connection handler for webrick, at least partially because it''s a big can of worms that essentially has nothing to do with making Puppet better, so it''d be wasted development. I can instead spend some time supporting Mongrel -- which might be wasted development but considering it''s now basically the "standard" hosting platform for ruby, I doubt it -- and then tell you to throw a proxy up, and you can use whatever proxy you want as long as it provides the right data to Mongrel. In other words, this seems like the easiest of the available options. Webrick will always be supported, and I''m committed to always having Puppetmasterd work with no configuration out of the box. I just need options for higher performance beyond single cores, and Mongrel seems like the best bet. -- I have learned to use the word ''impossible'' with the greatest caution. -- Wernher von Braun --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com