Hi, Luke asked me if I was willing to share what I am doing with the Puppet user community. So here goes: I am using puppet to manage one host right now. While that isn''t a good advertisement for Puppet''s scaling capabilities, I find it interesting because I built my single node (personal mail, svn, ci and web server) with puppet where I could (clearly there was some bootstrapping going on). So I started with a VPS that was Debian stable, installed Puppet (had to download some packages by hand from unstable, etc.) I keep all the config under source control and use puppet to ship that to the machine. I probably complicated things for myself by having the Puppet managed node and Puppetmaster on the same host, but them''s the breaks. I could probably distill my config down now that I have learned more about Puppet, and now that the tool has moved on. I also use Cruise Control[1] to hook up Subversion and Puppet: Changes get detected in Cruise, a rake[2] script runs some tests on my config[3], and then makes the updated configs available to Puppetmaster. Cruisecontrol acts like a change aware cron daemon, but it also publishes changes - in a software development context this will alert the team immediately to integration issues, but using it to notify people about infrastructure changes or config issues seemed to me like a good idea. I''m using a Java based CruiseControl service but I plan to adopt a ruby based tool[4] as that would make the rest of my toolset Ruby-based (apart from Svn, etc:). I''m presenting this concept in more detail at a Unix conference in the UK this month, so I''d be interested in your feedback or questions: if you think it''s great or insane, and why. Best Julian [1] http://cruisecontrol.sourceforge.net/ [2] http://rubyforge.org/projects/rake/ [3] http://mail.madstop.com/pipermail/puppet-users/2007-February/001471.html [4] http://rossniemi.wordpress.com/2007/02/28/a-comparison-of-continuous-integration-tools-for-ruby-on-rails/ _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
On Mar 4, 2007, at 3:49 PM, Julian Simpson wrote:> > I also use Cruise Control[1] to hook up Subversion and Puppet: > Changes get detected in Cruise, a rake[2] script runs some tests > on my config[3], and then makes the updated configs available to > Puppetmaster. Cruisecontrol acts like a change aware cron daemon, > but it also publishes changes - in a software development context > this will alert the team immediately to integration issues, but > using it to notify people about infrastructure changes or config > issues seemed to me like a good idea.Just so everyone realizes what''s going on here: Julian has implemented the beginnings of using a tool to automatically test Puppet configurations as they are committed to Subversion. This is just what the continuous integration tools do for developers doing normal languages. The ability to test configurations before deployment will always be somewhat limited, but this is exactly the path we should be on: Finding ways to add validation layers to our configurations, and providing good feedback mechanisms when those validations fail. This is significantly better than just syntax checking committed manifests, both because it has a good interface for getting status, and because it provides hooks for other kinds of checks, like syntax checking any number of files with a consistent framework (i.e., write a unit test for each file you want to check, and CruiseControl will automatically run all of those tests, rather than having a separate Subversion hook for each file). It''d be especially cool if we could start sharing unit tests for these things -- how you validate the syntax of a given file, things like that. -- The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On 3/4/07, Julian Simpson <simpsonjulian@gmail.com> wrote:> I''m presenting this concept in more detail at a Unix conference in the UK > this month, so I''d be interested in your feedback or questions: if you > think it''s great or insane, and why.This is great. I could really use a setup like this in a couple different places. I would love to hear more details. Are you preparing a presentation on this? If so, would it be possible to post your slides or any other artifacts you might be using for the presentation? Thanks, Thomas -- Thomas Lockney | tlockney@gmail.com | http://opposable-thumbs.net
Hi Thomas, I had been meaning to write more and I bashed this out on the train: In my day job, I work with developers on Agile software projects. Two key practises used in Agile are unit testing and Continuous Integration, which go hand in hand together. Developers write unit tests that specify the desired behaviour of the code, and when the tests pass, they know they are done. In reality of course there is a lot more to it than that. Once we have a body of tests that exercise most of the bespoke software code, we can easily find out if there is unintended side effects of a change we might be about to commit to our VCS. So developers should always come up to date and run unit tests before committing changes. Continuous Integration takes this idea further: when the developer commits, the CI server checks out the updated changes and runs all the tests. If the code is broken (for example, a developer may not have added a file to version control), the CI server will fail and show that project as broken. There is a strong cultural imperative not to make this happen, often the state of the build is made highly visible with lava lamps or large status indicators and the developer is hopefully incented to fix the build immediately. The benefit of these two practises is that: 1) you don''t have drawn out integration phases (you integrate in small steps, many times a day), 2) you always have code that is ready to deploy, 3) you have a safety net of tests to give you confidence to make changes and 4) your tests give a strong hint as to the intent behind the code. Ever since I started working in this environment (it''s my job to look after CI and deployment on these projects) I have been wondering about how to use Agile practises in an infrastructure context. In my previous role as a sysadmin I used to run cfengine, perforce and cron to manage my systems and although that did the trick I was never convinced that it was the complete toolset. Since discovering Puppet, I now have the following: Config files in subversion, unit tests that attempt to validate those config files, and cruisecontrol server (for CI)to take the changes and run the tests. It''s all wrapped up in a Rakefile (which cruise runs) to execute the tests and then update the puppet config on the server so that it can be made available to the Puppet clients. It''s been interesting. I don''t want to write code to parse config files unless they are the simplest type of file (I wrote a test to parse my aliases file and make sure that it was in the appropriate format, but I don''t ever want to write a parser for sendmail.cf): wherever possible I want to use the program that reads the config file to do that for me. It''s been invaluable having the ability to check my puppet recipes for syntactic correctness: I was making changes on the train to work this morning and managed to catch half a dozen typos that I made as I refactored my oldest recipe. All the validation happened locally on my laptop. I''d be interested in discussing this whole idea of testing further: we haven''t even touched on functional testing yet - ideally config would be tested on a small test system and then tagged once that passed, for consumption by the rest of the puppet clients. That way you could gain more confidence that your versioned configurations were ready for prime time. I am doing a presentation at the UKUUG Conference in Manchester this month. I''ll post something after the conference. Best Julian On 3/6/07, Thomas Lockney <tlockney@gmail.com> wrote:> > On 3/4/07, Julian Simpson <simpsonjulian@gmail.com> wrote: > > I''m presenting this concept in more detail at a Unix conference in the > UK > > this month, so I''d be interested in your feedback or questions: if you > > think it''s great or insane, and why. > > This is great. I could really use a setup like this in a couple > different places. I would love to hear more details. Are you preparing > a presentation on this? If so, would it be possible to post your > slides or any other artifacts you might be using for the presentation? > > Thanks, > Thomas > > -- > Thomas Lockney | tlockney@gmail.com | http://opposable-thumbs.net > _______________________________________________ > Puppet-users mailing list > Puppet-users@madstop.com > https://mail.madstop.com/mailman/listinfo/puppet-users >_______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
On Mar 6, 2007, at 2:49 PM, Julian Simpson wrote:> I''d be interested in discussing this whole idea of testing > further: we haven''t even touched on functional testing yet - > ideally config would be tested on a small test system and then > tagged once that passed, for consumption by the rest of the puppet > clients. That way you could gain more confidence that your > versioned configurations were ready for prime time.I''ve been thinking about this some, and you''re going to want to decouple your tests, just like you do in code. At the least, you''d want two sets of tests for any given class or definition: One for making sure a given set of inputs provided the correct output -- i.e., it got compiled to the correct resource specifications. Then, for a given set of resource specifications, you actually get the correct behaviour. The first step, verifying that your code compiles correctly, is pretty straightforward, and I''d be glad to work with you getting the unit test framework coded up. The second is clearly far more difficult, though, and the only real way I see to do it is via installation on test hosts. -- Freedom of speech in Usenet means that when you shout ''Fire!'' in a crowded theatre, half the crowd stands up and shouts, ''Wrong theatre!'' --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
> > I''ve been thinking about this some, and you''re going to want to > decouple your tests, just like you do in code.Like unit tests vs. functional tests in a software project? I agree. At the least, you''d want two sets of tests for any given class or> definition: One for making sure a given set of inputs provided the > correct output -- i.e., it got compiled to the correct resource > specifications. Then, for a given set of resource specifications, > you actually get the correct behaviour.Behaviour. That''s a good word. I want to be able to specify behaviour, and fail if the tests if that behaviour isn''t found. The first step, verifying that your code compiles correctly, is> pretty straightforward, and I''d be glad to work with you getting the > unit test framework coded up. The second is clearly far more > difficult, though, and the only real way I see to do it is via > installation on test hosts.Ok. I''m also happy to focus on the first and mull over the second. Best Julian. _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
On Wed, 2007-03-07 at 20:53 +0000, Julian Simpson wrote:> Behaviour. That''s a good word. I want to be able to specify > behaviour, and fail if the tests if that behaviour isn''t found.If you don''t already know about it, check out RSpec - which is designed to help you express your tests in the form of behaviour specifications. http://rspec.rubyforge.org/ J -- "If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilisation." - Gerald Weinberg
On Mar 7, 2007, at 2:53 PM, Julian Simpson wrote:> Like unit tests vs. functional tests in a software project? I agree.Partially, but also things like testing that a function calls the right function, rather than verifying the whole functional stack. I''ve got too much code that tries to verify the whole stack, which means my tests take forever to run (and write) and I have to refactor them all when mid- and bottom-layer functions change.> Behaviour. That''s a good word. I want to be able to specify > behaviour, and fail if the tests if that behaviour isn''t found.That''s tough on both accounts -- you''ll need to figure out how to correctly specify behaviour, and then how to test that behaviour. Specification is pretty difficult by itself, but the testing will almost definitely require some kind of external monitoring feedback.> The first step, verifying that your code compiles correctly, is > pretty straightforward, and I''d be glad to work with you getting the > unit test framework coded up. The second is clearly far more > difficult, though, and the only real way I see to do it is via > installation on test hosts. > > Ok. I''m also happy to focus on the first and mull over the second.Yeah, I think it''s the best place to start. -- Don''t tell me how hard you work. Tell me how much you get done. -- James Ling --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Mar 7, 2007, at 3:09 PM, John Arundel wrote:> On Wed, 2007-03-07 at 20:53 +0000, Julian Simpson wrote: > >> Behaviour. That''s a good word. I want to be able to specify >> behaviour, and fail if the tests if that behaviour isn''t found. > > If you don''t already know about it, check out RSpec - which is > designed > to help you express your tests in the form of behaviour > specifications. > > http://rspec.rubyforge.org/Yeah, I keep meaning to start using this some, but it hasn''t seemed as hugely beneficial as I would like. I''m assuming I''ll get around to it one of these days, but I''m secretly hoping that someone else (*cough*mpalmer*cough*) will start using it and I can just take advantage of it. -- If computers get too powerful, we can organize them into a committee -- that will do them in. -- Bradley''s Bromide --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Wed, 2007-03-07 at 15:59 -0600, Luke Kanies wrote:> Yeah, I keep meaning to start using this some, but it hasn''t seemed > as hugely beneficial as I would like. I''m assuming I''ll get around > to it one of these days, but I''m secretly hoping that someone else > (*cough*mpalmer*cough*) will start using it and I can just take > advantage of it.I really like it. I just started using it recently and I''m finding it a much more natural and expressive way to write my tests - ''specification, not verification''. Instead of merely verifying that your code behaves correctly, you are specifying what it means for your code to behave correctly. RSpec''s grammar encourages you to write specifications in a straightforward, human-readable way. It''s quite possible for a non-programmer to look at a well-written set of specs and understand what the application is supposed to do. I hear people asking ''So what does RSpec do that Test::Unit doesn''t?'' It''s the wrong question (the answer is ''Nothing, except integrate support for some useful tools like RCov and Heckle''). The right question is ''Am I writing better tests with RSpec, and am I enjoying it more?'' (''Yes, and yes.'') This is a good article explaining the merits of BDD: http://blog.daveastels.com/files/BDD_Intro.pdf J -- "That''s the thing about people who think they hate computers. What they really hate is lousy programmers." - Larry Niven and Jerry Pournelle in "Oath of Fealty"
On Mar 7, 2007, at 4:26 PM, John Arundel wrote:> > I really like it. I just started using it recently and I''m finding > it a > much more natural and expressive way to write my tests - > ''specification, > not verification''. Instead of merely verifying that your code behaves > correctly, you are specifying what it means for your code to behave > correctly. RSpec''s grammar encourages you to write specifications in a > straightforward, human-readable way. It''s quite possible for a > non-programmer to look at a well-written set of specs and understand > what the application is supposed to do. > > I hear people asking ''So what does RSpec do that Test::Unit doesn''t?'' > It''s the wrong question (the answer is ''Nothing, except integrate > support for some useful tools like RCov and Heckle''). > > The right question is ''Am I writing better tests with RSpec, and am I > enjoying it more?'' (''Yes, and yes.'')I think the problem that I have with RSpec is that it comes very close to being what I want without actually being it, which makes me afraid I''ll end up rewriting some portion of it. Silly, but true. Specifically, I''d like spec contexts to be nestable, and they aren''t quite. That is, you can nest them, but they don''t behave correctly when you do. I expect I''ll start using it one of these days, but the last few times I started to, it was when testing something else, so I was biting off a bit more than I wanted to chew. -- A little government and a little luck are necessary in life, but only a fool trusts either of them. -- P. J. O''Rourke --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
Hi John, I had BDD in mind when I mentioned behaviour in my previous email. It struck me as I wrote the email that specifying behaviour for infrastructure might be a better fit than a testing metaphor. I haven''t really used RSpec much - would like to find out more. Best Julian. If you don''t already know about it, check out RSpec - which is designed> to help you express your tests in the form of behaviour specifications. > > http://rspec.rubyforge.org/ > > J > -- > "If builders built buildings the way programmers wrote programs, then > the first woodpecker that came along would destroy civilisation." > - Gerald Weinberg > _______________________________________________ > Puppet-users mailing list > Puppet-users@madstop.com > https://mail.madstop.com/mailman/listinfo/puppet-users >_______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
Hello again, Here''s a very short screencast of things in action: http://www.youtube.com/watch?v=I18kjq4lAgE I''m planning to show it as part of the presentation at UKUUG on Tuesday. Best Julian On 3/6/07, Thomas Lockney <tlockney@gmail.com> wrote:> > On 3/4/07, Julian Simpson <simpsonjulian@gmail.com> wrote: > > I''m presenting this concept in more detail at a Unix conference in the > UK > > this month, so I''d be interested in your feedback or questions: if you > > think it''s great or insane, and why. > > This is great. I could really use a setup like this in a couple > different places. I would love to hear more details. Are you preparing > a presentation on this? If so, would it be possible to post your > slides or any other artifacts you might be using for the presentation? > > Thanks, > Thomas > > -- > Thomas Lockney | tlockney@gmail.com | http://opposable-thumbs.net > _______________________________________________ > Puppet-users mailing list > Puppet-users@madstop.com > https://mail.madstop.com/mailman/listinfo/puppet-users >_______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday 18 March 2007 21:57, Julian Simpson wrote:> Hello again, > > Here's a very short screencast of things in action: > > http://www.youtube.com/watch?v=I18kjq4lAgE > > I'm planning to show it as part of the presentation at UKUUG on Tuesday.Very slick :) Since you didn't use it in your screencast, here a little tip to save typing and screen real-estate: you can put multiple resources of the same type together in one definition: package { [ puppet, debsums ]: ensure => installed; cfengine: ensure => absent; } Regards, David - -- - - hallo... wie gehts heute? - - *hust* gut *rotz* *keuch* - - gott sei dank kommunizieren wir über ein septisches medium ;) -- Matthias Leeb, Uni f. angewandte Kunst, 2005-02-15 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFF/bD8/Pp1N6Uzh0URAuWxAJ9vzs//mvljjGDaMJk/n3l4ejRDaQCglhwS cAZsRgxi1YXplmmrnTIkXA4=EeUA -----END PGP SIGNATURE----- _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
Hi David, Very slick :) Thank you! It''s my first screencast. :) Since you didn''t use it in your screencast, here a little tip to save typing> and screen real-estate: > > you can put multiple resources of the same type together in one > definition: > > package { > [ puppet, debsums ]: ensure => installed; > cfengine: ensure => absent; > }That''s brilliant! Thank you for the hint - I feel a refactoring coming along: ''egrep "file|package" *.pp'' brought up too many matches! Best, Julian _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users