Gerardo Santana Gómez Garrido
2013-Apr-16 22:31 UTC
[Puppet Users] [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
FWIW, I''ve wrote puppet-cleaner to help me make comply thousands of lines of puppet 2.6 DSL code to puppet 2.7 style guide and expectations. I''m uploading it to github today for anyone to use. https://github.com/santana/puppet-cleaner Externally, you run puppet-clean file.pp and it can transform this: /* multiline comment trailing white space here -> */ class someclass($version = "5", $platform = "rhel6") { if ! ($version in ["4", "5"] or !($platform in ["rhel5", "rhel6"])) { fail("Version $version on $platform is not supported") } else { notice("Defining class for version \"$version\" on ''$platform''") package { [ "package-1", "package-2", ]: } sysctl2::conf {"${something}": settings => [ "set key1 ''value1''", "set key2 ''value2''", ] } file { "/file/a": # test source => ''puppet:///someclass/file/a'', owner => ''root'', ensure => ensure_a, group=> "${group}"; "/file/b": source => ''puppet:///someclass/file/b'', mode=>755, ensure => ensure_b, owner => ''root'', group=> "${group}"; } file { "/file/c": source => ''puppet:///someclass/file/c'', owner => ''root'', ensure => ensure_c, group=> "${group}"; "/file/d": source => ''puppet:///someclass/file/d'', mode=>"755", owner => ''root'', group=> "${group}" } file { "/file/e": source => ''puppet:///someclass/file/c'', owner => ''root'', alias => ["$hostname"], group=> "${group}", ensure => ''absent''; "/file/f": ensure => ''ensure_f''; } user { "jdoe": managed_home => "true", } } } into this: # # multiline comment # trailing white space here -> class someclass($version = ''5'', $platform = ''rhel6'') { if ! ($version in [''4'', ''5''] or !($platform in [''rhel5'', ''rhel6''])) { fail("Version ${version} on ${platform} is not supported") } else { notice("Defining class for version \"${version}\" on ''${platform}''") package { [ ''package-1'', ''package-2'', ]: } sysctl2::conf {$something: settings => [ ''set key1 \''value1\'''', ''set key2 \''value2\'''', ] } file { ''/file/a'': ensure => link, # test source => ''puppet:///someclass/file/a'', owner => ''root'', target => ensure_a, group => $group; ''/file/b'': ensure => link, source => ''puppet:///someclass/file/b'', mode => ''0755'', target => ensure_b, owner => ''root'', group => $group; } file { ''/file/c'': ensure => link, source => ''puppet:///someclass/file/c'', owner => ''root'', target => ensure_c, group => $group; ''/file/d'': source => ''puppet:///someclass/file/d'', mode => ''0755'', owner => ''root'', group => $group } file { ''/file/e'': ensure => ''absent'', source => ''puppet:///someclass/file/c'', owner => ''root'', alias => [$hostname], group => $group; ''/file/f'': ensure => link, target => ''ensure_f''; } user { ''jdoe'': managed_home => true, } } } Internally, it monkey patches puppet 2.7.11 to take advantage of puppet''s tokenizer and iterates over them like in a production line, where each transformation algorithm (worker) awaits for the pattern of tokens it knows how to transform. I haven''t had the time to refactor some stuff; feel free to ignore it, improve it, send me patches, etc. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Gerardo Santana Gómez Garrido
2013-Apr-18 16:52 UTC
[Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
A small follow up: the gem has just been pushed to rubygems, and you can read more about the motivation for this at http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html El martes, 16 de abril de 2013 17:31:38 UTC-5, Gerardo Santana Gómez Garrido escribió:> > FWIW, > > I''ve wrote puppet-cleaner to help me make comply thousands of lines of > puppet 2.6 DSL code to puppet 2.7 style guide and expectations. I''m > uploading it to github today for anyone to use. > > https://github.com/santana/puppet-cleaner > > Externally, you run puppet-clean file.pp and it can transform this: > > /* > multiline comment > trailing white space here -> > */ > > class someclass($version = "5", $platform = "rhel6") > { > if ! ($version in ["4", "5"] or !($platform in ["rhel5", "rhel6"])) { > fail("Version $version on $platform is not supported") > } > else { > notice("Defining class for version \"$version\" on ''$platform''") > package { > [ > "package-1", > "package-2", > ]: > } > > sysctl2::conf {"${something}": > settings => [ > "set key1 ''value1''", > "set key2 ''value2''", > ] > } > file { > "/file/a": # test > source => ''puppet:///someclass/file/a'', > owner => ''root'', > ensure => ensure_a, > group=> "${group}"; > "/file/b": > source => ''puppet:///someclass/file/b'', > mode=>755, > ensure => ensure_b, > owner => ''root'', > group=> "${group}"; > } > > file { > "/file/c": > source => ''puppet:///someclass/file/c'', > owner => ''root'', > ensure => ensure_c, > group=> "${group}"; > "/file/d": > source => ''puppet:///someclass/file/d'', > mode=>"755", > owner => ''root'', > group=> "${group}" > } > file { > "/file/e": > source => ''puppet:///someclass/file/c'', > owner => ''root'', > alias => ["$hostname"], > group=> "${group}", > ensure => ''absent''; > "/file/f": > ensure => ''ensure_f''; > } > > user { > "jdoe": > managed_home => "true", > } > } > } > > into this: > > # > # multiline comment > # trailing white space here -> > > > class someclass($version = ''5'', $platform = ''rhel6'') > { > if ! ($version in [''4'', ''5''] or !($platform in [''rhel5'', ''rhel6''])) { > fail("Version ${version} on ${platform} is not supported") > } > else { > notice("Defining class for version \"${version}\" on ''${platform}''") > package { > [ > ''package-1'', > ''package-2'', > ]: > } > > sysctl2::conf {$something: > settings => [ > ''set key1 \''value1\'''', > ''set key2 \''value2\'''', > ] > } > file { > ''/file/a'': > ensure => link, # test > source => ''puppet:///someclass/file/a'', > owner => ''root'', > target => ensure_a, > group => $group; > ''/file/b'': > ensure => link, > source => ''puppet:///someclass/file/b'', > mode => ''0755'', > target => ensure_b, > owner => ''root'', > group => $group; > } > > file { > ''/file/c'': > ensure => link, > source => ''puppet:///someclass/file/c'', > owner => ''root'', > target => ensure_c, > group => $group; > ''/file/d'': > source => ''puppet:///someclass/file/d'', > mode => ''0755'', > owner => ''root'', > group => $group > } > file { > ''/file/e'': > ensure => ''absent'', > source => ''puppet:///someclass/file/c'', > owner => ''root'', > alias => [$hostname], > group => $group; > ''/file/f'': > ensure => link, > target => ''ensure_f''; > } > > user { > ''jdoe'': > managed_home => true, > } > } > } > > Internally, it monkey patches puppet 2.7.11 to take advantage of puppet''s > tokenizer and iterates over them like in a production line, where each > transformation algorithm (worker) awaits for the pattern of tokens it knows > how to transform. > > I haven''t had the time to refactor some stuff; feel free to ignore it, > improve it, send me patches, etc. > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Mohit Chawla
2013-Apr-18 17:08 UTC
Re: [Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
Hello, On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido < gerardo.santana@gmail.com> wrote:> A small follow up: the gem has just been pushed to rubygems, and you can > read more about the motivation for this at > http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html > >This is nice, gotta check it out, thanks for writing it. Just a note though, there was some recent work in puppet-lint ( http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint) to autofix simple issues too. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Dan White
2013-Apr-18 17:33 UTC
Re: [Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
I just tried it and the "fix" option does not. [root manifests]# puppet-lint -v Puppet-lint 0.4.0.pre1 [root manifests]# puppet-lint --fix init.pp WARNING: double quoted string containing no variables on line 21 The example clearly shows this warning cause getting corrected “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.” Bill Waterson (Calvin & Hobbes) ----- Original Message ----- From: "Mohit Chawla" <mohit.chawla.binary@gmail.com> To: puppet-users@googlegroups.com Sent: Thursday, April 18, 2013 1:08:25 PM Subject: Re: [Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide Hello, On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido < gerardo.santana@gmail.com > wrote: A small follow up: the gem has just been pushed to rubygems, and you can read more about the motivation for this at http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html This is nice, gotta check it out, thanks for writing it. Just a note though, there was some recent work in puppet-lint ( http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint ) to autofix simple issues too. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en . For more options, visit https://groups.google.com/groups/opt_out . -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Gerardo Santana Gómez Garrido
2013-Apr-18 17:45 UTC
Re: [Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
Hi Alcy, that''s nice! I used puppet-lint before writing puppet-cleaner and was very useful at the time. Thanks to the author for writing it. puppet-lint has an issue though that may make it more difficult to implement those corrections: it uses regular expressions to tokenize the source code. This was my first approach too, but it''s limiting. Having puppet''s own tokenizer on your side is not just more helpful but exact, there''s no chance for errors at tokenizing the code, it''s faster, and you gain more introspection because you have the whole source code tokenized. El jueves, 18 de abril de 2013 12:08:25 UTC-5, alcy escribió:> > Hello, > > On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido < > gerardo...@gmail.com <javascript:>> wrote: > >> A small follow up: the gem has just been pushed to rubygems, and you can >> read more about the motivation for this at >> http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html >> >> > > This is nice, gotta check it out, thanks for writing it. Just a note > though, there was some recent work in puppet-lint ( > http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint) > to autofix simple issues too. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Tim Sharpe
2013-Apr-20 10:08 UTC
Re: [Puppet Users] [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
Hi Dan, Could you please file a bug report at https://github.com/rodjek/puppet-lint including the manifest you used below? Thanks, Tim On 19/04/2013, at 3:33 AM, Dan White <ygor@comcast.net> wrote:> I just tried it and the "fix" option does not. > > [root manifests]# puppet-lint -v > Puppet-lint 0.4.0.pre1 > > [root manifests]# puppet-lint --fix init.pp > WARNING: double quoted string containing no variables on line 21 > > The example clearly shows this warning cause getting corrected > > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.” > Bill Waterson (Calvin & Hobbes) > > ----- Original Message ----- > From: "Mohit Chawla" <mohit.chawla.binary@gmail.com> > To: puppet-users@googlegroups.com > Sent: Thursday, April 18, 2013 1:08:25 PM > Subject: Re: [Puppet Users] Re: [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide > > Hello, > > > On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido < gerardo.santana@gmail.com > wrote: > > > A small follow up: the gem has just been pushed to rubygems, and you can read more about the motivation for this at http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html > > > > This is nice, gotta check it out, thanks for writing it. Just a note though, there was some recent work in puppet-lint ( http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint ) to autofix simple issues too. > > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. > To post to this group, send email to puppet-users@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-users?hl=en . > For more options, visit https://groups.google.com/groups/opt_out . > > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. > To post to this group, send email to puppet-users@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Tim Sharpe
2013-Apr-20 10:22 UTC
Re: [Puppet Users] [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
Hi Gerardo, Nice project! A couple of things though, from someone who''s been down this path before: Developing against a moving target (i.e. the Puppet tokeniser) will end up causing you a non-trivial amount of pain. The point where I decided to roll my own tokeniser rather than use Puppet''s (puppet-lint 0.2.x) was when variables containing dashes was "fixed", requiring me to add yet another monkeypatch to Puppet''s tokeniser to reimplement the old behaviour, so that it could be detected. If you test puppet-cleaner against Puppet 0.25.x through 3.1.x, you''ll see what I mean :) Puppet''s tokeniser and puppet-lint''s tokeniser are actually very similar and work in much the same way (both are regex based and tokenise the entire file). Puppet-lint''s adds some additional tokens that Puppet''s does not need to concern itself with (formatting related whitespace mostly). Cheers, Tim On 19/04/2013, at 3:45 AM, Gerardo Santana Gómez Garrido <gerardo.santana@gmail.com> wrote:> Hi Alcy, > > that''s nice! I used puppet-lint before writing puppet-cleaner and was very useful at the time. Thanks to the author for writing it. > > puppet-lint has an issue though that may make it more difficult to implement those corrections: it uses regular expressions to tokenize the source code. This was my first approach too, but it''s limiting. > > Having puppet''s own tokenizer on your side is not just more helpful but exact, there''s no chance for errors at tokenizing the code, it''s faster, and you gain more introspection because you have the whole source code tokenized. > > El jueves, 18 de abril de 2013 12:08:25 UTC-5, alcy escribió: > Hello, > > On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido <gerardo...@gmail.com> wrote: > A small follow up: the gem has just been pushed to rubygems, and you can read more about the motivation for this at http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html > > > > This is nice, gotta check it out, thanks for writing it. Just a note though, there was some recent work in puppet-lint (http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint) to autofix simple issues too. > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. > To post to this group, send email to puppet-users@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Gerardo Santana Gómez Garrido
2013-Apr-22 18:55 UTC
Re: [Puppet Users] [ANN] puppet-cleaner: makes puppet DSL code comply with a subset of the style guide
Thanks for sharing your thoughts! I see your point and agree. I''ve just e-mailed the developer''s mailing list about this. I hope to get green light on making the necessary adaptations to the lexer so anybody can reuse or extend it safely, as the OO Gods intended ;) Regards. El sábado, 20 de abril de 2013 05:22:14 UTC-5, Tim Sharpe escribió:> > Hi Gerardo, > > Nice project! > > A couple of things though, from someone who''s been down this path before: > > Developing against a moving target (i.e. the Puppet tokeniser) will end up > causing you a non-trivial amount of pain. The point where I decided to roll > my own tokeniser rather than use Puppet''s (puppet-lint 0.2.x) was when > variables containing dashes was "fixed", requiring me to add yet another > monkeypatch to Puppet''s tokeniser to reimplement the old behaviour, so that > it could be detected. If you test puppet-cleaner against Puppet 0.25.x > through 3.1.x, you''ll see what I mean :) > > Puppet''s tokeniser and puppet-lint''s tokeniser are actually very similar > and work in much the same way (both are regex based and tokenise the entire > file). Puppet-lint''s adds some additional tokens that Puppet''s does not > need to concern itself with (formatting related whitespace mostly). > > Cheers, > Tim > > On 19/04/2013, at 3:45 AM, Gerardo Santana Gómez Garrido < > gerardo...@gmail.com <javascript:>> wrote: > > Hi Alcy, > > that''s nice! I used puppet-lint before writing puppet-cleaner and was very > useful at the time. Thanks to the author for writing it. > > puppet-lint has an issue though that may make it more difficult to > implement those corrections: it uses regular expressions to tokenize the > source code. This was my first approach too, but it''s limiting. > > Having puppet''s own tokenizer on your side is not just more helpful but > exact, there''s no chance for errors at tokenizing the code, it''s faster, > and you gain more introspection because you have the whole source code > tokenized. > > El jueves, 18 de abril de 2013 12:08:25 UTC-5, alcy escribió: >> >> Hello, >> >> On Thu, Apr 18, 2013 at 10:22 PM, Gerardo Santana Gómez Garrido < >> gerardo...@gmail.com> wrote: >> >>> A small follow up: the gem has just been pushed to rubygems, and you can >>> read more about the motivation for this at >>> http://santanatechnotes.blogspot.mx/2013/04/puppet-cleaner-010-released.html >>> >>> >> >> This is nice, gotta check it out, thanks for writing it. Just a note >> though, there was some recent work in puppet-lint ( >> http://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint) >> to autofix simple issues too. >> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users...@googlegroups.com <javascript:>. > To post to this group, send email to puppet...@googlegroups.com<javascript:> > . > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.