Trevor Vaughan
2011-Dec-28 14:19 UTC
[Puppet Users] How are people regression testing their code for variable scoping?
I think I''ve found all of the cases, but I''d like a definitive way to be sure without trying to call every piece of code that I''ve written. Thanks, Trevor -- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 tvaughan@onyxpoint.com -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Trevor Vaughan
2012-Jan-03 17:15 UTC
[Puppet Users] Re: How are people regression testing their code for variable scoping?
Any takers? On Wed, Dec 28, 2011 at 9:19 AM, Trevor Vaughan <tvaughan@onyxpoint.com> wrote:> I think I''ve found all of the cases, but I''d like a definitive way to > be sure without trying to call every piece of code that I''ve written. > > Thanks, > > Trevor > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > tvaughan@onyxpoint.com > > -- This account not approved for unencrypted proprietary information ---- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 tvaughan@onyxpoint.com -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Felix Frank
2012-Jan-05 13:27 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
On 01/03/2012 06:15 PM, Trevor Vaughan wrote:> Any takers?Please elaborate on the question. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Henrik Lindberg
2012-Jan-05 14:09 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
Geppetto and puppet-lint are two useful tools for checking for problems. Hope that helps. Regards - henrik On 2012-03-01 18:15, Trevor Vaughan wrote:> Any takers? > > On Wed, Dec 28, 2011 at 9:19 AM, Trevor Vaughan<tvaughan@onyxpoint.com> wrote: >> I think I''ve found all of the cases, but I''d like a definitive way to >> be sure without trying to call every piece of code that I''ve written. >> >> Thanks, >> >> Trevor >> >> -- >> Trevor Vaughan >> Vice President, Onyx Point, Inc >> (410) 541-6699 >> tvaughan@onyxpoint.com >> >> -- This account not approved for unencrypted proprietary information -- > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Trevor Vaughan
2012-Jan-06 13:31 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ah, So, if you have an existing code base that you''ve inherited and you''re going to be trying the 2.8 series of Puppet, where variable scoping is mandatory, how would you go about finding all out of scope variables without actually writing code to call every portion of the puppet code. I''ve used puppetdoc to find issues in the past but it doesn''t seem to care about variable scope from what I can tell. I''ll try puppet-lint and see if it picks up on the issues but I''m suspecting that it won''t since it''s probably not compiling the code. Trevor On 01/05/2012 08:27 AM, Felix Frank wrote:> On 01/03/2012 06:15 PM, Trevor Vaughan wrote: >> Any takers? > > Please elaborate on the question. >- -- Trevor Vaughan Vice President, Onyx Point, Inc. email: tvaughan@onyxpoint.com phone: 410-541-ONYX (6699) pgp: 0x6C701E94 - -- This account not approved for unencrypted sensitive information -- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJPBvewAAoJECNCGV1OLcypG84H/1U/Ythc0aK9Ie6RTCT1LYkL ZUAQP+3NVobFovevE99P1J90suyZPn5dp9KPF/eW4GPiwoJ+oz0Zx5CdbH9untIX EZ+XlMIxS0y8mAUmwQzBoLu8ca7BjUTJeQBZBweiwxDwvzhnlzTsYPiGSs7iL85w PQS26DFP+LHJCRZh/kKi2LC3mG/WfGvI5CeK1CXJp9Ieetq2lMafFLztZRoOq1/W 5pHRHmCdvj7TiwGtvjdPbUsuimhF+FHgZu+X+QXf6kxSdIcwnEMVUJmHv/v4BfT5 mkAT4r6PHhK/WcRW3toE/Lt4vW/Jqga7LTg9zLp0NfS/tL7/O7Eax9wDLp2e54g=V56a -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
R.I.Pienaar
2012-Jan-06 13:40 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
----- Original Message -----> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ah, > > So, if you have an existing code base that you''ve inherited and > you''re going to be trying the 2.8 series of Puppet, where variable > scoping is > mandatory, how would you go about finding all out of scope variables > without actually writing code to call every portion of the puppet > code. > > I''ve used puppetdoc to find issues in the past but it doesn''t seem to > care about variable scope from what I can tell. > > I''ll try puppet-lint and see if it picks up on the issues but I''m > suspecting that it won''t since it''s probably not compiling the code.The only sane way imho is to diff the catalogs. The catalogs are based on the variables, vars are interpolated into strings in the catalog and anything like versions, names etc that came from variables would be in the catalog already parsed. So if you diff''d a 2.6 and 2.8 catalog you would know if variable changes affected you. I did some extremely hacky thing to diff catalogs in the past[1] but I have no idea if this still works with 2.7 or 2.8 but I think this is the only real way to tell if you will have changes due to master changes. Would be nice if there was a migration tool written by someone who actually know the catalog internals :) Of course this does not tell you if the agent will interpret the catalog correctly... [1] https://github.com/ripienaar/puppet-catalog-diff> > Trevor > > On 01/05/2012 08:27 AM, Felix Frank wrote: > > On 01/03/2012 06:15 PM, Trevor Vaughan wrote: > >> Any takers? > > > > Please elaborate on the question. > > > > - -- > Trevor Vaughan > Vice President, Onyx Point, Inc. > email: tvaughan@onyxpoint.com > phone: 410-541-ONYX (6699) > pgp: 0x6C701E94 > > - -- This account not approved for unencrypted sensitive information > -- > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEcBAEBAgAGBQJPBvewAAoJECNCGV1OLcypG84H/1U/Ythc0aK9Ie6RTCT1LYkL > ZUAQP+3NVobFovevE99P1J90suyZPn5dp9KPF/eW4GPiwoJ+oz0Zx5CdbH9untIX > EZ+XlMIxS0y8mAUmwQzBoLu8ca7BjUTJeQBZBweiwxDwvzhnlzTsYPiGSs7iL85w > PQS26DFP+LHJCRZh/kKi2LC3mG/WfGvI5CeK1CXJp9Ieetq2lMafFLztZRoOq1/W > 5pHRHmCdvj7TiwGtvjdPbUsuimhF+FHgZu+X+QXf6kxSdIcwnEMVUJmHv/v4BfT5 > mkAT4r6PHhK/WcRW3toE/Lt4vW/Jqga7LTg9zLp0NfS/tL7/O7Eax9wDLp2e54g> =V56a > -----END PGP SIGNATURE----- > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- R.I.Pienaar -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Trevor Vaughan
2012-Jan-06 13:52 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for the response. puppet-lint certainly finds the manifest-based issues but doesn''t seem to pick up issues in the templates that I can tell so far. Any suggestions from the Puppet Labs'' guys on automated techniques that pick up items in templates, functions, and custom types would be most appreciated since, well, they''re the only ones I can think of that''ll know how to do that. Thanks, Trevor On 01/06/2012 08:40 AM, R.I.Pienaar wrote:> > > ----- Original Message ----- > Ah, > > So, if you have an existing code base that you''ve inherited and > you''re going to be trying the 2.8 series of Puppet, where variable > scoping is > mandatory, how would you go about finding all out of scope variables > without actually writing code to call every portion of the puppet > code. > > I''ve used puppetdoc to find issues in the past but it doesn''t seem to > care about variable scope from what I can tell. > > I''ll try puppet-lint and see if it picks up on the issues but I''m > suspecting that it won''t since it''s probably not compiling the code. > > >> The only sane way imho is to diff the catalogs. > >> The catalogs are based on the variables, vars are interpolated into >> strings in the catalog and anything like versions, names etc that came >> from variables would be in the catalog already parsed. > >> So if you diff''d a 2.6 and 2.8 catalog you would know if variable changes >> affected you. > >> I did some extremely hacky thing to diff catalogs in the past[1] but I have >> no idea if this still works with 2.7 or 2.8 but I think this is the only >> real way to tell if you will have changes due to master changes. Would be >> nice if there was a migration tool written by someone who actually know >> the catalog internals :) > >> Of course this does not tell you if the agent will interpret the catalog >> correctly... > >> [1] https://github.com/ripienaar/puppet-catalog-diff > > > Trevor > > On 01/05/2012 08:27 AM, Felix Frank wrote: >>>> On 01/03/2012 06:15 PM, Trevor Vaughan wrote: >>>>> Any takers? >>>> >>>> Please elaborate on the question. >>>> > >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> >> >- -- Trevor Vaughan Vice President, Onyx Point, Inc. email: tvaughan@onyxpoint.com phone: 410-541-ONYX (6699) pgp: 0x6C701E94 - -- This account not approved for unencrypted sensitive information -- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJPBvyfAAoJECNCGV1OLcyp4IAH+wYxC34BUnI54Ie9E4GbCD0C hPkJwtFBf0isKqv9FHstKrZ7N8v8duPhVXbL9JbjL4VBlVkHHz1x20OAUcoGGa76 oVu01eHkPBRrTCPvEpXZMC/DkZaz97GKq4e/5XxNgsEwfPijht+jIiCpo3i3JHV0 qra/VFjW0kwVFSCNlT5CSN8OF9j2TH+qIFZfDcOfBtV5LydiQkr0i5EMgQF7cOrf pk303NzDL76i9hEZI0faqZxosn/5pB92joEGwp5SNTqPL7kuUCuk6GGl6UtpoyEQ NHmHS7MvpEjKCq7b0entwqaGORwebooI5oTroHROUpThPtOeF4WbKrQRDvzEIKw=bY+t -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Felix Frank
2012-Jan-06 13:54 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
On 01/06/2012 02:40 PM, R.I.Pienaar wrote:> The only sane way imho is to diff the catalogs.Maybe, but a different (not necessarily claiming "sane") choice is to 1. run puppetmaster 2.8 on a non-standard port 2. have each agent perform a --noop run using this masterport 3. collect reports from the agents, see if they would erase your SSH keys etc. That''s basically what I like to do whenever upgrading my master. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
R.I.Pienaar
2012-Jan-06 14:29 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
----- Original Message -----> On 01/06/2012 02:40 PM, R.I.Pienaar wrote: > > The only sane way imho is to diff the catalogs. > > Maybe, but a different (not necessarily claiming "sane") choice is to > 1. run puppetmaster 2.8 on a non-standard port > 2. have each agent perform a --noop run using this masterport > 3. collect reports from the agents, see if they would erase your SSH > keys etc. > > That''s basically what I like to do whenever upgrading my master.yes, you''d then also need to do a detailed audit of the reports to make sure that no resources are suddenly unmanaged - imagine: if $foo { # manage package bar } the behavior of that simple if changed between major versions of puppet some time ago and the report based approach wouldnt pick that up unless you very carefully audit and compare every reported resource. When I wrote the differ the puppet reports didnt have the level of detail required to do the audit purely on reports they do now, so if you were going between 2 versions of puppet that supported the detailed reporting this would indeed be a better approach. Still though - relationships can change, be missing etc and those owuldnt be in reports either. It''s basically just extremely complex and I dont think there''s a truly right answer but there should be one if the 2.7.x series is anything to go by thanks to the ever changing landscape of features and behaviors -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Walter Heck
2012-Jan-06 15:17 UTC
Re: [Puppet Users] Re: How are people regression testing their code for variable scoping?
You could also use rip''s excellent parselocalconfig script to see if the same resources will be applied. Then you''d combine that with other more in-depth tests: http://www.devco.net/code/parselocalconfig.rb cheers, Walter On Fri, Jan 6, 2012 at 15:29, R.I.Pienaar <rip@devco.net> wrote:> > > ----- Original Message ----- >> On 01/06/2012 02:40 PM, R.I.Pienaar wrote: >> > The only sane way imho is to diff the catalogs. >> >> Maybe, but a different (not necessarily claiming "sane") choice is to >> 1. run puppetmaster 2.8 on a non-standard port >> 2. have each agent perform a --noop run using this masterport >> 3. collect reports from the agents, see if they would erase your SSH >> keys etc. >> >> That''s basically what I like to do whenever upgrading my master. > > yes, you''d then also need to do a detailed audit of the reports to make > sure that no resources are suddenly unmanaged - imagine: > > if $foo { > # manage package bar > } > > the behavior of that simple if changed between major versions of puppet > some time ago and the report based approach wouldnt pick that up unless > you very carefully audit and compare every reported resource. > > When I wrote the differ the puppet reports didnt have the level of detail > required to do the audit purely on reports they do now, so if you were going > between 2 versions of puppet that supported the detailed reporting this would > indeed be a better approach. > > Still though - relationships can change, be missing etc and those owuldnt > be in reports either. > > It''s basically just extremely complex and I dont think there''s a truly right > answer but there should be one if the 2.7.x series is anything to go by thanks > to the ever changing landscape of features and behaviors > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- Walter Heck -- follow @walterheck on twitter to see what I''m up to! -- Check out my new startup: Server Monitoring as a Service @ http://tribily.com Follow @tribily on Twitter and/or ''Like'' our Facebook page at http://www.facebook.com/tribily -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.