Anyone know if there''s a plan to allow definitions to be used in external nodes? Not having that ability is a major pain in the ass. It really means that all that really works with external nodes is really simple cases of a single piece of software being installed and configured, like apache for example. I have a number of web packages that I need to install on a system. Using external nodes, I have to explicitly set everything as a variable, and then have a specialised class to handle that, which means that each time I add a new customer, I have to add a new class. Uhm... new logic to handle new customers.... not very scalable. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Aug 26, 2010, at 12:58 PM, Douglas Garstang wrote:> Anyone know if there''s a plan to allow definitions to be used in > external nodes? Not having that ability is a major pain in the ass. It > really means that all that really works with external nodes is really > simple cases of a single piece of software being installed and > configured, like apache for example.I''ve used external nodes all along and never found this to be a limitation. Classes can do much much more than one thing, so I''m not sure what you mean. External nodes can have classes and variables. Classes can call definitions. Facts and variables can be passed to those definitions. What additional functionality would you get? -- Rob McBroom <http://www.skurfer.com/> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Thu, Aug 26, 2010 at 11:17 AM, Rob McBroom <mailinglist0@skurfer.com> wrote:> On Aug 26, 2010, at 12:58 PM, Douglas Garstang wrote: > >> Anyone know if there''s a plan to allow definitions to be used in >> external nodes? Not having that ability is a major pain in the ass. It >> really means that all that really works with external nodes is really >> simple cases of a single piece of software being installed and >> configured, like apache for example. > > I''ve used external nodes all along and never found this to be a limitation. Classes can do much much more than one thing, so I''m not sure what you mean. > > External nodes can have classes and variables. Classes can call definitions. Facts and variables can be passed to those definitions. > > What additional functionality would you get?I mean that you need to create a new class for each customer you add. That does not scale. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang <doug.garstang@gmail.com> wrote:> On Thu, Aug 26, 2010 at 11:17 AM, Rob McBroom <mailinglist0@skurfer.com> wrote: >> On Aug 26, 2010, at 12:58 PM, Douglas Garstang wrote: >> >>> Anyone know if there''s a plan to allow definitions to be used in >>> external nodes? Not having that ability is a major pain in the ass. It >>> really means that all that really works with external nodes is really >>> simple cases of a single piece of software being installed and >>> configured, like apache for example. >> >> I''ve used external nodes all along and never found this to be a limitation. Classes can do much much more than one thing, so I''m not sure what you mean. >> >> External nodes can have classes and variables. Classes can call definitions. Facts and variables can be passed to those definitions. >> >> What additional functionality would you get? > > I mean that you need to create a new class for each customer you add. > That does not scale.Why can''t you have a class that instantiates your defined type?> > Doug. > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten wrote:> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang > <doug.garstang@gmail.com> wrote:>> I mean that you need to create a new class for each customer you add. >> That does not scale. > > Why can''t you have a class that instantiates your defined type?I don''t know Douglas'' circumstances, but assume you have a web hotel. You then probably have a database of your customers, telling you things like which domain(s) they have, what features they pay for (static pages only, CGI scripts, mod_python, database access, and so on), and how much storage and bandwidth they are paying for. To map all these customers onto your physical machines, you write a Puppet define that is used something like this: virtual_web_host { "www.example.com": avg_bandwidth => 4096, # Mbyte/month peak_bandwidth => 2000, # Kbit/s storage => 1536, # Mbyte disk space cgi => true, mod_python => false, postgresql => true; } Since you have your customers in a database, it would be nice to use an external nodes script to basically generate one such resource for each customer domain, and it will probably use some information in the database to balance the virtual hosts over the physical machines. But external nodes scripts *can''t* generate resources. You of course don''t want to write a new class for each customer, because if you are a largish web hotel you may get dozens of new customers per day (and possibly loose as many), so that would be too much work. But external node classifiers *can''t* generate resources. They can only use resources that already exist in the manifests, and affect them by setting variables. There is a way to work around this. You can let the external node classifier set a variable with a value like this: $vhosts = [ "dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...", "dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...", "dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...", ... ] (although the script needs to output it using YAML syntax, of course), and tell Puppet to include a class "all-vhosts". The manifest files would contain code looking something like this: class all-vhosts { ext_virtual_web_host { $vhosts: ; } } define ext_virtual_web_host() { $domain = extract_field(''dom'', $name) $avgbw = extract_field(''avgbw'', $name) $peakbw = extract_field(''peakbw'', $name) $storage = extract_field(''storage'', $name) $cgi = extract_field(''cgi'', $name) $mod_python = extract_field(''mod_python'', $name) $postgresql = extract_field(''postgresql'', $name) virtual_web_host { $domain: avg_bandwidth => $avgbw, peak_bandwidth => $peakbw, storage => $storage, cgi => $cgi, mod_python => $mod_python, postgresql => $postgresql; } } And then a custom function extract_field() for parsing the strings. Doable, but not very pretty... Those strings are of course a kind of dictionary/hash/mapping, and Puppet 2.6 has support for dictionaries directly. But that would only help you if 2.6 allows you to have a dictionary as the title/name for a resource; I haven''t checked, but I would guess that you can only use strings for titles/names. It would be much easier if external node classifiers could define resources directly, instead of having to do something like the above. But I seem to remember Luke saying that they are planning to add that functionality sometime in the future; I don''t think they have set a timeline for it yet, though -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sat, Aug 28, 2010 at 3:49 PM, Thomas Bellman <bellman@nsc.liu.se> wrote:> Nigel Kersten wrote: > >> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang >> <doug.garstang@gmail.com> wrote: > >>> I mean that you need to create a new class for each customer you add. >>> That does not scale. >> >> Why can''t you have a class that instantiates your defined type? > > I don''t know Douglas'' circumstances, but assume you have a web > hotel. You then probably have a database of your customers, > telling you things like which domain(s) they have, what features > they pay for (static pages only, CGI scripts, mod_python, > database access, and so on), and how much storage and bandwidth > they are paying for. > > To map all these customers onto your physical machines, you write > a Puppet define that is used something like this: > > virtual_web_host { > "www.example.com": > avg_bandwidth => 4096, # Mbyte/month > peak_bandwidth => 2000, # Kbit/s > storage => 1536, # Mbyte disk space > cgi => true, > mod_python => false, > postgresql => true; > } > > Since you have your customers in a database, it would be nice to > use an external nodes script to basically generate one such > resource for each customer domain, and it will probably use some > information in the database to balance the virtual hosts over the > physical machines. But external nodes scripts *can''t* generate > resources. You of course don''t want to write a new class for > each customer, because if you are a largish web hotel you may get > dozens of new customers per day (and possibly loose as many), so > that would be too much work. > > But external node classifiers *can''t* generate resources. They > can only use resources that already exist in the manifests, and > affect them by setting variables. > > > There is a way to work around this. You can let the external > node classifier set a variable with a value like this: > > $vhosts = [ > > "dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...", > > "dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...", > "dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...", > ... > ] > > (although the script needs to output it using YAML syntax, of > course), and tell Puppet to include a class "all-vhosts". The > manifest files would contain code looking something like this: > > class all-vhosts > { > ext_virtual_web_host { > $vhosts: > ; > } > } > > define ext_virtual_web_host() > { > $domain = extract_field(''dom'', $name) > $avgbw = extract_field(''avgbw'', $name) > $peakbw = extract_field(''peakbw'', $name) > $storage = extract_field(''storage'', $name) > $cgi = extract_field(''cgi'', $name) > $mod_python = extract_field(''mod_python'', $name) > $postgresql = extract_field(''postgresql'', $name) > > virtual_web_host { > $domain: > avg_bandwidth => $avgbw, > peak_bandwidth => $peakbw, > storage => $storage, > cgi => $cgi, > mod_python => $mod_python, > postgresql => $postgresql; > } > } > > And then a custom function extract_field() for parsing the > strings. Doable, but not very pretty... > > Those strings are of course a kind of dictionary/hash/mapping, > and Puppet 2.6 has support for dictionaries directly. But that > would only help you if 2.6 allows you to have a dictionary as the > title/name for a resource; I haven''t checked, but I would guess > that you can only use strings for titles/names. > > It would be much easier if external node classifiers could define > resources directly, instead of having to do something like the > above. But I seem to remember Luke saying that they are planning > to add that functionality sometime in the future; I don''t think > they have set a timeline for it yet, thoughThomas, thanks for the detailed description. Nigel, Thomas pretty much nailed the issue on the head. It''s not practical to add a new class every time you add a customer. The extract_field customer function doesn''t exist yet, and I''m not a ruby programmer, so it won''t exist any time soon. Even if it did, it''s an ugly hack. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Thu, Aug 26, 2010 at 9:58 AM, Douglas Garstang <doug.garstang@gmail.com> wrote:> Anyone know if there''s a plan to allow definitions to be used in > external nodes? Not having that ability is a major pain in the ass. It > really means that all that really works with external nodes is really > simple cases of a single piece of software being installed and > configured, like apache for example.This is feature [#2408]. Please watch the ticket and paste your use cases there. http://projects.puppetlabs.com/issues/2408 Thanks, -Jeff -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sat, Aug 28, 2010 at 4:48 PM, Douglas Garstang <doug.garstang@gmail.com> wrote:> On Sat, Aug 28, 2010 at 3:49 PM, Thomas Bellman <bellman@nsc.liu.se> wrote: >> Nigel Kersten wrote: >> >>> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang >>> <doug.garstang@gmail.com> wrote: >> >>>> I mean that you need to create a new class for each customer you add. >>>> That does not scale. >>> >>> Why can''t you have a class that instantiates your defined type? >> >> I don''t know Douglas'' circumstances, but assume you have a web >> hotel. You then probably have a database of your customers, >> telling you things like which domain(s) they have, what features >> they pay for (static pages only, CGI scripts, mod_python, >> database access, and so on), and how much storage and bandwidth >> they are paying for. >> >> To map all these customers onto your physical machines, you write >> a Puppet define that is used something like this: >> >> virtual_web_host { >> "www.example.com": >> avg_bandwidth => 4096, # Mbyte/month >> peak_bandwidth => 2000, # Kbit/s >> storage => 1536, # Mbyte disk space >> cgi => true, >> mod_python => false, >> postgresql => true; >> } >> >> Since you have your customers in a database, it would be nice to >> use an external nodes script to basically generate one such >> resource for each customer domain, and it will probably use some >> information in the database to balance the virtual hosts over the >> physical machines. But external nodes scripts *can''t* generate >> resources. You of course don''t want to write a new class for >> each customer, because if you are a largish web hotel you may get >> dozens of new customers per day (and possibly loose as many), so >> that would be too much work. >> >> But external node classifiers *can''t* generate resources. They >> can only use resources that already exist in the manifests, and >> affect them by setting variables. >> >> >> There is a way to work around this. You can let the external >> node classifier set a variable with a value like this: >> >> $vhosts = [ >> >> "dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...", >> >> "dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...", >> "dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...", >> ... >> ] >> >> (although the script needs to output it using YAML syntax, of >> course), and tell Puppet to include a class "all-vhosts". The >> manifest files would contain code looking something like this: >> >> class all-vhosts >> { >> ext_virtual_web_host { >> $vhosts: >> ; >> } >> } >> >> define ext_virtual_web_host() >> { >> $domain = extract_field(''dom'', $name) >> $avgbw = extract_field(''avgbw'', $name) >> $peakbw = extract_field(''peakbw'', $name) >> $storage = extract_field(''storage'', $name) >> $cgi = extract_field(''cgi'', $name) >> $mod_python = extract_field(''mod_python'', $name) >> $postgresql = extract_field(''postgresql'', $name) >> >> virtual_web_host { >> $domain: >> avg_bandwidth => $avgbw, >> peak_bandwidth => $peakbw, >> storage => $storage, >> cgi => $cgi, >> mod_python => $mod_python, >> postgresql => $postgresql; >> } >> } >> >> And then a custom function extract_field() for parsing the >> strings. Doable, but not very pretty... >> >> Those strings are of course a kind of dictionary/hash/mapping, >> and Puppet 2.6 has support for dictionaries directly. But that >> would only help you if 2.6 allows you to have a dictionary as the >> title/name for a resource; I haven''t checked, but I would guess >> that you can only use strings for titles/names. >> >> It would be much easier if external node classifiers could define >> resources directly, instead of having to do something like the >> above. But I seem to remember Luke saying that they are planning >> to add that functionality sometime in the future; I don''t think >> they have set a timeline for it yet, though > > Thomas, thanks for the detailed description. Nigel, Thomas pretty much > nailed the issue on the head. It''s not practical to add a new class > every time you add a customer. The extract_field customer function > doesn''t exist yet, and I''m not a ruby programmer, so it won''t exist > any time soon. Even if it did, it''s an ugly hack.This is much clearer now. Can you not solve this problem with extlookup now rather than having to write custom functions? Either key the relevant data off a base variable, or set the key to lookup info for in the external node provider? I like this approach because it keeps the data and the model nicely separated, rather than whacking data into the external node provider.> > Doug. > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 8/30/2010 9:44 AM, Nigel Kersten wrote:> This is much clearer now. > > Can you not solve this problem with extlookup now rather than having > to write custom functions? Either key the relevant data off a base > variable, or set the key to lookup info for in the external node > provider? > > I like this approach because it keeps the data and the model nicely > separated, rather than whacking data into the external node provider. >If the extlookup() function supported other than just CSV files now, it would be more attractive. My instance, most of this information has to be derived in the external node classifier, this would entail some sort of periodic extraction to generate the extra data source. Besides, isn''t the node classifier intended to provide the driving data (e.g., parameters) for the models? As well as calling out the nodes classes? -Alan -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Aug 30, 2010 at 5:35 PM, Alan Sparks <asparks@doublesparks.net> wrote:> On 8/30/2010 9:44 AM, Nigel Kersten wrote: >> This is much clearer now. >> >> Can you not solve this problem with extlookup now rather than having >> to write custom functions? Either key the relevant data off a base >> variable, or set the key to lookup info for in the external node >> provider? >> >> I like this approach because it keeps the data and the model nicely >> separated, rather than whacking data into the external node provider. >> > > If the extlookup() function supported other than just CSV files now, it > would be more attractive. My instance, most of this information has to > be derived in the external node classifier, this would entail some sort > of periodic extraction to generate the extra data source. Besides, > isn''t the node classifier intended to provide the driving data (e.g., > parameters) for the models? As well as calling out the nodes classes?I''m not disagreeing with you. It could be better, just pointing out an option. I have custom functions for pulling data out of yaml files, and strongly prefer to keep that data separate to the actual code in my node classifier, so it doesn''t really matter to me whether the data is directly interrogated by the node classifier, or pulled out by functions inside manifests based upon a key that the node classifier sets.> -Alan > > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Alan Sparks wrote:> > If the extlookup() function supported other than just CSV files now, it > would be more attractive. My instance, most of this information has to > be derived in the external node classifier, this would entail some sort > of periodic extraction to generate the extra data source. Besides, > isn''t the node classifier intended to provide the driving data (e.g., > parameters) for the models? As well as calling out the nodes classes? > -Alan > >On the roadmap for extlookup are JSON and YAML inputs. That''s targeted at 2.7. Regards James Turnbull -- Puppet Labs - http://www.puppetlabs.com C: 503-734-8571 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.