Re: [Puppet Users] Definitions in External Nodes

Thomas Bellman
Sat, 28 Aug 2010 22:49:55 -0000

Nigel Kersten wrote:

> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang
> <doug.garstang@gmail.com> wrote:

>> I mean that you need to create a new class for each customer you add.
>> That does not scale.
>
> Why can''t you have a class that instantiates your defined type?

I don''t know Douglas'' circumstances, but assume you have a web
hotel.  You then probably have a database of your customers,
telling you things like which domain(s) they have, what features
they pay for (static pages only, CGI scripts, mod_python,
database access, and so on), and how much storage and bandwidth
they are paying for.

To map all these customers onto your physical machines, you write
a Puppet define that is used something like this:

     virtual_web_host {
         "www.example.com":
             avg_bandwidth => 4096,      # Mbyte/month
             peak_bandwidth => 2000,     # Kbit/s
             storage => 1536,            # Mbyte disk space
             cgi => true,
             mod_python => false,
             postgresql => true;
     }

Since you have your customers in a database, it would be nice to
use an external nodes script to basically generate one such
resource for each customer domain, and it will probably use some
information in the database to balance the virtual hosts over the
physical machines.  But external nodes scripts *can''t* generate
resources.  You of course don''t want to write a new class for
each customer, because if you are a largish web hotel you may get
dozens of new customers per day (and possibly loose as many), so
that would be too much work.

But external node classifiers *can''t* generate resources.  They
can only use resources that already exist in the manifests, and
affect them by setting variables.


There is a way to work around this.  You can let the external
node classifier set a variable with a value like this:

     $vhosts = [

"dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...",

"dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...",

"dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...",
         ...
     ]

(although the script needs to output it using YAML syntax, of
course), and tell Puppet to include a class "all-vhosts".  The
manifest files would contain code looking something like this:

     class all-vhosts
     {
         ext_virtual_web_host {
             $vhosts:
                 ;
         }
     }

     define ext_virtual_web_host()
     {
         $domain = extract_field(''dom'', $name)
         $avgbw = extract_field(''avgbw'', $name)
         $peakbw = extract_field(''peakbw'', $name)
         $storage = extract_field(''storage'', $name)
         $cgi = extract_field(''cgi'', $name)
         $mod_python = extract_field(''mod_python'', $name)
         $postgresql = extract_field(''postgresql'', $name)

         virtual_web_host {
             $domain:
                 avg_bandwidth => $avgbw,
                 peak_bandwidth => $peakbw,
                 storage => $storage,
                 cgi => $cgi,
                 mod_python => $mod_python,
                 postgresql => $postgresql;
         }
     }

And then a custom function extract_field() for parsing the
strings.  Doable, but not very pretty...

Those strings are of course a kind of dictionary/hash/mapping,
and Puppet 2.6 has support for dictionaries directly.  But that
would only help you if 2.6 allows you to have a dictionary as the
title/name for a resource; I haven''t checked, but I would guess
that you can only use strings for titles/names.

It would be much easier if external node classifiers could define
resources directly, instead of having to do something like the
above.  But I seem to remember Luke saying that they are planning
to add that functionality sometime in the future; I don''t think
they have set a timeline for it yet, though