Re: [Puppet Users] Definitions in External Nodes

Douglas Garstang
Sat, 28 Aug 2010 23:48:07 -0000

On Sat, Aug 28, 2010 at 3:49 PM, Thomas Bellman <bellman@nsc.liu.se>
wrote:
> Nigel Kersten wrote:
>
>> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang
>> <doug.garstang@gmail.com> wrote:
>
>>> I mean that you need to create a new class for each customer you
add.
>>> That does not scale.
>>
>> Why can''t you have a class that instantiates your defined
type?
>
> I don''t know Douglas'' circumstances, but assume you have
a web
> hotel.  You then probably have a database of your customers,
> telling you things like which domain(s) they have, what features
> they pay for (static pages only, CGI scripts, mod_python,
> database access, and so on), and how much storage and bandwidth
> they are paying for.
>
> To map all these customers onto your physical machines, you write
> a Puppet define that is used something like this:
>
>    virtual_web_host {
>        "www.example.com":
>            avg_bandwidth => 4096,      # Mbyte/month
>            peak_bandwidth => 2000,     # Kbit/s
>            storage => 1536,            # Mbyte disk space
>            cgi => true,
>            mod_python => false,
>            postgresql => true;
>    }
>
> Since you have your customers in a database, it would be nice to
> use an external nodes script to basically generate one such
> resource for each customer domain, and it will probably use some
> information in the database to balance the virtual hosts over the
> physical machines.  But external nodes scripts *can''t* generate
> resources.  You of course don''t want to write a new class for
> each customer, because if you are a largish web hotel you may get
> dozens of new customers per day (and possibly loose as many), so
> that would be too much work.
>
> But external node classifiers *can''t* generate resources.  They
> can only use resources that already exist in the manifests, and
> affect them by setting variables.
>
>
> There is a way to work around this.  You can let the external
> node classifier set a variable with a value like this:
>
>    $vhosts = [
>
>
 "dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...",
>
>
 "dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...",
>
 "dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...",
>        ...
>    ]
>
> (although the script needs to output it using YAML syntax, of
> course), and tell Puppet to include a class "all-vhosts".  The
> manifest files would contain code looking something like this:
>
>    class all-vhosts
>    {
>        ext_virtual_web_host {
>            $vhosts:
>                ;
>        }
>    }
>
>    define ext_virtual_web_host()
>    {
>        $domain = extract_field(''dom'', $name)
>        $avgbw = extract_field(''avgbw'', $name)
>        $peakbw = extract_field(''peakbw'', $name)
>        $storage = extract_field(''storage'', $name)
>        $cgi = extract_field(''cgi'', $name)
>        $mod_python = extract_field(''mod_python'', $name)
>        $postgresql = extract_field(''postgresql'', $name)
>
>        virtual_web_host {
>            $domain:
>                avg_bandwidth => $avgbw,
>                peak_bandwidth => $peakbw,
>                storage => $storage,
>                cgi => $cgi,
>                mod_python => $mod_python,
>                postgresql => $postgresql;
>        }
>    }
>
> And then a custom function extract_field() for parsing the
> strings.  Doable, but not very pretty...
>
> Those strings are of course a kind of dictionary/hash/mapping,
> and Puppet 2.6 has support for dictionaries directly.  But that
> would only help you if 2.6 allows you to have a dictionary as the
> title/name for a resource; I haven''t checked, but I would guess
> that you can only use strings for titles/names.
>
> It would be much easier if external node classifiers could define
> resources directly, instead of having to do something like the
> above.  But I seem to remember Luke saying that they are planning
> to add that functionality sometime in the future; I don''t think
> they have set a timeline for it yet, though

Thomas, thanks for the detailed description. Nigel, Thomas pretty much
nailed the issue on the head. It''s not practical to add a new class
every time you add a customer. The extract_field customer function
doesn''t exist yet, and I''m not a ruby programmer, so it
won''t exist
any time soon. Even if it did, it''s an ugly hack.

Doug.