Re: [Puppet Users] Definitions in External Nodes

Nigel Kersten
Mon, 30 Aug 2010 15:44:37 -0000

On Sat, Aug 28, 2010 at 4:48 PM, Douglas Garstang
<doug.garstang@gmail.com> wrote:
> On Sat, Aug 28, 2010 at 3:49 PM, Thomas Bellman <bellman@nsc.liu.se>
wrote:
>> Nigel Kersten wrote:
>>
>>> On Sat, Aug 28, 2010 at 9:04 AM, Douglas Garstang
>>> <doug.garstang@gmail.com> wrote:
>>
>>>> I mean that you need to create a new class for each customer
you add.
>>>> That does not scale.
>>>
>>> Why can''t you have a class that instantiates your defined
type?
>>
>> I don''t know Douglas'' circumstances, but assume you
have a web
>> hotel.  You then probably have a database of your customers,
>> telling you things like which domain(s) they have, what features
>> they pay for (static pages only, CGI scripts, mod_python,
>> database access, and so on), and how much storage and bandwidth
>> they are paying for.
>>
>> To map all these customers onto your physical machines, you write
>> a Puppet define that is used something like this:
>>
>>    virtual_web_host {
>>        "www.example.com":
>>            avg_bandwidth => 4096,      # Mbyte/month
>>            peak_bandwidth => 2000,     # Kbit/s
>>            storage => 1536,            # Mbyte disk space
>>            cgi => true,
>>            mod_python => false,
>>            postgresql => true;
>>    }
>>
>> Since you have your customers in a database, it would be nice to
>> use an external nodes script to basically generate one such
>> resource for each customer domain, and it will probably use some
>> information in the database to balance the virtual hosts over the
>> physical machines.  But external nodes scripts *can''t*
generate
>> resources.  You of course don''t want to write a new class for
>> each customer, because if you are a largish web hotel you may get
>> dozens of new customers per day (and possibly loose as many), so
>> that would be too much work.
>>
>> But external node classifiers *can''t* generate resources.
 They
>> can only use resources that already exist in the manifests, and
>> affect them by setting variables.
>>
>>
>> There is a way to work around this.  You can let the external
>> node classifier set a variable with a value like this:
>>
>>    $vhosts = [
>>
>>
 "dom=www.example.com|avgbw=4096|peakbw=2000|storage=1536|cgi=true|...",
>>
>>
 "dom=www.example.org|avgbw=65536|peakbw=512|storage=600|cgi=false|...",
>>
 "dom=www.example.net|avgbw=512|peakbw=100|storage=2500|cgi=true|...",
>>        ...
>>    ]
>>
>> (although the script needs to output it using YAML syntax, of
>> course), and tell Puppet to include a class "all-vhosts".
 The
>> manifest files would contain code looking something like this:
>>
>>    class all-vhosts
>>    {
>>        ext_virtual_web_host {
>>            $vhosts:
>>                ;
>>        }
>>    }
>>
>>    define ext_virtual_web_host()
>>    {
>>        $domain = extract_field(''dom'', $name)
>>        $avgbw = extract_field(''avgbw'', $name)
>>        $peakbw = extract_field(''peakbw'', $name)
>>        $storage = extract_field(''storage'', $name)
>>        $cgi = extract_field(''cgi'', $name)
>>        $mod_python = extract_field(''mod_python'',
$name)
>>        $postgresql = extract_field(''postgresql'',
$name)
>>
>>        virtual_web_host {
>>            $domain:
>>                avg_bandwidth => $avgbw,
>>                peak_bandwidth => $peakbw,
>>                storage => $storage,
>>                cgi => $cgi,
>>                mod_python => $mod_python,
>>                postgresql => $postgresql;
>>        }
>>    }
>>
>> And then a custom function extract_field() for parsing the
>> strings.  Doable, but not very pretty...
>>
>> Those strings are of course a kind of dictionary/hash/mapping,
>> and Puppet 2.6 has support for dictionaries directly.  But that
>> would only help you if 2.6 allows you to have a dictionary as the
>> title/name for a resource; I haven''t checked, but I would
guess
>> that you can only use strings for titles/names.
>>
>> It would be much easier if external node classifiers could define
>> resources directly, instead of having to do something like the
>> above.  But I seem to remember Luke saying that they are planning
>> to add that functionality sometime in the future; I don''t
think
>> they have set a timeline for it yet, though
>
> Thomas, thanks for the detailed description. Nigel, Thomas pretty much
> nailed the issue on the head. It''s not practical to add a new
class
> every time you add a customer. The extract_field customer function
> doesn''t exist yet, and I''m not a ruby programmer, so it
won''t exist
> any time soon. Even if it did, it''s an ugly hack.

This is much clearer now.

Can you not solve this problem with extlookup now rather than having
to write custom functions?  Either key the relevant data off a base
variable, or set the key to lookup info for in the external node
provider?

I like this approach because it keeps the data and the model nicely
separated, rather than whacking data into the external node provider.

>
> Doug.
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
>



--
nigel