<Derek.Whayman@barclayscapital.com>
2007-Jun-11 14:00 UTC
Thoughts on arrays and generic functions
Working more on my cluster class, I can see other things coming up.
I''ve ended up creating custom functions to perform basic array-handling
tasks (like ''join'' and extracting a single element).
I''ve also had to
create a template for /etc/hosts because I can''t use the
''host'' type and
iterate over an array.
I know Puppet is trying very hard not to be a programming language, but
it would certainly be easier to build my classes with either (i) better
array handling or (ii) more functions or generate access to the ruby
language.
The below class which illustrates this take a compact cluster defintion
and write various config files from this ''golden'' data. It
also shells
out to a script to call lots of the ha* commands.
Also one thing would be nice is really compact test harness for custom
functions!
Any thoughts anyone?
Cheers,
Derek
define clusterconf($members, $clustername, $clusterno,
$vips, $vipmasks, $loprinic, $hiprinic, $primarynic) {
$hostlist = array2string($members) # <-- Custom fn - join(" ")
$vip1 = array_elt($members, 0) # <-- Custom fn - return element
$mask1 = array_elt($vipmasks, 0)
file { llthosts:
owner => root,
group => root,
mode => 0644,
content => template("llthosts_conf"),
path => "/etc/llthosts"
}
file { llttab:
owner => root,
group => root,
mode => 0644,
content => template("llttab_conf"),
path => "/etc/llttab"
}
file { gabtab:
owner => root,
group => root,
mode => 0755,
content => template("gabtab_conf"),
path => "/etc/gabtab"
}
# vcs_cmc script:
file { vcs_cmc:
path => "/var/lib/puppet/clientscripts/vcs_cmc",
owner => root,
group => root,
mode => 0755,
source => "puppet://puppet/clientscripts/vcs_cmc"
}
exec { vcs_cmc:
command => "vcs_cmc $clustername $primarynic $vip1 $mask1
$hostlist",
path => "/var/lib/puppet/clientscripts:/bin/usr/bin",
logoutput => true,
require => [ File[clientscripts],
File[vcs_cmc],
File[llthosts],
File[llttab],
File[gabtab]
]
}
# Would really prefer to use ''host'' type, but how to
iterate over
array?
file { hosts:
owner => root,
group => root,
mode => 0644,
content => template("cluster_hosts_conf"),
path => "/etc/hosts"
}
}
# Individual Cluster Definitions.
class cluster {
case $hostname {
engpsr0022, engpsr0026: {
clusterconf { clusterconf:
members => [ "engpsr0022", "engpsr0026"
],
clustername => "engclu01",
clusterno => 0,
loprinic => "eth0",
hiprinic => "eth1",
primarynic => "eth0",
vips => [ "30.96.3.201", "30.96.3.202",
"30.96.3.203",
"30.96.3.204", "30.96.3.205" ],
vipmasks => [ "255.255.255.0",
"255.255.255.0",
"255.255.255.0", "255.255.255.0", "255.255.255.0"
],
}
}
}
}
------------------------------------------------------------------------
For important statutory and regulatory disclosures and more information about
Barclays Capital, please visit our web site at http://www.barcap.com.
Internet communications are not secure and therefore the Barclays Group does not
accept legal responsibility for the contents of this message. Although the
Barclays Group operates anti-virus programmes, it does not accept responsibility
for any damage whatsoever that is caused by viruses being passed. Any views or
opinions presented are solely those of the author and do not necessarily
represent those of the Barclays Group. Replies to this email may be monitored
by the Barclays Group for operational or business reasons.
Barclays Capital is the investment banking division of Barclays Bank PLC, a
company registered in England (number 1026167) with its registered office at 1
Churchill Place, London, E14 5HP. This email may relate to or be sent from other
members of the Barclays Group.
------------------------------------------------------------------------
On Jun 11, 2007, at 9:00 AM, <Derek.Whayman@barclayscapital.com> <Derek.Whayman@barclayscapital.com> wrote:> Working more on my cluster class, I can see other things coming up. > > I''ve ended up creating custom functions to perform basic array- > handling > tasks (like ''join'' and extracting a single element). I''ve also had to > create a template for /etc/hosts because I can''t use the ''host'' > type and > iterate over an array. > > I know Puppet is trying very hard not to be a programming language, > but > it would certainly be easier to build my classes with either (i) > better > array handling or (ii) more functions or generate access to the ruby > language.The primary reason Puppet doesn''t have these features is that the only way I can reasonably add them is to just copy Ruby''s functionality, which means Puppet quickly loses its distinction from Ruby itself. I''d be interested in your trying out the dsl.rb stuff to see if it does (or could do) what you want. My guess is that people who want more functionality will be happier with pure ruby and skipping Puppet''s language entirely. It''s not that I''m opposed to pure ruby, specifically; it''s just a close result if we start adding arrays (and hashes, and regexes, etc.) to Puppet''s language. That being said, nearly every case I''ve seen where iteration was needed involved exactly what you''re doing here: Focusing on file contents instead of resources. You do appear to have some specific counter-examples -- the join() you''re doing is used by an exec, for instance. It looks like you''ve got three additional resource types here (one for each llt file you''re generating). If you modeled those directly as resources, instead of generating the files as you do, you likely would not need iteration at all. Note that I''m not saying it''s a perfect solution, and I do know that Puppet''s facilities for doing this are apparently not user-friendly, but this is the "right" solution, and if people would start doing this -- and pushing me toward making it more user-friendly -- then we''d get a lot more functionality in a much more usable way. As an example, if you had definitions or native types that modeled the contents of llthosts.conf (as an example), you could use Puppet''s pseudo-iteration: llthost { $members: ensure => present } I''ve no actual idea what goes in that file, so it''s likely that you need name/value iteration, but it looks from your configuration like the IP addresses are not localized to individual hosts (either that, or you''re using file order to determine which host gets which IP addresses). The current pattern for doing this follows this basic flow: class llthosts { # Monitor the directory we''re using file { ''/etc/llthosts.d'': purge => true, notify => Exec[rebuild- llthosts] } # Rebuild when necessary Exec { rebuild-llthosts: command => "cat /etc/llthosts.d/* > / etc/hosts", refreshonly => true } } define llthost($ensure = present) { include llthosts # No idea of the actual content here file { "/etc/llthosts.d/$name": ensure => $ensure, content => "$name\n" } } Note that, IMO, this isn''t very good -- it''s just the best thing we have right now, if you don''t want to write a native type, and no one seems to want to write a native type. I''ve been trying to come up with a better way to do this for, um, ages, and I apparently have not succeeded, even though the ParsedFile base provider class works pretty well for simple cases (see the sshkey parsed provider for an example, although even it''s more complicated than seems necessary).> The below class which illustrates this take a compact cluster > defintion > and write various config files from this ''golden'' data. It also > shells > out to a script to call lots of the ha* commands. > > Also one thing would be nice is really compact test harness for custom > functions!What are you looking for, specifically? I assume you''re looking to test away from svn; if not, you should be able to just add new unit tests to language/functions.rb, or create language/functions/ custom.rb in a checked-out tree. I haven''t really thought about making it easy for people to test their custom code.> Any thoughts anyone? > > Cheers, > Derek > > > define clusterconf($members, $clustername, $clusterno, > $vips, $vipmasks, $loprinic, $hiprinic, $primarynic) { > > $hostlist = array2string($members) # <-- Custom fn - join(" ") > $vip1 = array_elt($members, 0) # <-- Custom fn - return > element > $mask1 = array_elt($vipmasks, 0)Ok, I see how these are used by the exec. [...]> # Would really prefer to use ''host'' type, but how to iterate over > array? > file { hosts: > owner => root, > group => root, > mode => 0644, > content => template("cluster_hosts_conf"), > path => "/etc/hosts" > } > }How does the template know which host gets which IP address? -- To define recursion, we must first define recursion. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com