Puppet users - Sep 2013 - default data lookup w/ hiera2? "Data in Modules"

I am reading over the release notes for Puppet 3.3, and buried under "data 
in modules" is a link to ARM-9.
 
There is a good chance that I don''t "get it" - Puppet is
moving pretty fast
these days and I am still trying to get caught up with all of the changes.

I have only read the latest Puppet code and the ARM docs- I have not tried 
it on a real system yet. However, I am concerned about the usefullness of 
the default lookup syntax. I find the examples in the ARM document a bit 
obtuse.
https://github.com/puppetlabs/armatures/blob/master/arm-9.data_in_modules/index.md#lookup-with-default

With hiera1, we can (very simply) do:
  hiera("namespace::key_name", <default value> )

Hiera2 requires some more work:
 $x = lookup(''something'')
 $looked_up = $x ? { undef => ''nothing'', default => $x }

This is ugly. Checking if variables are undefined, and then setting them to 
a default is what we did in bash. Not good.

On the positive side, the lookup function adds a static type checker, but 
it also fails to be completely useful because we cannot (?) define our own 
static types or validators. If we had the ability to define our own type 
check via lambda or other function, this would be useful. We could do 
complex validation on data, more than just ''string'' or
''array''.

I would like to see something like:
lookup("namespace::key_name", custom_validator_function)

Where the custom_validator_function could be any puppet parser function OR 
since we now worship the* *great *λ, *we should be able to use an anonymous
function
in-place to validate the input (and in this fictitious example, set a 
default value):
lookup("namespace::key_name", |$val| {$val or "default
value"} )


To sum it up, as someone who writes a lot of Puppet DSL code, I find the 
hiera1 syntax much more useful than the lookup syntax in ARM-9.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Thank you for the feedback, very good comments.

See more inline...

On 2013-11-09 11:02, robbyt wrote:
> I am reading over the release notes for Puppet 3.3, and buried under
> "data in modules" is a link to ARM-9.
> There is a good chance that I don''t "get it" - Puppet is
moving pretty
> fast these days and I am still trying to get caught up with all of the
> changes.
>
> I have only read the latest Puppet code and the ARM docs- I have not
> tried it on a real system yet. However, I am concerned about the
> usefullness of the default lookup syntax. I find the examples in the ARM
> document a bit obtuse.
>
https://github.com/puppetlabs/armatures/blob/master/arm-9.data_in_modules/index.md#lookup-with-default
>
> With hiera1, we can (very simply) do:
>    hiera("namespace::key_name", <default value> )
>
> Hiera2 requires some more work:
> $x = lookup(''something'')
>   $looked_up = $x ? { undef => ''nothing'', default
=> $x }
>
> This is ugly. Checking if variables are undefined, and then setting them
> to a default is what we did in bash. Not good.
>
The challenge here was to have a function that works for 3x as well as 
for what is being worked on for 4x (where the "data-in-modules" 
supposedly is on by default along "parser future"). When we reach that
point, it is possible to reference a type directly without having to 
encode it in a string. i.e. it will be possible to do like this:

     lookup(''mykey'', Integer)

For 3.x this was not possible and the signature of the lookup function 
is now lookup(String key, String type), or just lookup(String key), and 
thus difficult to handle both type and default value as optional (given 
two strings, it that key and type or key and default).

We have a couple of options before 4x:
a) Add a third argument, and if a default is wanted, type must be specified.

b) Add the ability to pass arguments as a hash, it can either be a third 
argument, or used instead of the type argument. (i.e. giving arguments 
by name instead).

The first is simple to add, the second does not work well unless the 
future parser is also used.

Examples:
    # a
    lookup(''mykey'', ''Integer'', 42)

    # b (requires future parser since 3x does not support direct passing
    # of a hash
    lookup(''mykey'', ''Integer'', { default
=> 42})
    lookup(''mykey'', { type => ''Integer'',
default => 42})

    # In 4x. this will is possible:
    lookup(''mykey'', ''Integer'', 42)
    lookup(''mykey'', Integer, 42)
    lookup(''mykey'', 42)
    lookup(''mykey'', Integer)

If future parser is used, it is also possible to use a lambda, there is 
no need to first assign variable and then check - i.e.

     $x = lookup(''something'') {|$x| $x ? {
       undef => ''nothing'', default => $x }

which is just slightly better, but still bulky.

I am in favor of adding the 3d argument for default, and until 4x the 
lookup must then also use type.

What do you think about that?

> On the positive side, the lookup function adds a static type checker,
> but it also fails to be completely useful because we cannot (?) define
> our own static types or validators. If we had the ability to define our
> own type check via lambda or other function, this would be useful. We
> could do complex validation on data, more than just
''string'' or ''array''.
>
> I would like to see something like:
> lookup("namespace::key_name", custom_validator_function)
>
> Where the custom_validator_function could be any puppet parser function
> OR since we now worship the**great *λ, *we should be able to use an
> anonymous function in-place to validate the input (and in this
> fictitious example, set a default value):
> lookup("namespace::key_name", |$val| {$val or "default
value"} )
>
There is a short and a long answer.

Short answer, the lookup already takes a lambda which can do validation 
and handle default value. This works now with --parser future.

Longer answer. The idea is to support specification of types in the 
Puppet Language. Your example would be

     lookup(''namespace::key'', SomeNamespace::SomeType)

Puppet Types is described in ARM-7 
(https://github.com/puppetlabs/armatures/blob/master/arm-7.puppet_types/puppet_types.md).

ARM-7 may need a revision since the work on Type in ARM-9 took the type 
system a bit further.
>
> To sum it up, as someone who writes a lot of Puppet DSL code, I find the
> hiera1 syntax much more useful than the lookup syntax in ARM-9.
>
Does adding a 3d argument to lookup for default cut it for you?

Regards
- henrik


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Henrik,

Thank you for your quick response. Sorry mine wasn''t so quick- Google
ate
my message, perhaps it contained too many curly braces. :)

Anyhow, adding a default argument option to lookup() would be great. But 
adding the option to pass a hash with keyword arguments would be even 
better!

Python solved the API UI for this quite well with **, and *.
http://docs.python.org/release/2.7.4/tutorial/controlflow.html#keyword-arguments
and
http://docs.python.org/release/2.7.4/tutorial/controlflow.html#unpacking-argument-lists

Imo, the 2nd positional argument of the lookup() function should accept 
ONLY a hash. Otherwise, what would I do if I want to set a hash to BE the 
default argument?

e.g.,

1st Positional argument as a hash?
Ambiguous positional argument (error!)
lookup(''namespace::some_hash'', {''my_key''
=> ''my_value''})

More explicit, better:
lookup(''namespace::some_hash'', { type =>
''Hash'', default => {''my_key'' =>
''my_value''} })



On Thursday, September 12, 2013 12:13:37 AM UTC+3, Henrik Lindberg
wrote:
>
> Thank you for the feedback, very good comments. 
>
> See more inline... 
>
> On 2013-11-09 11:02, robbyt wrote: 
> > I am reading over the release notes for Puppet 3.3, and buried under 
> > "data in modules" is a link to ARM-9. 
> > There is a good chance that I don''t "get it" -
Puppet is moving pretty
> > fast these days and I am still trying to get caught up with all of the
> > changes. 
> > 
> > I have only read the latest Puppet code and the ARM docs- I have not 
> > tried it on a real system yet. However, I am concerned about the 
> > usefullness of the default lookup syntax. I find the examples in the
ARM
> > document a bit obtuse. 
> > 
>
https://github.com/puppetlabs/armatures/blob/master/arm-9.data_in_modules/index.md#lookup-with-default
> > 
> > With hiera1, we can (very simply) do: 
> >    hiera("namespace::key_name", <default value> ) 
> > 
> > Hiera2 requires some more work: 
> > $x = lookup(''something'') 
> >   $looked_up = $x ? { undef => ''nothing'', default
=> $x }
> > 
> > This is ugly. Checking if variables are undefined, and then setting
them
> > to a default is what we did in bash. Not good. 
> > 
>
> The challenge here was to have a function that works for 3x as well as 
> for what is being worked on for 4x (where the "data-in-modules" 
> supposedly is on by default along "parser future"). When we reach
that
> point, it is possible to reference a type directly without having to 
> encode it in a string. i.e. it will be possible to do like this: 
>
>      lookup(''mykey'', Integer) 
>
> For 3.x this was not possible and the signature of the lookup function 
> is now lookup(String key, String type), or just lookup(String key), and 
> thus difficult to handle both type and default value as optional (given 
> two strings, it that key and type or key and default). 
>
> We have a couple of options before 4x: 
> a) Add a third argument, and if a default is wanted, type must be 
> specified. 
>
> b) Add the ability to pass arguments as a hash, it can either be a third 
> argument, or used instead of the type argument. (i.e. giving arguments 
> by name instead). 
>
> The first is simple to add, the second does not work well unless the 
> future parser is also used. 
>
> Examples: 
>     # a 
>     lookup(''mykey'', ''Integer'', 42) 
>
>     # b (requires future parser since 3x does not support direct passing 
>     # of a hash 
>     lookup(''mykey'', ''Integer'', {
default => 42})
>     lookup(''mykey'', { type =>
''Integer'', default => 42})
>
>     # In 4x. this will is possible: 
>     lookup(''mykey'', ''Integer'', 42) 
>     lookup(''mykey'', Integer, 42) 
>     lookup(''mykey'', 42) 
>     lookup(''mykey'', Integer) 
>
> If future parser is used, it is also possible to use a lambda, there is 
> no need to first assign variable and then check - i.e. 
>
>      $x = lookup(''something'') {|$x| $x ? { 
>        undef => ''nothing'', default => $x } 
>
> which is just slightly better, but still bulky. 
>
> I am in favor of adding the 3d argument for default, and until 4x the 
> lookup must then also use type. 
>
> What do you think about that? 
>
>
> > On the positive side, the lookup function adds a static type checker, 
> > but it also fails to be completely useful because we cannot (?) define
> > our own static types or validators. If we had the ability to define
our
> > own type check via lambda or other function, this would be useful. We 
> > could do complex validation on data, more than just
''string'' or ''array''.
> > 
> > I would like to see something like: 
> > lookup("namespace::key_name", custom_validator_function) 
> > 
> > Where the custom_validator_function could be any puppet parser
function
> > OR since we now worship the**great *οΏ½, *we should be able to use an 
> > anonymous function in-place to validate the input (and in this 
> > fictitious example, set a default value): 
> > lookup("namespace::key_name", |$val| {$val or "default
value"} )
> > 
>
> There is a short and a long answer. 
>
> Short answer, the lookup already takes a lambda which can do validation 
> and handle default value. This works now with --parser future. 
>
> Longer answer. The idea is to support specification of types in the 
> Puppet Language. Your example would be 
>
>      lookup(''namespace::key'', SomeNamespace::SomeType) 
>
> Puppet Types is described in ARM-7 
> (
>
https://github.com/puppetlabs/armatures/blob/master/arm-7.puppet_types/puppet_types.md).
>
>
> ARM-7 may need a revision since the work on Type in ARM-9 took the type 
> system a bit further. 
>
> > 
> > To sum it up, as someone who writes a lot of Puppet DSL code, I find
the
> > hiera1 syntax much more useful than the lookup syntax in ARM-9. 
> > 
>
> Does adding a 3d argument to lookup for default cut it for you? 
>
> Regards 
> - henrik 
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

And yes, a 3rd positional argument for the default would be fantastic. (For 
now...)

On Thursday, September 12, 2013 12:13:37 AM UTC+3, Henrik Lindberg wrote:
> Does adding a 3d argument to lookup for default cut it for you? 
>
> Regards 
> - henrik 
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 2013-16-09 20:34, robbyt wrote:
> Henrik,
>
> Thank you for your quick response. Sorry mine wasn''t so quick-
Google
> ate my message, perhaps it contained too many curly braces. :)
>
> Anyhow, adding a default argument option to lookup() would be great. But
> adding the option to pass a hash with keyword arguments would be even
> better!
>
> Python solved the API UI for this quite well with **, and *.
>
http://docs.python.org/release/2.7.4/tutorial/controlflow.html#keyword-arguments
> and
>
http://docs.python.org/release/2.7.4/tutorial/controlflow.html#unpacking-argument-lists
>
> Imo, the 2nd positional argument of the lookup() function should accept
> ONLY a hash. Otherwise, what would I do if I want to set a hash to BE
> the default argument?
>
> e.g.,
>
> 1st Positional argument as a hash?
> Ambiguous positional argument (error!)
> lookup(''namespace::some_hash'',
{''my_key'' => ''my_value''})
>
> More explicit, better:
> lookup(''namespace::some_hash'', { type =>
''Hash'', default => {''my_key'' =>
> ''my_value''} })
>
>
Here are the possible alterantives:

arity 1: String key
arity 1: Hash options
arity 2: String key, String | Type type
arity 2: String key, Hash options
arity 3: String key, String | Type type, Object default_value

Regards
- henrik

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Wednesday, September 18, 2013 3:15:18 PM UTC+3, Henrik Lindberg wrote:
> Here are the possible alterantives: 
>
> arity 1: String key 
> arity 1: Hash options 
> arity 2: String key, String | Type type 
> arity 2: String key, Hash options 
> arity 3: String key, String | Type type, Object default_value 
>
> Regards 
> - henrik 
>
Hi Henrik,
Yes, I think those options are reasonable.

In addition, I assume that we can still use a variable for all of the 
arguments? e.g.,
$default_data = { type => ''Integer'', default => 42 }
lookup(''mykey'', $default_data) 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 2013-19-09 13:48, robbyt wrote:
> On Wednesday, September 18, 2013 3:15:18 PM UTC+3, Henrik Lindberg wrote:
>
>     Here are the possible alterantives:
>
>     arity 1: String key
>     arity 1: Hash options
>     arity 2: String key, String | Type type
>     arity 2: String key, Hash options
>     arity 3: String key, String | Type type, Object default_value
>
>     Regards
>     - henrik
>
>
> Hi Henrik,
> Yes, I think those options are reasonable.
>
> In addition, I assume that we can still use a variable for all of the
> arguments? e.g.,
> $default_data = { type => ''Integer'', default => 42
}
> lookup(''mykey'', $default_data)
>
Sure, that will work without switching to future parser.

FYI - I am working on documentation, and have also documented a proposal 
to fix the lookup function.

I work on the documents here - some will be documentation, some parts 
probably will go into the ARM docs.

https://github.com/hlindberg/misc-puppet-docs

In particular, the Lookup is described in this document:
https://github.com/hlindberg/misc-puppet-docs/blob/master/improvements-arm-9.md

along with a bunch of other improvements.

Regards
- henrik
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.