Puppet users - Jul 2012 - how to conditionally add users to a virtualized group?

I''m fighting with a ticklish issue.  We have some groups and users that
only belong on some systems. So we made all users virtual and then realize them
in classes specific to those system types.  This works quite well for the users,
but not for the groups. When you specify a user, you have to list all the groups
they are in.
	 groups     =>
[''support'',ops'',''dev''],

 Obviously some groups aren''t realized on all systems, so this produces
an error when usermod is run.
	''/usr/sbin/usermod -G support,ops,dev jrhett'' returned 6:
usermod: unknown group dev
	usermod: unknown group dev

So I tried to get smarter, and put logic to add the group to each member under
the appropriate class
	Class users::dev inherits users { 
		User[''jrhett''] { groups +> [''dev''] }
	}

This works… almost. It works for all instances where the user is only subclassed
once. But if I do the same technique in multiple classes I get

err: Could not retrieve catalog from remote server: Error 400 on SERVER:
Parameter ''groups'' is already set on User_and_key[jrhett] by
#<Puppet::Resource::Type:0x7f4feed2d828> at
/etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
/etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com

So how can this be achieved, short of using an exec with an unless doing another
exec to determine if the group exists?

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet users and groups are fiddly. My current not implemented thinking is to
use ldap and manage pam_groups via puppet on the hosts to get the granularity.

More thinking out loud than anything else.

Den

On 12/07/2012, at 6:03, Jo Rhett <jrhett@netconsonance.com> wrote:
> I''m fighting with a ticklish issue.  We have some groups and users
that only belong on some systems. So we made all users virtual and then realize
them in classes specific to those system types.  This works quite well for the
users, but not for the groups. When you specify a user, you have to list all the
groups they are in.
> 	 groups     =>
[''support'',ops'',''dev''],
> 
>  Obviously some groups aren''t realized on all systems, so this
produces an error when usermod is run.
> 	''/usr/sbin/usermod -G support,ops,dev jrhett'' returned
6: usermod: unknown group dev
> 	usermod: unknown group dev
> 
> So I tried to get smarter, and put logic to add the group to each member
under the appropriate class
> 	Class users::dev inherits users { 
> 		User[''jrhett''] { groups +> [''dev'']
}
> 	}
> 
> This works… almost. It works for all instances where the user is only
subclassed once. But if I do the same technique in multiple classes I get
> 
> err: Could not retrieve catalog from remote server: Error 400 on SERVER:
Parameter ''groups'' is already set on User_and_key[jrhett] by
#<Puppet::Resource::Type:0x7f4feed2d828> at
/etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
/etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com
> 
> So how can this be achieved, short of using an exec with an unless doing
another exec to determine if the group exists?
> 
> -- 
> Jo Rhett
> Net Consonance : net philanthropy to improve open source and internet
projects.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I use nss-pam-ldapd and pam_ldap depending on the system, using an ldap filter
to allow only certain groups per system. I prefer nss-pam-ldapd.

nss-pam-ldapd:

CentOS 6
Debian 6
Ubuntu 10.04

pam_ldap:

CentOS 5
FreeBSD 9

(Solaris is more like pam_ldap in configuration, but fairly unique.)

The manifests to deal with the above are essentially OS-specific.

On Thu, Jul 12, 2012 at 05:52:24PM +1000, Denmat wrote:
>    Puppet users and groups are fiddly. My current not implemented thinking
is
>    to use ldap and manage pam_groups via puppet on the hosts to get the
>    granularity. 
>    More thinking out loud than anything else.
>    Den
> 
>    On 12/07/2012, at 6:03, Jo Rhett <[1]jrhett@netconsonance.com>
wrote:
> 
>      I''m fighting with a ticklish issue.  We have some groups and
users that
>      only belong on some systems. So we made all users virtual and then
>      realize them in classes specific to those system types.  This works
>      quite well for the users, but not for the groups. When you specify a
>      user, you have to list all the groups they are in. 
>               groups     =>
[''support'',ops'',''dev''],
>       Obviously some groups aren''t realized on all systems, so
this produces
>      an error when usermod is run.
>              ''/usr/sbin/usermod -G support,ops,dev
jrhett'' returned 6:
>      usermod: unknown group dev
>              usermod: unknown group dev
>      So I tried to get smarter, and put logic to add the group to each
member
>      under the appropriate class
>              Class users::dev inherits users { 
>                      User[''jrhett''] { groups +>
[''dev''] }
>              }
>      This works� almost. It works for all instances where the user is only
>      subclassed once. But if I do the same technique in multiple classes I
>      get 
>      err: Could not retrieve catalog from remote server: Error 400 on
SERVER:
>      Parameter ''groups'' is already set on
User_and_key[jrhett] by
>      #<Puppet::Resource::Type:0x7f4feed2d828> at
>      /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
>      /etc/puppet/modules/users/manifests/dev.pp:27 on node
>      [2]s2-d1.company.com
>      So how can this be achieved, short of using an exec with an unless
doing
>      another exec to determine if the group exists?
>      -- 
>      Jo Rhett
>      Net Consonance : net philanthropy to improve open source and internet
>      projects.
> 
>      --
>      You received this message because you are subscribed to the Google
>      Groups "Puppet Users" group.
>      To post to this group, send email to [3]puppet-users@googlegroups.com.
>      To unsubscribe from this group, send email to
>      [4]puppet-users+unsubscribe@googlegroups.com.
>      For more options, visit this group at
>      [5]http://groups.google.com/group/puppet-users?hl=en.
> 
>    --
>    You received this message because you are subscribed to the Google
Groups
>    "Puppet Users" group.
>    To post to this group, send email to puppet-users@googlegroups.com.
>    To unsubscribe from this group, send email to
>    puppet-users+unsubscribe@googlegroups.com.
>    For more options, visit this group at
>    http://groups.google.com/group/puppet-users?hl=en.
> 
> References
> 
>    Visible links
>    1. mailto:jrhett@netconsonance.com
>    2. http://s2-d1.company.com/
>    3. mailto:puppet-users@googlegroups.com
>    4. mailto:puppet-users+unsubscribe@googlegroups.com
>    5. http://groups.google.com/group/puppet-users?hl=en
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi,

On 07/11/2012 10:03 PM, Jo Rhett wrote:
> So I tried to get smarter, and put logic to add the group to each member
> under the appropriate class
> Class users::dev inherits users { 
> User[''jrhett''] { groups +> [''dev''] }
> }
> 
> This works… almost. It works for all instances where the user is only
> subclassed once. But if I do the same technique in multiple classes I get 
sound approach, but I''ve hit this wall a couple of times as well.

I''ve resorted to horrors that would add items to array variables that
are declared in a central, well-known class, and use the final value for
the resources in question. Depending on how much flexibility is
required, this may not be feasible at all.

Perhaps hiera can be used to do something clever here?

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wednesday, July 11, 2012 3:03:14 PM UTC-5, Jo wrote:
>
> I''m fighting with a ticklish issue.  We have some groups and users
that
> only belong on some systems. So we made all users virtual and then realize 
> them in classes specific to those system types.  This works quite well for 
> the users, but not for the groups. When you specify a user, you have to 
> list all the groups they are in. 
> groups =>
[''support'',ops'',''dev''],
>
>  Obviously some groups aren''t realized on all systems, so this
produces an
> error when usermod is run.
> ''/usr/sbin/usermod -G support,ops,dev jrhett'' returned 6:
usermod: unknown
> group dev
> usermod: unknown group dev
>
> So I tried to get smarter, and put logic to add the group to each member 
> under the appropriate class
> Class users::dev inherits users { 
> User[''jrhett''] { groups +> [''dev''] }
> }
>
> This works… almost. It works for all instances where the user is only 
> subclassed once. But if I do the same technique in multiple classes I get 
>
> err: Could not retrieve catalog from remote server: Error 400 on SERVER: 
> Parameter ''groups'' is already set on User_and_key[jrhett]
by
> #<Puppet::Resource::Type:0x7f4feed2d828> at 
> /etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at 
> /etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com
>
> So how can this be achieved, short of using an exec with an unless doing 
> another exec to determine if the group exists?
>
>
If it is the case that each user always has the same potential secondary 
groups, and you need to narrow the actual secondary groups to those that 
are actually present, then I think you could do it without too much pain.  
The main ingredients would be a list (array) of the groups that are 
supposed to be present, and a custom function that forms the intersection 
of two arrays.  (Or you could use an inline template and split(), but yuck!)

Hiera would probably provide a good means for building the list of 
available groups, which you could then use not only to filter user 
definitions but also to drive virtual group realization.  Here''s a
skeleton
of how it might work:

class auth::constants {
  $available_groups = hiera(''groups'')
}

class auth::groups::virtual {
  # Virtual group declarations, such as
  @group { ''dev'': 
    gid => 4242,
    ensure => present
  }
}

define auth::concrete_group () {
  include ''auth::groups::virtual''
  realize Group[$name]
}

class auth::groups {
  include ''auth::constants''

  auth::concrete_group { $auth::constants::available_groups: }
}

class auth::users::virtual {
  include ''auth::constants''

  # Virtual user declarations, such as
  @user { ''jbolling'':
    uid => 4200,
    gid => 4200,
    groups => intersect([''dev'',
''support'', ''ops''],
$auth::constants::available_groups)
  }
}

A few bits are omitted, most notably user realization.  The main concept is 
to declare what you want in the first place, rather than throwing up 
something and trying to tweak it afterward, or trying to build values 
incrementally.  The latter two approaches tends to work poorly in Puppet 
(with certain caveats).

Note also that the above is completely hypothetical.  I think it would 
work, but it''s not based on anything I have actually implemented.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/uo9sWOQTJyMJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

That''s great if you have centralized and co-hosted infrastructure and
are willing to accept the dependancy. Given that this is a small need for a
small number of users on a very small amount of systems (like 3 out of hundreds)
without a centralized backbone between them, implementing LDAP makes little
sense.

On Jul 12, 2012, at 12:52 AM, Denmat wrote:
> Puppet users and groups are fiddly. My current not implemented thinking is
to use ldap and manage pam_groups via puppet on the hosts to get the
granularity.
> 
> More thinking out loud than anything else.
> 
> Den
> 
> On 12/07/2012, at 6:03, Jo Rhett <jrhett@netconsonance.com> wrote:
> 
>> I''m fighting with a ticklish issue.  We have some groups and
users that only belong on some systems. So we made all users virtual and then
realize them in classes specific to those system types.  This works quite well
for the users, but not for the groups. When you specify a user, you have to list
all the groups they are in.
>> 	 groups     =>
[''support'',ops'',''dev''],
>> 
>>  Obviously some groups aren''t realized on all systems, so this
produces an error when usermod is run.
>> 	''/usr/sbin/usermod -G support,ops,dev jrhett''
returned 6: usermod: unknown group dev
>> 	usermod: unknown group dev
>> 
>> So I tried to get smarter, and put logic to add the group to each
member under the appropriate class
>> 	Class users::dev inherits users { 
>> 		User[''jrhett''] { groups +>
[''dev''] }
>> 	}
>> 
>> This works… almost. It works for all instances where the user is only
subclassed once. But if I do the same technique in multiple classes I get
>> 
>> err: Could not retrieve catalog from remote server: Error 400 on
SERVER: Parameter ''groups'' is already set on
User_and_key[jrhett] by #<Puppet::Resource::Type:0x7f4feed2d828> at
/etc/puppet/modules/users/manifests/support.pp:22; cannot redefine at
/etc/puppet/modules/users/manifests/dev.pp:27 on node s2-d1.company.com
>> 
>> So how can this be achieved, short of using an exec with an unless
doing another exec to determine if the group exists?
>> 
>> -- 
>> Jo Rhett
>> Net Consonance : net philanthropy to improve open source and internet
projects.
>> 
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
>> To post to this group, send email to puppet-users@googlegroups.com.
>> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
>> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 12, 2012, at 4:30 AM, Felix Frank wrote:
> On 07/11/2012 10:03 PM, Jo Rhett wrote:
>> So I tried to get smarter, and put logic to add the group to each
member
>> under the appropriate class
>> Class users::dev inherits users { 
>> User[''jrhett''] { groups +>
[''dev''] }
>> }
>> 
>> This works… almost. It works for all instances where the user is only
>> subclassed once. But if I do the same technique in multiple classes I
get
> 
> sound approach, but I''ve hit this wall a couple of times as well.
> 
> I''ve resorted to horrors that would add items to array variables
that
> are declared in a central, well-known class, and use the final value for
> the resources in question. Depending on how much flexibility is
> required, this may not be feasible at all.
Hm. That might work, but seems even uglier :(
> Perhaps hiera can be used to do something clever here?

This is actually something that hiera seems perfect for, but we simply
don''t have any backend dataset from which to derive hiera data at this
time. That is going to change, and I''m looking forward to having hiera
access at that point.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 12, 2012, at 6:46 AM, jcbollinger wrote:
> If it is the case that each user always has the same potential secondary
groups, and you need to narrow the actual secondary groups to those that are
actually present, then I think you could do it without too much pain.  The main
ingredients would be a list (array) of the groups that are supposed to be
present, and a custom function that forms the intersection of two arrays.  (Or
you could use an inline template and split(), but yuck!)
> 
> Hiera would probably provide a good means for building the list of
available groups, which you could then use not only to filter user definitions
but also to drive virtual group realization.  Here''s a skeleton of how
it might work:
> 
> class auth::constants {
>   $available_groups = hiera(''groups'')
> }
Interesting idea, but depends on an external datasource that tells us which
groups are valid.  Since all of these groups are already defined in puppet, I
simply don''t see the value of managing intersections of data between a
hiera data source and puppet.
>   # Virtual user declarations, such as
>   @user { ''jbolling'':
>     uid => 4200,
>     gid => 4200,
>     groups => intersect([''dev'',
''support'', ''ops''],
$auth::constants::available_groups)
>   }
> }

I think the intersect idea is valid, as long as I can find out if a parameter is
realized or not.  Basically, write a function that removes from the array any
group which isn''t realized. This removes any need for heira.  However
I''m poking around and the docs don''t show any methods to
determine if something has been realized or not.

If I am reading this right, intersect is provided by stdlib, right?  So I really
just need to write a function to determine if something is realized or not. I
suspect this is going to fall back to the same issues as defined() unless I can
delay execution until the end.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 12, 2012 1:42:28 PM UTC-5, Jo wrote:
>
> On Jul 12, 2012, at 6:46 AM, jcbollinger wrote:
>
> If it is the case that each user always has the same potential secondary 
> groups, and you need to narrow the actual secondary groups to those that 
> are actually present, then I think you could do it without too much pain.  
> The main ingredients would be a list (array) of the groups that are 
> supposed to be present, and a custom function that forms the intersection 
> of two arrays.  (Or you could use an inline template and split(), but
yuck!)
>
> Hiera would probably provide a good means for building the list of 
> available groups, which you could then use not only to filter user 
> definitions but also to drive virtual group realization.  Here''s a
skeleton
> of how it might work:
>
> class auth::constants {
>   $available_groups = hiera(''groups'')
> }
>
>
> Interesting idea, but depends on an external datasource that tells us 
> which groups are valid.  Since all of these groups are already defined in 
> puppet, I simply don''t see the value of managing intersections of
data
> between a hiera data source and puppet.
>
No, it doesn''t depend on an external datasource; rather, It depends on 
up-front knowledge of which groups are supposed to be realized for the 
node.  Although I proposed using an external datasource to provide that 
data, it could just as well be provided by an ENC or determined via DSL 
code based on conditionals, node facts, etc.  Even class parameters.

 
>
>   # Virtual user declarations, such as
>   @user { ''jbolling'':
>     uid => 4200,
>     gid => 4200,
>     groups => intersect([''dev'',
''support'', ''ops''],
> $auth::constants::available_groups)
>   }
> }
>
>
> I think the intersect idea is valid, as long as I can find out if a 
> parameter is realized or not.  Basically, write a function that removes 
> from the array any group which isn''t realized. This removes any
need for
> heira.  However I''m poking around and the docs don''t show
any methods to
> determine if something has been realized or not.
>
> If I am reading this right, intersect is provided by stdlib, right?
>
If so, then I''m somehow overlooking it.  My suggestion and expectation
was
that you would create it yourself, but it seems sufficiently 
general-purpose that you might find something suitable already made.  You 
can also, of course, jerry-rig something based on inline_template().
 
>  So I really just need to write a function to determine if something is 
> realized or not. I suspect this is going to fall back to the same issues as
> defined() unless I can delay execution until the end.
>
>
I would avoid that variation on this approach if at all possible.  You 
would sidestep multiple pitfalls if you could determine up front, based on 
node name and facts, which groups are *supposed* to be present, instead of 
attempting to determine after the fact which were realized.  Indeed, you 
might even find it convenient to use that information to drive group 
realization.  If nothing else, doing so would ensure that users aren''t 
assigned to secondary groups that don''t get realized.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/tO-mgaYJ7-sJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 12, 2012, at 2:26 PM, jcbollinger wrote:
> I would avoid that variation on this approach if at all possible.  You
would sidestep multiple pitfalls if you could determine up front, based on node
name and facts, which groups are supposed to be present, instead of attempting
to determine after the fact which were realized.  Indeed, you might even find it
convenient to use that information to drive group realization.
> If nothing else, doing so would ensure that users aren''t assigned
to secondary groups that don''t get realized.
This is what policy as expressed in the puppet manifests does. I don''t
see how to avoid the unrealized problem here.

What''s funny is that you are expressing exactly what puppet does today,
but it appears you are suggesting that I need to create another data source and
mirror the information out of puppet manifests into that for comparison
purposes. Huh?

I''m a bit baffled by the fairly constant suggestion by people here that
I keep spreading out the places where information is stored. The point is to
centralize the data, not provide more sources to grow inconsistent with each
other.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 12, 2012 6:04:17 PM UTC-5, Jo wrote:
>
> On Jul 12, 2012, at 2:26 PM, jcbollinger wrote:
>
> I would avoid that variation on this approach if at all possible.  You 
> would sidestep multiple pitfalls if you could determine up front, based on 
> node name and facts, which groups are *supposed* to be present, instead 
> of attempting to determine after the fact which were realized.  Indeed, you
> might even find it convenient to use that information to drive group 
> realization.
>
> If nothing else, doing so would ensure that users aren''t assigned
to
> secondary groups that don''t get realized.
>
>
> This is what policy as expressed in the puppet manifests does. I
don''t see
> how to avoid the unrealized problem here.
>
> What''s funny is that you are expressing exactly what puppet does
today,
> but it appears you are suggesting that I need to create another data source
> and mirror the information out of puppet manifests into that for comparison
> purposes. Huh?
>
> I''m a bit baffled by the fairly constant suggestion by people here
that I
> keep spreading out the places where information is stored. The point is to 
> centralize the data, not provide more sources to grow inconsistent with 
> each other.
>
>  
Relying on a single source of information is *exactly* what what I have 
suggested you do, specifically by using an up-front group list both to 
filter users'' declared secondary groups and to drive which groups get 
realized.  I have described that three times now, and it''s included in
the
example code I posted earlier.  You can populate such a list by whatever 
means you want and from whatever source you want, and you can store it 
wherever you want, so long as you produce the entire list before any part 
of it is needed.

So no, I''m not suggesting you mirror information from your puppet 
manifests.  Rather, I am suggesting that you *move* implicit information 
out of your manifests to someplace more accessible.  Study my example if 
you still don''t understand what I mean by that.  The
"someplace" where the
information lands could be an explicit expression elsewhere in your 
manifests, or it could be external, as seems best to you. The information 
implicitly encoded in the structure of your manifests and/or developed 
during compilation is inherently difficult to use from within the manifests 
themselves, and if you insist on using it anyway then you''re choosing
to be
stuck in an uncomfortable position.

More generally, people recommending various possible data sources to you -- 
hiera, ENC, etc. -- are not implying that you should "spread out" your
data.  That''s a function of your own manifest designs and how you use
the
data.  You do a disservice to those volunteering their help to you by 
criticizing them for deficiencies in *your imagined applications* of their 
suggestions.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/ldsUjX6CCy0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/13/2012 05:02 PM, jcbollinger wrote:
> More generally, people recommending various possible data sources to you
> -- hiera, ENC, etc. -- are not implying that you should "spread
out"
> your data.  That''s a function of your own manifest designs and how
you
> use the data.  You do a disservice to those volunteering their help to
> you by criticizing them for deficiencies in /your imagined applications/
> of their suggestions.
Though he did put it quite bluntly, I do believe that Jo has a point.

The thing is, I generally want my manifests to be clever about some
things. When I include my mysql development class, I may want to realize
a couple of users and groups as a result (hypothetically speaking - I
have no such class nor such requirements, but there are other things in
this vein).

I would tell hiera to have puppet include the mysql development class,
not each single user and group. That would strike me as silly.

So there *is* value in constructing information inside the manifest from
outside information, or at least that is my firm belief.

Configuring roles and other settings of larger granularity is what I
want to do in hiera. Picking single user accounts - probably not so much.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Friday, July 13, 2012 10:17:25 AM UTC-5, Felix.Frank
wrote:
>
> On 07/13/2012 05:02 PM, jcbollinger wrote: 
> > More generally, people recommending various possible data sources to
you
> > -- hiera, ENC, etc. -- are not implying that you should "spread
out"
> > your data.  That''s a function of your own manifest designs
and how you
> > use the data.  You do a disservice to those volunteering their help to
> > you by criticizing them for deficiencies in /your imagined
applications/
> > of their suggestions. 
>
> Though he did put it quite bluntly, I do believe that Jo has a point. 
>
Do you mean that it would be useful to have a reliable way for manifests to 
extract information about declarations appearing some unspecified 
elsewhere?  I don''t dispute that, but the fact is that Puppet does not
have
such a mechanism, and I don''t see any likelihood that it will have one 
soon.  The point is that that limitation does not create a need for data 
duplication, despite Jo''s assertions to the contrary.
 
>
> The thing is, I generally want my manifests to be clever about some 
> things. When I include my mysql development class, I may want to realize 
> a couple of users and groups as a result (hypothetically speaking - I 
> have no such class nor such requirements, but there are other things in 
> this vein).
>
I don''t mean to suggest that Puppet already does the best it is
possible to
do in this area.  Indeed, my comments to Jo are entirely about working 
within Puppet''s current constraints, not about wishlist features.

Nevertheless, the essential problem is a hard one: to determine what 
declarations satisfying certain criteria will have been made by the end of 
catalog compilation.  That''s the underlying problem with using
defined(),
using a hypothetical realized() function, building values cooperatively, 
and perhaps other potentially useful things.  It would be convenient to be 
able to do those safely and reliably, but solving the key problem would 
likely require a fundamental change in the manifest compiler''s
architecture.
 
>
> I would tell hiera to have puppet include the mysql development class, 
> not each single user and group. That would strike me as silly. 
>
Sure, but I''m not seeing how that relates.  A more parallel situation
would
be if in addition, some unrelated class wanted to be able to determine 
which users and groups the mysql development class had declared.  As Puppet 
now stands, the best way would be for the mysql development class to 
provide that data in class variables, or else to have obtained it from some 
shared source in the first place.  The point is that neither of those 
options requires that data to be duplicated in the structure of the class.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/OvZbRRPlwpMJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 13, 2012, at 8:02 AM, jcbollinger wrote:
> Relying on a single source of information is exactly what what I have
suggested you do, specifically by using an up-front group list both to filter
users'' declared secondary groups and to drive which groups get
realized.  I have described that three times now, and it''s included in
the example code I posted earlier.  You can populate such a list by whatever
means you want and from whatever source you want, and you can store it wherever
you want, so long as you produce the entire list before any part of it is
needed.
I did not see that from what you showed. Your example didn''t show how
to aggregate or use the data at all. I saw six classes that were significantly
more complex and appeared to require defining the data in multiple places. There
was certainly no obvious way this would reduce my data sources.
> So no, I''m not suggesting you mirror information from your puppet
manifests.  Rather, I am suggesting that you move implicit information out of
your manifests to someplace more accessible.  Study my example if you still
don''t understand what I mean by that.  The "someplace" where
the information lands could be an explicit expression elsewhere in your
manifests, or it could be external, as seems best to you. The information
implicitly encoded in the structure of your manifests and/or developed during
compilation is inherently difficult to use from within the manifests themselves,
and if you insist on using it anyway then you''re choosing to be stuck
in an uncomfortable position.
I hear what you are saying, but I really don''t see how your example
makes this idea clear. I saw multiple sets of classes relying on each
other''s data in an unreadable manner.

I would argue that even if it does do what I meant, the very fact that I
couldn''t read it to understand this ensures nobody else here has a
chance at maintaining it. "More complex" is not a desired trait here.

In general I see ENCs as eventually providing a way to simplify the data input,
but that''s not what I''ve seen recommended or demonstrated. The
case for ENCs would be made a lot stronger if some good examples of ways to
simply via the use of ENCs were posted.
> More generally, people recommending various possible data sources to you --
hiera, ENC, etc. -- are not implying that you should "spread out" your
data.  That''s a function of your own manifest designs and how you use
the data.  You do a disservice to those volunteering their help to you by
criticizing them for deficiencies in your imagined applications of their
suggestions.

I could go back and make a line by line review of every single time people have
told me that I should take data from the puppet manifests and reinforce it /
control it via data from an ENC. There hasn''t been a single situation
where someone said what you are suggesting -- hey, pull this all out of puppet
and incorporate it this way (x) so that you can get what you want.  It has
always been how to do half the job in puppet and half the job in something else,
and manually manage the dependancies between the two.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

John,

On 07/13/2012 06:38 PM, jcbollinger wrote:
> 
>     I would tell hiera to have puppet include the mysql development class,
>     not each single user and group. That would strike me as silly.
> 
> 
> Sure, but I''m not seeing how that relates.  A more parallel
situation
> would be if in addition, some unrelated class wanted to be able to
> determine which users and groups the mysql development class had
> declared.  As Puppet now stands, the best way would be for the mysql
> development class to provide that data in class variables, or else to
> have obtained it from some shared source in the first place.  The point
> is that neither of those options requires that data to be duplicated in
> the structure of the class.
I''m not sure I concur with your conclusion, nor with what you boil the
problem at hand down to.

Puppet *does* have means to implement business logic that adds resources
as implications of roles or aspects, and that is by the singleton nature
of classes. You can add users via implication of different possible
aspects of nodes by having the "aspect" class include the approprate
user class(es) for example.

Puppet even allows to make adjustments to such implied resources by
means of subclasses. If a node has a very certain role, the respective
class will include a subclass of some other feature to specialize its
resources appropriately.

All this is according to what puppet has so far been designed to do.

The problem at hand can be handled to a degree with these language
features, as the OP has successfully done already. The approach has
scaling issues, so a workaround is currently needed. I don''t see why
the
approach should be discarded as "not currently implemented" out of
hand,
seeing as the better part of it *is* in fact possible.

Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Felix,

On Monday, July 16, 2012 2:44:58 AM UTC-5, Felix.Frank
wrote:
>
> John, 
>
> On 07/13/2012 06:38 PM, jcbollinger wrote: 
> > 
> >     I would tell hiera to have puppet include the mysql development 
> class, 
> >     not each single user and group. That would strike me as silly. 
> > 
> > 
> > Sure, but I''m not seeing how that relates.  A more parallel
situation
> > would be if in addition, some unrelated class wanted to be able to 
> > determine which users and groups the mysql development class had 
> > declared.  As Puppet now stands, the best way would be for the mysql 
> > development class to provide that data in class variables, or else to 
> > have obtained it from some shared source in the first place.  The
point
> > is that neither of those options requires that data to be duplicated
in
> > the structure of the class. 
>
> I''m not sure I concur with your conclusion, nor with what you boil
the
> problem at hand down to. 
>
I didn''t define the problem, Jo did.  He defined at least two different
problems, in fact, so perhaps that''s where you and I have gotten out of
sync.  The first problem was to build the value of a resource property 
based on declarations in multiple unrelated classes.  Jo then went on to 
object to some proposed alternative solutions on the basis that they 
introduced duplication of data already implicit in his class and resource 
declarations, making avoiding such duplication the second problem.
 
>
> Puppet *does* have means to implement business logic that adds resources 
> as implications of roles or aspects, and that is by the singleton nature 
> of classes. You can add users via implication of different possible 
> aspects of nodes by having the "aspect" class include the
approprate
> user class(es) for example. 
>
> Puppet even allows to make adjustments to such implied resources by 
> means of subclasses. If a node has a very certain role, the respective 
> class will include a subclass of some other feature to specialize its 
> resources appropriately. 
>
> All this is according to what puppet has so far been designed to do. 
>
> The problem at hand can be handled to a degree with these language 
> features, as the OP has successfully done already. The approach has 
> scaling issues, so a workaround is currently needed. I don''t see
why the
> approach should be discarded as "not currently implemented" out
of hand,
> seeing as the better part of it *is* in fact possible.
>
The fact that the approach doesn''t work isn''t sufficient? 
Until now, my
focus has been on something that Jo could actually use, rather than on 
advocacy for new Puppet features.  If you have a good way to make that 
approach work for Jo, with current Puppet, then I would be genuinely 
interested in hearing it.  I suppose Jo would be even more so.

I''m not suggesting that Puppet shouldn''t have a feature that
allowed Jo to
do what he wants, approximately how he wants, without parse-order 
dependencies or other serious drawbacks.  I would love it if there were 
such a feature.  In fact, I think it would serve a lot of people well, and 
allow clearer, simpler manifests.

That feature isn''t available today, as far as I know, but I have an
idea
how it might look.  In fact, I had it about six months ago.  Remember 
resource constraints 
(https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?  
The idea arose as an approach to cross-module dependencies, but it would 
address this problem as well.  In fact, this problem is pretty similar to a 
cross-module dependency issue, and might even be one.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/2A8pqYpNEeoJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/16/2012 04:14 PM, jcbollinger wrote:
> The fact that the approach doesn''t work isn''t sufficient?
Until now, my
> focus has been on something that Jo could actually use, rather than on
> advocacy for new Puppet features.  If you have a good way to make that
> approach work for Jo, with current Puppet, then I would be genuinely
> interested in hearing it.  I suppose Jo would be even more so.
I cannot, of course, but I do sympathize with Jo''s notion that in order
to solve the apparently small problem of making resource overrides
scale, he is now required to rework most if not all of his manifests to
play with a hiera based approach.
I''m not saying there is a simpler solution right now. And
we''ve already
agreed there should be:
> I''m not suggesting that Puppet shouldn''t have a feature
that allowed Jo
> to do what he wants, approximately how he wants, without parse-order
> dependencies or other serious drawbacks.  I would love it if there were
> such a feature.  In fact, I think it would serve a lot of people well,
> and allow clearer, simpler manifests.
> 
> That feature isn''t available today, as far as I know, but I have
an idea
> how it might look.  In fact, I had it about six months ago.  Remember
> resource constraints
>
(https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?
> The idea arose as an approach to cross-module dependencies, but it would
> address this problem as well.  In fact, this problem is pretty similar
> to a cross-module dependency issue, and might even be one.
I do. Sometimes even wondered if that made it to some pin-board down at
puppetlabs ;-)

But I''m not sure they''re the best solution for the problem at
hand. My
understanding is that you would have each class declare constraints on
given users'' group memberships, e.g. user jo needs to be in the devs
group.
The only real advantage I see here is that, yes, you can make the
compilation fail if your hiera data is not consistent with your business
logic. That doesn''t really simplify the way to get to the desired
catalog.

Am I missing a wholly different implied intention of your''s here?

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 16, 2012, at 8:42 AM, Felix Frank wrote:
> I cannot, of course, but I do sympathize with Jo''s notion that in
order
> to solve the apparently small problem of making resource overrides
> scale, he is now required to rework most if not all of his manifests to
> play with a hiera based approach.

Well, more matter of factly, that shifting to a hiera-based approach would
require us to manage very carefully the balance of data between puppet and
hiera, and manage by eyeball the dependencies between the two. There is
considerable resistance to this idea here.

If it was possible to put all the user and group information in hiera and then
put the assignment/management of that information into puppet then we could
probably manage that. But having to edit "this host gets the sql server
info" in puppet and then "these users get put in mysql group on this
host" in hiera is completely nonfunctional, and I''ve seen no
examples of ways to bridge that gap.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

(inline)

On Mon, Jul 16, 2012 at 11:19:02AM -0700, Jo Rhett
wrote:
>    On Jul 16, 2012, at 8:42 AM, Felix Frank wrote:
> 
>      I cannot, of course, but I do sympathize with Jo''s notion
that in order
>      to solve the apparently small problem of making resource overrides
>      scale, he is now required to rework most if not all of his manifests
to
>      play with a hiera based approach.
> 
>    Well, more matter of factly, that shifting to a hiera-based approach
would
>    require us to manage very carefully the balance of data between puppet
and
>    hiera, and manage by eyeball the dependencies between the two. There is
>    considerable resistance to this idea here.
>    If it was possible to put all the user and group information in hiera
and
>    then put the assignment/management of that information into puppet then
we
>    could probably manage that. But having to edit "this host gets the
sql
>    server info" in puppet and then "these users get put in mysql
group on
>    this host" in hiera is completely nonfunctional, and I''ve
seen no examples
>    of ways to bridge that gap.
Possibly something like the following pseudocode example? The main point being
to only include a puppet class if there''s a certain piece of data in
hiera.

node default {
  if hiera(''usemysql'') {
    include mysql::service
  }
  if hiera_array(''users'') {
    include users
  }
}

(I haven''t tested the above myself. We''re still not using
hiera at work, more''s the pity.)
>    -- 
>    Jo Rhett
>    Net Consonance : net philanthropy to improve open source and internet
>    projects.
> 
>    --
>    You received this message because you are subscribed to the Google
Groups
>    "Puppet Users" group.
>    To post to this group, send email to puppet-users@googlegroups.com.
>    To unsubscribe from this group, send email to
>    puppet-users+unsubscribe@googlegroups.com.
>    For more options, visit this group at
>    http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 16, 2012, at 11:55 AM, Christopher Wood wrote:
> Possibly something like the following pseudocode example? The main point
being to only include a puppet class if there''s a certain piece of data
in hiera.
> 
> node default {
>  if hiera(''usemysql'') {
>    include mysql::service
>  }
>  if hiera_array(''users'') {
>    include users
>  }
> }
I''m not sure how this would work.  So you''re now talking about
putting all the if/then logic inside hiera?
> (I haven''t tested the above myself. We''re still not using
hiera at work, more''s the pity.)

We aren''t, because we have no external datasource for this stuff and
every example we''ve seen (like yours above) indicates that
we''re going to have to put half of the logic engine of puppet inside
the data source, which means it needs to be a very complex thing that enforces
the structure and somehow ties it with the puppet logic. Our analysis so far is
that to implement hiera we''re going to have to write our own software
platform which manages hiera data and writes out puppet policies on the fly when
the data changes.

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/16/2012 10:29 PM, Jo Rhett wrote:
> We aren''t, because we have no external datasource for this stuff
and
> every example we''ve seen (like yours above) indicates that
we''re going
> to have to put half of the logic engine of puppet inside the data
> source, which means it needs to be a very complex thing that enforces
> the structure and somehow ties it with the puppet logic. Our analysis so
> far is that to implement hiera we''re going to have to write our
own
> software platform which manages hiera data and writes out puppet
> policies on the fly when the data changes.
Not quite. I believe that the canonical approach is to move your node ->
roles relation into hiera. This way you need little "individual"
manifest code per node.
You certainly need a means to manage hiera''s datastores, but I
don''t
think generating manifests is required.

Anyway, as participants in this thread seem to agree, the approach is
limited insofar that puppet has some shortcomings in the area of
constructing complex derived data (e.g. groups for given users) from
complex and disjoint pieces of input data.

Since you''re going to end up with some sort of kludge anyway (imho), I
believe it''s perfectly fine for you to forego the hiera backend at this
time and apply the kludge inside your comfort zone, for what it''s
worth.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Monday, July 16, 2012 10:42:11 AM UTC-5, Felix.Frank
wrote:
>
> On 07/16/2012 04:14 PM, jcbollinger wrote: 
> > That feature isn''t available today, as far as I know, but I
have an idea
> > how it might look.  In fact, I had it about six months ago.  Remember 
> > resource constraints 
> > (
>
https://groups.google.com/forum/?fromgroups#!topic/puppet-users/Fvl0aOe4RPE)?
>
> > The idea arose as an approach to cross-module dependencies, but it
would
> > address this problem as well.  In fact, this problem is pretty similar
> > to a cross-module dependency issue, and might even be one. 
>
> I do. Sometimes even wondered if that made it to some pin-board down at 
> puppetlabs ;-) 
>
> But I''m not sure they''re the best solution for the
problem at hand. My
> understanding is that you would have each class declare constraints on 
> given users'' group memberships, e.g. user jo needs to be in the
devs
> group. 
> The only real advantage I see here is that, yes, you can make the 
> compilation fail if your hiera data is not consistent with your business 
> logic. That doesn''t really simplify the way to get to the desired
catalog.
>
> Am I missing a wholly different implied intention of your''s here? 
>
Apparently so.  I don''t want to drag this thread off into a rehash of
the
constraints idea, but one of the central ideas is that it allows 
cooperative specification of resource properties.  Constraints -- as I 
envision them -- are not a dynamic validation feature, but rather an 
indirect, deferred declaration feature.  In many cases, explicit resource 
declarations could be replaced by one or more constraints on the same 
resource, which could appear anywhere in the manifest set.  Everything gets 
resolved after all resources are compiled.

I''ll say no more about that here, but if anyone wants to discuss it
further
then I''d be likely to respond to a new thread on that topic.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/Q_fD1qLRUHwJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 17, 2012, at 12:54 AM, Felix Frank wrote:
> On 07/16/2012 10:29 PM, Jo Rhett wrote:
>> We aren''t, because we have no external datasource for this
stuff and
>> every example we''ve seen (like yours above) indicates that
we''re going
>> to have to put half of the logic engine of puppet inside the data
>> source, which means it needs to be a very complex thing that enforces
>> the structure and somehow ties it with the puppet logic. Our analysis
so
>> far is that to implement hiera we''re going to have to write
our own
>> software platform which manages hiera data and writes out puppet
>> policies on the fly when the data changes.
> 
> Not quite. I believe that the canonical approach is to move your node ->
> roles relation into hiera. This way you need little "individual"
> manifest code per node.
> You certainly need a means to manage hiera''s datastores, but I
don''t
> think generating manifests is required.

Is this not the epitome of diverse and redundant dependancies? I can''t
edit my hiera data without evaluating puppet manifests, I can''t edit
the puppet manifests without editing the hiera data…

In short, I''ll look at hiera as soon as I have time to build out a
whole new infrastructure for data management. And trust me, free time is
something I have lots of. (not)

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 17, 2012, at 6:30 AM, jcbollinger wrote:
> Apparently so.  I don''t want to drag this thread off into a rehash
of the constraints idea, but one of the central ideas is that it allows
cooperative specification of resource properties.  Constraints -- as I envision
them -- are not a dynamic validation feature, but rather an indirect, deferred
declaration feature.  In many cases, explicit resource declarations could be
replaced by one or more constraints on the same resource, which could appear
anywhere in the manifest set.  Everything gets resolved after all resources are
compiled.
Sounds like treating hiera data as virtualized to me (and sounds like a
functional way to deal with the issues we are discussing).  How would you
implement this today?
> I''ll say no more about that here, but if anyone wants to discuss
it further then I''d be likely to respond to a new thread on that topic.
Seems like a thread that you should name, unless you want a thread labelled
"Ask John" … ;-)

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tuesday, July 17, 2012 12:35:34 PM UTC-5, Jo wrote:
>
> On Jul 17, 2012, at 6:30 AM, jcbollinger wrote:
>
> Apparently so.  I don''t want to drag this thread off into a rehash
of the
> constraints idea, but one of the central ideas is that it allows 
> cooperative specification of resource properties.  Constraints -- as I 
> envision them -- are not a dynamic validation feature, but rather an 
> indirect, deferred declaration feature.  In many cases, explicit resource 
> declarations could be replaced by one or more constraints on the same 
> resource, which could appear anywhere in the manifest set.  Everything gets
> resolved after all resources are compiled.
>
>
> Sounds like treating hiera data as virtualized to me (and sounds like a 
> functional way to deal with the issues we are discussing).  How would you 
> implement this today?
>
And now I''m breaking my promise, but it *is* your thread... 

The resource constraints idea has no direct relationship to hiera. 
There''s
no reason why the two shouldn''t interoperate, but neither depends on
the
other.

I am not at all sure there is a clean way to build it on the (intended) 
extension points available today, however.  As I conceive it, it involves a 
catalog tweaking step between initial compilation and delivery to the 
agent, and as such it would require modifications to the Puppet core.  
Perhaps one could approximate it, however.  The most likely way I can think 
of to do that involves custom functions.

At least two functions would be needed: one to cache data somewhere in the 
master (invoked potentially many times) and one to read back and apply that 
data (probably called fewer times, and very late in the compilation -- such 
as in a final run stage).  I apologize for how abstract and vague that 
description is, but there is a great deal more design effort needed than I 
am prepared to exert at the moment.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/TQBINg-GU4MJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jul 17, 2012, at 12:42 PM, jcbollinger wrote:
> I apologize for how abstract and vague that description is, but there is a
great deal more design effort needed than I am prepared to exert at the moment.

No worries. It confirms what is necessary for me right now which is that
it''s not something I should be testing anytime soon ;-)

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/17/2012 07:31 PM, Jo Rhett wrote:
>
> Is this not the epitome of diverse and redundant dependancies? I
can''t
> edit my hiera data without evaluating puppet manifests, I can''t
edit the
> puppet manifests without editing the hiera data…
Hmm, by "managing your hiera data" I wasn''t implying that
puppet should
manage them. I don''t think that can work.

Rather, if you''re not intending to scribble YAML files by hand (which
is
entirely possible), you would have to write a web frontend or similar.

On 07/17/2012 09:42 PM, jcbollinger wrote:
> At least two functions would be needed: one to cache data somewhere in
> the master (invoked potentially many times) and one to read back and
> apply that data (probably called fewer times, and very late in the
> compilation -- such as in a final run stage).  I apologize for how
> abstract and vague that description is, but there is a great deal more
> design effort needed than I am prepared to exert at the moment.
This seems reminiscent of the "futures" idea that puppetlabs bounced
around a year back or so. Implementing such things is really a lot
harder than it sounds, because you need to make sure that your compiler
does not take x^n (x raised to the power of n) loops, where n is e.g.
the number of resources in your manifest.

I may be overthinking this, though. Also this branch of discussion
probably belongs on the dev list. Kudos for thinking this up in the
first place, anyhow.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 07/17/2012 07:31 PM, Jo Rhett wrote:
>> Is this not the epitome of diverse and redundant dependancies? I
can''t
>> edit my hiera data without evaluating puppet manifests, I
can''t edit the
>> puppet manifests without editing the hiera data…
On Jul 18, 2012, at 1:19 AM, Felix Frank wrote:
> Rather, if you''re not intending to scribble YAML files by hand
(which is
> entirely possible), you would have to write a web frontend or similar.

Understood. The problem is that every example I''ve seen to date you
have to know the puppet module code well to edit the data. The value of a
front-end would be to allow users other than ruby coders to manage the data. I
haven''t seen any examples with enough differentiation for that.

Ultimately I guess you build an entire ecosystem where you tie the front-end
data management code to the puppet manifests in git and update both at the same
time--but that''s one hell of an infrastructure creation that I
don''t have enough free time for.  In short, I believe that hiera is
only useful for companies who already have a large information schema from which
to draw their data, and you are only editing the hiera adapters to get the data
as you update the puppet manifests.  Anyone else has a huge project to get to
"useful".

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.