Puppet users - Jul 2012 - What is the intention of thin_storeconfigs?

I started doing some experiments with the configuration option
''thin_storeconfigs=true'' by adding this option to one of my
Puppet masters. However, I could not determine any change in behavior. I
expected to have the resources collected faster, but Puppet still takes some
15min to do the job. So maybe I misunderstood something. Should this option
instead be placed in the client''s configuration to make them export
only the @@-resources?

Couldn''t find any documentation. So any help is appreciated.

Thanks
Bernd

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi,

Thin_storeconfig stores less detail in the database, that won''t
necessarily speed it up but will reduce its footprint.

The actual differences I''m a bit fuzzy on.

Den

On 12/07/2012, at 18:29, Bernd Adamowicz <Bernd.Adamowicz@esailors.de>
wrote:
> I started doing some experiments with the configuration option
''thin_storeconfigs=true'' by adding this option to one of my
Puppet masters. However, I could not determine any change in behavior. I
expected to have the resources collected faster, but Puppet still takes some
15min to do the job. So maybe I misunderstood something. Should this option
instead be placed in the client''s configuration to make them export
only the @@-resources?
> 
> Couldn''t find any documentation. So any help is appreciated.
> 
> Thanks
> Bernd
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thin stored configs only store exported resources in the database Instead
of all resources. There may be more differences though.

Sent from my galaxy tab
On Jul 12, 2012 6:01 PM, "Denmat" <tu2bgone@gmail.com> wrote:
> Hi,
>
> Thin_storeconfig stores less detail in the database, that won''t
> necessarily speed it up but will reduce its footprint.
>
> The actual differences I''m a bit fuzzy on.
>
> Den
>
> On 12/07/2012, at 18:29, Bernd Adamowicz
<Bernd.Adamowicz@esailors.de>
> wrote:
>
> > I started doing some experiments with the configuration option
> ''thin_storeconfigs=true'' by adding this option to one of
my Puppet masters.
> However, I could not determine any change in behavior. I expected to have
> the resources collected faster, but Puppet still takes some 15min to do the
> job. So maybe I misunderstood something. Should this option instead be
> placed in the client''s configuration to make them export only the
> @@-resources?
> >
> > Couldn''t find any documentation. So any help is appreciated.
> >
> > Thanks
> > Bernd
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 12, 2012 3:29:15 AM UTC-5, badamowicz
wrote:
>
> I started doing some experiments with the configuration option 
> ''thin_storeconfigs=true'' by adding this option to one of
my Puppet masters.
> However, I could not determine any change in behavior. I expected to have 
> the resources collected faster, but Puppet still takes some 15min to do the
> job. So maybe I misunderstood something. Should this option instead be 
> placed in the client''s configuration to make them export only the 
> @@-resources? 
>
> Couldn''t find any documentation. So any help is appreciated. 
>
>
Google gave me this as my number one hit: 
http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration.  
It''s a bit dated, but it nevertheless contains the key information:
"[thin
storeconfigs] only collects and stores to the database exported resources, 
tags and host facts."

''Thick'' storeconfigs additionally stores non-exported
resources, and
possibly other things -- overall, a relational representation of nodes''
entire catalogs.

I would expect thin storeconfigs to accelerate resource collection relative 
to thick only if the *overall* database size were slowing the DBMS.  If the 
problem is specifically with the number and/or complexity of exported 
resources, then thin vs. thick should not make a difference in that area.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/ehNmEJw6uVUJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thursday, July 12, 2012 3:29:15 AM UTC-5, badamowicz
wrote:
>
> Should [thin_storeconfigs] instead be placed in the client''s
configuration
> to make them export only the @@-resources? 
>
No, (thin_)storeconfigs is a function of the master, at catalog compilation 
time.

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/7_ug4xm5OFAJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks to all who gave feedback on this topic. I will have a deeper look at this
issue throughout the next days and weeks and will share my experience then.

Bernd
> -----Ursprüngliche Nachricht-----
> Von: Bernd Adamowicz
> Gesendet: Donnerstag, 12. Juli 2012 10:29
> An: ''puppet-users@googlegroups.com''
> Betreff: What is the intention of thin_storeconfigs?
> 
> I started doing some experiments with the configuration option
> ''thin_storeconfigs=true'' by adding this option to one of
my Puppet
> masters. However, I could not determine any change in behavior. I
> expected to have the resources collected faster, but Puppet still takes
> some 15min to do the job. So maybe I misunderstood something. Should
> this option instead be placed in the client''s configuration to
make
> them export only the @@-resources?
> 
> Couldn''t find any documentation. So any help is appreciated.
> 
> Thanks
> Bernd
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 12/07/12 10:29, Bernd Adamowicz wrote:
> I started doing some experiments with the configuration option
> ''thin_storeconfigs=true'' by adding this option to one of
my Puppet
> masters. However, I could not determine any change in behavior. 
As others already have explained, with thin_storeconfigs, only exported
resources, facts and nodes are persisted to the DB. With regular (thick)
storeconfigs every resources are persisted to the database.
> I expected to have the resources collected faster, but Puppet still
> takes some 15min to do the job. 
The thing is that if you had run with regular storeconfigs before
activating the thin_storeconfigs option, then your database is already
populated with all the resources definitions and parameters. So the
first time you run with thin_storeconfigs you end up collecting as if
thick was activated, then after the first catalog run (for a given
node), puppet should remove all the un-needed resources (ie the non
exported ones) from the database.
If that doesn''t happen, I would suggest you to cleanup the database for
your nodes so that only exported resources are persisted and collected.
> So maybe I misunderstood something.
> Should this option instead be placed in the client''s configuration
to
> make them export only the @@-resources?
No, it''s a master option.
-- 
Brice Figureau
My Blog: http://www.masterzen.fr/


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Brice,

Thanks for this detailed answer. As mentioned, I will try this throughout the
next weeks (will not find the time before.)

Thanks
Bernd
> -----Ursprüngliche Nachricht-----
> Von: puppet-users@googlegroups.com [mailto:puppet-
> users@googlegroups.com] Im Auftrag von Brice Figureau
> Gesendet: Freitag, 13. Juli 2012 11:08
> An: puppet-users@googlegroups.com
> Betreff: Re: [Puppet Users] What is the intention of thin_storeconfigs?
> 
> On 12/07/12 10:29, Bernd Adamowicz wrote:
> > I started doing some experiments with the configuration option
> > ''thin_storeconfigs=true'' by adding this option to
one of my Puppet
> > masters. However, I could not determine any change in behavior.
> 
> As others already have explained, with thin_storeconfigs, only exported
> resources, facts and nodes are persisted to the DB. With regular
> (thick) storeconfigs every resources are persisted to the database.
> 
> > I expected to have the resources collected faster, but Puppet still
> > takes some 15min to do the job.
> 
> The thing is that if you had run with regular storeconfigs before
> activating the thin_storeconfigs option, then your database is already
> populated with all the resources definitions and parameters. So the
> first time you run with thin_storeconfigs you end up collecting as if
> thick was activated, then after the first catalog run (for a given
> node), puppet should remove all the un-needed resources (ie the non
> exported ones) from the database.
> If that doesn''t happen, I would suggest you to cleanup the
database for
> your nodes so that only exported resources are persisted and collected.
> 
> > So maybe I misunderstood something.
> > Should this option instead be placed in the client''s
configuration to
> > make them export only the @@-resources?
> 
> No, it''s a master option.
> --
> Brice Figureau
> My Blog: http://www.masterzen.fr/
> 
> 
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to puppet-
> users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Jul 13, 2012 at 2:08 AM, Brice Figureau
<brice-puppet@daysofwonder.com> wrote:
> On 12/07/12 10:29, Bernd Adamowicz wrote:
>> I started doing some experiments with the configuration option
>> ''thin_storeconfigs=true'' by adding this option to one
of my Puppet
>> masters. However, I could not determine any change in behavior.
>
> As others already have explained, with thin_storeconfigs, only exported
> resources, facts and nodes are persisted to the DB. With regular (thick)
> storeconfigs every resources are persisted to the database.
...and to follow up on this late: when you use PuppetDB, not only is
performance better than full storeconfigs, it is usually better than
thin storeconfigs.  PuppetDB delivers this *without* giving up any
information.

I can''t recommend strongly enough that you look to PuppetDB before you
look to thin storeconfigs in production.

-- 
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.