Hey all, I''m hoping I can get some information from other users on the list in relationship ak0ska''s problem listed below. I thought I would start a new thread so more users would see this message and not loose it in the original thread which is already pretty long: https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/yRMI1J_y6Ps Basically I want to see if anyone is seeing the same constraint errors at all in your postgresql log: [2013-03-12 05:51:19 CET]DETAIL: Key (catalog)=(c93825c7157cba8c10641b54d9dce17eb0ce618b) is not present in table "catalogs". [2013-03-12 05:51:19 CET]STATEMENT: INSERT INTO certname_catalogs (certname,catalog,timestamp) VALUES ($1,$2,$3) RETURNING * It seems to me this problem is enacted under heavy load or an attenuated scenario like ''load'', but I''m having trouble trying to reproduce. I''m curious basically if this is happening the wild to anyone else other than ak0ska. This is specifically happening during database garbage collection. If anyone sees this, or does not see it - I''d like to know, I''m just interested to see how prolific or non-prolific this problem is so I can get an understanding of impact. I guess I''d be also interested in rough understand what load your PuppetDB is under either way - # of nodes and frequency of ''replace catalog'' actions - if this is something you can release publicly that is :-). If someone is seeing it I may have follow up questions, but for now I''m really interested in seeing if this is an isolated scenario (therefore potentially ''environmental'') or if its prolific and therefore something we need to be more worried about. Thanks in advance all :-). ken. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
On Thursday, March 14, 2013 4:51:25 PM UTC-5, Ken Barber wrote:> > Hey all, > > I''m hoping I can get some information from other users on the list in > relationship ak0ska''s problem listed below. I thought I would start a > new thread so more users would see this message and not loose it in > the original thread which is already pretty long: > > > https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/yRMI1J_y6Ps > > Basically I want to see if anyone is seeing the same constraint errors > at all in your postgresql log: > > [2013-03-12 05:51:19 CET]DETAIL: Key > (catalog)=(c93825c7157cba8c10641b54d9dce17eb0ce618b) is not present in > table "catalogs". > [2013-03-12 05:51:19 CET]STATEMENT: INSERT INTO certname_catalogs > (certname,catalog,timestamp) VALUES ($1,$2,$3) RETURNING * > > It seems to me this problem is enacted under heavy load or an > attenuated scenario like ''load'', but I''m having trouble trying to > reproduce. I''m curious basically if this is happening the wild to > anyone else other than ak0ska. > > This is specifically happening during database garbage collection. > > If anyone sees this, or does not see it - I''d like to know, I''m just > interested to see how prolific or non-prolific this problem is so I > can get an understanding of impact. I guess I''d be also interested in > rough understand what load your PuppetDB is under either way - # of > nodes and frequency of ''replace catalog'' actions - if this is > something you can release publicly that is :-). If someone is seeing > it I may have follow up questions, but for now I''m really interested > in seeing if this is an isolated scenario (therefore potentially > ''environmental'') or if its prolific and therefore something we need to > be more worried about. > > Thanks in advance all :-). > >Not seeing the problem here, still on 1.0.1 (but hoping to upgrade monday) 48 nodes, default 30 min run interval command processing sec/command: 0.129 command processing command/sec 0.0147 enqueing: 0.0371 collection queries: 0.0106> ken. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
On Thu, Mar 14, 2013 at 3:51 PM, Ken Barber <ken@puppetlabs.com> wrote:> Hey all, > > I''m hoping I can get some information from other users on the list in > relationship ak0ska''s problem listed below. I thought I would start a > new thread so more users would see this message and not loose it in > the original thread which is already pretty long: > > > https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/yRMI1J_y6Ps > > Basically I want to see if anyone is seeing the same constraint errors > at all in your postgresql log: > > [2013-03-12 05:51:19 CET]DETAIL: Key > (catalog)=(c93825c7157cba8c10641b54d9dce17eb0ce618b) is not present in > table "catalogs". > [2013-03-12 05:51:19 CET]STATEMENT: INSERT INTO certname_catalogs > (certname,catalog,timestamp) VALUES ($1,$2,$3) RETURNING * > > It seems to me this problem is enacted under heavy load or an > attenuated scenario like ''load'', but I''m having trouble trying to > reproduce. I''m curious basically if this is happening the wild to > anyone else other than ak0ska. > > This is specifically happening during database garbage collection. > > If anyone sees this, or does not see it - I''d like to know, I''m just > interested to see how prolific or non-prolific this problem is so I > can get an understanding of impact. I guess I''d be also interested in > rough understand what load your PuppetDB is under either way - # of > nodes and frequency of ''replace catalog'' actions - if this is > something you can release publicly that is :-). If someone is seeing > it I may have follow up questions, but for now I''m really interested > in seeing if this is an isolated scenario (therefore potentially > ''environmental'') or if its prolific and therefore something we need to > be more worried about. > > Thanks in advance all :-).Worth mentioning that this issue isn''t related to correctness...PuppetDB has built-in retry w/exponential backoff for all incoming commands, so unless the discard/failure counters in your puppetdb dashboard are continually ticking upwards, these errors aren''t a concern in terms of correctness or data loss. :) But as Ken says, it would be interesting to see just how often we run into these kinds of things in the wild! deepak -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
On Thursday, March 14, 2013 4:51:25 PM UTC-5, Ken Barber wrote:> > Hey all, > > I''m hoping I can get some information from other users on the list in > relationship ak0ska''s problem listed below. I thought I would start a > new thread so more users would see this message and not loose it in > the original thread which is already pretty long: > > > https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/yRMI1J_y6Ps > > Basically I want to see if anyone is seeing the same constraint errors > at all in your postgresql log: > >I''m not seeing the errors, but I attached an analysis to your gist. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.