Christopher Wood
2014-Nov-11 14:13 UTC
[Puppet Users] troubleshooting constraint violations in puppetdb?
Do any of you have troubleshooting tips for the puppetdb error log message about violating database constraints? How would I tell which constraint and which resource, short of logging all sql queries? 2014-11-11 08:29:34,814 ERROR [c.p.p.command] [6133d0c4-3350-4567-abb0-476980a9616e] [replace catalog] Retrying after attempt 4, due to: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "resource_params_pkey" The full error: https://gist.github.com/anonymous/3a169a2b9380196983c6 More details... Per our logs, we've been seeing this with puppetdb 1.6.3 and 2.2.2. So far I've tried, with little success: -raising the storage log level to debug -digging through /var/log/puppetdb/puppetdb.log before/after debug; couldn't see a pattern -stop puppetdb, purge files under /var/lib/puppetdb/mq, start puppetdb; to start clean and see if maybe something was corrupt/cached, but didn't help The kahadb page: https://docs.puppetlabs.com/puppetdb/latest/trouble_kahadb_corruption.html We rebuild preprod machines at intervals and we do use some exported resources but I'd like to know what's happening before I start firing off "puppet node deactivate randomhost" until the problem fixes itself. https://ask.puppetlabs.com/question/88/how-can-i-purge-exported-resources-from-puppetdb/ Logging all sql queries: http://stackoverflow.com/questions/722221/how-to-log-postgres-sql-queries The stack: java 1.7.0 openjdk postgresql 8.4 (it came with CentOS) puppet 3.7.2 puppetmasters and on puppetdb host, but 3.6.2 on most nodes puppetdb 2.2.2 puppetdb-terminus 2.2.2 on all puppetmasters -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20141111141320.GA6861%40iniquitous.heresiarch.ca. For more options, visit https://groups.google.com/d/optout.