F. Y.
2013-Sep-20 21:17 UTC
[Puppet Users] Puppet nodes unable to send report to Puppet Master running under Passenger
Hello all,
I recently switched to running puppet master using Passenger. However, I am
seeing agents unable to send reports after a bunch of ''master''
processes
starting to accumulate and eat all the RAM and swap space.
Error: Could not send report: Error 500 on SERVER: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
[no address given] and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.2.22 (Ubuntu) Server at puppetmaster.example.com Port
8140</address>
</body></html>
These are the ''master'' processes launched:
root@puppetmaster:/etc/puppet# ps aufx | grep master
root 4352 0.0 0.0 6504 616 pts/1 S+ 17:07 0:00 |
\_ grep --color=auto master
puppet 2100 14.5 7.2 4703940 1793068 ? D 15:06 17:34 master
puppet 2145 4.4 7.6 4434900 1893028 ? D 15:08 5:17 master
puppet 2156 3.4 8.0 7484288 1986164 ? D 15:08 4:08 master
puppet 2182 2.9 8.7 3612508 2169308 ? D 15:08 3:29 master
puppet 2191 4.0 10.7 4893500 2661364 ? D 15:08 4:49 master
puppet 2200 24.8 16.0 12025276 3953904 ? D 15:08 29:39 master
puppet 3585 0.1 0.3 178984 81944 ? S 16:08 0:06 master
puppet 3603 0.2 0.3 178892 88320 ? S 16:08 0:07 master
puppet 3612 0.1 0.3 178196 84316 ? S 16:08 0:03 master
Puppet Master is running on Ubuntu 12.04, and is running Puppet 3.3.0.
Can anyone please advise how to troubleshoot this? I am at my wit''s end
here.
Thanks.
F.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Ellison Marks
2013-Sep-23 17:26 UTC
[Puppet Users] Re: Puppet nodes unable to send report to Puppet Master running under Passenger
Anything showing up in your logs? On Friday, September 20, 2013 2:17:32 PM UTC-7, F. Y. wrote:> > Hello all, > > I recently switched to running puppet master using Passenger. However, I > am seeing agents unable to send reports after a bunch of ''master'' processes > starting to accumulate and eat all the RAM and swap space. > > Error: Could not send report: Error 500 on SERVER: <!DOCTYPE HTML PUBLIC > "-//IETF//DTD HTML 2.0//EN"> > <html><head> > <title>500 Internal Server Error</title> > </head><body> > <h1>Internal Server Error</h1> > <p>The server encountered an internal error or > misconfiguration and was unable to complete > your request.</p> > <p>Please contact the server administrator, > [no address given] and inform them of the time the error occurred, > and anything you might have done that may have > caused the error.</p> > <p>More information about this error may be available > in the server error log.</p> > <hr> > <address>Apache/2.2.22 (Ubuntu) Server at puppetmaster.example.com Port > 8140</address> > </body></html> > > These are the ''master'' processes launched: > > root@puppetmaster:/etc/puppet# ps aufx | grep master > root 4352 0.0 0.0 6504 616 pts/1 S+ 17:07 0:00 | > \_ grep --color=auto master > puppet 2100 14.5 7.2 4703940 1793068 ? D 15:06 17:34 master > > > puppet 2145 4.4 7.6 4434900 1893028 ? D 15:08 5:17 master > > > puppet 2156 3.4 8.0 7484288 1986164 ? D 15:08 4:08 master > > > puppet 2182 2.9 8.7 3612508 2169308 ? D 15:08 3:29 master > > > puppet 2191 4.0 10.7 4893500 2661364 ? D 15:08 4:49 master > > > puppet 2200 24.8 16.0 12025276 3953904 ? D 15:08 29:39 master > > > puppet 3585 0.1 0.3 178984 81944 ? S 16:08 0:06 master > > > puppet 3603 0.2 0.3 178892 88320 ? S 16:08 0:07 master > > > puppet 3612 0.1 0.3 178196 84316 ? S 16:08 0:03 master > > Puppet Master is running on Ubuntu 12.04, and is running Puppet 3.3.0. > > Can anyone please advise how to troubleshoot this? I am at my wit''s end > here. > > Thanks. > > F. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Forrie
2013-Sep-24 01:42 UTC
[Puppet Users] Re: Puppet nodes unable to send report to Puppet Master running under Passenger
I ran into similar problems before, which ended up being the permissions of the rack config.ru file -- it needs to be owned and readable by the puppet user -- check that? Mine was in a path like /etc/puppet/rack/puppetmaster/config.ru. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
F. Y.
2013-Sep-24 19:53 UTC
[Puppet Users] Re: Puppet nodes unable to send report to Puppet Master running under Passenger
Nothing in the logs. But there were two problems: 1. I needed to disable the Apache VHOST file for Puppet Dashboard, then Puppet Master was able to work 2. A silly mistake a manifest caused a recursive loop in a manifest that was used in the majority of our node types. Figured that out by looking at machines and what the puppet agent was doing on them (they were all hung!) and the reported number of resources for the hosts were a huge number in the dashboard reports. Killed them, and restarted them but they did the same (i.e. memory continued to leak and ''master'' processes started to go out of control). But once the manifest was fixed, everything was resolved. I hope this helps someone from making the mistakes I did. Thanks. On Monday, 23 September 2013 13:26:00 UTC-4, Ellison Marks wrote:> > Anything showing up in your logs? > > On Friday, September 20, 2013 2:17:32 PM UTC-7, F. Y. wrote: >> >> Hello all, >> >> I recently switched to running puppet master using Passenger. However, I >> am seeing agents unable to send reports after a bunch of ''master'' processes >> starting to accumulate and eat all the RAM and swap space. >> >> Error: Could not send report: Error 500 on SERVER: <!DOCTYPE HTML PUBLIC >> "-//IETF//DTD HTML 2.0//EN"> >> <html><head> >> <title>500 Internal Server Error</title> >> </head><body> >> <h1>Internal Server Error</h1> >> <p>The server encountered an internal error or >> misconfiguration and was unable to complete >> your request.</p> >> <p>Please contact the server administrator, >> [no address given] and inform them of the time the error occurred, >> and anything you might have done that may have >> caused the error.</p> >> <p>More information about this error may be available >> in the server error log.</p> >> <hr> >> <address>Apache/2.2.22 (Ubuntu) Server at puppetmaster.example.com Port >> 8140</address> >> </body></html> >> >> These are the ''master'' processes launched: >> >> root@puppetmaster:/etc/puppet# ps aufx | grep master >> root 4352 0.0 0.0 6504 616 pts/1 S+ 17:07 0:00 | >> \_ grep --color=auto master >> puppet 2100 14.5 7.2 4703940 1793068 ? D 15:06 17:34 master >> >> >> puppet 2145 4.4 7.6 4434900 1893028 ? D 15:08 5:17 master >> >> >> puppet 2156 3.4 8.0 7484288 1986164 ? D 15:08 4:08 master >> >> >> puppet 2182 2.9 8.7 3612508 2169308 ? D 15:08 3:29 master >> >> >> puppet 2191 4.0 10.7 4893500 2661364 ? D 15:08 4:49 master >> >> >> puppet 2200 24.8 16.0 12025276 3953904 ? D 15:08 29:39 master >> >> >> puppet 3585 0.1 0.3 178984 81944 ? S 16:08 0:06 master >> >> >> puppet 3603 0.2 0.3 178892 88320 ? S 16:08 0:07 master >> >> >> puppet 3612 0.1 0.3 178196 84316 ? S 16:08 0:03 master >> >> Puppet Master is running on Ubuntu 12.04, and is running Puppet 3.3.0. >> >> Can anyone please advise how to troubleshoot this? I am at my wit''s end >> here. >> >> Thanks. >> >> F. >> >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.