this is really weird. i''ve upgraded test machine to 2.6 and started
getting those. virtually any file pulled from puppetmaster has zero
size. it''s even true for internal objects like custom functions and
facts. whenever puppet decides that file has actually changed (this
logic works fine) it pulls a new file but somehow ends up with empty
object. see (please compare md5s):
<0>root@judy(1)~$ puppetd --test
notice: Ignoring --listen on onetime run
info: Retrieving plugin
info: /File[/var/lib/puppet/lib]: Storing newly-audited value for
content
info: /File[/var/lib/puppet/lib/facter]: Storing newly-audited value
for content
info: /File[/var/lib/puppet/lib/puppet]: Storing newly-audited value
for content
info: /File[/var/lib/puppet/lib/puppet/parser]: Storing newly-audited
value for content
info: /File[/var/lib/puppet/lib/puppet/parser/functions]: Storing
newly-audited value for content
notice: /File[/var/lib/puppet/lib/puppet/parser/functions/vlanfind.rb]/
content: content changed
''{md5}d41d8cd98f00b204e9800998ecf8427e'' to
''{md5}2ff6a0929541c75de18fa51ad420fdca''
info: /File[/var/lib/puppet/lib/puppet/provider]: Storing newly-
audited value for content
info: /File[/var/lib/puppet/lib/puppet/provider/sysctl]: Storing newly-
audited value for content
info: /File[/var/lib/puppet/lib/puppet/type]: Storing newly-audited
value for content
info: Loading downloaded plugin /var/lib/puppet/lib/puppet/parser/
functions/vlanfind.rb
notice: Caught INT; calling stop
<0>root@judy(2)~$ cat /var/lib/puppet/lib/puppet/parser/functions/
vlanfind.rb
<0>root@judy(3)~$ md5sum /var/lib/puppet/lib/puppet/parser/functions/
vlanfind.rb
d41d8cd98f00b204e9800998ecf8427e /var/lib/puppet/lib/puppet/parser/
functions/vlanfind.rb
<0>root@judy(4)~$ >/tmp/EMPTY
<0>root@judy(5)~$ md5sum /tmp/EMPTY
d41d8cd98f00b204e9800998ecf8427e /tmp/EMPTY
i''m talking to passenger puppetmaster seated on localhost.
anyone has seen anything similiar?
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 07:19 UTC
Re: [Puppet Users] getting empty files from fileserver
On Mon, 2010-08-02 at 12:15 -0700, asq wrote:> this is really weird. i''ve upgraded test machine to 2.6 and started > getting those. virtually any file pulled from puppetmaster has zero > size. it''s even true for internal objects like custom functions and > facts. whenever puppet decides that file has actually changed (this > logic works fine) it pulls a new file but somehow ends up with empty > object. see (please compare md5s): > > <0>root@judy(1)~$ puppetd --test > notice: Ignoring --listen on onetime run > info: Retrieving plugin > info: /File[/var/lib/puppet/lib]: Storing newly-audited value for > content > info: /File[/var/lib/puppet/lib/facter]: Storing newly-audited value > for content > info: /File[/var/lib/puppet/lib/puppet]: Storing newly-audited value > for content > info: /File[/var/lib/puppet/lib/puppet/parser]: Storing newly-audited > value for content > info: /File[/var/lib/puppet/lib/puppet/parser/functions]: Storing > newly-audited value for content > notice: /File[/var/lib/puppet/lib/puppet/parser/functions/vlanfind.rb]/ > content: content changed ''{md5}d41d8cd98f00b204e9800998ecf8427e'' to > ''{md5}2ff6a0929541c75de18fa51ad420fdca'' > info: /File[/var/lib/puppet/lib/puppet/provider]: Storing newly- > audited value for content > info: /File[/var/lib/puppet/lib/puppet/provider/sysctl]: Storing newly- > audited value for content > info: /File[/var/lib/puppet/lib/puppet/type]: Storing newly-audited > value for content > info: Loading downloaded plugin /var/lib/puppet/lib/puppet/parser/ > functions/vlanfind.rb > notice: Caught INT; calling stop > <0>root@judy(2)~$ cat /var/lib/puppet/lib/puppet/parser/functions/ > vlanfind.rb > <0>root@judy(3)~$ md5sum /var/lib/puppet/lib/puppet/parser/functions/ > vlanfind.rb > d41d8cd98f00b204e9800998ecf8427e /var/lib/puppet/lib/puppet/parser/ > functions/vlanfind.rb > <0>root@judy(4)~$ >/tmp/EMPTY > <0>root@judy(5)~$ md5sum /tmp/EMPTY > d41d8cd98f00b204e9800998ecf8427e /tmp/EMPTY > > i''m talking to passenger puppetmaster seated on localhost.Since 2.6, puppet now streams the file content through rack. It is well possible you''re hitting a bug with this new feature. Can you let us know what version of passenger and rack you are using. It would also be worth upgrading those versions to see if that helps. Then, you should also run your master with --debug --trace, to see if the master logs anything interesting during one of this specific file transfer. -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 3 Sie, 09:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote:> Since 2.6, puppet now streams the file content through rack. It is well > possible you''re hitting a bug with this new feature. > > Can you let us know what version of passenger and rack you are using. It > would also be worth upgrading those versions to see if that helps. > > Then, you should also run your master with --debug --trace, to see if > the master logs anything interesting during one of this specific file > transfer.rack 1.0.1 and 1.2.1 passenger 2.2.7 ruby: ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux], MBARI 0x6770, Ruby Enterprise Edition 20090928 same thing with both ruby and enterprise ruby interpreters. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 10:19 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On Tue, 2010-08-03 at 03:09 -0700, asq wrote:> On 3 Sie, 09:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > > > Since 2.6, puppet now streams the file content through rack. It is well > > possible you''re hitting a bug with this new feature. > > > > Can you let us know what version of passenger and rack you are using. It > > would also be worth upgrading those versions to see if that helps. > > > > Then, you should also run your master with --debug --trace, to see if > > the master logs anything interesting during one of this specific file > > transfer. > > rack 1.0.1 and 1.2.1 > passenger 2.2.7 > ruby: > ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] > ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux], MBARI 0x6770, > Ruby Enterprise Edition 20090928 > > same thing with both ruby and enterprise ruby interpreters.Ruby version shouldn''t matter, I more think this is a puppet rack versioning issue. Would it be possible to test rack 1.1.0 (which is what I tried when adding this feature)? Do you have any interesting --debug --trace log on the master side? -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 3 Sie, 12:09, asq <asqu...@gmail.com> wrote:> On 3 Sie, 09:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > > > Since 2.6, puppet now streams the file content through rack. It is well > > possible you''re hitting a bug with this new feature. > > > Can you let us know what version of passenger and rack you are using. It > > would also be worth upgrading those versions to see if that helps. > > > Then, you should also run your master with --debug --trace, to see if > > the master logs anything interesting during one of this specific file > > transfer. > > rack 1.0.1 and 1.2.1 > passenger 2.2.7 > ruby: > ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] > ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux], MBARI 0x6770, > Ruby Enterprise Edition 20090928 > > same thing with both ruby and enterprise ruby interpreters.logs from debug shows something interesting: Could not find file_content for ''/plugins/puppet/parser/functions/ template_search.rb'' see logs: http://pastie.org/private/tx8intebzefhymcu5miqw config: http://pastie.org/private/7eugfclhbhsdwuq5gj6za file exists: http://pastie.org/private/ms67m4z4rhzdsucpawfja -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thorsten Biel
2010-Aug-03 10:22 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
Hi, On 3 Aug 2010, at 12:09, asq <asquelt@gmail.com> wrote:> On 3 Sie, 09:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > >> Since 2.6, puppet now streams the file content through rack. It is well >> possible you''re hitting a bug with this new feature. >> >> Can you let us know what version of passenger and rack you are using. It >> would also be worth upgrading those versions to see if that helps. >> >> Then, you should also run your master with --debug --trace, to see if >> the master logs anything interesting during one of this specific file >> transfer. > > rack 1.0.1 and 1.2.1 > passenger 2.2.7 > ruby: > ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] > ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux], MBARI 0x6770, > Ruby Enterprise Edition 20090928 > > same thing with both ruby and enterprise ruby interpreters.Try deleting the old Rack version (gem cleanup). Had a similar problem, it looks as if rack 1.0.1 doesn''t play nice with Puppet 2.6. Cheers, Thorsten -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 3 Sie, 12:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote:> Ruby version shouldn''t matter, I more think this is a puppet rack > versioning issue. > > Would it be possible to test rack 1.1.0 (which is what I tried when > adding this feature)?same thing with rack 1.1.0 (i did gem cleanup)> Do you have any interesting --debug --trace log on the master side?attached in previous mail. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 11:01 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On Tue, 2010-08-03 at 03:22 -0700, asq wrote:> On 3 Sie, 12:09, asq <asqu...@gmail.com> wrote: > > On 3 Sie, 09:19, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > > > > > Since 2.6, puppet now streams the file content through rack. It is well > > > possible you''re hitting a bug with this new feature. > > > > > Can you let us know what version of passenger and rack you are using. It > > > would also be worth upgrading those versions to see if that helps. > > > > > Then, you should also run your master with --debug --trace, to see if > > > the master logs anything interesting during one of this specific file > > > transfer. > > > > rack 1.0.1 and 1.2.1 > > passenger 2.2.7 > > ruby: > > ruby 1.8.6 (2008-08-11 patchlevel 287) [x86_64-linux] > > ruby 1.8.7 (2009-06-12 patchlevel 174) [x86_64-linux], MBARI 0x6770, > > Ruby Enterprise Edition 20090928 > > > > same thing with both ruby and enterprise ruby interpreters. > > logs from debug shows something interesting: > > Could not find file_content for ''/plugins/puppet/parser/functions/ > template_search.rb'' > > see logs: http://pastie.org/private/tx8intebzefhymcu5miqw > config: http://pastie.org/private/7eugfclhbhsdwuq5gj6za > file exists: http://pastie.org/private/ms67m4z4rhzdsucpawfjaI think you''re seeing bug #4319: http://projects.puppetlabs.com/issues/4319 Can you try the latest 2.6.1rc1 and confirm that it solves your issue? Or alternatively try what Thorsten Biel suggested in another e-mail in this thread (ie getting rid of rack 1.0.1)? -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 3 Sie, 13:01, Brice Figureau <brice-pup...@daysofwonder.com> wrote:> I think you''re seeing bug #4319:http://projects.puppetlabs.com/issues/4319 > > Can you try the latest 2.6.1rc1 and confirm that it solves your issue? > Or alternatively try what Thorsten Biel suggested in another e-mail in > this thread (ie getting rid of rack 1.0.1)?with patch applied (http://github.com/masterzen/puppet/tree/tickets/ master/4319) i''m getting passenger error: [Tue Aug 03 16:25:16 2010] [error] [client 10.100.101.15] Premature end of script headers: vlanfind.rb [ pid=3397 file=ext/apache2/Hooks.cpp:711 time=2010-08-03 16:25:16.910 ]: The backend application (process 3608) didn''t send a valid HTTP response. It might have crashed during the middle of sending an HTTP response, so please check whether there are crashing problems in your application. This is the data that it sent: [Content-Type] *** Exception NoMethodError in PhusionPassenger::Rack::ApplicationSpawner (undefined method `each'' for 4741:Fixnum) (process 3608): from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/request_handler.rb:103:in `process_request'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/request_handler.rb:99:in `each'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/request_handler.rb:99:in `process_request'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/abstract_request_handler.rb:207:in `main_loop'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/application_spawner.rb:118:in `run'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/application_spawner.rb:69:in `spawn_application'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/utils.rb:184:in `safe_fork'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/application_spawner.rb:62:in `spawn_application'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/rack/application_spawner.rb:45:in `spawn_application'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/spawn_manager.rb:159:in `spawn_application'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/spawn_manager.rb:287:in `handle_spawn_application'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/abstract_server.rb:352:in `__send__'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/abstract_server.rb:352:in `main_loop'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/lib/ phusion_passenger/abstract_server.rb:196:in `start_synchronously'' from /usr/lib/ruby/gems/1.8/gems/passenger-2.2.7/bin/passenger- spawn-server:61 rack 1.1.0 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 16:00 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On Tue, 2010-08-03 at 07:34 -0700, asq wrote:> On 3 Sie, 13:01, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > > > I think you''re seeing bug #4319:http://projects.puppetlabs.com/issues/4319 > > > > Can you try the latest 2.6.1rc1 and confirm that it solves your issue? > > Or alternatively try what Thorsten Biel suggested in another e-mail in > > this thread (ie getting rid of rack 1.0.1)? > > with patch applied (http://github.com/masterzen/puppet/tree/tickets/ > master/4319) i''m getting passenger error:Which means it now tries to send the file. You just hit another issue which is discussed in this thread: http://groups.google.com/group/puppet-users/browse_thread/thread/543510e15f06074c# -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 3 Sie, 18:00, Brice Figureau <brice-pup...@daysofwonder.com> wrote:> Which means it now tries to send the file. You just hit another issue > which is discussed in this thread:http://groups.google.com/group/puppet-users/browse_thread/thread/5435...i''ve tried 2.2.11/1.2.1 combo and unfortunately results are pretty much the same. i''ll try 2.6.1rc1 tomorrow, maybe there were some other fixes on passenger. if there is no success what is current best practice to run large scale puppetmasters? (so far we were quite happy on passenger and enterprise-ruby memory management tweaks) -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 17:56 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On 03/08/10 18:56, asq wrote:> On 3 Sie, 18:00, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > >> Which means it now tries to send the file. You just hit another issue >> which is discussed in this thread:http://groups.google.com/group/puppet-users/browse_thread/thread/5435... > > i''ve tried 2.2.11/1.2.1 combo and unfortunately results are pretty > much the same. > i''ll try 2.6.1rc1 tomorrow, maybe there were some other fixes on > passenger. > if there is no success what is current best practice to run large > scale puppetmasters? > (so far we were quite happy on passenger and enterprise-ruby memory > management tweaks)Passenger is still the preferred way to scale a puppetmaster (until I progress on the JRuby front). Unfortunately rack or passenger tends to break on each new release (and it is worst in case of Puppet _and_ rack upgrades :)) The best to fix this issue is to collect the more log possible (including stack trace) so that we can understand what''s going wrong. Did someone file a puppet bug about this issue? -- Brice Figureau My Blog: http://www.masterzen.fr/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Richard Crowley
2010-Aug-03 18:24 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
> Passenger is still the preferred way to scale a puppetmaster (until I > progress on the JRuby front).I''ve had great success running Puppet 0.25.4, 0.25.5, and 2.6.0 under Nginx and Unicorn. Alas, http://projects.puppetlabs.com/issues/4319 is still in play. I use the highly suspect "give up and use wget" workaround. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-03 18:44 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On 03/08/10 20:24, Richard Crowley wrote:>> Passenger is still the preferred way to scale a puppetmaster (until I >> progress on the JRuby front). > > I''ve had great success running Puppet 0.25.4, 0.25.5, and 2.6.0 under > Nginx and Unicorn. Alas, http://projects.puppetlabs.com/issues/4319 > is still in play. I use the highly suspect "give up and use wget" > workaround.Use either 2.6.1rc1 or my single-line patch linked to the bug report to get rid of the issue :) IMHO it''s better than refactoring your manifests to wget stuff. -- Brice Figureau My Blog: http://www.masterzen.fr/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nicolas Brisac
2010-Aug-04 13:34 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
I''m getting the same error with a new custom fact in <module>/lib. The puppetmaster is running on Apache2 + Passenger (setup works fine in production with 0.24.8). With Puppet 2.6.0 (Debian package) the file is created empty. With Puppet 2.6.0 + patch from Brice, I get a ''Could not set file on ensure: Error 500'' on the client side and the apache logs on the server shows: [Wed Aug 04 23:01:46 2010] [error] [client 192.168.200.199] Premature end of script headers: vzbackup.rb [ pid=28420 file=ext/apache2/Hooks.cpp:735 time=2010-08-04 23:01:46.242 ]: The backend application (process 28465) didn''t send a valid HTTP response. It might have crashed during the middle of sending an HTTP response, so please check whether there are crashing problems in your application. This is the data that it sent: [Content-Type] *** Exception NoMethodError in PhusionPassenger::Rack::ApplicationSpawner (undefined method `each'' for 1268:Fixnum) (process 28465): from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/request_handler.rb:100:in `process_request'' from /var/lib/gems/1.8/gems/rack-1.2.1/lib/rack/utils.rb:305:in `each'' from /var/lib/gems/1.8/gems/rack-1.2.1/lib/rack/utils.rb:304:in `each'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/abstract_request_handler.rb:207:in `main_loop'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/application_spawner.rb:120:in `run'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/application_spawner.rb:65:in `spawn_application'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/utils.rb:252:in `safe_fork'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/application_spawner.rb:58:in `spawn_application'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/rack/application_spawner.rb:41:in `spawn_application'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/spawn_manager.rb:150:in `spawn_application'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/spawn_manager.rb:278:in `handle_spawn_application'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/abstract_server.rb:352:in `__send__'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/abstract_server.rb:352:in `main_loop'' from /var/lib/gems/1.8/gems/passenger-2.2.15/lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'' from /var/lib/gems/1.8/gems/passenger-2.2.15/bin/passenger-spawn-server:61 Same thing with 2.6.1rc1 from git unfortunately... Apart from this, the manifests are compiled and run fine, and the facts that already exist on the client are evaluated correctly. The server I use for testing is running Ubuntu 10.04, the client is 8.04. I''ve tried various combinations of Passenger (2.2.7, 2.2.11, 2.2.15) and Rack (1.1.0, 1.2.1), from gems and .deb, without success. Here is my current gems list on the server: actionmailer (2.3.8) actionpack (2.3.8) activerecord (2.3.8) activeresource (2.3.8) activesupport (2.3.8) fastthread (1.0.7) passenger (2.2.15) rack (1.2.1) rails (2.3.8) rake (0.8.7) I''ll take any suggestion! Thanks, Nico On 03/08/10 20:24, Richard Crowley wrote:>> Passenger is still the preferred way to scale a puppetmaster (until I >> progress on the JRuby front). > > I''ve had great success running Puppet 0.25.4, 0.25.5, and 2.6.0 under > Nginx and Unicorn. Alas, http://projects.puppetlabs.com/issues/4319 > is still in play. I use the highly suspect "give up and use wget" > workaround.Use either 2.6.1rc1 or my single-line patch linked to the bug report to get rid of the issue :) IMHO it''s better than refactoring your manifests to wget stuff. -- Brice Figureau My Blog: http://www.masterzen.fr/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Brice Figureau
2010-Aug-04 13:47 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On Wed, 2010-08-04 at 23:34 +1000, Nicolas Brisac wrote:> I''m getting the same error with a new custom fact in <module>/lib. > The puppetmaster is running on Apache2 + Passenger (setup works fine > in production with 0.24.8). > > With Puppet 2.6.0 (Debian package) the file is created empty. > With Puppet 2.6.0 + patch from Brice, I get a ''Could not set file on > ensure: Error 500'' on the client side and the apache logs on the > server shows: > > [Wed Aug 04 23:01:46 2010] [error] [client 192.168.200.199] Premature > end of script headers: vzbackup.rb > [ pid=28420 file=ext/apache2/Hooks.cpp:735 time=2010-08-04 > 23:01:46.242 ]: > The backend application (process 28465) didn''t send a valid HTTP > response. It might have crashed during the middle of sending an HTTP > response, so please check whether there are crashing problems in your > application. This is the data that it sent: [Content-Type] > *** Exception NoMethodError in > PhusionPassenger::Rack::ApplicationSpawner (undefined method `each'' > for 1268:Fixnum) (process 28465):OK, I just had a look to the passenger code and this is insane. I think the problem is that we return the content-length as an int, but passenger chokes if it gets anything except a string. Can you try this (untested) patch: diff --git a/lib/puppet/network/http/rack/rest.rb b/lib/puppet/network/http/rack/rest.rb index e5f50c4..b7e1d97 100644 --- a/lib/puppet/network/http/rack/rest.rb +++ b/lib/puppet/network/http/rack/rest.rb @@ -41,7 +41,7 @@ class Puppet::Network::HTTP::RackREST < Puppet::Network::HTTP::RackHttpHandler unless result.is_a?(File) response.write result else - response["Content-Length"] = result.stat.size + response["Content-Length"] = result.stat.size.to_s response.body = RackFile.new(result) end end Let me know if that fixes the problem, -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Markus Roberts
2010-Aug-04 15:26 UTC
Re: Re: [Puppet Users] Re: getting empty files from fileserver
Brice -- If that doesn''t pan out could the problem be with our format_to_mime (i.e. the content type, rather than the content length)? That seems equally odd, but it would explain why there isn''t widespread discussion of the problem (which I would expect if passenger were requiring content length to be an string)--if we''re somehow returning an integer for the Content-Type it would be a problem that only we would see. -- Markus ----------------------------------------------------------- The power of accurate observation is commonly called cynicism by those who have not got it. ~George Bernard Shaw ------------------------------------------------------------ -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com. To unsubscribe from this group, send email to puppet-dev+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
Nicolas Brisac
2010-Aug-04 23:03 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
That fixes it for me! Merci :) Current setup: Puppet 2.6.0 (.deb from Sid) Passenger 2.2.15 and Rack 1.2.1 (gems) Cheers, Nico On Wed, 2010-08-04 at 23:34 +1000, Nicolas Brisac wrote:> I''m getting the same error with a new custom fact in <module>/lib. > The puppetmaster is running on Apache2 + Passenger (setup works fine > in production with 0.24.8). > > With Puppet 2.6.0 (Debian package) the file is created empty. > With Puppet 2.6.0 + patch from Brice, I get a ''Could not set file on > ensure: Error 500'' on the client side and the apache logs on the > server shows: > > [Wed Aug 04 23:01:46 2010] [error] [client 192.168.200.199] Premature > end of script headers: vzbackup.rb > [ pid=28420 file=ext/apache2/Hooks.cpp:735 time=2010-08-04 > 23:01:46.242 ]: > The backend application (process 28465) didn''t send a valid HTTP > response. It might have crashed during the middle of sending an HTTP > response, so please check whether there are crashing problems in your > application. This is the data that it sent: [Content-Type] > *** Exception NoMethodError in > PhusionPassenger::Rack::ApplicationSpawner (undefined method `each'' > for 1268:Fixnum) (process 28465):OK, I just had a look to the passenger code and this is insane. I think the problem is that we return the content-length as an int, but passenger chokes if it gets anything except a string. Can you try this (untested) patch: diff --git a/lib/puppet/network/http/rack/rest.rb b/lib/puppet/network/http/rack/rest.rb index e5f50c4..b7e1d97 100644 --- a/lib/puppet/network/http/rack/rest.rb +++ b/lib/puppet/network/http/rack/rest.rb @@ -41,7 +41,7 @@ class Puppet::Network::HTTP::RackREST < Puppet::Network::HTTP::RackHttpHandler unless result.is_a?(File) response.write result else - response["Content-Length"] = result.stat.size + response["Content-Length"] = result.stat.size.to_s response.body = RackFile.new(result) end end Let me know if that fixes the problem, -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi,
My setup:
gems:
rack (1.0.1)
rails (2.3.4)
rake (0.8.7)
activerecord (2.3.8, 2.3.4)
activeresource (2.3.8, 2.3.4)
activesupport (2.3.8, 2.3.4)
mongrel (1.1.5)
RPMs:
puppet-server-2.6.0-0.7.el5 (tmz''s)
I patched the files installed via the RPM to test out nginx+mongrel as
follows (taken from http://projects.puppetlabs.com/issues/4319 ):
http://pastie.org/1114966
The problem I''m facing is:
My modules have ''source'' like:
source => [
"puppet:///imapproxy/$project/imapproxy.conf.
$hostname",
"puppet:///imapproxy/$project/imapproxy.conf-$class",
"puppet:///imapproxy/imapproxy.conf.$hostname",
"puppet:///imapproxy/imapproxy.conf",
],
in my files directory, I have:
/projectA/imapproxy.conf-webmail
/projectA/imapproxy.conf
In my site manifest, I have a few webmail servers which have $class
''webmail'' in them and a few which don''t have any
$class set to
something else. Before I migrated to the nginx+mongrel setup (and
after I reverted back from it), for the nodes that have a $class
!''webmail'', the default imapproxy.conf was being served, but
now, I''m
served an emtpy file instead.
The hosts with $class = ''webmail'' face no such problem - they
receive
the file they were supposed to receive.
($project is set to ''projectA'' for all nodes in the manifest.)
Is anybody else able to replicate this?
-Naresh.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Hi, I gave this another attempt recently with 2.6.2rc1, nginx, and thin via config.ru. (not nginx + mongrels like earlier) Now I know a little more about the problem than I did a month ago: I use a lot of variables such as $project in my file path array I set for the source parameter for file type resources:> source => [ > "puppet:///imapproxy/$project/imapproxy.conf.$hostname",And in my site manifest, I tend to have class foo { $project = "blah" include imapproxy } node bar { include foo } I get the correct file served when I use the stock WEBrick/ puppetmasterd and I don''t get the correct file served with nginx+thin +config.ru unless I do: node bar { $project = "blah" include imapproxy } FWIW, my server-side logs don''t show me the $project part in the GETs at all - the $hostname (and other facts are) is shown. [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / production/file_metadata/users//sudoers.xen-3 HTTP/1.1" 404 48 [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / production/file_metadata/users//sudoers- HTTP/1.1" 404 43 [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / production/file_metadata/users//sudoers HTTP/1.1" 200 336 There should''ve been a "GET /production/file_metadata/users/blah/ sudoers.xen-3" etc.) Thank you, Naresh. On Aug 25, 6:23 pm, "Naresh V." <nares...@gmail.com> wrote:> Hi, > > My setup: > > gems: > rack (1.0.1) > rails (2.3.4) > rake (0.8.7) > activerecord (2.3.8, 2.3.4) > activeresource (2.3.8, 2.3.4) > activesupport (2.3.8, 2.3.4) > mongrel (1.1.5) > > RPMs: > puppet-server-2.6.0-0.7.el5 (tmz''s) > > I patched the files installed via the RPM to test out nginx+mongrel as > follows (taken fromhttp://projects.puppetlabs.com/issues/4319): > > http://pastie.org/1114966 > > The problem I''m facing is: > My modules have ''source'' like: > > source => [ > "puppet:///imapproxy/$project/imapproxy.conf. > $hostname", > "puppet:///imapproxy/$project/imapproxy.conf-$class", > "puppet:///imapproxy/imapproxy.conf.$hostname", > "puppet:///imapproxy/imapproxy.conf", > ], > > in my files directory, I have: > > /projectA/imapproxy.conf-webmail > /projectA/imapproxy.conf > > In my site manifest, I have a few webmail servers which have $class > ''webmail'' in them and a few which don''t have any $class set to > something else. Before I migrated to the nginx+mongrel setup (and > after I reverted back from it), for the nodes that have a $class !> ''webmail'', the default imapproxy.conf was being served, but now, I''m > served an emtpy file instead. > > The hosts with $class = ''webmail'' face no such problem - they receive > the file they were supposed to receive. > > ($project is set to ''projectA'' for all nodes in the manifest.) > > Is anybody else able to replicate this? > > -Naresh.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Peter Meier
2010-Oct-06 08:03 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
> FWIW, my server-side logs don''t show me the $project part in the GETs > at all - the $hostname (and other facts are) is shown. > > [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / > production/file_metadata/users//sudoers.xen-3 HTTP/1.1" 404 48 > [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / > production/file_metadata/users//sudoers- HTTP/1.1" 404 43 > [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / > production/file_metadata/users//sudoers HTTP/1.1" 200 336 > > There should''ve been a "GET /production/file_metadata/users/blah/ > sudoers.xen-3" etc.)http://projects.puppetlabs.com/issues/4922 ~pete -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
John Hawkes-Reed
2010-Oct-06 09:15 UTC
Re: [Puppet Users] Re: getting empty files from fileserver
On 6/10/10 09:03, "Peter Meier" <peter.meier@immerda.ch> wrote:>> FWIW, my server-side logs don''t show me the $project part in the GETs >> at all - the $hostname (and other facts are) is shown. >> >> [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / >> production/file_metadata/users//sudoers.xen-3 HTTP/1.1" 404 48 >> [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / >> production/file_metadata/users//sudoers- HTTP/1.1" 404 43 >> [2010-10-06 00:04:35] bar - - [06/Oct/2010:00:04:35 CDT] "GET / >> production/file_metadata/users//sudoers HTTP/1.1" 200 336 >> >> There should''ve been a "GET /production/file_metadata/users/blah/ >> sudoers.xen-3" etc.) > > http://projects.puppetlabs.com/issues/4922 > > ~peteAha! I suspect that''s what I''ve been seeing with 2.6.(0|1) + unicorn + nginx + 2.6.0 clients and also with 2.6.(0|1) + passenger + apache. Glad (FAVO ''glad'', obviously) it''s not just me. -- John Hawkes-Reed -- Future Publishing Limited (registered company number 2008885) and Future Publishing (Overseas) Limited (registered company number 06202940) are wholly owned subsidiaries of Future plc (registered company number 3757874). Future Publishing Limited, Future Publishing (Overseas) Limited and Future plc are all incorporated in England and Wales and share the same registered address at Beauford Court, 30 Monmouth Street, Bath BA1 2BW. This email and any files transmitted with it are confidential. If you have received this email in error please notify the sender and then delete it immediately. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Future. The recipient should check this email and any attachments for the presence of viruses. Future accepts no liability for any damage caused by any virus transmitted by this email. Future may regularly and randomly monitor outgoing and incoming emails (including the content of them) and other telecommunications on its email and telecommunications systems. By replying to this email you give your consent to such monitoring. ***** Save resources: think before you print. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.