Bobby Santiago
2008-May-05 03:02 UTC
[Backgroundrb-devel] Deploying to a staging server using Capistrano: how to start up BackgrounDRb?
Hi.
I am using BackgrounDRb to process thumbnails and upload to S3 -
things are hunky-dory in development (thumbs are generated, these are
uploaded to S3, the metadata is saved to trhe DB, and I get a nice
status page updated by periodic calls via ask_status), but when I
tried to deploy to our staging server and stop/start BackgrounDRb via
Capistrano, things blew up - well, not exactly, but when the app tried
to hand off the thumbnail generation, I got a "Could not connect to
the BackgrounDRb server" error.
Initially, when I didn''t start the BDRb server manually, I got:
* executing `after_update_code''
* executing `set_env_staging''
ENV[''RAILS_ENV''] = staging
* executing `restart_backgroundrb''
* executing `stop_backgroundrb''
* executing "cd /var/www/apps/my_app/releases/20080501133415 && ./
script/backgroundrb -e staging stop"
servers: ["staging.my_app.com"]
[staging.my_app.com] executing command
** [out :: staging.my_app.com] ./script/backgroundrb:46:in `initialize''
** [out :: staging.my_app.com] :
** [out :: staging.my_app.com] No such file or directory - /var/www/
apps/my_app/releases/20080501133415/tmp/pids/backgroundrb_22222.pid
** [out :: staging.my_app.com] (
** [out :: staging.my_app.com] Errno::ENOENT
** [out :: staging.my_app.com] )
** [out :: staging.my_app.com] from ./script/backgroundrb:46:in `open''
** [out :: staging.my_app.com] from ./script/backgroundrb:46
command finished
*** [deploy:update_code] rolling back
* executing "rm -rf /var/www/apps/my_app/releases/20080501133415;
true"
servers: ["staging.my_app.com"]
[staging.my_app.com] executing command
command finished
So I tried manually starting backgroundrb on the staging server,, and
the deployment went through successfully. But when I tried generating
thumbs with a background worker, I got the "Could not connect to the
BackgrounDRb server" error.
I check out from GitHub, and use Capistrano-ext for multistage
deployments:
cap staging deploy
Before I deploy, I SSH into the VPS and do
script/backgroundrb -e staging start
in my config/deploy/staging.rb file:
----------------
set :rails_env, ''staging''
...
task :after_update_code do
set_env_staging
copy_mongrel_cluster_config
restart_backgroundrb
end
desc "Sets the environment variable
RAILS_ENV=''staging''."
task :set_env_staging do
ENV[''RAILS_ENV''] = ''staging''
puts "ENV[''RAILS_ENV''] =
#{ENV[''RAILS_ENV'']}"
end
desc "Copying the right mongrel cluster config for the current stage
environment."
task :copy_mongrel_cluster_config do
run "cp -f #{release_path}/config/deploy/staging/mongrel_cluster.yml
#{release_path}/config/mongrel_cluster.yml"
end
desc "Start the backgroundrb server"
task :start_backgroundrb , :roles => :app do
run "cd #{current_path} && nohup ./script/backgroundrb start --
-r
staging > #{current_path}/log/backgroundrb-cap.log 2>&1"
end
desc "Stop the backgroundrb server"
task :stop_backgroundrb, :roles => :app do
run "cd #{release_path} && ./script/backgroundrb -e staging
stop"
end
desc "Restart the backgroundrb server"
task :restart_backgroundrb, :roles => :app do
stop_backgroundrb
start_backgroundrb
end
-------
Here''s the exception_notifier backtrace:
[RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:81:in
`dump_object''
[RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:98:in
`new_worker''
[RAILS_ROOT]/app/models/uploader.rb:54:in `start_thumbnail_worker''
[RAILS_ROOT]/app/controllers/photos_controller.rb:49:in `create''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
mime_responds.rb:106:in `call''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
mime_responds.rb:106:in `respond_to''
[RAILS_ROOT]/app/controllers/photos_controller.rb:47:in `create''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
base.rb:1158:in `send''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
base.rb:1158:in `perform_action_without_filters''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
filters.rb:697:in `call_filters''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
filters.rb:689:in `perform_action_without_benchmark''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
benchmarking.rb:68:in `perform_action_without_rescue''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
benchmarking.rb:68:in `perform_action_without_rescue''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
rescue.rb:199:in `perform_action_without_caching''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
caching.rb:678:in `perform_action''
/usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/
connection_adapters/abstract/query_cache.rb:33:in `cache''
/usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/
query_cache.rb:8:in `cache''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
caching.rb:677:in `perform_action''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
base.rb:524:in `send''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
base.rb:524:in `process_without_filters''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
filters.rb:685:in `process_without_session_management_support''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
session_management.rb:123:in `sass_old_process''
/usr/lib/ruby/gems/1.8/gems/haml-1.8.1/lib/sass/plugin/rails.rb:
15:in `process''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
base.rb:388:in `process''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
dispatcher.rb:171:in `handle_request''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
dispatcher.rb:115:in `dispatch''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
dispatcher.rb:126:in `dispatch_cgi''
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
dispatcher.rb:9:in `dispatch''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:76:in
`process''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
`synchronize''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
`process''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:159:in
`process_client''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
`each''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
`process_client''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
`initialize''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in `new''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
`initialize''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in `new''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
configurator.rb:282:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
configurator.rb:281:in `each''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
configurator.rb:281:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:128:in
`run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/command.rb:
212:in `run''
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:281
/usr/bin/mongrel_rails:19:in `load''
/usr/bin/mongrel_rails:19
My backgroundrb.yml file:
:backgroundrb:
:ip: 0.0.0.0
:development:
:backgroundrb:
:port: 11006 # use port 11006
:log: foreground # foreground mode,print log messages on console
:debug_log: true
:staging:
:backgroundrb:
:port: 22222 # use port 22222
:lazy_load: true # do not load models eagerly
:debug_log: false # disable log workers and other logging
:production:
:backgroundrb:
:port: 33333 # use port 33333
:lazy_load: true # do not load models eagerly
:debug_log: false # disable log workers and other logging
My worker:
class ThumbnailGeneratorWorker < BackgrounDRb::MetaWorker
set_worker_name :thumbnail_generator_worker
set_no_auto_load(true)
def create(args = nil)
register_status(:percent_complete => 0)
args.each_with_index do |uploader_id, index|
@uploader = Uploader.find(uploader_id)
@uploader.generate_thumbnails
percent_complete = ((index + 1) * 100) / args.length
logger.info "Thumbnail generation is #{percent_complete}%
complete..."
register_status(:percent_complete => percent_complete)
end
exit
end
end
Any hints/help/ideas would be much appreciated.
Bobby
Stevie Clifton
2008-May-05 16:14 UTC
[Backgroundrb-devel] Deploying to a staging server using Capistrano: how to start up BackgrounDRb?
Hey Bobby,
I think you''re dealing with two separate issues here. The first error
you''re seeing with capistrano is b/c capistrano doesn''t
auto-link the
/RAILS_ROOT/tmp/pids folder, so backgroundrb is unable to create a pid
file there. I do something like the following (untested):
after "deploy:update_code", "deploy:symlink_pids"
namespace :deploy
desc "Symlink tmp/pids folder"
task :symlink_pids do
run "mkdir -p #{release_path}/tmp"
run "ln -nfs #{shared_path}/pids #{release_path}/tmp/pids"
end
end
For the second issue, it''s a little difficult knowing why rails
can''t
see your worker without seeing some bdrb log output. Have you
verified that the worker is actually running? (ps aux | grep
<worker_name>). Also, have you checked your backgroundrb_server.log
to see if it''s dying on instantiation because of some other
dependencies? I''d recommend launching backgroundrb manually on
staging and setting :log: foreground in backgroundrb.yml so you can
see what''s going on with your worker.
stevie
On Sun, May 4, 2008 at 11:02 PM, Bobby Santiago
<bobby.santiago at gmail.com> wrote:> Hi.
>
> I am using BackgrounDRb to process thumbnails and upload to S3 - things
are
> hunky-dory in development (thumbs are generated, these are uploaded to S3,
> the metadata is saved to trhe DB, and I get a nice status page updated by
> periodic calls via ask_status), but when I tried to deploy to our staging
> server and stop/start BackgrounDRb via Capistrano, things blew up - well,
> not exactly, but when the app tried to hand off the thumbnail generation, I
> got a "Could not connect to the BackgrounDRb server" error.
>
> Initially, when I didn''t start the BDRb server manually, I got:
>
> * executing `after_update_code''
> * executing `set_env_staging''
> ENV[''RAILS_ENV''] = staging
> * executing `restart_backgroundrb''
> * executing `stop_backgroundrb''
> * executing "cd /var/www/apps/my_app/releases/20080501133415
&&
> ./script/backgroundrb -e staging stop"
> servers: ["staging.my_app.com"]
> [staging.my_app.com] executing command
> ** [out :: staging.my_app.com] ./script/backgroundrb:46:in
`initialize''
> ** [out :: staging.my_app.com] :
> ** [out :: staging.my_app.com] No such file or directory -
>
/var/www/apps/my_app/releases/20080501133415/tmp/pids/backgroundrb_22222.pid
> ** [out :: staging.my_app.com] (
> ** [out :: staging.my_app.com] Errno::ENOENT
> ** [out :: staging.my_app.com] )
> ** [out :: staging.my_app.com] from ./script/backgroundrb:46:in
`open''
> ** [out :: staging.my_app.com] from ./script/backgroundrb:46
> command finished
> *** [deploy:update_code] rolling back
> * executing "rm -rf /var/www/apps/my_app/releases/20080501133415;
true"
> servers: ["staging.my_app.com"]
> [staging.my_app.com] executing command
> command finished
>
> So I tried manually starting backgroundrb on the staging server,, and the
> deployment went through successfully. But when I tried generating thumbs
> with a background worker, I got the "Could not connect to the
BackgrounDRb
> server" error.
>
> I check out from GitHub, and use Capistrano-ext for multistage
deployments:
>
> cap staging deploy
>
> Before I deploy, I SSH into the VPS and do
> script/backgroundrb -e staging start
>
> in my config/deploy/staging.rb file:
> ----------------
> set :rails_env, ''staging''
> ...
>
> task :after_update_code do
> set_env_staging
> copy_mongrel_cluster_config
> restart_backgroundrb
> end
>
> desc "Sets the environment variable
RAILS_ENV=''staging''."
> task :set_env_staging do
> ENV[''RAILS_ENV''] = ''staging''
> puts "ENV[''RAILS_ENV''] =
#{ENV[''RAILS_ENV'']}"
> end
>
> desc "Copying the right mongrel cluster config for the current stage
> environment."
> task :copy_mongrel_cluster_config do
> run "cp -f #{release_path}/config/deploy/staging/mongrel_cluster.yml
> #{release_path}/config/mongrel_cluster.yml"
> end
>
> desc "Start the backgroundrb server"
> task :start_backgroundrb , :roles => :app do
> run "cd #{current_path} && nohup ./script/backgroundrb
start -- -r
> staging > #{current_path}/log/backgroundrb-cap.log 2>&1"
> end
>
>
> desc "Stop the backgroundrb server"
> task :stop_backgroundrb, :roles => :app do
> run "cd #{release_path} && ./script/backgroundrb -e staging
stop"
> end
>
> desc "Restart the backgroundrb server"
> task :restart_backgroundrb, :roles => :app do
> stop_backgroundrb
> start_backgroundrb
> end
>
> -------
> Here''s the exception_notifier backtrace:
>
> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:81:in
> `dump_object''
> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:98:in
> `new_worker''
> [RAILS_ROOT]/app/models/uploader.rb:54:in
`start_thumbnail_worker''
> [RAILS_ROOT]/app/controllers/photos_controller.rb:49:in `create''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/mime_responds.rb:106:in
> `call''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/mime_responds.rb:106:in
> `respond_to''
> [RAILS_ROOT]/app/controllers/photos_controller.rb:47:in `create''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:1158:in
> `send''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:1158:in
> `perform_action_without_filters''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:697:in
> `call_filters''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:689:in
> `perform_action_without_benchmark''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/benchmarking.rb:68:in
> `perform_action_without_rescue''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/benchmarking.rb:68:in
> `perform_action_without_rescue''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/rescue.rb:199:in
> `perform_action_without_caching''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/caching.rb:678:in
> `perform_action''
>
>
/usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in
> `cache''
>
>
/usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/query_cache.rb:8:in
> `cache''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/caching.rb:677:in
> `perform_action''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:524:in
> `send''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:524:in
> `process_without_filters''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:685:in
> `process_without_session_management_support''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/session_management.rb:123:in
> `sass_old_process''
> /usr/lib/ruby/gems/1.8/gems/haml-1.8.1/lib/sass/plugin/rails.rb:15:in
> `process''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:388:in
> `process''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:171:in
> `handle_request''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:115:in
> `dispatch''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:126:in
> `dispatch_cgi''
>
>
/usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:9:in
> `dispatch''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:76:in
> `process''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
> `synchronize''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
> `process''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:159:in
> `process_client''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
`each''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
> `process_client''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
`run''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
> `initialize''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
`new''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
`run''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
> `initialize''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
`new''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
`run''
>
>
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/configurator.rb:282:in
> `run''
>
>
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/configurator.rb:281:in
> `each''
>
>
/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/configurator.rb:281:in
> `run''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:128:in
`run''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/command.rb:212:in
> `run''
> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:281
> /usr/bin/mongrel_rails:19:in `load''
> /usr/bin/mongrel_rails:19
>
> My backgroundrb.yml file:
> :backgroundrb:
> :ip: 0.0.0.0
>
> :development:
> :backgroundrb:
> :port: 11006 # use port 11006
> :log: foreground # foreground mode,print log messages on console
> :debug_log: true
>
> :staging:
> :backgroundrb:
> :port: 22222 # use port 22222
> :lazy_load: true # do not load models eagerly
> :debug_log: false # disable log workers and other logging
>
> :production:
> :backgroundrb:
> :port: 33333 # use port 33333
> :lazy_load: true # do not load models eagerly
> :debug_log: false # disable log workers and other logging
>
> My worker:
> class ThumbnailGeneratorWorker < BackgrounDRb::MetaWorker
> set_worker_name :thumbnail_generator_worker
> set_no_auto_load(true)
>
> def create(args = nil)
> register_status(:percent_complete => 0)
> args.each_with_index do |uploader_id, index|
> @uploader = Uploader.find(uploader_id)
> @uploader.generate_thumbnails
> percent_complete = ((index + 1) * 100) / args.length
> logger.info "Thumbnail generation is #{percent_complete}%
complete..."
> register_status(:percent_complete => percent_complete)
> end
> exit
>
> end
> end
>
> Any hints/help/ideas would be much appreciated.
>
> Bobby
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
Bobby Santiago
2008-May-06 00:27 UTC
[Backgroundrb-devel] Deploying to a staging server using Capistrano: how to start up BackgrounDRb?
Hey, Stevie, When I changed the port number in backgroundrb.yml to match the mongrel ports in my conf file, the connection error went away. I looked into /var/www/apps/my_app/shared/pids, and backgroundrb_9201.pid is there. In the /var/www/apps/my_app/shared/log dir, I have backgroundrb_9201.log backgroundrb_9201_debug.log backgroundrb_server_9201.log When I did ps aux | grep thumbnail_generator_worker, I got: deploy 12279 0.0 0.1 1816 496 pts/0 R+ 20:07 0:00 grep thumbnail_generator_worker So I guess it''s not running. After setting :log: foreground in backgroundrb.yml and deploying, I tailed the backgroundrb_9201.log, backgroundrb_9201_debug.log, and backgroundrb_server_9201.log files. No output there. In development, the backgroundrb_1106.log file has the correct output: BillingWorker#create called... billing_worker started Schedules for worker loaded ThumbnailGeneratorWorker#create called... thumbnail_generator_worker started Schedules for worker loaded generate 37 There is no feedback from the ask_status method, I get nil when I inspect the variable in staging.log. I''m running out of ideas. Bobby On May 6, 2008, at 12:14 AM, Stevie Clifton wrote:> Hey Bobby, > > I think you''re dealing with two separate issues here. The first error > you''re seeing with capistrano is b/c capistrano doesn''t auto-link the > /RAILS_ROOT/tmp/pids folder, so backgroundrb is unable to create a pid > file there. I do something like the following (untested): > > after "deploy:update_code", "deploy:symlink_pids" > > namespace :deploy > desc "Symlink tmp/pids folder" > task :symlink_pids do > run "mkdir -p #{release_path}/tmp" > run "ln -nfs #{shared_path}/pids #{release_path}/tmp/pids" > end > end > > For the second issue, it''s a little difficult knowing why rails can''t > see your worker without seeing some bdrb log output. Have you > verified that the worker is actually running? (ps aux | grep > <worker_name>). Also, have you checked your backgroundrb_server.log > to see if it''s dying on instantiation because of some other > dependencies? I''d recommend launching backgroundrb manually on > staging and setting :log: foreground in backgroundrb.yml so you can > see what''s going on with your worker. > > stevie > > > On Sun, May 4, 2008 at 11:02 PM, Bobby Santiago > <bobby.santiago at gmail.com> wrote: >> Hi. >> >> I am using BackgrounDRb to process thumbnails and upload to S3 - >> things are >> hunky-dory in development (thumbs are generated, these are uploaded >> to S3, >> the metadata is saved to trhe DB, and I get a nice status page >> updated by >> periodic calls via ask_status), but when I tried to deploy to our >> staging >> server and stop/start BackgrounDRb via Capistrano, things blew up - >> well, >> not exactly, but when the app tried to hand off the thumbnail >> generation, I >> got a "Could not connect to the BackgrounDRb server" error. >> >> Initially, when I didn''t start the BDRb server manually, I got: >> >> * executing `after_update_code'' >> * executing `set_env_staging'' >> ENV[''RAILS_ENV''] = staging >> * executing `restart_backgroundrb'' >> * executing `stop_backgroundrb'' >> * executing "cd /var/www/apps/my_app/releases/20080501133415 && >> ./script/backgroundrb -e staging stop" >> servers: ["staging.my_app.com"] >> [staging.my_app.com] executing command >> ** [out :: staging.my_app.com] ./script/backgroundrb:46:in >> `initialize'' >> ** [out :: staging.my_app.com] : >> ** [out :: staging.my_app.com] No such file or directory - >> /var/www/apps/my_app/releases/20080501133415/tmp/pids/ >> backgroundrb_22222.pid >> ** [out :: staging.my_app.com] ( >> ** [out :: staging.my_app.com] Errno::ENOENT >> ** [out :: staging.my_app.com] ) >> ** [out :: staging.my_app.com] from ./script/backgroundrb:46:in >> `open'' >> ** [out :: staging.my_app.com] from ./script/backgroundrb:46 >> command finished >> *** [deploy:update_code] rolling back >> * executing "rm -rf /var/www/apps/my_app/releases/20080501133415; >> true" >> servers: ["staging.my_app.com"] >> [staging.my_app.com] executing command >> command finished >> >> So I tried manually starting backgroundrb on the staging server,, >> and the >> deployment went through successfully. But when I tried generating >> thumbs >> with a background worker, I got the "Could not connect to the >> BackgrounDRb >> server" error. >> >> I check out from GitHub, and use Capistrano-ext for multistage >> deployments: >> >> cap staging deploy >> >> Before I deploy, I SSH into the VPS and do >> script/backgroundrb -e staging start >> >> in my config/deploy/staging.rb file: >> ---------------- >> set :rails_env, ''staging'' >> ... >> >> task :after_update_code do >> set_env_staging >> copy_mongrel_cluster_config >> restart_backgroundrb >> end >> >> desc "Sets the environment variable RAILS_ENV=''staging''." >> task :set_env_staging do >> ENV[''RAILS_ENV''] = ''staging'' >> puts "ENV[''RAILS_ENV''] = #{ENV[''RAILS_ENV'']}" >> end >> >> desc "Copying the right mongrel cluster config for the current stage >> environment." >> task :copy_mongrel_cluster_config do >> run "cp -f #{release_path}/config/deploy/staging/mongrel_cluster.yml >> #{release_path}/config/mongrel_cluster.yml" >> end >> >> desc "Start the backgroundrb server" >> task :start_backgroundrb , :roles => :app do >> run "cd #{current_path} && nohup ./script/backgroundrb start -- -r >> staging > #{current_path}/log/backgroundrb-cap.log 2>&1" >> end >> >> >> desc "Stop the backgroundrb server" >> task :stop_backgroundrb, :roles => :app do >> run "cd #{release_path} && ./script/backgroundrb -e staging stop" >> end >> >> desc "Restart the backgroundrb server" >> task :restart_backgroundrb, :roles => :app do >> stop_backgroundrb >> start_backgroundrb >> end >> >> ------- >> Here''s the exception_notifier backtrace: >> >> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:81:in >> `dump_object'' >> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:98:in >> `new_worker'' >> [RAILS_ROOT]/app/models/uploader.rb:54:in `start_thumbnail_worker'' >> [RAILS_ROOT]/app/controllers/photos_controller.rb:49:in `create'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> mime_responds.rb:106:in >> `call'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> mime_responds.rb:106:in >> `respond_to'' >> [RAILS_ROOT]/app/controllers/photos_controller.rb:47:in `create'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> base.rb:1158:in >> `send'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> base.rb:1158:in >> `perform_action_without_filters'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> filters.rb:697:in >> `call_filters'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> filters.rb:689:in >> `perform_action_without_benchmark'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> benchmarking.rb:68:in >> `perform_action_without_rescue'' >> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> benchmarking.rb:68:in >> `perform_action_without_rescue'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> rescue.rb:199:in >> `perform_action_without_caching'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> caching.rb:678:in >> `perform_action'' >> >> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/ >> connection_adapters/abstract/query_cache.rb:33:in >> `cache'' >> >> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/ >> query_cache.rb:8:in >> `cache'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> caching.rb:677:in >> `perform_action'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> base.rb:524:in >> `send'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> base.rb:524:in >> `process_without_filters'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> filters.rb:685:in >> `process_without_session_management_support'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> session_management.rb:123:in >> `sass_old_process'' >> /usr/lib/ruby/gems/1.8/gems/haml-1.8.1/lib/sass/plugin/rails.rb: >> 15:in >> `process'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> base.rb:388:in >> `process'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> dispatcher.rb:171:in >> `handle_request'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> dispatcher.rb:115:in >> `dispatch'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> dispatcher.rb:126:in >> `dispatch_cgi'' >> >> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/ >> dispatcher.rb:9:in >> `dispatch'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:76:in >> `process'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in >> `synchronize'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in >> `process'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:159:in >> `process_client'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in >> `each'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in >> `process_client'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in >> `run'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in >> `initialize'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in >> `new'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in >> `run'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in >> `initialize'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in >> `new'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in >> `run'' >> >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/ >> configurator.rb:282:in >> `run'' >> >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/ >> configurator.rb:281:in >> `each'' >> >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/ >> configurator.rb:281:in >> `run'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:128:in >> `run'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/command.rb: >> 212:in >> `run'' >> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:281 >> /usr/bin/mongrel_rails:19:in `load'' >> /usr/bin/mongrel_rails:19 >> >> My backgroundrb.yml file: >> :backgroundrb: >> :ip: 0.0.0.0 >> >> :development: >> :backgroundrb: >> :port: 11006 # use port 11006 >> :log: foreground # foreground mode,print log messages on console >> :debug_log: true >> >> :staging: >> :backgroundrb: >> :port: 22222 # use port 22222 >> :lazy_load: true # do not load models eagerly >> :debug_log: false # disable log workers and other logging >> >> :production: >> :backgroundrb: >> :port: 33333 # use port 33333 >> :lazy_load: true # do not load models eagerly >> :debug_log: false # disable log workers and other logging >> >> My worker: >> class ThumbnailGeneratorWorker < BackgrounDRb::MetaWorker >> set_worker_name :thumbnail_generator_worker >> set_no_auto_load(true) >> >> def create(args = nil) >> register_status(:percent_complete => 0) >> args.each_with_index do |uploader_id, index| >> @uploader = Uploader.find(uploader_id) >> @uploader.generate_thumbnails >> percent_complete = ((index + 1) * 100) / args.length >> logger.info "Thumbnail generation is #{percent_complete}% >> complete..." >> register_status(:percent_complete => percent_complete) >> end >> exit >> >> end >> end >> >> Any hints/help/ideas would be much appreciated. >> >> Bobby >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>