Backgroundrb devel - Mar 2008 - Upgrading from older version - how?

Hi there,

We''ve been using previous version of bdrb (drb based one) for a while
and it was running relatively OK. We recently attempted to upgrade to
the new version to keep up / use supported version but we ended up
reverting to the older version.


Here are the things that moved us to revert :

1) With the workers we never manage to pass this error:
''''
You have a nil object when you didn''t expect it!
The error occurred while evaluating nil.send_request
/opt/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in
`ask_worker''
''''

2) It is not clear the difference between Middleman.worker(:my_worker,
key).ask_status and using the nonexistent method
Middleman.worker(:my_worker, key).mumbo_jumbo - they both return nil.
I understand that is related to one way invocation (asynch) but this
just too loose and easy to mixup things.

3) Unit testing - we used backgroundrb_mock.rb that mocks new_worker
and other framework api that manage the creation of workers, so
testing was simple as testing any other Rails classes. This is
actually pretty big one for us.

4) No RDoc  - older version has RDoc.

5) ./script/background start | stop are unreliable. Stop throws if the
pid file is not found. Invoking start multiple times start multiple
processes - not sure what that means. Invoking stop afterwards stops
only the one process obviously.

6) Finally it would be great to have some sort of migration cookbook -
how to change the older workers and migrate to the new version. The
number of changes are pretty big and we were never sure whether we
completed the migration.

Hope this does not sound too negative - those are our ''wishes''
for new
bdrdb to have.

thanks,
emil

On Fri, Mar 28, 2008 at 2:13 PM, Emil Marceta <emarceta at gmail.com>
wrote:
[snip]
>  5) ./script/background start | stop are unreliable. Stop throws if the
>  pid file is not found. Invoking start multiple times start multiple
>  processes - not sure what that means. Invoking stop afterwards stops
>  only the one process obviously.
I''m unable to reproduce the your issue of multiple processes when
invoking start multiple times.  Did you delete your
script/backgroundrb file and run rake backgroundrb:setup?

username at host ~/current_app_name $ ps aux | grep ruby &&
./script/backgroundrb start && ps aux | grep ruby
username  30073  0.0  2.0  48452 35288 ?        Sl   17:39   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3000 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3000.pid -l
/var/www/rails/app_name/current/log/mongrel.3000.log
username  30076  0.0  2.0  47808 34952 ?        Sl   17:39   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3001 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3001.pid -l
/var/www/rails/app_name/current/log/mongrel.3001.log
username  30079  0.0  2.1  51284 38292 ?        Sl   17:39   0:03
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3002 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3002.pid -l
/var/www/rails/app_name/current/log/mongrel.3002.log
username  30082  0.0  1.9  46684 33820 ?        Sl   17:40   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3003 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3003.pid -l
/var/www/rails/app_name/current/log/mongrel.3003.log
username  30402  0.0  1.7  43000 30396 ?        S    18:28   0:00 ruby
/var/www/rails/app_name/current/script/backgroundrb start
username  30403  0.0  1.7  42864 29860 ?        S    18:28   0:00 ruby
log_worker
username  30409  0.1  1.9  44972 33576 ?        S    18:28   0:01 ruby
queue_processing_worker
username  30484  0.0  0.0   1700   504 pts/0    R+   18:37   0:00 grep
--colour=auto ruby
username  30073  0.0  2.0  48452 35288 ?        Sl   17:39   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3000 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3000.pid -l
/var/www/rails/app_name/current/log/mongrel.3000.log
username  30076  0.0  2.0  47808 34952 ?        Sl   17:39   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3001 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3001.pid -l
/var/www/rails/app_name/current/log/mongrel.3001.log
username  30079  0.0  2.1  51284 38292 ?        Sl   17:39   0:03
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3002 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3002.pid -l
/var/www/rails/app_name/current/log/mongrel.3002.log
username  30082  0.0  1.9  46684 33820 ?        Sl   17:40   0:02
/usr/bin/ruby18 /usr/bin/mongrel_rails start -d -e production -a
10.252.63.80 -c /var/www/rails/app_name/current -p 3003 -P
/var/www/rails/app_
name/current/tmp/pids/mongrel.3003.pid -l
/var/www/rails/app_name/current/log/mongrel.3003.log
username  30402  0.0  1.7  43008 30404 ?        S    18:28   0:00 ruby
/var/www/rails/app_name/current/script/backgroundrb start
username  30403  0.0  1.7  42864 29868 ?        S    18:28   0:00 ruby
log_worker
username  30409  0.1  1.9  44972 33576 ?        S    18:28   0:01 ruby
queue_processing_worker
username  30488  0.0  1.6  41724 29224 pts/0    R    18:37   0:00 ruby
log_worker
username  30490  0.0  0.0   1700   504 pts/0    R+   18:37   0:00 grep
--colour=auto ruby


I use capistrano and have added the following tasks to my
config/deploy.rb for handling the thrown error where a pid file
doesn''t exist.

 task :restart do
    run "mongrel_rails cluster::restart -C #{mongrel_conf}"
    restart_backgroundrb
  end

  task :restart_backgroundrb do
    begin stop_backgroundrb; rescue; end #this catches the bdrb error
where a PID file doesn''t exist
    start_backgroundrb
  end

  task :stop_backgroundrb do
    run "#{deploy_to}/current/script/backgroundrb stop"
  end

  task :start_backgroundrb do
    run "#{deploy_to}/current/script/backgroundrb start"
  end

Good luck!
  Jonathan

On Fri, Mar 28, 2008 at 7:44 PM, Jonathan Wallace
<jonathan.wallace at gmail.com> wrote:
> On Fri, Mar 28, 2008 at 2:13 PM, Emil Marceta <emarceta at gmail.com>
wrote:
>  [snip]
>
> >  5) ./script/background start | stop are unreliable. Stop throws if
the
>  >  pid file is not found. Invoking start multiple times start multiple
>  >  processes - not sure what that means. Invoking stop afterwards stops
>  >  only the one process obviously.
>
>  I''m unable to reproduce the your issue of multiple processes when
>  invoking start multiple times.  Did you delete your
>  script/backgroundrb file and run rake backgroundrb:setup?
Yes. Here is on vanilla Rails project created from scratch with plugin
installed at the same time.


wahoo:bdrb_test emil$ ./script/backgroundrb start
wahoo:bdrb_test emil$ ps ax | grep ruby
 3075 s003  S      0:00.08 ruby ./script/backgroundrb start
 3076 s003  S      0:00.05 ruby log_worker
TERM_PROGRAM=iTerm.app
 3078 s003  R+     0:00.00 grep ruby
wahoo:bdrb_test emil$ ./script/backgroundrb start
wahoo:bdrb_test emil$ ps ax | grep ruby
 3075 s003  S      0:00.21 ruby ./script/backgroundrb start
 3076 s003  S      0:00.14 ruby log_worker
TERM_PROGRAM=iTerm.app
 3083 s003  R+     0:00.00 grep ruby
wahoo:bdrb_test emil$ ./script/backgroundrb stop
Deleting pid file
wahoo:bdrb_test emil$ ps ax | grep ruby
 3075 s003  S      0:00.37 ruby ./script/backgroundrb start
 3076 s003  S      0:00.25 ruby log_worker
TERM_PROGRAM=iTerm.app
 3086 s003  R+     0:00.00 grep ruby
wahoo:bdrb_test emil$ ./script/backgroundrb stop
./script/backgroundrb:46:in `initialize'': No such file or directory -
/Users/emil/tmp/bdrb_test/tmp/pids/backgroundrb_11006.pid
(Errno::ENOENT)
        from ./script/backgroundrb:46:in `open''
        from ./script/backgroundrb:46
wahoo:bdrb_test emil$ ps ax | grep ruby
 3075 s003  S      0:00.95 ruby ./script/backgroundrb start
 3076 s003  S      0:00.61 ruby log_worker
TERM_PROGRAM=iTerm.app
 3089 s003  R+     0:00.00 grep ruby
wahoo:bdrb_test emil$

Maybe is OS X specific. I added a guard in script/backgroundrb that
tests for the pid file existence and that fixes double stop but he
double start is still there. Could it be that the 2nd start is
rewriting the pid file?

The start/stop issue is a less important one though. Migration process
/ tasks from old version to this version is our major issue. We were
unsuccessful to convert workers and framework to this version.

emil

On Fri, Mar 28, 2008 at 8:25 PM, Jonathan Wallace
<jonathan.wallace at gmail.com> wrote:
> On Fri, Mar 28, 2008 at 3:01 PM, Emil Marceta <emarceta at gmail.com>
wrote:
<snip/>
>  All of my workers are basically just calls into my models.  I''m
not
>  using the dynamic worker creation.  This keeps all testing in
>  Test::Unit.  I''m using bdrb to run jobs that are queued to the
>  database by controllers.

Yes. however to do integration or functional testing - where
controller starts the worker  - fully mocked framework is needed.
backroundrb_mock.rb in the older version is perfect, it does not
require backgroundrb server and runs the worker class synchronously.

However I''m still stuck at the step 1 - to invoke the worker. I get
this exception :
''''
The error occurred while evaluating nil.send_request
/opt/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in
`ask_worker''
''''

emil

On Fri, Mar 28, 2008 at 8:01 PM, Emil Marceta <emarceta at gmail.com>
wrote:
> On Fri, Mar 28, 2008 at 7:44 PM, Jonathan Wallace
>  <jonathan.wallace at gmail.com> wrote:
>  > On Fri, Mar 28, 2008 at 2:13 PM, Emil Marceta <emarceta at
gmail.com> wrote:
[snip]
>  Maybe is OS X specific.
just for a reference here is the old version on OS X.

wahoo:steam emil$ ./script/backgroundrb start
wahoo:steam emil$ ./script/backgroundrb start
ERROR: there is already one or more instance(s) of the program running

emil

Hello,

I''d just like to update that my capistrano deploy tasks for
backgroundrb were incorrect for my situation.  In my case, the current
working directory for backgroundrb on deploy was the home directory of
the deploy user.  This caused some issues in my app.  Here are the
updated tasks which correct the issue.

  task :restart do
    run "mongrel_rails cluster::restart -C #{mongrel_conf}"
    restart_backgroundrb
  end

  task :restart_backgroundrb do
    begin stop_backgroundrb; rescue; end #this catches the bdrb error
where a PID file doesn''t exist
    start_backgroundrb
  end

  task :stop_backgroundrb do
    run "cd #{deploy_to}/current &&
#{deploy_to}/current/script/backgroundrb stop"
  end

  task :start_backgroundrb do
    run "cd #{deploy_to}/current &&
#{deploy_to}/current/script/backgroundrb start"
  end

Jonathan

On Fri, Mar 28, 2008 at 2:44 PM, Jonathan Wallace
<jonathan.wallace at gmail.com> wrote:
>  I use capistrano and have added the following tasks to my
>  config/deploy.rb for handling the thrown error where a pid file
>  doesn''t exist.
>
>   task :restart do
>     run "mongrel_rails cluster::restart -C #{mongrel_conf}"
>     restart_backgroundrb
>   end
>
>   task :restart_backgroundrb do
>     begin stop_backgroundrb; rescue; end #this catches the bdrb error
>  where a PID file doesn''t exist
>     start_backgroundrb
>   end
>
>   task :stop_backgroundrb do
>     run "#{deploy_to}/current/script/backgroundrb stop"
>   end
>
>   task :start_backgroundrb do
>     run "#{deploy_to}/current/script/backgroundrb start"
>   end
>
>  Good luck!
>   Jonathan
>