Backgroundrb devel - Aug 2008 - purpose of delayed persistent jobs

We used to use persistent jobs implemented ourselves for an older  
version of backgroundrb (not sure what version, something from 2006),  
but they weren''t delayed and didn''t require polling. Polling
isn''t
really that big of a deal as far as hits to the database go, but I see  
negatives of this system, and can''t think of the benefits. The  
downsides to delayed persistent jobs are your background tasks don''t  
execute immediately,  and you have more hits to your database  
(although small). Shopify released a plugin to do exactly this, so it  
has to be useful for something, but when you have scheduling software  
like bdrb that does stuff immediately and without polling (and you can  
make it save results to the database if you want), what would be an  
example of something that benefits from this model of delayed database- 
backed jobs?

-Woody

(and sorry for the flood of questions. hopefully someone else in the  
vastness of the internet has the same ones :-)

On Fri, Aug 15, 2008 at 12:50 AM, Woody Peterson
<woody at crystalcommerce.com> wrote:
> We used to use persistent jobs implemented ourselves for an older version
of
> backgroundrb (not sure what version, something from 2006), but they
weren''t
> delayed and didn''t require polling. Polling isn''t really
that big of a deal
> as far as hits to the database go, but I see negatives of this system, and
> can''t think of the benefits. The downsides to delayed persistent
jobs are
> your background tasks don''t execute immediately,  and you have
more hits to
> your database (although small). Shopify released a plugin to do exactly
> this, so it has to be useful for something, but when you have scheduling
> software like bdrb that does stuff immediately and without polling (and you
> can make it save results to the database if you want), what would be an
> example of something that benefits from this model of delayed
> database-backed jobs?
Something thats super critical, shouldn''t be lost if backgroundrb
server crashes or anything like that. Can be run manually if that
kinda thing happens.

On Fri, Aug 15, 2008 at 4:14 AM, hemant <gethemant at gmail.com> wrote:
> On Fri, Aug 15, 2008 at 12:50 AM, Woody Peterson
> what would be an
> > example of something that benefits from this model of delayed
> > database-backed jobs?
>
> Something thats super critical, shouldn''t be lost if backgroundrb
> server crashes or anything like that. Can be run manually if that
> kinda thing happens.
>
I''ve forked the main repository here:

git://github.com/tatyree/backgroundrb.git

and made a couple of changes to make the job_queue consume all outstanding
jobs at once, which is what I understand Woody to be commenting on (and
something I''ve needed for a while).

I''m afraid I''ve got a conflict on the specs as I have a hard
dependency on
Rspec''s internal mocks, and so I''m having some trouble making
the specs run
in my environment (mocha just causes everything to fall apart badly, which
is ashame).  As a result, I''m afraid the level of testing is a little
(OK,
completely...it works for me) unsophisticated.  That said, I''m pretty
new to
Rspec, and am probably missing/misunderstanding something.

I started to re-add the test for RAILS_ENV that KieranP removed ( Line 27 of
bdrb_config.rb -  Object.const_set("RAILS_ENV",environment) unless
defined?(RAILS_ENV) ) as bdrb complains on startup without it.  However,
I''m
not at all clear why he removed it so I decided to leave it alone.

Hope I''ve done everything all right.  I''m new to Git and
Rspec, but not new
to development or Rails.

Best,
Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080817/c468fd07/attachment.html>

On Sun, Aug 17, 2008 at 7:43 PM, Todd Tyree <todd at snappl.co.uk>
wrote:
>
> I''ve forked the main repository here:
>
> git://github.com/tatyree/backgroundrb.git
>
> and made a couple of changes to make the job_queue consume all outstanding
> jobs at once, which is what I understand Woody to be commenting on (and
> something I''ve needed for a while).
>
> I''m afraid I''ve got a conflict on the specs as I have a
hard dependency on
> Rspec''s internal mocks, and so I''m having some trouble
making the specs run
> in my environment (mocha just causes everything to fall apart badly, which
> is ashame).  As a result, I''m afraid the level of testing is a
little (OK,
> completely...it works for me) unsophisticated.  That said, I''m
pretty new to
> Rspec, and am probably missing/misunderstanding something.
>
> I started to re-add the test for RAILS_ENV that KieranP removed ( Line 27
of
> bdrb_config.rb -  Object.const_set("RAILS_ENV",environment)
unless
> defined?(RAILS_ENV) ) as bdrb complains on startup without it.  However,
I''m
> not at all clear why he removed it so I decided to leave it alone.
>
> Hope I''ve done everything all right.  I''m new to Git and
Rspec, but not new
> to development or Rails.
We can''t add this, mainly because pulling all tasks out of queue at
once will essentially block the worker for a real long time. I do not
think, that will be a good idea, also if your worker is doing nothing
except running tasks from queue, you are essentially achieving the
same result(except for the part where worker is actually responding to
external events).

Thoughts?


PS: Good work, btw. Also, I am using test/spec not rspec there. ;)

Hello Todd,

gnufied actually removed that line, and for good reason. When RAILS_ENV is
set, which it appears is the case when it gets to that line, then it
won''t
overwrite it with that you want from the config file or the -e command line
option, so you''re always stuck in development mode, no matter what you
specify.

That complaining it makes is just a warning, and can be safely ignored.

Regards
KieranP


On Mon, Aug 18, 2008 at 2:13 AM, Todd Tyree <todd at snappl.co.uk>
wrote:
>
> I started to re-add the test for RAILS_ENV that KieranP removed ( Line 27
> of bdrb_config.rb -  Object.const_set("RAILS_ENV",environment)
unless
> defined?(RAILS_ENV) ) as bdrb complains on startup without it.  However,
I''m
> not at all clear why he removed it so I decided to leave it alone.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080818/2c819225/attachment-0001.html>