Backgroundrb devel - Jun 2006 - New release!

Hey Folks-

	I have just pushed a new release of BackgrounDRb to rubyforge. It  
has some nice new features to allow for usage of your ActiveRecord  
Models within your worker classes. I also added a config file for  
setting hosts and ports and if you want to load rails or not.

	Please have a look and let me know if anyone runs into any issues.  
If you already have an older version installed then you need to  
delete the script/backgroundrb dir from your rails app and reinstall  
the plugin. You can leave your lib/workers dir alone and it won''t get  
clobbered. Once you have run rake backgroundrb:setup with the new  
plugin you need to look into config/backlgroundrb.yml for your  
settings. The defaults are fine but you need to be aware of them. You  
can set whether ActiveRecord connects to your development or  
production databases in the conf file. It will use your existing  
database.ymkl to make the connection.

         This is a new improved way to create your worker classes.
         You just need to inherit from BackgrounDRb::Rails and define
         a do_work(args) method. This method will automatically get
         called in its own thread when you create a new worker from
         rails. So when you say:

	MiddleMan.new_worker :class => :foo_worker, :args => "Hello!"

	The "Hello" argument gets sent to your do_work method.

	# Put your code that runs your task inside the do_work method
	# it will be run automatically in a thread. You have access to
	# all of your rails models if you set load_rails to true in the
	# config file. You also get @logger inside of this class by default.
	class FooWorker < BackgrounDRb::Rails

	  def do_work(args)
	    # This method is called in it''s own new thread when you
	    # call new worker. args is set to :args.
             @logger.debug Post.find(:all).to_yaml
	  end

	end

	So you no longer have to deal with THreads yourself. Just inherit  
from the right class and this is taken care of for you.


Cheers-
-Ezra

> Please have a look and let me know if anyone runs into any issues.
Ezra,
This is fantastic!
However, it''s not working for me.  I think it''s because my
app''s
custom config (in config/environments/stuff.rb) does not appear to be  
loaded along with the rest of Rails.  Could that be?

	uninitialized constant StuffModule

Thanks for your awesome work,
pmark

On Jun 13, 2006, at 12:38 AM, P.Mark Anderson wrote:
>
>> Please have a look and let me know if anyone runs into any issues.
>
> Ezra,
> This is fantastic!
> However, it''s not working for me.  I think it''s because
my app''s
> custom config (in config/environments/stuff.rb) does not appear to be
> loaded along with the rest of Rails.  Could that be?
>
> 	uninitialized constant StuffModule
>
> Thanks for your awesome work,
> pmark
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

Hey PMark-

	Yeah that''s what it is. When backgroundrb starts up and the  
load_rails config is set to true, all I do is require boot.rb and  
then set up the db connection. If you have other classes that don''t  
get loaded when just boot.rb is required then you will need to  
explicitly require them.

	BackgrounDRb doesn''t need all of rails loaded. THat would make it  
consume way more memory then it needs to run.  I am open for  
suggestions though on how to better handle this. Maybe I can create a  
config file option where you can list the classes you need to  
require. For now you should just open script/backgroundrb/start and  
add a require for the file you need. USe rails root to qualify its  
path. So say you have RAILS_ROOT/lib/stuff_module.rb that you need to  
require.

Look for this line and add your require right after it.

# Require all worker classes in lib/workers/
Dir["#{RAILS_ROOT}/lib/workers/*"].each{ |worker| require worker }

require "#{RAILS_ROOT}/lib/stuff_module"

	Let me know if you run into any more problems.

Cheers-
-Ezra

>
> 	BackgrounDRb doesn''t need all of rails loaded. THat would make it
> consume way more memory then it needs

Hi Ezra - I got it to work for me.  At first I changed the start  
script and config to include only the libs I needed, but then I  
needed a plugin (acts_as_taggable), so I included that too, but then  
the ''logger'' instance used by my models wasn''t
defined, so I decided
to simply include the whole Rails environment and now all is well.   
Is it the most efficient use of memory?  No.  But oh well.  I need it.

To include the whole Rails environment I made this change in script/ 
backgroundrb/start:

if CONFIG[''load_rails'']
   require "#{RAILS_ROOT}/config/environment"
end


Alternatively, you can pick and choose environment config libs and  
plugins by adding this to your config/backgroundrb.yml:

libs:
     environments:
         pocket_config
	some_other_config
     plugins:
         acts_as_taggable
	some_other_plugin


And then change script/backgroundrb/start:

# Require all worker classes in lib/workers/
Dir["#{RAILS_ROOT}/lib/workers/*"].each{ |worker| require worker }

# Load other libraries
unless CONFIG[''libs''].nil?
    env_libs =
CONFIG[''libs''][''environments'']
    plugin_libs = CONFIG[''libs''][''plugins'']
    env_libs.each {|lib| require "#{RAILS_ROOT}/config/environments/# 
{lib}"} unless env_libs.nil?
    plugin_libs.each {|plugin| require "#{RAILS_ROOT}/vendor/plugins/# 
{plugin}/init"} unless plugin_libs.nil?
end


HTH.
I might name my firstborn Ezra now.
pmark

Hi-
>>
>> 	BackgrounDRb doesn''t need all of rails loaded. THat would
make it
>> consume way more memory then it needs
>
>
> Hi Ezra - I got it to work for me.  At first I changed the start
> script and config to include only the libs I needed, but then I
> needed a plugin (acts_as_taggable), so I included that too, but then
> the ''logger'' instance used by my models wasn''t
defined, so I decided
> to simply include the whole Rails environment and now all is well.
> Is it the most efficient use of memory?  No.  But oh well.  I need it.
>
> To include the whole Rails environment I made this change in script/
> backgroundrb/start:
>
> if CONFIG[''load_rails'']
>    require "#{RAILS_ROOT}/config/environment"
> end
>
>
> Alternatively, you can pick and choose environment config libs and
> plugins by adding this to your config/backgroundrb.yml:
>
> libs:
>      environments:
>          pocket_config
> 	some_other_config
>      plugins:
>          acts_as_taggable
> 	some_other_plugin
>
>
> And then change script/backgroundrb/start:
>
> # Require all worker classes in lib/workers/
> Dir["#{RAILS_ROOT}/lib/workers/*"].each{ |worker| require worker
}
>
> # Load other libraries
> unless CONFIG[''libs''].nil?
>     env_libs =
CONFIG[''libs''][''environments'']
>     plugin_libs =
CONFIG[''libs''][''plugins'']
>     env_libs.each {|lib| require "#{RAILS_ROOT}/config/environments/#
> {lib}"} unless env_libs.nil?
>     plugin_libs.each {|plugin| require "#{RAILS_ROOT}/vendor/plugins/#
> {plugin}/init"} unless plugin_libs.nil?
> end
>
>
> HTH.
> I might name my firstborn Ezra now.
> pmark

	Cool man I''m glad its working good for you. And thanks for the  
additions. Now that we have a real config file stuff like this is  
easy to add. I''ll check this in when I get a chance.

	Can you do me a favor? Start up your drb server that requires all  
your rails stuff and use top to see how much mem it takes up. If you  
can just copy paste some top output that would be cool. I''m curious  
to see how much ram it takes up with everything loaded. When I run it  
without ActiveRecord or anything other then itself and a few workers  
it only takes up 2-3Mb of ram. ActiveRecord adds a bit to this  
bringing it up to 8Mb or so.



Cheers-
-Ezra

>>>
>>> BackgrounDRb doesn''t need all of rails loaded. THat would
make it
>>> consume way more memory then it needs
>>
>>
>> ...I decided to simply include the whole Rails environment...
>
> 	Can you do me a favor? Start up your drb server that requires all  
> your rails stuff and use top to see how much mem it takes up. If  
> you can just copy paste some top output that would be cool. I''m  
> curious to see how much ram it takes up with everything loaded.  
> When I run it without ActiveRecord or anything other then itself  
> and a few workers it only takes up 2-3Mb of ram. ActiveRecord adds  
> a bit to this bringing it up to 8Mb or so.
>

602M free goes to 581M free, so it consumes ~20 megabytes.  Running  
BackgrounDRb this way is equivalent to running the console.  Too bad,  
since the only thing missing after loading my AR models and extra  
libs is the @logger instance (I think).

Mark