Backgroundrb devel - Nov 2007 - BackgrounDRb version 1.0RC1 available now

Hi Folks,


BackgrounDRb is a Ruby job server and scheduler. Its main intent is to be
used with Ruby on Rails applications for offloading long-running tasks.
Since a Rails application blocks while serving a request it is best to
move long-running tasks off into a background process that is divorced
from http request/response cycle.

This new release of BackgrounDRb is also modular and can be used without
Rails. So any Ruby program or framework can use it.

Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)

== Usage

=== Installation
Getting the code:
  svn co http://svn.devjavu.com/backgroundrb/branches/version10/

Installation with svn externals:
  svn propedit svn:externals vendor/plugins
  [add the following line:]
  backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
  [exit editor]

  svn ci -m ''updating svn:external svn property for
backgroundrb'' vendor/plugins
  svn up vendor/plugins
  rake backgroundrb:setup

Installation with piston:
  piston import
http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb

=== Configuration
  Use rake task for initial configuration:

* Cron style scheduling and config

  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  |
  | :schedules:
  |   :foo_worker:
  |     :worker_method: foobar
  |     :trigger_args: */5 * * * * * *

* Normal Unix scheduler
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :start: <%= Time.now + 5.seconds %>
  |     :end: <%= Time.now + 10.minutes %>
  |     :repeat_interval: <%= 1.minute %>

* Plain config
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006

=== Scheduling
  There are three schemes for periodic execution and scheduling.
  - Cron Scheduling
    You can use configuration file for cron scheduling of workers.
Method specified in configuration
    file would be called periodically. You should take care of the
fact that, time gap between periodic
    invocation of a method should be more than the time thats actually
required to execute the method.
    If a method takes longer time than the time window specified, your
method invocations would lag
    perpetually.
  - Normal Scheduler
    You can use second form of scheduling as shown in config file.
  - add_periodic_timer method
    A third and very basic form of scheduling that you can use is,
"add_periodic_timer" method. You can call
    method from anywhere in your worker.

           def create
             add_periodic_timer(5) { say_hello }
           end

    Above snippet would register the proc for periodic execution at
every 5 seconds.

=== Code
  Install the plugin, and run setup task. Create a worker, using
worker generator.

          ./script/generatr worker bar

  You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/(
called WORKER_ROOT henceforth ).
  Generated code will look like this:

  class BarWorker < BackgrounDRb::MetaWorker
    set_worker_name :bar_worker
    def create
      # this method is called, when worker is loaded for the first time
      puts "starting a bar worker"
    end

    def process_request(p_data)
      user_input = p_data[:data]
      result = self.send(user_input[:method],user_input[:data])
      send_response(p_data,result)
    end
  end

  ''create'' method gets called, when worker is loaded and
created. Each
worker runs in its
  own process and you can use ''create'' for initializing worker
specific stuff.
  All the requests, sent from rails, to bdrb would be received in
process_request method.
  Received data, contains client address as well, which will be used
to send the response
  back to client. You just need to extract the data from received data
and call appropriate
  method. Once, you are done with executing your method, you can use
"send_response" to send
  the result back to client ( rails in our case ).

  It should be noted that, you must pass original data as an parameter
with send_response
  method because original data contains client address.

  Following code snippet, would ask bdrb to execute method
''add_values'' in ''foo_worker'' with
  arguments ''10+10'' and return the result.

       MiddleMan.send_request(:worker => :foo_worker, :method =>
:add_values,:data => "10+10")

  You can also use register_status as described in following snippet
to register status of
  your worker with master, which can be directly queried from rails.

       register_status(some_status_data)

  From rails, you can query status object using following code:

       MiddleMan.ask_status(:worker => :foo_worker)

  Above code would return status object of ''foo_worker''. When
you call
register_status
  from a worker, it replaces older state of the worker with master.
Since, master process
  stores status of the worker, all the status queries are served by
master itself. It can be
  used to store result hashes and stuff. Unlike previous versions of
bdrb, there shouldn''t be
  any data corruption.

=== Legacy and deprecated stuff

   Although, You need to wrap your head a bit for understanding
"evented" model of network programming,
   but it gets easier once you get hang of it. Much of the older stuff
is deprecated. Here is a brief list:

   - ACL : gone, trust to thy firewalls.
   - Runtime scheduling through MiddleMan proxy object.
     I don''t know if many people used this feature, and is probably
easy to implement back.
   - Threads: find .|grep ''Thread'' , gone
   - Passing of arguments from configuration file.
     Again, I am not sure many people used it either, if you are
passing arguments to periodic
     methods from configuration files, you may as well hardcode that
argument in worker itself.

   Some people, asked, if we can do "progress bar" or
"file_upload"
with new code base. The answer is yes, but
   again, changes would be required. "progress bar" example would look
like this:

     class ProgressWorker < BackgrounDRb::MetaWorker
       set_worker_name :progress_worker
       def create
         @counter = 0
         add_peridic_timer(2) { increment_counter }
       end
       def increment_counter
         @counter += 1
         register_status(@counter)
       end
     end

   And using MiddleMan proxy, you can keep queering status of your progress bar:

       MiddleMan.ask_status(:worker => :progress_worker)

   I would welcome, anyone who contributes more examples back. You can
even use callbacks
   or Deferable pattern for invoking callbacks.

=== Exciting new stuff
  * Rock solid stable ( or will be , after few bug reports )
  * Master Process is using a hash for storing status of each worker.
I am thinking of adding something like this:

        #backgroundrb.yml
        status_storage:
          storage: db
          database: worker_status
    Above code would fullfill thy wishes of persitent worker status
storage. Any more ideas, welcome.
  * Each worker comes with Event loop of its own and can potentially
do lots of fancy stuff. Two noteworthy methods are:

         connect(ip,port,Handler)
         start_worker(ip,port,Handler)

    If you are familiar with EventMachine or Twisted style of network
programming, above methods allow you to
    start tcp servers inside your workers or lets you connect to
external tcp servers. For Each accepted client or
    connected socket a instance of Handler class would be created and
integrated with main event loop.
    This can be used for worker to worker communication between
backgroundrb servers running on two machines.

    You are encouraged to look into framework directory, and see the
code that implements all this stuff.Guts of
    new bdrb is based on this library, which would be released soon as
separately.

== Online Resources
- http://svn.devjavu.com/backgroundrb/branches/version10/
- http://backgroundrb.devjavu.com (trac)
- http://backgroundrb.rubyforge.org (rdoc)




-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Heya all,

A coworker and I have checked out the new version of BackgrounDRb in
order to build one of our latest projects, so far we''ve just tried to
replicate some of the functionality we had when testing the old trunk
version.

I may be missing something obvious, but from the code it doesn''t appear
that you can set off multiple copies of the same worker like you could
in the old trunk version. We intend to use BackgrounDRb to set off snmp
scans of various devices on our network, some of which take some time to
complete. It is likely that this will happen more than once at any
particular moment in time.

At the moment, using the sample code we''ve moved the reply to the
MiddleMan to before the work, therefore actually letting the calling
application carry on (which is what we need). However a second call to
the same worker obviously hangs until the first call has completed.

I think what I''m essentially saying is that we want multiple instances
of the same worker which can be identified by a key (as it was in the
trunk). Is this possible with the new version of BackgrounDRb, or is it
on the map?

Peter.

Peter A. Wood                     
Network Security Specialist       
Technical Services Group          
Lancaster University

PGP Fingerprint:
C3B5 376B 16C5 F8D1 E3AE  617F 5718 C338 1D06 0689

-----Original Message-----
From: backgroundrb-devel-bounces at rubyforge.org
[mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of hemant
Sent: 14 November 2007 05:02
To: backgroundrb-devel at rubyforge.org
Subject: [Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now

Hi Folks,


BackgrounDRb is a Ruby job server and scheduler. Its main intent is to
be used with Ruby on Rails applications for offloading long-running
tasks.
Since a Rails application blocks while serving a request it is best to
move long-running tasks off into a background process that is divorced
from http request/response cycle.

This new release of BackgrounDRb is also modular and can be used without
Rails. So any Ruby program or framework can use it.

Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)

== Usage

=== Installation
Getting the code:
  svn co http://svn.devjavu.com/backgroundrb/branches/version10/

Installation with svn externals:
  svn propedit svn:externals vendor/plugins
  [add the following line:]
  backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
  [exit editor]

  svn ci -m ''updating svn:external svn property for
backgroundrb''
vendor/plugins
  svn up vendor/plugins
  rake backgroundrb:setup

Installation with piston:
  piston import
http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb

=== Configuration
  Use rake task for initial configuration:

* Cron style scheduling and config

  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  |
  | :schedules:
  |   :foo_worker:
  |     :worker_method: foobar
  |     :trigger_args: */5 * * * * * *

* Normal Unix scheduler
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :start: <%= Time.now + 5.seconds %>
  |     :end: <%= Time.now + 10.minutes %>
  |     :repeat_interval: <%= 1.minute %>

* Plain config
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006

=== Scheduling
  There are three schemes for periodic execution and scheduling.
  - Cron Scheduling
    You can use configuration file for cron scheduling of workers.
Method specified in configuration
    file would be called periodically. You should take care of the fact
that, time gap between periodic
    invocation of a method should be more than the time thats actually
required to execute the method.
    If a method takes longer time than the time window specified, your
method invocations would lag
    perpetually.
  - Normal Scheduler
    You can use second form of scheduling as shown in config file.
  - add_periodic_timer method
    A third and very basic form of scheduling that you can use is,
"add_periodic_timer" method. You can call
    method from anywhere in your worker.

           def create
             add_periodic_timer(5) { say_hello }
           end

    Above snippet would register the proc for periodic execution at
every 5 seconds.

=== Code
  Install the plugin, and run setup task. Create a worker, using worker
generator.

          ./script/generatr worker bar

  You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/( called
WORKER_ROOT henceforth ).
  Generated code will look like this:

  class BarWorker < BackgrounDRb::MetaWorker
    set_worker_name :bar_worker
    def create
      # this method is called, when worker is loaded for the first time
      puts "starting a bar worker"
    end

    def process_request(p_data)
      user_input = p_data[:data]
      result = self.send(user_input[:method],user_input[:data])
      send_response(p_data,result)
    end
  end

  ''create'' method gets called, when worker is loaded and
created. Each
worker runs in its
  own process and you can use ''create'' for initializing worker
specific
stuff.
  All the requests, sent from rails, to bdrb would be received in
process_request method.
  Received data, contains client address as well, which will be used to
send the response
  back to client. You just need to extract the data from received data
and call appropriate
  method. Once, you are done with executing your method, you can use
"send_response" to send
  the result back to client ( rails in our case ).

  It should be noted that, you must pass original data as an parameter
with send_response
  method because original data contains client address.

  Following code snippet, would ask bdrb to execute method
''add_values''
in ''foo_worker'' with
  arguments ''10+10'' and return the result.

       MiddleMan.send_request(:worker => :foo_worker, :method =>
:add_values,:data => "10+10")

  You can also use register_status as described in following snippet to
register status of
  your worker with master, which can be directly queried from rails.

       register_status(some_status_data)

  From rails, you can query status object using following code:

       MiddleMan.ask_status(:worker => :foo_worker)

  Above code would return status object of ''foo_worker''. When
you call
register_status
  from a worker, it replaces older state of the worker with master.
Since, master process
  stores status of the worker, all the status queries are served by
master itself. It can be
  used to store result hashes and stuff. Unlike previous versions of
bdrb, there shouldn''t be
  any data corruption.

=== Legacy and deprecated stuff

   Although, You need to wrap your head a bit for understanding
"evented" model of network programming,
   but it gets easier once you get hang of it. Much of the older stuff
is deprecated. Here is a brief list:

   - ACL : gone, trust to thy firewalls.
   - Runtime scheduling through MiddleMan proxy object.
     I don''t know if many people used this feature, and is probably
easy
to implement back.
   - Threads: find .|grep ''Thread'' , gone
   - Passing of arguments from configuration file.
     Again, I am not sure many people used it either, if you are passing
arguments to periodic
     methods from configuration files, you may as well hardcode that
argument in worker itself.

   Some people, asked, if we can do "progress bar" or
"file_upload"
with new code base. The answer is yes, but
   again, changes would be required. "progress bar" example would look
like this:

     class ProgressWorker < BackgrounDRb::MetaWorker
       set_worker_name :progress_worker
       def create
         @counter = 0
         add_peridic_timer(2) { increment_counter }
       end
       def increment_counter
         @counter += 1
         register_status(@counter)
       end
     end

   And using MiddleMan proxy, you can keep queering status of your
progress bar:

       MiddleMan.ask_status(:worker => :progress_worker)

   I would welcome, anyone who contributes more examples back. You can
even use callbacks
   or Deferable pattern for invoking callbacks.

=== Exciting new stuff
  * Rock solid stable ( or will be , after few bug reports )
  * Master Process is using a hash for storing status of each worker.
I am thinking of adding something like this:

        #backgroundrb.yml
        status_storage:
          storage: db
          database: worker_status
    Above code would fullfill thy wishes of persitent worker status
storage. Any more ideas, welcome.
  * Each worker comes with Event loop of its own and can potentially do
lots of fancy stuff. Two noteworthy methods are:

         connect(ip,port,Handler)
         start_worker(ip,port,Handler)

    If you are familiar with EventMachine or Twisted style of network
programming, above methods allow you to
    start tcp servers inside your workers or lets you connect to
external tcp servers. For Each accepted client or
    connected socket a instance of Handler class would be created and
integrated with main event loop.
    This can be used for worker to worker communication between
backgroundrb servers running on two machines.

    You are encouraged to look into framework directory, and see the
code that implements all this stuff.Guts of
    new bdrb is based on this library, which would be released soon as
separately.

== Online Resources
- http://svn.devjavu.com/backgroundrb/branches/version10/
- http://backgroundrb.devjavu.com (trac)
- http://backgroundrb.rubyforge.org (rdoc)




--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org
_______________________________________________
Backgroundrb-devel mailing list
Backgroundrb-devel at rubyforge.org
http://rubyforge.org/mailman/listinfo/backgroundrb-devel

On Nov 14, 2007 6:50 PM, Wood, Peter <p.wood at lancaster.ac.uk>
wrote:
> Heya all,
>
> A coworker and I have checked out the new version of BackgrounDRb in
> order to build one of our latest projects, so far we''ve just tried
to
> replicate some of the functionality we had when testing the old trunk
> version.
>
> I may be missing something obvious, but from the code it doesn''t
appear
> that you can set off multiple copies of the same worker like you could
> in the old trunk version. We intend to use BackgrounDRb to set off snmp
> scans of various devices on our network, some of which take some time to
> complete. It is likely that this will happen more than once at any
> particular moment in time.
>
> At the moment, using the sample code we''ve moved the reply to the
> MiddleMan to before the work, therefore actually letting the calling
> application carry on (which is what we need). However a second call to
> the same worker obviously hangs until the first call has completed.
>
> I think what I''m essentially saying is that we want multiple
instances
> of the same worker which can be identified by a key (as it was in the
> trunk). Is this possible with the new version of BackgrounDRb, or is it
> on the map?
>
With new version you can''t have two instances of a worker active at
the same time. Although, I do think of having that facility soon, but
for the time being you will have to put up with writing copies of
workers. BTW, how many copies do you need and how much time, does
it take to complete scan of devices on network?


>
> -----Original Message-----
> From: backgroundrb-devel-bounces at rubyforge.org
> [mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of hemant
> Sent: 14 November 2007 05:02
> To: backgroundrb-devel at rubyforge.org
> Subject: [Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now
>
> Hi Folks,
>
>
> BackgrounDRb is a Ruby job server and scheduler. Its main intent is to
> be used with Ruby on Rails applications for offloading long-running
> tasks.
> Since a Rails application blocks while serving a request it is best to
> move long-running tasks off into a background process that is divorced
> from http request/response cycle.
>
> This new release of BackgrounDRb is also modular and can be used without
> Rails. So any Ruby program or framework can use it.
>
> Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
> Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)
>
> == Usage
>
> === Installation
> Getting the code:
>   svn co http://svn.devjavu.com/backgroundrb/branches/version10/
>
> Installation with svn externals:
>   svn propedit svn:externals vendor/plugins
>   [add the following line:]
>   backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
>   [exit editor]
>
>   svn ci -m ''updating svn:external svn property for
backgroundrb''
> vendor/plugins
>   svn up vendor/plugins
>   rake backgroundrb:setup
>
> Installation with piston:
>   piston import
> http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb
>
> === Configuration
>   Use rake task for initial configuration:
>
> * Cron style scheduling and config
>
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>   |
>   | :schedules:
>   |   :foo_worker:
>   |     :worker_method: foobar
>   |     :trigger_args: */5 * * * * * *
>
> * Normal Unix scheduler
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>   | :schedules:
>   |   :foo_worker:
>   |     :start: <%= Time.now + 5.seconds %>
>   |     :end: <%= Time.now + 10.minutes %>
>   |     :repeat_interval: <%= 1.minute %>
>
> * Plain config
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>
> === Scheduling
>   There are three schemes for periodic execution and scheduling.
>   - Cron Scheduling
>     You can use configuration file for cron scheduling of workers.
> Method specified in configuration
>     file would be called periodically. You should take care of the fact
> that, time gap between periodic
>     invocation of a method should be more than the time thats actually
> required to execute the method.
>     If a method takes longer time than the time window specified, your
> method invocations would lag
>     perpetually.
>   - Normal Scheduler
>     You can use second form of scheduling as shown in config file.
>   - add_periodic_timer method
>     A third and very basic form of scheduling that you can use is,
> "add_periodic_timer" method. You can call
>     method from anywhere in your worker.
>
>            def create
>              add_periodic_timer(5) { say_hello }
>            end
>
>     Above snippet would register the proc for periodic execution at
> every 5 seconds.
>
> === Code
>   Install the plugin, and run setup task. Create a worker, using worker
> generator.
>
>           ./script/generatr worker bar
>
>   You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/( called
> WORKER_ROOT henceforth ).
>   Generated code will look like this:
>
>   class BarWorker < BackgrounDRb::MetaWorker
>     set_worker_name :bar_worker
>     def create
>       # this method is called, when worker is loaded for the first time
>       puts "starting a bar worker"
>     end
>
>     def process_request(p_data)
>       user_input = p_data[:data]
>       result = self.send(user_input[:method],user_input[:data])
>       send_response(p_data,result)
>     end
>   end
>
>   ''create'' method gets called, when worker is loaded and
created. Each
> worker runs in its
>   own process and you can use ''create'' for initializing
worker specific
> stuff.
>   All the requests, sent from rails, to bdrb would be received in
> process_request method.
>   Received data, contains client address as well, which will be used to
> send the response
>   back to client. You just need to extract the data from received data
> and call appropriate
>   method. Once, you are done with executing your method, you can use
> "send_response" to send
>   the result back to client ( rails in our case ).
>
>   It should be noted that, you must pass original data as an parameter
> with send_response
>   method because original data contains client address.
>
>   Following code snippet, would ask bdrb to execute method
''add_values''
> in ''foo_worker'' with
>   arguments ''10+10'' and return the result.
>
>        MiddleMan.send_request(:worker => :foo_worker, :method =>
> :add_values,:data => "10+10")
>
>   You can also use register_status as described in following snippet to
> register status of
>   your worker with master, which can be directly queried from rails.
>
>        register_status(some_status_data)
>
>   From rails, you can query status object using following code:
>
>        MiddleMan.ask_status(:worker => :foo_worker)
>
>   Above code would return status object of ''foo_worker''.
When you call
> register_status
>   from a worker, it replaces older state of the worker with master.
> Since, master process
>   stores status of the worker, all the status queries are served by
> master itself. It can be
>   used to store result hashes and stuff. Unlike previous versions of
> bdrb, there shouldn''t be
>   any data corruption.
>
> === Legacy and deprecated stuff
>
>    Although, You need to wrap your head a bit for understanding
> "evented" model of network programming,
>    but it gets easier once you get hang of it. Much of the older stuff
> is deprecated. Here is a brief list:
>
>    - ACL : gone, trust to thy firewalls.
>    - Runtime scheduling through MiddleMan proxy object.
>      I don''t know if many people used this feature, and is
probably easy
> to implement back.
>    - Threads: find .|grep ''Thread'' , gone
>    - Passing of arguments from configuration file.
>      Again, I am not sure many people used it either, if you are passing
> arguments to periodic
>      methods from configuration files, you may as well hardcode that
> argument in worker itself.
>
>    Some people, asked, if we can do "progress bar" or
"file_upload"
> with new code base. The answer is yes, but
>    again, changes would be required. "progress bar" example would
look
> like this:
>
>      class ProgressWorker < BackgrounDRb::MetaWorker
>        set_worker_name :progress_worker
>        def create
>          @counter = 0
>          add_peridic_timer(2) { increment_counter }
>        end
>        def increment_counter
>          @counter += 1
>          register_status(@counter)
>        end
>      end
>
>    And using MiddleMan proxy, you can keep queering status of your
> progress bar:
>
>        MiddleMan.ask_status(:worker => :progress_worker)
>
>    I would welcome, anyone who contributes more examples back. You can
> even use callbacks
>    or Deferable pattern for invoking callbacks.
>
> === Exciting new stuff
>   * Rock solid stable ( or will be , after few bug reports )
>   * Master Process is using a hash for storing status of each worker.
> I am thinking of adding something like this:
>
>         #backgroundrb.yml
>         status_storage:
>           storage: db
>           database: worker_status
>     Above code would fullfill thy wishes of persitent worker status
> storage. Any more ideas, welcome.
>   * Each worker comes with Event loop of its own and can potentially do
> lots of fancy stuff. Two noteworthy methods are:
>
>          connect(ip,port,Handler)
>          start_worker(ip,port,Handler)
>
>     If you are familiar with EventMachine or Twisted style of network
> programming, above methods allow you to
>     start tcp servers inside your workers or lets you connect to
> external tcp servers. For Each accepted client or
>     connected socket a instance of Handler class would be created and
> integrated with main event loop.
>     This can be used for worker to worker communication between
> backgroundrb servers running on two machines.
>
>     You are encouraged to look into framework directory, and see the
> code that implements all this stuff.Guts of
>     new bdrb is based on this library, which would be released soon as
> separately.
>
> == Online Resources
> - http://svn.devjavu.com/backgroundrb/branches/version10/
> - http://backgroundrb.devjavu.com (trac)
> - http://backgroundrb.rubyforge.org (rdoc)
>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

On Nov 14, 2007, at 8:55 AM, hemant wrote:
> On Nov 14, 2007 6:50 PM, Wood, Peter <p.wood at lancaster.ac.uk>
wrote:
>> Heya all,
>>
>> A coworker and I have checked out the new version of BackgrounDRb in
>> order to build one of our latest projects, so far we''ve just
tried to
>> replicate some of the functionality we had when testing the old trunk
>> version.
>>
>> I may be missing something obvious, but from the code it
doesn''t
>> appear
>> that you can set off multiple copies of the same worker like you  
>> could
>> in the old trunk version. We intend to use BackgrounDRb to set off  
>> snmp
>> scans of various devices on our network, some of which take some  
>> time to
>> complete. It is likely that this will happen more than once at any
>> particular moment in time.
>>
>> At the moment, using the sample code we''ve moved the reply to
the
>> MiddleMan to before the work, therefore actually letting the calling
>> application carry on (which is what we need). However a second call  
>> to
>> the same worker obviously hangs until the first call has completed.
>>
>> I think what I''m essentially saying is that we want multiple  
>> instances
>> of the same worker which can be identified by a key (as it was in the
>> trunk). Is this possible with the new version of BackgrounDRb, or  
>> is it
>> on the map?
>>
>
> With new version you can''t have two instances of a worker active
at
> the same time. Although, I do think of having that facility soon, but
> for the time being you will have to put up with writing copies of
> workers. BTW, how many copies do you need and how much time, does
> it take to complete scan of devices on network?
This is a show stopper for me.  The only thing I''ve ever used  
BackgrounDRb for it firing off workers at will to do various  
processing tasks, such as transferring large files or transcoding  
video.  If I have a quad core CPU, I want 4 instances of the same  
worker running in order to take full advantage of the CPU.

Brandon 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url :
http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071114/c9476221/attachment.bin

On Nov 14, 2007 7:57 PM, Brandon Keepers <brandon at opensoul.org>
wrote:
>
>
> On Nov 14, 2007, at 8:55 AM, hemant wrote:
>
> > On Nov 14, 2007 6:50 PM, Wood, Peter <p.wood at lancaster.ac.uk>
wrote:
> >> Heya all,
> >>
> >> A coworker and I have checked out the new version of BackgrounDRb
in
> >> order to build one of our latest projects, so far we''ve
just tried to
> >> replicate some of the functionality we had when testing the old
trunk
> >> version.
> >>
> >> I may be missing something obvious, but from the code it
doesn''t
> >> appear
> >> that you can set off multiple copies of the same worker like you
> >> could
> >> in the old trunk version. We intend to use BackgrounDRb to set off
> >> snmp
> >> scans of various devices on our network, some of which take some
> >> time to
> >> complete. It is likely that this will happen more than once at any
> >> particular moment in time.
> >>
> >> At the moment, using the sample code we''ve moved the
reply to the
> >> MiddleMan to before the work, therefore actually letting the
calling
> >> application carry on (which is what we need). However a second
call
> >> to
> >> the same worker obviously hangs until the first call has
completed.
> >>
> >> I think what I''m essentially saying is that we want
multiple
> >> instances
> >> of the same worker which can be identified by a key (as it was in
the
> >> trunk). Is this possible with the new version of BackgrounDRb, or
> >> is it
> >> on the map?
> >>
> >
> > With new version you can''t have two instances of a worker
active at
> > the same time. Although, I do think of having that facility soon, but
> > for the time being you will have to put up with writing copies of
> > workers. BTW, how many copies do you need and how much time, does
> > it take to complete scan of devices on network?
>
> This is a show stopper for me.  The only thing I''ve ever used
> BackgrounDRb for it firing off workers at will to do various
> processing tasks, such as transferring large files or transcoding
> video.  If I have a quad core CPU, I want 4 instances of the same
> worker running in order to take full advantage of the CPU.
>
Ok, then folks, I will be implementing it. Will be available soonish.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

On Nov 14, 2007 8:13 PM, hemant <gethemant at gmail.com>
wrote:
>
> On Nov 14, 2007 7:57 PM, Brandon Keepers <brandon at opensoul.org>
wrote:
> >
> >
> > On Nov 14, 2007, at 8:55 AM, hemant wrote:
> >
> > > On Nov 14, 2007 6:50 PM, Wood, Peter <p.wood at
lancaster.ac.uk> wrote:
> > >> Heya all,
> > >>
> > >> A coworker and I have checked out the new version of
BackgrounDRb in
> > >> order to build one of our latest projects, so far
we''ve just tried to
> > >> replicate some of the functionality we had when testing the
old trunk
> > >> version.
> > >>
> > >> I may be missing something obvious, but from the code it
doesn''t
> > >> appear
> > >> that you can set off multiple copies of the same worker like
you
> > >> could
> > >> in the old trunk version. We intend to use BackgrounDRb to
set off
> > >> snmp
> > >> scans of various devices on our network, some of which take
some
> > >> time to
> > >> complete. It is likely that this will happen more than once
at any
> > >> particular moment in time.
> > >>
> > >> At the moment, using the sample code we''ve moved the
reply to the
> > >> MiddleMan to before the work, therefore actually letting the
calling
> > >> application carry on (which is what we need). However a
second call
> > >> to
> > >> the same worker obviously hangs until the first call has
completed.
> > >>
> > >> I think what I''m essentially saying is that we want
multiple
> > >> instances
> > >> of the same worker which can be identified by a key (as it
was in the
> > >> trunk). Is this possible with the new version of
BackgrounDRb, or
> > >> is it
> > >> on the map?
> > >>
> > >
> > > With new version you can''t have two instances of a
worker active at
> > > the same time. Although, I do think of having that facility soon,
but
> > > for the time being you will have to put up with writing copies of
> > > workers. BTW, how many copies do you need and how much time, does
> > > it take to complete scan of devices on network?
> >
> > This is a show stopper for me.  The only thing I''ve ever used
> > BackgrounDRb for it firing off workers at will to do various
> > processing tasks, such as transferring large files or transcoding
> > video.  If I have a quad core CPU, I want 4 instances of the same
> > worker running in order to take full advantage of the CPU.
> >
>
> Ok, then folks, I will be implementing it. Will be available soonish.
Ok, on second thoughts, I was curious about the fact, that can''t you
have 4 different workers running at the same time?
Why you need copies of same workers? I know, you don''t have to write
additional worker code, if you use the same worker, but
apart from that what?

You can very well have 4 four workers like:

video_encoder_worker.rb
transfer_data_worker.rb
snmp_scanner_worker.rb
port_scanner_worker.rb

You can very well do, whatever you want inside them. Another question
is, how do you schedule workers ( which are instance of same class?).
You
use cron based scheduling or Scheduling using MiddleMan  or what?

I am not trying to dodge, but just want to understand full use case.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

Ok, whoever interested on discussing about future of backgroundrb is
welcome on #backgroundrb between 15.00 - 22.00 (UTC) on freenode.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

On Nov 14, 2007 8:32 PM, hemant <gethemant at gmail.com>
wrote:
> Ok, whoever interested on discussing about future of backgroundrb is
> welcome on #backgroundrb between 15.00 - 22.00 (UTC) on freenode.
Ok, we did had a sort of discussion and decided to have following features:

1. Log worker.
2. Ability to start & stop workers at runtime.
3. Currently, waiting for long responses in rails is not supported.
People apparently need it.
4. Ability to cancel wrong running tasks.
5. Better examples demonstrating file uploads, progress bar and stuff.

Now, If anyone commits few patches and is ready to work on things, I
would be happy to give commit access.

>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
>


-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org