Backgroundrb devel - Aug 2008 - Strange behaviour (worker only running on 3 or more attempts)

Dear Mailing List,

I am trying to use backgroundRB to process an uploaded CSV file and save 
it in the database. I have successfully done this, with only one problem 
that seems very unusual. After starting the server, it takes three tries 
before my script is run.

For example, in my controller I have the code:

def authenticate_import
         @dataset = Dataset.new params[:dataset]
         @dataset.campaign = @campaign
         #Check the normal info is valid
         if !@dataset.valid?
            render :action => ''import''
         else
            #Start the worker
             MiddleMan.worker(:dataset_worker).async_process(:arg => 
@dataset)
            @progress = 
MiddleMan.worker(:dataset_worker).ask_result(:progress)
             render :action => ''progress''
         end
    end

This progress is then fed into a progress bar which periodically updates 
itself by calling:

@progress = MiddleMan.worker(:dataset_worker).ask_result(:progress)

The problem is that the first two times I attempt uploading (.i.e 
calling this code) nothing happens. No output happens at all. After that 
every call works fine and all the debugging info etc is written to the 
log files etc, everything is saved fine.

Does anyone have any ideas why this may be happening? any help would be 
greatly appreciated

My worker is below:

class DatasetWorker < BackgrounDRb::MetaWorker
  set_worker_name :dataset_worker

  def create(args = nil)
      logger.info "Dataset Worker setup"
  end

  def process(dataset)
    thread_pool.defer(:process_csv, dataset)
  end
   
  def process_csv(dataset)
    logger.info "Started to Process Dataset (title: #{dataset.name})"
    cache[:progress] = 0.00
    .
    .  #Code to save stuff into database and updating cache[:progress]
    .
  end

end

On Thu, Aug 14, 2008 at 5:11 PM, Iain <iain.adams.1985 at googlemail.com>
wrote:
> I am trying to use backgroundRB to process an uploaded CSV file and save it
> in the database. I have successfully done this, with only one problem that
> seems very unusual. After starting the server, it takes three tries before
> my script is run.
Which OS and what version of packet?

On 8/14/08, Iain <iain.adams.1985 at googlemail.com>
wrote:
> hemant wrote:
>  I''m using Packet-0.1.10 and OS is ubuntu 8.04 and yep bdrb 1.0.4
It shouldn''t happen, I will check later today and reply.

Hi,

>>> hemant wrote:
>>>  I''m using Packet-0.1.10 and OS is ubuntu 8.04 and yep
bdrb 1.0.4
>>>
>>
>> It shouldn''t happen, I will check later today and reply.
>>
>
> Thanks very much Herman. I am using default config except for :log:
> foreground
>

I tried to reproduce the problem, But I couldn''t, here is the code, I
tried:

class WowWorker < BackgrounDRb::MetaWorker
  set_worker_name :wow_worker

  def create(args = nil)
    # we are using memcache for storing results and hence, lets reset the cache
    # memcache will persist the results between backgroundrb server restarts
  end

  def run_wow user_args
    thread_pool.defer(:some_crap,user_args)
  end

  def some_crap user_args
    # we are using memcache for storing results and hence, lets reset the cache.
    # memcache will persist the results between backgroundrb server restarts
    # Also, if you are running lots of tasks parallely, don''t forget
to use unique keys
    cache[:progress] = 0.00
    5.times do |i|
      sleep(2)
      logger.info "starting some_crap #{i}"
      cache[:progress] = ((i+1)/5.00)*100
    end
  end
end

# And I called using:
MiddleMan.worker(:wow_worker).async_run_wow(:arg => "hello world")
MiddleMan.worker(:wow_worker).ask_result(:progress)

So, I wonder, whats wrong there. You can send me a minimized version
of your application in tar ball and i will see, if I can reproduce the
problem.

Also, upgrade to git version if you haven''t and don''t use
inbuilt
cache, try replacing it with memcache (again refer the docs).