Backgroundrb devel - Oct 2008 - Performance problems running Prawn under BackgrounDRb

I have a RoR application which I''m using Prawn via BackgrounDRb to
generate PDF''s.  I''ve determined there seems to be a 5-6x
performance
hit compared to running Prawn directly as a script from the command
line (~12sec to 60+sec) on my OS X system.

Anyone have any suggestions on how to improve performance?

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix &
Windows
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.  -- Benjamin Franklin

On Mon, Oct 27, 2008 at 10:49 AM, Aaron Turner <synfinatic at gmail.com>
wrote:
> I have a RoR application which I''m using Prawn via BackgrounDRb to
> generate PDF''s.  I''ve determined there seems to be a 5-6x
performance
> hit compared to running Prawn directly as a script from the command
> line (~12sec to 60+sec) on my OS X system.
>
> Anyone have any suggestions on how to improve performance?
Hi Aaron,

1. How are you benchmarking this?
2. Are you using thread pool?
3. If you do not need large thread pool, you can reduce size using
"thread_pool 1", which will give some boost in performance because
having 20 threads active inside interpretor is a performance penalty.

On Mon, Oct 27, 2008 at 8:04 PM, hemant <gethemant at gmail.com>
wrote:
> On Mon, Oct 27, 2008 at 10:49 AM, Aaron Turner <synfinatic at
gmail.com> wrote:
>> I have a RoR application which I''m using Prawn via
BackgrounDRb to
>> generate PDF''s.  I''ve determined there seems to be a
5-6x performance
>> hit compared to running Prawn directly as a script from the command
>> line (~12sec to 60+sec) on my OS X system.
>>
>> Anyone have any suggestions on how to improve performance?
>
> Hi Aaron,
>
> 1. How are you benchmarking this?
> 2. Are you using thread pool?
> 3. If you do not need large thread pool, you can reduce size using
> "thread_pool 1", which will give some boost in performance
because
> having 20 threads active inside interpretor is a performance penalty.
>
I tried the thread_pool thing, it helped a little, but backgroundrb is
still multiple times slower then running the same code from a ruby
script.  Anyways, I created a standalone rails app which showcases
this problem:

http://synfin.net/code/prawn_demo.tar.gz

I put some info in the README, but if you have any further questions
let me know.

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix &
Windows
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.  -- Benjamin Franklin

On Wed, Oct 29, 2008 at 3:03 AM, Aaron Turner <synfinatic at gmail.com>
wrote:
> On Mon, Oct 27, 2008 at 8:04 PM, hemant <gethemant at gmail.com>
wrote:
>> On Mon, Oct 27, 2008 at 10:49 AM, Aaron Turner <synfinatic at
gmail.com> wrote:
>>> I have a RoR application which I''m using Prawn via
BackgrounDRb to
>>> generate PDF''s.  I''ve determined there seems to
be a 5-6x performance
>>> hit compared to running Prawn directly as a script from the command
>>> line (~12sec to 60+sec) on my OS X system.
>>>
>>> Anyone have any suggestions on how to improve performance?
>>
>> Hi Aaron,
>>
>> 1. How are you benchmarking this?
>> 2. Are you using thread pool?
>> 3. If you do not need large thread pool, you can reduce size using
>> "thread_pool 1", which will give some boost in performance
because
>> having 20 threads active inside interpretor is a performance penalty.
>>
>
> I tried the thread_pool thing, it helped a little, but backgroundrb is
> still multiple times slower then running the same code from a ruby
> script.  Anyways, I created a standalone rails app which showcases
> this problem:
>
> http://synfin.net/code/prawn_demo.tar.gz
>
Hey there,

two things:

1. I see you are passing a lot of data around (the whole YAML dump).
Can you not just pass some id or some other key, which can be used to
read the data in worker and populate the object? That will speed up
things by fair margin.

2. That update() hack is ugly. You got to use cache object of bdrb,
make sure you are using memcache backend there.

http://backgroundrb.rubyforge.org/workers/#result_caching

Thats for now. I will get back, once I get to see things more closely.

On Tue, Oct 28, 2008 at 10:39 PM, hemant <gethemant at gmail.com>
wrote:
> On Wed, Oct 29, 2008 at 3:03 AM, Aaron Turner <synfinatic at
gmail.com> wrote:
>> On Mon, Oct 27, 2008 at 8:04 PM, hemant <gethemant at gmail.com>
wrote:
>>> On Mon, Oct 27, 2008 at 10:49 AM, Aaron Turner <synfinatic at
gmail.com> wrote:
>>>> I have a RoR application which I''m using Prawn via
BackgrounDRb to
>>>> generate PDF''s.  I''ve determined there seems
to be a 5-6x performance
>>>> hit compared to running Prawn directly as a script from the
command
>>>> line (~12sec to 60+sec) on my OS X system.
>>>>
>>>> Anyone have any suggestions on how to improve performance?
>>>
>>> Hi Aaron,
>>>
>>> 1. How are you benchmarking this?
>>> 2. Are you using thread pool?
>>> 3. If you do not need large thread pool, you can reduce size using
>>> "thread_pool 1", which will give some boost in
performance because
>>> having 20 threads active inside interpretor is a performance
penalty.
>>>
>>
>> I tried the thread_pool thing, it helped a little, but backgroundrb is
>> still multiple times slower then running the same code from a ruby
>> script.  Anyways, I created a standalone rails app which showcases
>> this problem:
>>
>> http://synfin.net/code/prawn_demo.tar.gz
>>
>
> Hey there,
>
> two things:
>
> 1. I see you are passing a lot of data around (the whole YAML dump).
> Can you not just pass some id or some other key, which can be used to
> read the data in worker and populate the object? That will speed up
> things by fair margin.
That could be done, but my testing shows that the startup cost of
passing all the data from RoR to backgroundrb is insignificant
compared to the time spent in the while loop processing it in Prawn.
> 2. That update() hack is ugly. You got to use cache object of bdrb,
> make sure you are using memcache backend there.
>
> http://backgroundrb.rubyforge.org/workers/#result_caching
That''s a good suggestion.  I''ll have to start using memcache.

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix &
Windows
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.  -- Benjamin Franklin

On Wed, Oct 29, 2008 at 11:46 AM, Aaron Turner <synfinatic at gmail.com>
wrote:
> I have a RoR application which I''m using Prawn via BackgrounDRb to
> generate PDF''s.  I''ve determined there seems to be a 5-6x
performance
> hit compared to running Prawn directly as a script from the command
> line (~12sec to 60+sec) on my OS X system.

Okay, I profiled your worker code and results can be found here:

http://backgroundrb.gnufied.org/worker_result.html

What this indicates is, worker is spending quite a bit of time inside,
ActiveSupport::Dependencies#new_constants_in, what this means is,
ActiveSupport really screws constant loading in a big way and slows it
down.

I do not have a solution since this is hardly bdrb problem.