Prateek Dayal
2010-Jan-06 05:53 UTC
[Backgroundrb-devel] Weird thread behavior while using mp3info
Hi,
I have been using Backgroundrb for processing mp3 files in background
for a long time but lately have been having a lot of problems. Here is
my setup
Rails --> Background Worker (using persistent queue) --> Defer to
thread pool and process mp3
I am using ruby-mp3info gem to open the mp3 file and extract bitrate
etc from it. This setup works fine but once in a while randomly stops
working. Once one job fails, all subsequent jobs fails until I restart
backgroundrb server. Here is the backtrace for all failed jobs
/usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:209:in
`reload''
/usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:204:in
`initialize''
/usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:327:in
`new''
/usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:327:in
`open''
/home/rails/apps/muziboo/releases/20091230043630/lib/workers/background_worker.rb:178:in
`process_mp3''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:94:in
`send''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:94:in
`run_task''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:76:in
`add_thread''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in
`initialize''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in
`new''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in
`add_thread''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in
`initialize''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in
`times''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in
`initialize''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:115:in
`new''
/home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:115:in
`worker_init''
/usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/../lib/packet/packet_worker.rb:19:in
`start_worker''
/usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:33:in
`load_worker''
/usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:26:in
`initialize''
/usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:47:in
`new''
/usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:47
/usr/bin/packet_worker_runner:19:in `load''
/usr/bin/packet_worker_runner:19
Line 209 in mp3info.rb is
# reload (or load for the first time) the file from disk
def reload
raise(Mp3InfoError, "empty file") unless File.size?(@filename) #
Line 209
@header = {}
The Mp3 File is never size 0 (this is already checked in
attachment_fu) so the exception should never be raised. I have tried
checking hte filesize in console for failing files and the size is
reported back correctly. As I said earlier, every subsequent job fails
with the same error message untill I restart backgroundrb. Once I
restart the server, these jobs all go through without a problem.
I would appreciate any help with this as I have not been able to find
much info on the net. I assumed that most of ruby''s basic library
functions (like File.size?) etc would work fine in the threaded
environment.
Regards,
--
Prateek Dayal
Co-Founder Muziboo.com
Personal Blog: http://www.prateekdayal.net/blog
Muziboo Dev Blog: http://devblog.muziboo.com
Prateek Dayal
2010-Jan-06 06:34 UTC
[Backgroundrb-devel] Weird thread behavior while using mp3info
Hi Raghavendra, I am rescuing the exception (and thats where I send the backtrace to the logger). Even with the exception rescued, all later tasks fail with the same exception Regards Prateek On Wed, Jan 6, 2010 at 12:02 PM, raghavendra gada <raghugada at gmail.com> wrote:> Hi, > > You can rescue the exception. > > Regards, > Raghu > > On Wed, Jan 6, 2010 at 11:23 AM, Prateek Dayal <prateek at muziboo.com> wrote: >> >> Hi, >> >> I have been using Backgroundrb for processing mp3 files in background >> for a long time but lately have been having a lot of problems. Here is >> my setup >> >> Rails --> Background Worker (using persistent queue) --> Defer to >> thread pool and process mp3 >> >> I am using ruby-mp3info gem to open the mp3 file and extract bitrate >> etc from it. This setup works fine but once in a while randomly stops >> working. Once one job fails, all subsequent jobs fails until I restart >> backgroundrb server. Here is the backtrace for all failed jobs >> >> /usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:209:in >> `reload'' >> /usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:204:in >> `initialize'' >> /usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:327:in >> `new'' >> /usr/lib/ruby/gems/1.8/gems/ruby-mp3info-0.6.13/lib/mp3info.rb:327:in >> `open'' >> >> /home/rails/apps/muziboo/releases/20091230043630/lib/workers/background_worker.rb:178:in >> `process_mp3'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:94:in >> `send'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:94:in >> `run_task'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:76:in >> `add_thread'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in >> `initialize'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in >> `new'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:62:in >> `add_thread'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in >> `initialize'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in >> `times'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/bdrb_thread_pool.rb:27:in >> `initialize'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:115:in >> `new'' >> >> /home/rails/apps/muziboo/releases/20091230043630/vendor/plugins/backgroundrb/server/lib/meta_worker.rb:115:in >> `worker_init'' >> >> /usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/../lib/packet/packet_worker.rb:19:in >> `start_worker'' >> /usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:33:in >> `load_worker'' >> /usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:26:in >> `initialize'' >> /usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:47:in >> `new'' >> /usr/lib/ruby/gems/1.8/gems/packet-0.1.15/bin/packet_worker_runner:47 >> /usr/bin/packet_worker_runner:19:in `load'' >> /usr/bin/packet_worker_runner:19 >> >> Line 209 in mp3info.rb is >> >> >> ?# reload (or load for the first time) the file from disk >> ?def reload >> ? ?raise(Mp3InfoError, "empty file") unless File.size?(@filename) # Line >> 209 >> >> ? ?@header = {} >> >> The Mp3 File is never size 0 (this is already checked in >> attachment_fu) so the exception should never be raised. I have tried >> checking hte filesize in console for failing files and the size is >> reported back correctly. As I said earlier, every subsequent job fails >> with the same error message untill I restart backgroundrb. ?Once I >> restart the server, these jobs all go through without a problem. >> >> I would appreciate any help with this as I have not been able to find >> much info on the net. I assumed that most of ruby''s basic library >> functions (like File.size?) etc would work fine in the threaded >> environment. >> >> >> Regards, >> >> -- >> Prateek Dayal >> Co-Founder Muziboo.com >> >> Personal Blog: ? ? ? http://www.prateekdayal.net/blog >> Muziboo Dev Blog: http://devblog.muziboo.com >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >-- Prateek Dayal Co-Founder Muziboo.com Personal Blog: http://www.prateekdayal.net/blog Muziboo Dev Blog: http://devblog.muziboo.com