Jerrod Blavos
2006-Oct-27 14:02 UTC
[Backgroundrb-devel] Error after moving to production server
so i have an after_create method on my track model that kicks off a background process that converts an mp3 to a swf. this is working fine locally on my machine, but upon moving it to my production server, i am getting the following error. I''ve only been using backgroundRB for like 48 hours, so i''m not really sure what this means or where to go. any ideas? DRb::DRbConnError (too large packet 1008813135): /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' /app/models/track.rb:55:in `save_swf'' /app/models/track.rb:40:in `process''
* Jerrod Blavos (jerrod at indierockmedia.com) [061027 11:05]:> so i have an after_create method on my track model that kicks off a > background process that converts an mp3 to a swf. > this is working fine locally on my machine, but upon moving it to my > production server, i am getting the following error. I''ve only been > using backgroundRB for like 48 hours, so i''m not really sure what > this means or where to go. any ideas?the stack strace tells that you are attempting to send a 960MB or so payload, and load_limit is 25MB by default. Are you sure that you are sending what you intend to send? /skaar> > DRb::DRbConnError (too large packet 1008813135): > /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' > /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' > /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' > /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' > /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' > /app/models/track.rb:55:in `save_swf'' > /app/models/track.rb:40:in `process'' > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel-- ---------------------------------------------------------------------- |\|\ where in the | s_u_b_s_t_r_u_c_t_i_o_n | | >=========== W.A.S.T.E. | genarratologies |/|/ (_) is the wisdom | skaar at waste.org ----------------------------------------------------------------------
Erik Morton
2006-Oct-27 16:17 UTC
[Backgroundrb-devel] Error after moving to production server
There must be a limit on the size of the packet that DRB will accept. Can you have your worker read the file from a database or the file system, rather than over the DRB socket? Erik On Oct 27, 2006, at 10:02 AM, Jerrod Blavos wrote:> so i have an after_create method on my track model that kicks off a > background process that converts an mp3 to a swf. > this is working fine locally on my machine, but upon moving it to my > production server, i am getting the following error. I''ve only been > using backgroundRB for like 48 hours, so i''m not really sure what > this means or where to go. any ideas? > > DRb::DRbConnError (too large packet 1008813135): > /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' > /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' > /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' > /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' > /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' > /app/models/track.rb:55:in `save_swf'' > /app/models/track.rb:40:in `process'' > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
Ezra Zygmuntowicz
2006-Oct-27 16:24 UTC
[Backgroundrb-devel] Error after moving to production server
On Oct 27, 2006, at 7:02 AM, Jerrod Blavos wrote:> so i have an after_create method on my track model that kicks off a > background process that converts an mp3 to a swf. > this is working fine locally on my machine, but upon moving it to my > production server, i am getting the following error. I''ve only been > using backgroundRB for like 48 hours, so i''m not really sure what > this means or where to go. any ideas? > > DRb::DRbConnError (too large packet 1008813135): > /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' > /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' > /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' > /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' > /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' > /app/models/track.rb:55:in `save_swf'' > /app/models/track.rb:40:in `process''Hey Jerrod- What are you sending across the wire? This error means you are sending too much data for the drb server to handle with its current settings. Are you sending something really big as :args to your workers ? -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273)
Charles Brian Quinn
2006-Oct-27 16:25 UTC
[Backgroundrb-devel] Error after moving to production server
Are you passing the file? Better to pass an id or name of the file and let backgrounDRb load it up. If not, disregard this message, as it appears the actual DRB server got something too big sent to it.... On 10/27/06, Jerrod Blavos <jerrod at indierockmedia.com> wrote:> so i have an after_create method on my track model that kicks off a > background process that converts an mp3 to a swf. > this is working fine locally on my machine, but upon moving it to my > production server, i am getting the following error. I''ve only been > using backgroundRB for like 48 hours, so i''m not really sure what > this means or where to go. any ideas? > > DRb::DRbConnError (too large packet 1008813135): > /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' > /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' > /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' > /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' > /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' > /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' > /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' > /app/models/track.rb:55:in `save_swf'' > /app/models/track.rb:40:in `process'' > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel >-- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com
Jerrod Blavos
2006-Oct-27 18:51 UTC
[Backgroundrb-devel] Error after moving to production server
interesting. Ok here is what is happening. im asking backgrounDrb to call an ffmpeg command. the command is to convert a 1.9mb mp3 into what end up being a .9mb swf file. sounds like perhaps the ffmpeg command is not right? here is the method on the model thatgets called after_create def save_swf command_ffmpeg = "ffmpeg -i ''#{path}'' -ab 96 -f swf ''# {swf_path}'' 2>&1" self.job_key = MiddleMan.new_worker :class=>:audio_worker, :args=>command_ffmpeg end and here is the worker method >>>>>>>>> def do_work(args) @progress = nil @results = nil IO.popen(args) do |pipe| pipe.each do |line| if line =~ /Overwrite ?/ @results = "Problem Encountered. File Already exists." end if line =~ /Duration: (\d\d):(\d\d):(\d\d).(\d)/ @duration = (($1.to_i * 60 + $2.to_i) * 60 + $3.to_i) * 10 + $4.to_i end time_exp = /time=(\d+).(\d+)/ match = time_exp.match(line) if !match == nil? p = ($1.to_i * 10 + $2.to_i) * 100 / @duration p = 100 if p > 98 if progress != p @progress = p if p == 100 @results = "SWF Successfully Created!" end end end end end end >>>>> On Oct 27, 2006, at 12:15 PM, skaar wrote:> * Jerrod Blavos (jerrod at indierockmedia.com) [061027 11:05]: >> so i have an after_create method on my track model that kicks off a >> background process that converts an mp3 to a swf. >> this is working fine locally on my machine, but upon moving it to my >> production server, i am getting the following error. I''ve only been >> using backgroundRB for like 48 hours, so i''m not really sure what >> this means or where to go. any ideas? > > the stack strace tells that you are attempting to send a 960MB or so > payload, and load_limit is 25MB by default. Are you sure that you are > sending what you intend to send? > > /skaar > >> >> DRb::DRbConnError (too large packet 1008813135): >> /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' >> /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' >> /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' >> /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' >> /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' >> /app/models/track.rb:55:in `save_swf'' >> /app/models/track.rb:40:in `process'' >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- > ---------------------------------------------------------------------- > |\|\ where in the | s_u_b_s_t_r_u_c_t_i_o_n > | | >=========== W.A.S.T.E. | genarratologies > |/|/ (_) is the wisdom | skaar at waste.org > ----------------------------------------------------------------------
Jerrod Blavos
2006-Oct-27 18:53 UTC
[Backgroundrb-devel] Error after moving to production server
im pretty sure it should be reading from the file system. the worker should just be calling the ffmpeg command using IO.popen (args) do |pipe| On Oct 27, 2006, at 12:17 PM, Erik Morton wrote:> There must be a limit on the size of the packet that DRB will > accept. Can you have your worker read the file from a database or > the file system, rather than over the DRB socket? > > Erik > On Oct 27, 2006, at 10:02 AM, Jerrod Blavos wrote: > >> so i have an after_create method on my track model that kicks off a >> background process that converts an mp3 to a swf. >> this is working fine locally on my machine, but upon moving it to my >> production server, i am getting the following error. I''ve only been >> using backgroundRB for like 48 hours, so i''m not really sure what >> this means or where to go. any ideas? >> >> DRb::DRbConnError (too large packet 1008813135): >> /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' >> /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' >> /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' >> /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' >> /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' >> /app/models/track.rb:55:in `save_swf'' >> /app/models/track.rb:40:in `process'' >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >
Jerrod Blavos
2006-Oct-27 18:55 UTC
[Backgroundrb-devel] Error after moving to production server
all im sending through the :args is the text ffmpeg command that im calling through IO.popen(args) hmm. perhaps my command isnt right. On Oct 27, 2006, at 12:24 PM, Ezra Zygmuntowicz wrote:> > On Oct 27, 2006, at 7:02 AM, Jerrod Blavos wrote: > >> so i have an after_create method on my track model that kicks off a >> background process that converts an mp3 to a swf. >> this is working fine locally on my machine, but upon moving it to my >> production server, i am getting the following error. I''ve only been >> using backgroundRB for like 48 hours, so i''m not really sure what >> this means or where to go. any ideas? >> >> DRb::DRbConnError (too large packet 1008813135): >> /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' >> /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' >> /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' >> /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' >> /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' >> /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' >> /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' >> /app/models/track.rb:55:in `save_swf'' >> /app/models/track.rb:40:in `process'' > > > Hey Jerrod- > > > What are you sending across the wire? This error means you are > sending too much data for the drb server to handle with its current > settings. Are you sending something really big as :args to your > workers ? > > > > -- Ezra Zygmuntowicz-- Lead Rails Evangelist > -- ez at engineyard.com > -- Engine Yard, Serious Rails Hosting > -- (866) 518-YARD (9273) > >
Jerrod Blavos
2006-Oct-27 19:39 UTC
[Backgroundrb-devel] Error after moving to production server
Ok that seemed to help. that and i hadnt added the job_key field to the db. oops. either way it is now working beautifully. what will the update/rewrite change? -J On Oct 27, 2006, at 3:13 PM, skaar wrote:> just to check, can you make sure that do_work(args), doesn''t return > the > data, can you in the worker do > > def do_work(args) > ... your code ... > nil > end > > We''re very close to a new release (total re-write) of backgroundrb, > which might be a lot better for you. Stay tuned :) > > /skaar > > * Jerrod Blavos (jerrod at indierockmedia.com) [061027 14:08]: >> interesting. Ok here is what is happening. im asking backgrounDrb to >> call an ffmpeg command. the command is to convert a 1.9mb mp3 into >> what end up being a .9mb swf file. sounds like perhaps the ffmpeg >> command is not right? >> >> here is the method on the model thatgets called after_create >> >> def save_swf >> command_ffmpeg = "ffmpeg -i ''#{path}'' -ab 96 -f swf ''# >> {swf_path}'' 2>&1" >> self.job_key >> MiddleMan.new_worker :class=>:audio_worker, :args=>command_ffmpeg >> end >> >> and here is the worker method >> >>>>>>>>>>> >> def do_work(args) >> @progress = nil >> @results = nil >> IO.popen(args) do |pipe| >> pipe.each do |line| >> if line =~ /Overwrite ?/ >> @results = "Problem Encountered. File Already exists." >> end >> if line =~ /Duration: (\d\d):(\d\d):(\d\d).(\d)/ >> @duration = (($1.to_i * 60 + $2.to_i) * 60 + $3.to_i) * >> 10 + $4.to_i >> end >> time_exp = /time=(\d+).(\d+)/ >> match = time_exp.match(line) >> if !match == nil? >> p = ($1.to_i * 10 + $2.to_i) * 100 / @duration >> p = 100 if p > 98 >> if progress != p >> @progress = p >> if p == 100 >> @results = "SWF Successfully Created!" >> end >> end >> end >> end >> end >> end >>>>>>> >> >> >> >> >> >> On Oct 27, 2006, at 12:15 PM, skaar wrote: >> >>> * Jerrod Blavos (jerrod at indierockmedia.com) [061027 11:05]: >>>> so i have an after_create method on my track model that kicks off a >>>> background process that converts an mp3 to a swf. >>>> this is working fine locally on my machine, but upon moving it >>>> to my >>>> production server, i am getting the following error. I''ve only >>>> been >>>> using backgroundRB for like 48 hours, so i''m not really sure what >>>> this means or where to go. any ideas? >>> >>> the stack strace tells that you are attempting to send a 960MB or so >>> payload, and load_limit is 25MB by default. Are you sure that you >>> are >>> sending what you intend to send? >>> >>> /skaar >>> >>>> >>>> DRb::DRbConnError (too large packet 1008813135): >>>> /usr/lib/ruby/1.8/drb/drb.rb:570:in `load'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend'' >>>> /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing'' >>>> /app/models/track.rb:55:in `save_swf'' >>>> /app/models/track.rb:40:in `process'' >>>> _______________________________________________ >>>> Backgroundrb-devel mailing list >>>> Backgroundrb-devel at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >>> >>> -- >>> -------------------------------------------------------------------- >>> -- >>> |\|\ where in the | >>> s_u_b_s_t_r_u_c_t_i_o_n >>> | | >=========== W.A.S.T.E. | >>> genarratologies >>> |/|/ (_) is the wisdom | >>> skaar at waste.org >>> -------------------------------------------------------------------- >>> -- >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > -- > ---------------------------------------------------------------------- > |\|\ where in the | s_u_b_s_t_r_u_c_t_i_o_n > | | >=========== W.A.S.T.E. | genarratologies > |/|/ (_) is the wisdom | skaar at waste.org > ----------------------------------------------------------------------