my current wish list for EM would be timers per connection--i.e. you can ''cancel all timers associated with a particular connection'' all in one fell swoop. It makes sense for me--say a connection breaks up half way through, you want to close everything up cleanly. The others I mentioned previously :) Betters docs slightly (I''m working on it, in svn, and hope to integrate the changes soon). Thanks! -Roger
My wish for the wishing star... a function that is called when EM successfully writes data to a socket. "on_send_queue_length_decrease" or something like that. This would make queue monitoring consistent and easy. def on_send_queue_length_decrease if self.get_outbound_data_size == 0 # write some more stuff to it end end :) It seems to be the only ''missing event'' in EM, at least for now. Might be useful sometime. Thanks for reading! -Roger
latest wish: access to the intializer for EM connections. Or is this possible already? Thanks all! Merry Christmas. -Roger Go Mitt.
On Nov 26, 2007 1:03 PM, Roger Pack <rogerpack2005 at gmail.com> wrote:> My wish for the wishing star... > a function that is called when EM successfully writes data to a socket. > > "on_send_queue_length_decrease" or something like that. > > This would make queue monitoring consistent and easy. > > def on_send_queue_length_decrease > if self.get_outbound_data_size == 0 > # write some more stuff to it > end > endThe problem with something like this is that method dispatch isn''t terribly fast in Ruby (it''s terribly slow, in fact). Something like this would be doing an awful lot of method calls when there is a lot of IO occuring, and the vast majority of the time nobody is going to care what the outbound queue size is. If you do care, I think there are already ways to deal with this. Take a look at the streamer.rb library in the distribution, at the stream_one_chunk() method: if @connection.get_outbound_data_size > BackpressureLevel EventMachine::next_tick {stream_one_chunk} break end i.e. for those conditions where we don''t want to throw too much data into the outbound queue (so that the other queues aren''t starved), bail out of the method and have it revisited on the next reactor iterator. It achieves the desired effect, and doesn''t introduce extra method invocations which are useless for most protocols. Kirk Haines P.S. Francis, I have a patch for you for streamer.rb -- a change to stream_one_chunk() so that it will honor the @http_chunk flag to stream without chunking if @http_chunk is false.
> The problem with something like this is that method dispatch isn''t > terribly fast in Ruby (it''s terribly slow, in fact). Something like > this would be doing an awful lot of method calls when there is a lot > of IO occuring, and the vast majority of the time nobody is going to > care what the outbound queue size is. > > If you do care, I think there are already ways to deal with this. > > Take a look at the streamer.rb library in the distribution, at the > stream_one_chunk() method: > > if @connection.get_outbound_data_size > BackpressureLevel > EventMachine::next_tick {stream_one_chunk} > break > endI agree with both of the things you said--most people don''t need it, and it''s slow. At least two people thus far have, and anybody that wants to stream large amounts of data will want to. The method you suggest works, as does setting an often firing timer to accomplish the same. Both with drawbacks. Were you able to overcome its propensity for using 100% cpu? Kind regards, -Roger
On Dec 16, 2007 12:16 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> I agree with both of the things you said--most people don''t need it, > and it''s slow. At least two people thus far have, and anybody that > wants to stream large amounts of data will want to. The method you > suggest works, as does setting an often firing timer to accomplish the > same. Both with drawbacks. Were you able to overcome its propensity > for using 100% cpu?Swiftiply moves, depending on the server I test on, between 120 and 200+ megabytes per second using that technique. Transfer rate: 203675.98 [Kbytes/sec] received The CPU usage profile when it is moving that much data looks something like this (as reported by top): Cpu1 : 62.4% us, 28.7% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 8.9% si It would be interesting to insert an event like that which you describe, and then benchmark it to see how it compares. Kirk Haines Kirk Haines
With EventMachine 0.10.0 you can pass parameters to the connection''s initialize method by appending them to the start_server or connect calls, i.e.: EM.start_server(addr, port, MyConnection, arg1, arg2, ..., argN) On Dec 15, 2007 11:21 PM, Roger Pack <rogerpack2005 at gmail.com> wrote:> latest wish: access to the intializer for EM connections. Or is this > possible already? > Thanks all! > Merry Christmas. > -Roger > Go Mitt. > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071216/55ee9e80/attachment.html
On Dec 16, 2007 2:57 PM, Tony Arcieri <tony at clickcaster.com> wrote:> With EventMachine 0.10.0 you can pass parameters to the connection''s > initialize method by appending them to the start_server or connect calls, > i.e.: > > EM.start_server(addr, port, MyConnection, arg1, arg2, ..., argN) > > >And of course we need to profile the latest code additions to see what accounts for the nearly 20% performance drop that Kirk discovered. Another thing that changed in 0.10 was that I added an error-handling hook. Maybe it''s the culprit? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071217/ddf64f9d/attachment.html
Maybe the extra arguments for initialize on accept? GL! Maybe ruby-prof would help? -Roger On Dec 17, 2007 9:36 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> On Dec 16, 2007 2:57 PM, Tony Arcieri <tony at clickcaster.com> wrote: > > > With EventMachine 0.10.0 you can pass parameters to the connection''s > initialize method by appending them to the start_server or connect calls, > i.e.: > > > > EM.start_server(addr, port, MyConnection, arg1, arg2, ..., argN) > > > > > > > > > > And of course we need to profile the latest code additions to see what > accounts for the nearly 20% performance drop that Kirk discovered. Another > thing that changed in 0.10 was that I added an error-handling hook. Maybe > it''s the culprit?-- -Roger Pack For God hath not given us the spirit of fear; but of power, and of love, and of a sound mind" -- 2 Timothy 1:7
another addition to the wish list: the method ''reason_for_last_failure'' -- was it a rejected connection? was it a connection closed? Were there not enough file descriptors available? that type of thing Also would the ability to arbitrarily add socket options to EM''s sockets be useful? I don''t think I''d probably ever use it, but just wondering. -R
Portability suggests that the better thing to do here is just to have an escape hatch that exposes the raw handle, so you can do whatever with it. On 6/17/08, Roger Pack <rogerpack2005 at gmail.com> wrote:> another addition to the wish list: > > the method ''reason_for_last_failure'' -- was it a rejected connection? > was it a connection closed? Were there not enough file descriptors > available? that type of thing > > Also would the ability to arbitrarily add socket options to EM''s > sockets be useful? I don''t think I''d probably ever use it, but just > wondering. > > -R > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- --- Thomas H. Ptacek // matasano security read us on the web: http://www.matasano.com/log
Yet another addition to the wish list: file handle I/O, such that you can write to a file and ''forget about when the actual writing happens'' :) I''m not sure if this is even possible with win32, but hey, we can wish, right? [and I''m on OS X, anyway]. -R
Yet another addition to the old wish list would be: file handle I/O, such that you can write to an [evented] file and ''forget about when the actual writing happens'' :) I''m not sure if this is even possible with win32, but hey, we can wish, right? [and I''m on OS X, anyway]. I''m not even sure if I would actually help any programs, but hey, maybe somebody somewhere :) I guess if you want to read from a file and ''forget about when the reading occurs'' you could use fibers, or just pass it blocks. Cheers. -R
On Wed, Nov 14, 2007 at 9:51 AM, Roger Pack <rogerpack2005 at gmail.com> wrote:> my current wish list for EM would beAdd one to the wish list: ability to ''catch'' errors thrown during an EM::run [before it calls release_machine] a la def self.run begin run rescue Exception handle_em_error # user defined ensure release_machine end end Would be happy to submit patch. -R
Are you talking about the problem where any exception generated inside an event loop gets swallowed if anything bad happens in release_machine? Because I have the same problem, and had just assumed updating to HEAD would fix it. On 6/28/08, Roger Pack <rogerpack2005 at gmail.com> wrote:> On Wed, Nov 14, 2007 at 9:51 AM, Roger Pack <rogerpack2005 at gmail.com> wrote: > > my current wish list for EM would be > > > Add one to the wish list: > > ability to ''catch'' errors thrown during an EM::run [before it calls > release_machine] > > a la > > def self.run > begin > run > rescue Exception > handle_em_error # user defined > ensure > release_machine > end > end > > Would be happy to submit patch. > > -R > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- --- Thomas H. Ptacek // matasano security read us on the web: http://www.matasano.com/log
On Jun 28, 2008, at 7:45 PM, Thomas Ptacek wrote:> Are you talking about the problem where any exception generated inside > an event loop gets swallowed if anything bad happens in > release_machine? Because I have the same problem, and had just assumed > updating to HEAD would fix it.With that one I believe you can just put a begin..rescue block around EM::run and catch them. Or am I wrong? The ones I am referring to are that if you have anything bad happen within the event loop, it will call release_machine before propagating the error message out to your handlers. Then release_machine can throw ''connectionUnbound'' errors which mask the original exception. run is currently def self.run begin run_machine ensure release_machine # so any errors generated here will mask exception raised within the event loop, which errors can easily occur if there was an exception raised, since it might have prohibited a connection from being bound end end -R> > > On 6/28/08, Roger Pack <rogerpack2005 at gmail.com> wrote: >> On Wed, Nov 14, 2007 at 9:51 AM, Roger Pack >> <rogerpack2005 at gmail.com> wrote: >>> my current wish list for EM would be >> >> >> Add one to the wish list: >> >> ability to ''catch'' errors thrown during an EM::run [before it calls >> release_machine] >> >> a la >> >> def self.run >> begin >> run >> rescue Exception >> handle_em_error # user defined >> ensure >> release_machine >> end >> end >> >> Would be happy to submit patch. >> >> -R >> >> _______________________________________________ >> Eventmachine-talk mailing list >> Eventmachine-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/eventmachine-talk >> > > > -- > --- > Thomas H. Ptacek // matasano security > read us on the web: http://www.matasano.com/log > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk