All, I need some ideas. I''ve been working with Jeff''s EventDispatcher framework (see experiments/machine in SVN), and it feels like the right way to proceed. So far there are two EventDispatchers (one for Timers and one for I/O objects) and it''s been pretty easy to subclass them to do extra needed things (like nonblocking TCP connects and accepts). We''re going to need a handful of new EventDispatchers (like processes and signals, threadpools, GUI events, and a message router). But what I''m thinking about now is Protocol handlers. This applies particularly in the case of I/O EventDispatchers like socket connections, files, StringIOs and UDP handlers. These objects now all fire :read events when they get receive data. But I want to define a full stack of protocol handlers something like Twisted has, above the basic :read event. For example, a line-oriented handler that handles the :read event and fires :read_line events when it detects a line feed in the data. And in turn, you could write an RFC-822 header handler on top of that, which would fire :read_header and :read_data events. Above that, you''d have a state-machine protocol such as the one James Edward Gray is thinking about. And above that, SMTP and HTTP handlers. Somewhere in the middle of the stack would be SSL/TLS. You get the idea. I''m looking for the right API for creating these nested handlers. They can probably be implemented as Module mixins (and of course they can mix in each other) but they will need internal state so they might be better done as Delegates. I''m hoping you guys can come up with some suggestions. We clearly need to be able to add protocol handlers both to objects and to classes. We might need to define a class method and an instance method in EventDispatcher in case include and extend aren''t good enough. Something like this has to happen: class ServerClass < EventableIO push_protocol_handler HttpsHandler end and then all instances of this class would automatically generate something like an :http_request event, that you could add a custom handler for. Does my question make any sense? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060526/71301498/attachment.htm
On May 26, 2006, at 7:25 AM, Francis Cianfrocca wrote:> Above that, you''d have a state-machine protocol such as the one > James Edward Gray is thinking about.I really will get this in eventually. This week has been beyond crazy at work. I''m building a proof-of-concept to show everyone. I would say it''s about half done. Just FYI, it''s currently designed to work with the bare bones EventMachine. James Edward Gray II
Francis Cianfrocca wrote:> All, > you may have noticed that Ruby 1.8.4 now has proper nonblocking I/O functions.Awesome. This is great. So does this mean pure-ruby can actually mean pure-ruby now? (Although it might not hurt to have compatibility bindings around for older ruby installs...)> We''re going to need a handful of new EventDispatchers (like processes > and signals, threadpools, GUI events, and a message router).Signals should be in soon...> But what I''m thinking about now is Protocol handlers. This applies > particularly in the case of I/O EventDispatchers like socket > connections, files, StringIOs and UDP handlers. These objects now all > fire :read events when they get receive data. But I want to define a > full stack of protocol handlers something like Twisted has, above the > basic :read event. For example, a line-oriented handler that handles the > :read event and fires :read_line events when it detects a line feed in > the data. And in turn, you could write an RFC-822 header handler on top > of that, which would fire :read_header and :read_data events. Above > that, you''d have a state-machine protocol such as the one James Edward > Gray is thinking about. And above that, SMTP and HTTP handlers. > Somewhere in the middle of the stack would be SSL/TLS. You get the idea.Building up some protocols like you describe is probably going to be the real test of our latest event model. Maybe we should try a couple variations on the way to compose protocols to see what we like? It seemed like experimenting with the event model helped a lot to understand it better, maybe we should continue with the same methodology? These are the core components of the whole system so it''s probably worth it.> I''m looking for the right API for creating these nested handlers. They > can probably be implemented as Module mixins (and of course they can mix > in each other) but they will need internal state so they might be better > done as Delegates. I''m hoping you guys can come up with some suggestions.I''ve got some meta programming stuff laying around that I used before, which makes it easy to do things like your push_protocol_handler example. For starters though, I think it might be nice to try a simple version with regular classes that hold a reference to lower level classes. Each layer will be an EventDispatcher. At the app level you instantiate an SMTP protocol, for example, and it will in turn instantiate the RFC-822 layer, and on down the line. Each will subscribe to the lower levels events that it cares about. This is how GUI toolkits are built up and it seems to work pretty well. If we can get the guts of a couple of these layers into the repository so we have some example layers to experiment with that would be great. Do you already have something written? -Jeff
Thanks James, looking forward to it. I expect the current, extension-based eventmachine will continue to be supported. And I also expect that we''ll implement its existing API as a sugaring on top of the pure-ruby version, so existing code will continue to work. On 5/26/06, James Edward Gray II <james at grayproductions.net> wrote:> > On May 26, 2006, at 7:25 AM, Francis Cianfrocca wrote: > > > Above that, you''d have a state-machine protocol such as the one > > James Edward Gray is thinking about. > > I really will get this in eventually. This week has been beyond > crazy at work. > > I''m building a proof-of-concept to show everyone. I would say it''s > about half done. > > Just FYI, it''s currently designed to work with the bare bones > EventMachine. > > James Edward Gray II > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060526/50d70bb2/attachment.htm
It really does look like we can get pure Ruby now. Hurray! We most certainly will need some error handlers around to work with earlier Ruby builds, and possibly unsupport some features. Looking forward to the signals EventDispatcher. Will be fun to watch you make it work on Windows ;-). I started working on a message-router dispatcher because I need one for a current project. I suppose we might think of it as a more-interoperable, much faster version of DRb. I like the incremental methodology we''re using to propose and refine the API! I''d like to see what you can come up with for nested handlers. It''s basically a parallel stack to the hierarchy of EventDispatchers. Any kind of implementation strategy is ok with me as long as it is extensible and is graceful and intuitive for programmers to use. I''ve got a lot of protocol guts around to use, but it would be really simple to start with a LineProtocol, and maybe layer a MIME-header handler above that. So, one layer would look at each data event and fire :read_line events (obviously remembering partial lines between event calls), and the next layer would handle :read_line and parse the lines into header:content, and fire :read_header. That ought to be enough to test your API ideas. If you want me to write you the guts of those, let me know. They won''t be big! On 5/26/06, Jeff Rose <rosejn at gmail.com> wrote:> > Francis Cianfrocca wrote: > > All, > > you may have noticed that Ruby 1.8.4 now has proper nonblocking I/O > functions. > > Awesome. This is great. So does this mean pure-ruby can actually mean > pure-ruby now? (Although it might not hurt to have compatibility > bindings around for older ruby installs...) > > > We''re going to need a handful of new EventDispatchers (like processes > > and signals, threadpools, GUI events, and a message router). > > Signals should be in soon... > > > But what I''m thinking about now is Protocol handlers. This applies > > particularly in the case of I/O EventDispatchers like socket > > connections, files, StringIOs and UDP handlers. These objects now all > > fire :read events when they get receive data. But I want to define a > > full stack of protocol handlers something like Twisted has, above the > > basic :read event. For example, a line-oriented handler that handles the > > :read event and fires :read_line events when it detects a line feed in > > the data. And in turn, you could write an RFC-822 header handler on top > > of that, which would fire :read_header and :read_data events. Above > > that, you''d have a state-machine protocol such as the one James Edward > > Gray is thinking about. And above that, SMTP and HTTP handlers. > > Somewhere in the middle of the stack would be SSL/TLS. You get the idea. > > Building up some protocols like you describe is probably going to be the > real test of our latest event model. Maybe we should try a couple > variations on the way to compose protocols to see what we like? It > seemed like experimenting with the event model helped a lot to > understand it better, maybe we should continue with the same > methodology? These are the core components of the whole system so it''s > probably worth it. > > > I''m looking for the right API for creating these nested handlers. They > > can probably be implemented as Module mixins (and of course they can mix > > in each other) but they will need internal state so they might be better > > done as Delegates. I''m hoping you guys can come up with some > suggestions. > > I''ve got some meta programming stuff laying around that I used before, > which makes it easy to do things like your push_protocol_handler > example. For starters though, I think it might be nice to try a simple > version with regular classes that hold a reference to lower level > classes. Each layer will be an EventDispatcher. At the app level you > instantiate an SMTP protocol, for example, and it will in turn > instantiate the RFC-822 layer, and on down the line. Each will > subscribe to the lower levels events that it cares about. This is how > GUI toolkits are built up and it seems to work pretty well. > > If we can get the guts of a couple of these layers into the repository > so we have some example layers to experiment with that would be great. > Do you already have something written? > > -Jeff > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060526/1f6e4f5b/attachment-0001.htm
> Looking forward to the signals EventDispatcher. Will be fun to watch you > make it work on Windows ;-). I started working on a message-router > dispatcher because I need one for a current project. I suppose we might > think of it as a more-interoperable, much faster version of DRb.The Machine::Signaller is checked in. I didn''t want to call it just Signal because that is the name of ruby''s built-in class for handling signals... Not sure what I think about Signaller, but it works for now. Ruby basically gives you the ability to handle whatever signals are available on the current platform. What I do is build the list of supported signals on class definition, but then the user has to use a signal that is currently supported. Also, I found my bug in the Timeout code where we were missing one event from a periodic timer. I was incorrectly registering the handler which is supposed to do the periodic setup... It works as expected now.> I like the incremental methodology we''re using to propose and refine the > API! I''d like to see what you can come up with for nested handlers. It''s > basically a parallel stack to the hierarchy of EventDispatchers. Any > kind of implementation strategy is ok with me as long as it is > extensible and is graceful and intuitive for programmers to use. I''ve > got a lot of protocol guts around to use, but it would be really simple > to start with a LineProtocol, and maybe layer a MIME-header handler > above that.Alright, then over the weekend I''ll try to put together a couple variations so we can see what we like.> So, one layer would look at each data event and fire :read_line events > (obviously remembering partial lines between event calls), and the next > layer would handle :read_line and parse the lines into header:content, > and fire :read_header. That ought to be enough to test your API ideas. > If you want me to write you the guts of those, let me know. They won''t > be big!Sounds like a pretty straight forward stack that will be nice for experimenting. What differentiates a header from content? If you already have code laying around that you can check-in somewhere that would be great. Also, how is the test for eio supposed to work? -Jeff
How about Machine::SignalHandler? or SignalDispatcher? what does it do on Windows? Differentiating header from content will take some more work, depending on the protocol. SMTP, it''s all just lines, and the end is distinguished by a . on a line by itself. In SIP it goes by Content-length. In HTTP it''s a big mess. I''ll cobble together some stuff for you. I have to come up with real set of tests for EIO. What''s there now is just my dev-test garbage. Also, what do you think about SOAP? Wouldn''t it be interesting to have a complete stack for handling SOAP messages? As far as I know, there isn''t a "standard" Ruby method for running SOAP apps except for Webrick. That would be an important hole for us to fill. On 5/26/06, Jeff Rose <rosejn at gmail.com> wrote:> > > Looking forward to the signals EventDispatcher. Will be fun to watch you > > make it work on Windows ;-). I started working on a message-router > > dispatcher because I need one for a current project. I suppose we might > > think of it as a more-interoperable, much faster version of DRb. > > The Machine::Signaller is checked in. I didn''t want to call it just > Signal because that is the name of ruby''s built-in class for handling > signals... Not sure what I think about Signaller, but it works for now. > Ruby basically gives you the ability to handle whatever signals are > available on the current platform. What I do is build the list of > supported signals on class definition, but then the user has to use a > signal that is currently supported. > > Also, I found my bug in the Timeout code where we were missing one event > from a periodic timer. I was incorrectly registering the handler which > is supposed to do the periodic setup... It works as expected now. > > > I like the incremental methodology we''re using to propose and refine the > > API! I''d like to see what you can come up with for nested handlers. It''s > > basically a parallel stack to the hierarchy of EventDispatchers. Any > > kind of implementation strategy is ok with me as long as it is > > extensible and is graceful and intuitive for programmers to use. I''ve > > got a lot of protocol guts around to use, but it would be really simple > > to start with a LineProtocol, and maybe layer a MIME-header handler > > above that. > > Alright, then over the weekend I''ll try to put together a couple > variations so we can see what we like. > > > So, one layer would look at each data event and fire :read_line events > > (obviously remembering partial lines between event calls), and the next > > layer would handle :read_line and parse the lines into header:content, > > and fire :read_header. That ought to be enough to test your API ideas. > > If you want me to write you the guts of those, let me know. They won''t > > be big! > > Sounds like a pretty straight forward stack that will be nice for > experimenting. What differentiates a header from content? If you > already have code laying around that you can check-in somewhere that > would be great. > > Also, how is the test for eio supposed to work? > > -Jeff > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060526/2ffa1239/attachment.htm