I mistakenly specified CVS as the SCM for eventmachine in Rubyforge. I asked Tom Copeland to change it to SVN. If/when that gets done, then I''ll upload the source code. I also started sketching out a pure-ruby implementation. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060510/7bbd8bf2/attachment.htm
Francis Cianfrocca wrote:> I mistakenly specified CVS as the SCM for eventmachine in Rubyforge. I > asked Tom Copeland to change it to SVN. If/when that gets done, then > I''ll upload the source code. I also started sketching out a pure-ruby > implementation.I started trying to write a very minimal p2p chat app on top of EventMachine and ran into a couple things to think about. (It also made me realize how much easier this will be if we can stick to as much ruby as possible.) Accepting input from the keyboard, for example, would require a c++ implementation of a new EventableDescriptor implementation. Everything in Ruby sits on top of IO already, which is basically a beefed up version of EventableDescriptor, and if we can sit on top of that it will give us a lot for free. If the actual handling of IO objects in the interpreter (where it calls select) is what''s causing the problems, then maybe we should write our own version to replace it, or override a method somewhere so our version would get called rather than the default. It seems like if the whole project is going to be called EventMachine, then that needs to be the module encompassing everything. The actual event processor should maybe have a different name? Reactor seems to be the standard for that component... When starting on a bad IP address the server crashes with a "no acceptor" runtime exception. We are probably going to have to spend a fair amount of time making clear exceptions everywhere so errors like this are easy to track down. (It might be good to change the default example to use 127.0.0.1 also.) In reading the documentation it''s not totally clear whether absolutely everything is non-blocking or not. For example, send_data is actually not sending data right away. How do I know whether it sent successfully? That''s why they use the deferred''s in Twisted I guess... For some protocols (especially strict distributed computing algorithms) it''s really important to know exactly what went out and what didn''t. Why not put the outgoing stuff in a queue and do it on the next iteration? Memcpy''ing everything is pretty expensive, and it means you lose the knowledge of what exactly sent and what didn''t... Have you seen the IO-Reactor written a few years back on top of select? (http://www.deveiate.org/projects/IO-Reactor) It actually looks pretty clean, and could serve as a good basis for the bottom end of a ruby only version. It does the basic registering of IO objects (descriptors), and then looping over select. If timers and some kind of deferred mechanism were added it might be just what we want. -Jeff
Several points: 1) The rubyforge SCM has been changed to SVN. What''s your Rubyforge login name? 2) I totally see the point about being able to add EventableDescriptor subclasses in Ruby. Last night I started writing a pure-Ruby event machine and we''ll see where it goes. But if it doesn''t work, I can easily see a hybrid model where the base classes are in Ruby and some of the subclasses (the the socket handlers) are in C. And in another case of great-minds-think-alike, the pure-Ruby EventableDescriptor class is basically just IO. 3) Using the "reactor" name. Out of partisan pride I avoided that, but it''s probably better to use the word everyone is used to. Can you give me a code sample of how the basic loop would look with "reactor"? 4) Ah yes, error reporting, the bane of any young software project. It currently sux in EventMachine. Needs to get a whole lot better and a whole lot more consistent. Luckily you haven''t hit any of the internal asserts in the C++ code ;-) They just throw a C++ exception and you get nothing useful back on the screen. 5) send_data schedules data for transmission, and you really don''t know when it gets sent but in most protocols you don''t care too much. EventMachine currently buffers all outbound data, which is really wasteful, but works around the problem that the user doesn''t want to know about outbound EWOULDBLOCK errors. There are plenty of other subtleties here, though. However, I really like the idea of letting the user provide an error handler as Twisted does. If it matters for the user to know what was sent and what wasn''t, then you''re getting into guaranteed delivery, which is a very difficult problem because the dependencies are not all local. Can you give this some more thought and decide what you think the requirements should actually be? 6) Sounds like IO-Reactor is what I''ve already started to do in Ruby. Let me finish the first cut and then I''ll have a look and see how close I came to the other one. 7) Did you notice we''re on the Rubyforge most-active-this-week list? :-D On 5/11/06, Jeff Rose <rosejn at gmail.com> wrote:> > Francis Cianfrocca wrote: > > I mistakenly specified CVS as the SCM for eventmachine in Rubyforge. I > > asked Tom Copeland to change it to SVN. If/when that gets done, then > > I''ll upload the source code. I also started sketching out a pure-ruby > > implementation. > > I started trying to write a very minimal p2p chat app on top of > EventMachine and ran into a couple things to think about. (It also made > me realize how much easier this will be if we can stick to as much ruby > as possible.) Accepting input from the keyboard, for example, would > require a c++ implementation of a new EventableDescriptor > implementation. Everything in Ruby sits on top of IO already, which is > basically a beefed up version of EventableDescriptor, and if we can sit > on top of that it will give us a lot for free. If the actual handling > of IO objects in the interpreter (where it calls select) is what''s > causing the problems, then maybe we should write our own version to > replace it, or override a method somewhere so our version would get > called rather than the default. > > It seems like if the whole project is going to be called EventMachine, > then that needs to be the module encompassing everything. The actual > event processor should maybe have a different name? Reactor seems to be > the standard for that component... > > When starting on a bad IP address the server crashes with a "no > acceptor" runtime exception. We are probably going to have to spend a > fair amount of time making clear exceptions everywhere so errors like > this are easy to track down. (It might be good to change the default > example to use 127.0.0.1 also.) > > In reading the documentation it''s not totally clear whether absolutely > everything is non-blocking or not. For example, send_data is actually > not sending data right away. How do I know whether it sent > successfully? That''s why they use the deferred''s in Twisted I guess... > For some protocols (especially strict distributed computing > algorithms) it''s really important to know exactly what went out and what > didn''t. Why not put the outgoing stuff in a queue and do it on the next > iteration? Memcpy''ing everything is pretty expensive, and it means you > lose the knowledge of what exactly sent and what didn''t... > > Have you seen the IO-Reactor written a few years back on top of select? > (http://www.deveiate.org/projects/IO-Reactor) It actually looks pretty > clean, and could serve as a good basis for the bottom end of a ruby only > version. It does the basic registering of IO objects (descriptors), and > then looping over select. If timers and some kind of deferred mechanism > were added it might be just what we want. > > -Jeff > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20060511/3c2c644c/attachment-0001.htm
Francis Cianfrocca wrote:> Several points: > > 1) The rubyforge SCM has been changed to SVN. What''s your Rubyforge > login name?rosejn> 2) I totally see the point about being able to add EventableDescriptor > subclasses in Ruby. Last night I started writing a pure-Ruby event > machine and we''ll see where it goes. But if it doesn''t work, I can > easily see a hybrid model where the base classes are in Ruby and some of > the subclasses (the the socket handlers) are in C. And in another case > of great-minds-think-alike, the pure-Ruby EventableDescriptor class is > basically just IO.Cool. Maybe we can even just insert our own version of the socket handlers so that people can network as usual and be using our C backend rather than the default?> 3) Using the "reactor" name. Out of partisan pride I avoided that, but > it''s probably better to use the word everyone is used to. Can you give > me a code sample of how the basic loop would look with "reactor"?Haha, I know exactly the feeling. I''ve got some stuff to do this afternoon, but I''ll try to work on this afterward. Actually, Leslie Lamport, the father of Latex and Paxos, possibly the most famous algorithm in distributed computing, is speaking downstairs from me in 45 minutes :-)> 5) send_data schedules data for transmission, and you really don''t know > when it gets sent but in most protocols you don''t care too much. > EventMachine currently buffers all outbound data, which is really > wasteful, but works around the problem that the user doesn''t want to > know about outbound EWOULDBLOCK errors. There are plenty of other > subtleties here, though. However, I really like the idea of letting the > user provide an error handler as Twisted does. If it matters for the > user to know what was sent and what wasn''t, then you''re getting into > guaranteed delivery, which is a very difficult problem because the > dependencies are not all local. Can you give this some more thought and > decide what you think the requirements should actually be?When do you get EWOULDBLOCK errors? I''m not sure that we can or need to guarantee delivery, but at least if we can tell users that messages x & y went out onto the wire, but z got an error, that would be good. The errback scheme in Twisted seems pretty nice for this.> 6) Sounds like IO-Reactor is what I''ve already started to do in Ruby. > Let me finish the first cut and then I''ll have a look and see how close > I came to the other one.Ok. I also looked into how you integrate with glib or gtk. In Twisted they really integrate into the main loop, so everything from the IO objects being watched for events to the timeouts are handled by glib. (They basically have different implementations of the core reactor depending on whether you are raw on top of posix, using gtk, qt etc...) Otherwise you can insert code to be run from the glib main loop, or iterate over the glib main loop from your own. Depending on the processing being done that can probably lead to slow gui''s though, which brings us to threads. I don''t know how threading in ruby is implemented, and beyond that I''m not sure how you do threading when you are interacting with something like gtk from ruby. That is what I''ll look into next.> 7) Did you notice we''re on the Rubyforge most-active-this-week list? :-DHaha, that doesn''t say much for rubyforge activity, or at least for the way they determine it.> > On 5/11/06, *Jeff Rose* <rosejn at gmail.com <mailto:rosejn at gmail.com>> wrote: > > Francis Cianfrocca wrote: > > I mistakenly specified CVS as the SCM for eventmachine in > Rubyforge. I > > asked Tom Copeland to change it to SVN. If/when that gets done, then > > I''ll upload the source code. I also started sketching out a > pure-ruby > > implementation. > > I started trying to write a very minimal p2p chat app on top of > EventMachine and ran into a couple things to think about. (It also made > me realize how much easier this will be if we can stick to as much ruby > as possible.) Accepting input from the keyboard, for example, would > require a c++ implementation of a new EventableDescriptor > implementation. Everything in Ruby sits on top of IO already, which is > basically a beefed up version of EventableDescriptor, and if we can sit > on top of that it will give us a lot for free. If the actual handling > of IO objects in the interpreter (where it calls select) is what''s > causing the problems, then maybe we should write our own version to > replace it, or override a method somewhere so our version would get > called rather than the default. > > It seems like if the whole project is going to be called EventMachine, > then that needs to be the module encompassing everything. The actual > event processor should maybe have a different name? Reactor seems > to be > the standard for that component... > > When starting on a bad IP address the server crashes with a "no > acceptor" runtime exception. We are probably going to have to spend a > fair amount of time making clear exceptions everywhere so errors like > this are easy to track down. (It might be good to change the default > example to use 127.0.0.1 <http://127.0.0.1> also.) > > In reading the documentation it''s not totally clear whether absolutely > everything is non-blocking or not. For example, send_data is actually > not sending data right away. How do I know whether it sent > successfully? That''s why they use the deferred''s in Twisted I guess... > For some protocols (especially strict distributed computing > algorithms) it''s really important to know exactly what went out and what > didn''t. Why not put the outgoing stuff in a queue and do it on the next > iteration? Memcpy''ing everything is pretty expensive, and it means you > lose the knowledge of what exactly sent and what didn''t... > > Have you seen the IO-Reactor written a few years back on top of select? > (http://www.deveiate.org/projects/IO-Reactor > <http://www.deveiate.org/projects/IO-Reactor>) It actually looks pretty > clean, and could serve as a good basis for the bottom end of a ruby only > version. It does the basic registering of IO objects (descriptors), and > then looping over select. If timers and some kind of deferred > mechanism > were added it might be just what we want. > > -Jeff > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org <mailto:Eventmachine-talk at rubyforge.org> > http://rubyforge.org/mailman/listinfo/eventmachine-talk > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk
Francis Cianfrocca wrote: > Several points: > > 1) The rubyforge SCM has been changed to SVN. What''s your Rubyforge login name? rosejn > 2) I totally see the point about being able to add EventableDescriptor subclasses in Ruby. Last night I started writing a pure-Ruby event machine and we''ll see where it goes. But if it doesn''t work, I can easily see a hybrid model where the base classes are in Ruby and some of the subclasses (the the socket handlers) are in C. And in another case of great-minds-think-alike, the pure-Ruby EventableDescriptor class is basically just IO. Cool. Maybe we can even just insert our own version of the socket handlers so that people can network as usual and be using our C backend rather than the default? > 3) Using the "reactor" name. Out of partisan pride I avoided that, but it''s probably better to use the word everyone is used to. Can you give me a code sample of how the basic loop would look with "reactor"? Haha, I know exactly the feeling. I''ve got some stuff to do this afternoon, but I''ll try to work on this afterward. Actually, Leslie Lamport, the father of Latex and Paxos, possibly the most famous algorithm in distributed computing, is speaking downstairs from me in 45 minutes :-) > 5) send_data schedules data for transmission, and you really don''t know when it gets sent but in most protocols you don''t care too much. EventMachine currently buffers all outbound data, which is really wasteful, but works around the problem that the user doesn''t want to know about outbound EWOULDBLOCK errors. There are plenty of other subtleties here, though. However, I really like the idea of letting the user provide an error handler as Twisted does. If it matters for the user to know what was sent and what wasn''t, then you''re getting into guaranteed delivery, which is a very difficult problem because the dependencies are not all local. Can you give this some more thought and decide what you think the requirements should actually be? When do you get EWOULDBLOCK errors? I''m not sure that we can or need to guarantee delivery, but at least if we can tell users that messages x & y went out onto the wire, but z got an error, that would be good. The errback scheme in Twisted seems pretty nice for this. > 6) Sounds like IO-Reactor is what I''ve already started to do in Ruby. Let me finish the first cut and then I''ll have a look and see how close I came to the other one. Ok. I also looked into how you integrate with glib or gtk. In Twisted they really integrate into the main loop, so everything from the IO objects being watched for events to the timeouts are handled by glib. (They basically have different implementations of the core reactor depending on whether you are raw on top of posix, using gtk, qt etc...) Otherwise you can insert code to be run from the glib main loop, or iterate over the glib main loop from your own. Depending on the processing being done that can probably lead to slow gui''s though, which brings us to threads. I don''t know how threading in ruby is implemented, and beyond that I''m not sure how you do threading when you are interacting with something like gtk from ruby. That is what I''ll look into next. > 7) Did you notice we''re on the Rubyforge most-active-this-week list? :-D Haha, that doesn''t say much for rubyforge activity, or at least for the way they determine it.