Hello, I have been looking at the Ruby/EventMachine. First let me say it look very good. Reactor model with no threads makes for fast reliable server, and I have read about marvelous Twisted framework for Python and am glad to see something similar for Ruby. I am writing network app with Ruby threads now and it very slow, and I try new Ruby 1.9 with native threads that make it much slower. Plus I''m hit limit with number of concurrent connections. Ruby does not support more than around 1024, due to select bottleneck I think. But I have some questions before I use Ruby/EventMachine. How is the i/o multiplex handed? I read it does epoll? Is it custom library or is it something standard? I am mainly interested in bsd platform. In the past network servers I write use libevent project to handle i/o. This lets use bsd kqueue and linux epoll when running linux: http://monkey.org/~provos/libevent/ So far I see no Ruby/EventMachine support for kqueue, only epoll. I am trying to find way that make Ruby handle thousands of sockets with single process, on bsd platform. So, next question, why aren''t you using libevent to handle the i/o multiplex already? If so, EventMachine could support epoll for linux people and kqueue for bsd people, and other interfaces for other os. I was looking for libevent in Ruby and I found this: http://www.zedshaw.com/projects/ruby_event/index.html The project says it is dead due to Ruby thread. But Ruby getting new thread in a few weeks, right? Won''t that fix the problem? Could you use Ruby/Event project as libevent binding for EventMachine with new Ruby? Thank you for answering my questions. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071209/d9747563/attachment.html
On Dec 10, 2007 12:06 AM, Franki Rai <frankirai at gmail.com> wrote:> Hello, > > I have been looking at the Ruby/EventMachine. First let me say it look > very good. Reactor model with no threads makes for fast reliable server, > and I have read about marvelous Twisted framework for Python and am glad to > see something similar for Ruby. > > I am writing network app with Ruby threads now and it very slow, and I try > new Ruby 1.9 with native threads that make it much slower. Plus I''m hit > limit with number of concurrent connections. Ruby does not support more > than around 1024, due to select bottleneck I think. > > But I have some questions before I use Ruby/EventMachine. > > How is the i/o multiplex handed? I read it does epoll? Is it custom > library or is it something standard? > > I am mainly interested in bsd platform. In the past network servers I > write use libevent project to handle i/o. This lets use bsd kqueue and > linux epoll when running linux: > > http://monkey.org/~provos/libevent/<http://monkey.org/%7Eprovos/libevent/> > > So far I see no Ruby/EventMachine support for kqueue, only epoll. I am > trying to find way that make Ruby handle thousands of sockets with single > process, on bsd platform. > > So, next question, why aren''t you using libevent to handle the i/o > multiplex already? If so, EventMachine could support epoll for linux people > and kqueue for bsd people, and other interfaces for other os. > > I was looking for libevent in Ruby and I found this: > > http://www.zedshaw.com/projects/ruby_event/index.html > > The project says it is dead due to Ruby thread. But Ruby getting new > thread in a few weeks, right? Won''t that fix the problem? > > Could you use Ruby/Event project as libevent binding for EventMachine with > new Ruby? > > Thank you for answering my questions. >This question comes up every so often. EM''s reactor uses select(2), which is the most portable way to do it. It also supports epoll where available. It would not be difficult to add kqueue support to the EM reactor. It''s quite similar to epoll and it probably wouldn''t be a big job. Having programmed kqueue myself, I have to say that its performance and scalability benefits over select(2) seem minor on the BSD platform. It''s far less of an improvement than the enormous benefit that epoll provides on Linux. If anyone disagrees, I''m open to being convinced otherwise. I''m just as interested in reimplementing the reactor using IOCP, which would give a performance and scalability boost for Windows users. As far as libevent is concerned: I don''t particularly agree with the case that libevent is any faster, more stable, or more scalable than the EM implementation, which is based on many years of experience with high-speed networking. And there will be an ongoing need to continue supporting Ruby versions without native threads. Much more importantly, however, EM is not a competitor to libevent. The appropriate comparison is to Twisted, not to libevent. EM exists to provide a complete platform with a standard API, for creating fast, scalable applications. That''s why it includes a library of implementations of standard protocols, which will grow. The intention is not to enable low-level event access. It''s to provide high-level application support. I would have no objection at all to an implementation of the EM reactor based on libevent, provided that it passed all the existing unit tests, and guaranteed to run all existing EM code without changes. If anyone wants to work on this, I''ll be happy to support you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/73c6bcc5/attachment.html
On Dec 9, 2007 10:06 PM, Franki Rai <frankirai at gmail.com> wrote:> The project says it is dead due to Ruby thread. But Ruby getting new > thread in a few weeks, right? Won''t that fix the problem?Not directly. The Global Interpreter Lock needs to be released before making a blocking call. Check out the BLOCKING_REGION macro in Ruby 1.9''s thread.c. I''m sure there will be a more public API than that in the future. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/d85cea87/attachment.html
Upon closer inspection, the appropriate call seems to be:
VALUE
rb_thread_blocking_region(
rb_blocking_function_t *func, void *data1,
rb_unblock_function_t *ubf, void *data2)
...defined in intern.h
--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/b12d72a3/attachment.html
On Dec 10, 2007 1:26 PM, Tony Arcieri <tony at clickcaster.com> wrote:> Upon closer inspection, the appropriate call seems to be: > > VALUE > rb_thread_blocking_region( > rb_blocking_function_t *func, void *data1, > rb_unblock_function_t *ubf, void *data2) > > ...defined in intern.h > >Tony, given that, is it realistic to hope for a libevent-based reactor core for EM, for Ruby 1.9+ versions? If so, what would the benefit, apart from kqueue support? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/43510e4b/attachment.html
On Dec 10, 2007 12:15 PM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> Tony, given that, is it realistic to hope for a libevent-based reactor > core for EM, for Ruby 1.9+ versions? If so, what would the benefit, apart > from kqueue support? >It doesn''t have to be based on libevent. Where you''re currently making system calls like select or epoll and blocking at intervals less than the Ruby scheduler quantum (what is it, 10ms?), you can now just block indefinitely. This function will release the GIL and call the *func callback, so any other threads will continue to run (in their own separate native threads). When the blocking call completes, the GIL is reacquired and you can process the results. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/574d7f83/attachment.html
On Dec 10, 2007 2:59 PM, Tony Arcieri <tony at clickcaster.com> wrote:> On Dec 10, 2007 12:15 PM, Francis Cianfrocca <garbagecat10 at gmail.com> > wrote: > > > Tony, given that, is it realistic to hope for a libevent-based reactor > > core for EM, for Ruby 1.9+ versions? If so, what would the benefit, > > apart from kqueue support? > > > > It doesn''t have to be based on libevent. > > Where you''re currently making system calls like select or epoll and > blocking at intervals less than the Ruby scheduler quantum (what is it, > 10ms?), you can now just block indefinitely. This function will release the > GIL and call the *func callback, so any other threads will continue to run > (in their own separate native threads). When the blocking call completes, > the GIL is reacquired and you can process the results. > > >Ok, that would be a very minor change to the reactor, and it might not even increase performance at all. So it sounds like there really isn''t any compelling reason to go with libevent. In fact, it would be a pain because it''s yet another external component that users would have to provision and manage. The question matters primarily because people quite often stand up and say "we should use libevent because someone or other once thought it was a good idea." And if there are benefits to it, we should. If there are no benefits, we shouldn''t. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/3b73f338/attachment-0001.html
On Dec 10, 2007 1:28 PM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> Ok, that would be a very minor change to the reactor, and it might not > even increase performance at all. >It''s possible there will be no performance improvement. I think it''s worth a shot though. I''d be willing to give it a try with the epoll implementation, possibly. Does that still need select() for handling timers or is it 100% epoll driven now?> So it sounds like there really isn''t any compelling reason to go with > libevent. >I don''t know how you''d make libevent work with this API. libevent makes callbacks in C. If you wanted the callback handlers to make calls back into Ruby, they''d need to reacquire the GIL again, and I don''t know how to do that. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/477a295f/attachment.html
On Dec 10, 2007 4:36 PM, Tony Arcieri <tony at clickcaster.com> wrote:> On Dec 10, 2007 1:28 PM, Francis Cianfrocca <garbagecat10 at gmail.com> > wrote: > > > Ok, that would be a very minor change to the reactor, and it might not > > even increase performance at all. > > > > It''s possible there will be no performance improvement. I think it''s > worth a shot though. I''d be willing to give it a try with the epoll > implementation, possibly. > > Does that still need select() for handling timers or is it 100% epoll > driven now? >100% epoll, no select dependency for the timers.> > I don''t know how you''d make libevent work with this API. libevent makes > callbacks in C. If you wanted the callback handlers to make calls back into > Ruby, they''d need to reacquire the GIL again, and I don''t know how to do > that.I guess my bottom-line question is, do we lose anything by not supporting libevent? The only good answer I''ve heard to that so far is kqueue, which we can easily do. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/524a92df/attachment.html
On Dec 10, 2007 4:19 PM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> 100% epoll, no select dependency for the timers. >If that''s the case it should be fairly trivial to add a rb_thread_blocking_region() wrapper around the epoll call and let it block indefinitely.> I guess my bottom-line question is, do we lose anything by not supporting > libevent? The only good answer I''ve heard to that so far is kqueue, which we > can easily do. >libevent supports */dev/poll<http://access1.sun.com/techarticles/devpoll.html> *, *kqueue(2)<http://www.freebsd.org/cgi/man.cgi?query=kqueue&apropos=0&sektion=0&format=html> *, *event ports<http://developers.sun.com/solaris/articles/event_completion.html> *, *select(2)*, *poll(2)* and *epoll(4)<http://www.xmailserver.org/linux-patches/epoll.txt> * So, in addition to kqueue there''s additional support for Solaris 7+ (with /dev/poll) and Solaris 10 (event ports), as well as Irix and HP/UX (/dev/poll). It seems like people have been trying to create a binding to Win32 IOCP and mapping it onto the libevent bufferedevents API. However, all attempts to do this are only experimental at this point. The real win would be outsourcing the maintenance of the various platform-specific I/O mechanisms to others. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/d3e95460/attachment.html
I should add: I asked on ruby-core about using rb_thread_blocking_region() in conjunction to libevent. It should be fairly trivial if each blocking call runs the event loop once with the event_base_loop() function inside rb_thread_blocking_region(), then returns control back to Ruby for the actual event processing. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/c9692e2c/attachment-0001.html
On Dec 10, 2007 6:59 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> Having programmed kqueue myself, I have to say that its performance and > scalability benefits over select(2) seem minor on the BSD platform. It''s far > less of an improvement than the enormous benefit that epoll provides on > Linux. If anyone disagrees, I''m open to being convinced otherwise. >Main problem is number of connection. On the select, there is hard limit of 1024 maxfds. This limit not exist with kqueue. I understand the EventMachine epoll not have this limit, so why not kqueue? My program hit this limit in Ruby already. I need way to get past.> I would have no objection at all to an implementation of the EM reactor > based on libevent, provided that it passed all the existing unit tests, and > guaranteed to run all existing EM code without changes. If anyone wants to > work on this, I''ll be happy to support you. >I would be interest in having look. I have much luck with libevent server writing in C, so would like to stick with if possible when Ruby. I am new to Ruby C extension but have program C for many years, so it should not be big of change I hope. You have C extension already so could look at how hard would be to integrate. That get rid of Ruby extension stuff I don''t know at least. I have EventMachine source code and will have look through it. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/a5837752/attachment.html
On Dec 10, 2007 5:54 PM, Tony Arcieri <tony at clickcaster.com> wrote:> It should be fairly trivial if each blocking call runs the event loop once > with the event_base_loop() function inside rb_thread_blocking_region(), then > returns control back to Ruby for the actual event processing.Can you explain more? Usually use event_dispatch() not event_base_loop(). Why use it? I put the rb_thread_blocking_region() around blocking call? That fix thread problem? ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/4459bf4b/attachment.html
On Dec 10, 2007 7:44 PM, Franki Rai <frankirai at gmail.com> wrote:> Can you explain more? > > Usually use event_dispatch() not event_base_loop(). Why use it? >I think to write a proper libevent binding to Ruby you''d want to dispatch each event received as a Ruby callback. The problem with event_dispatch() is that it continues blocking until the event loop stops. When you invoke rb_thread_blocking_region(), it will release Ruby''s Global Interpreter Lock. The problem is, until the callback given to rb_thread_blocking_region() completes, there''s no way to reacquire this lock and call back into the Ruby interpreter. I haven''t used event_base_loop() myself, but it was suggested by someone on ruby-core. I believe it blocks until it receives a single event, then returns. This would allow you to receive the next event, have the rb_thread_blocking_region() call return (reacquiring the Global Interpreter Lock), then you can use rb_funcall() to dispatch the event back to the EventMachine reactor loop.> I put the rb_thread_blocking_region() around blocking call? That fix > thread problem? >Yes, using rb_thread_blocking_region() will allow you to safely make blocking calls for indefinite periods of time without worrying about affecting other threads. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/1d580b6e/attachment.html
On Dec 10, 2007 7:54 PM, Tony Arcieri <tony at clickcaster.com> wrote:> I should add: > > I asked on ruby-core about using rb_thread_blocking_region() in > conjunction to libevent. > > It should be fairly trivial if each blocking call runs the event loop once > with the event_base_loop() function inside rb_thread_blocking_region(), then > returns control back to Ruby for the actual event processing. > > >So you''re thinking about calling event_base_loop() and having its callbacks just record the actual events on a stack so they can be fed to the Ruby callbacks after the libevent loop runs? You might run into semantic issues with EM#next_tick. But let''s solve this one step at a time. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/ca6a66eb/attachment.html
On Dec 10, 2007 7:58 PM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> So you''re thinking about calling event_base_loop() and having its > callbacks just record the actual events on a stack so they can be fed to the > Ruby callbacks after the libevent loop runs? > > You might run into semantic issues with EM#next_tick. But let''s solve this > one step at a time. >Well, I certainly don''t want to try shoehorning libevent in as the base I/O package for EventMachine. I''d rather try wrapping the epoll call in rb_thread_blocking_region() and see if that improves performance, but I''ll probably wait until 1.9 proper is out before giving it a try. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/df78ea5b/attachment.html
On Dec 10, 2007 7:55 PM, Tony Arcieri <tony at clickcaster.com> wrote:> I think to write a proper libevent binding to Ruby you''d want to dispatch > each event received as a Ruby callback. > > The problem with event_dispatch() is that it continues blocking until the > event loop stops. When you invoke rb_thread_blocking_region(), it will > release Ruby''s Global Interpreter Lock. The problem is, until the callback > given to rb_thread_blocking_region() completes, there''s no way to reacquire > this lock and call back into the Ruby interpreter. > > I haven''t used event_base_loop() myself, but it was suggested by someone > on ruby-core. I believe it blocks until it receives a single event, then > returns. This would allow you to receive the next event, have the > rb_thread_blocking_region() call return (reacquiring the Global Interpreter > Lock), then you can use rb_funcall() to dispatch the event back to the > EventMachine reactor loop. >Okay, I think I get. The event_base_loop get next event inside blocking function, store event somewhere in variable, then use rb_funcall to call back into Ruby after the rb_thread_blocking_region has returned? If so that sound simple. Where do I find out more about rb_thread_blocking_region? Is there document? I am not finding many documentation about Ruby C extension. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/880a95b4/attachment.html
On Dec 10, 2007 8:02 PM, Tony Arcieri <tony at clickcaster.com> wrote:> Well, I certainly don''t want to try shoehorning libevent in as the base > I/O package for EventMachine. >Do you think it will be hard? Harder than adding kqueue without libevent? I am big fan for libevent. Would like to see it used in EventMachine. Maybe make faster? ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/f39a20d1/attachment.html
On Dec 10, 2007 9:46 PM, Franki Rai <frankirai at gmail.com> wrote:> Do you think it will be hard? >I really haven''t looked at the C/C++ in the EventMachine native extension enough to know. As far as I can tell though, switching to libevent would be a pretty major change. My C has gotten pretty rusty at this point and I wouldn''t really be confident in ripping out the guts of the EventMachine reactor. If you''re comfortable with C/C++, it might not be too hard.> Harder than adding kqueue without libevent? >It''d probably be a lot easier to just add kqueue support.> I am big fan for libevent. > > Would like to see it used in EventMachine. Maybe make faster? >There''s no way to know without benchmarks. -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/0651d338/attachment.html
On Dec 10, 2007 11:46 PM, Franki Rai <frankirai at gmail.com> wrote:> On Dec 10, 2007 8:02 PM, Tony Arcieri <tony at clickcaster.com> wrote: > > > Well, I certainly don''t want to try shoehorning libevent in as the base > > I/O package for EventMachine. > > > > Do you think it will be hard? > > Harder than adding kqueue without libevent? > > I am big fan for libevent. > > Would like to see it used in EventMachine. Maybe make faster? > >Franki, you''ve said that your problem is that you need more than 1024 descriptors in a single process, and that you need to use BSD instead of Linux. Tell us exactly what your application does and someone may be able to suggest a way to get around this problem. If so, then you should give EM a try. I think you''ll find that it''s pretty fast, and that libevent doesn''t give a performance advantage. I''ve replaced some carefully hand-written and hand-tuned C code with EM-based programs, and performance is just as fast or faster. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/a84024eb/attachment.html
On Dec 11, 2007 1:35 AM, Tony Arcieri <tony at clickcaster.com> wrote:> > It''d probably be a lot easier to just add kqueue support. > >Adding kqueue won''t be hard. I''ve used it before and it''s a lot like epoll. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/71184f34/attachment.html
On Dec 10, 2007 11:36 PM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> Franki, you''ve said that your problem is that you need more than 1024 > descriptors in a single process, and that you need to use BSD instead of > Linux. Tell us exactly what your application does and someone may be able to > suggest a way to get around this problem. >I do not think it possible. I cannot disclose specific information about program but can tell you general design. System provide near realtime event notification to thousands of clients, sending out many hundred messages per second. There is writer thread which send messages to all client and multiple reader thread which poll from different source and run delta algorithm to generate event message to different client. Client each have stateful event filter and can change filter depending on event, and also poll different source depending on if client ask. Must not hang writer if client is congested, and keep near realtime writeout to all client. Ruby is great for read and transform data from many different source, but writeout is bad with many client. Maybe it better to write C program with libevent that talk over socket and only use Ruby for read and transform data? But communication between C program and Ruby would be complex.> If so, then you should give EM a try. I think you''ll find that it''s pretty > fast, and that libevent doesn''t give a performance advantage. >EM look really good but need to get above 1024 client. Would like to get rid of threads too, they seem to make i/o slow. Ruby 1.9 only make things worse. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071210/822aef95/attachment-0001.html
On Dec 10, 2007 11:35 PM, Tony Arcieri <tony at clickcaster.com> wrote:> I really haven''t looked at the C/C++ in the EventMachine native extension > enough to know. As far as I can tell though, switching to libevent would be > a pretty major change. My C has gotten pretty rusty at this point and I > wouldn''t really be confident in ripping out the guts of the EventMachine > reactor. If you''re comfortable with C/C++, it might not be too hard. >I look at the EventMachine native extension and not see easy way to add in libevent. It''d probably be a lot easier to just add kqueue support.>I didn''t see easy way to add kqueue in either. Is really all i/o of EventMachine in em.cpp? ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/632f28c0/attachment.html
On Dec 11, 2007 1:58 AM, Franki Rai <frankirai at gmail.com> wrote:> On Dec 10, 2007 11:36 PM, Francis Cianfrocca <garbagecat10 at gmail.com> > wrote: > > > Franki, you''ve said that your problem is that you need more than 1024 > > descriptors in a single process, and that you need to use BSD instead of > > Linux. Tell us exactly what your application does and someone may be able to > > suggest a way to get around this problem. > > > > I do not think it possible. I cannot disclose specific information about > program but can tell you general design. > > System provide near realtime event notification to thousands of clients, > sending out many hundred messages per second. > > There is writer thread which send messages to all client and multiple > reader thread which poll from different source and run delta algorithm to > generate event message to different client. > > Client each have stateful event filter and can change filter depending on > event, and also poll different source depending on if client ask. > > Must not hang writer if client is congested, and keep near realtime > writeout to all client. > >How do the clients connect to the server? Does the server run a TCP acceptor on a well-known port that the clients make connections to? Or are you using UDP? Does this application run in the Internet across firewalls, or are all of the clients and servers within one network? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/8f05b99b/attachment.html
On Dec 10, 2007 11:58 PM, Franki Rai <frankirai at gmail.com> wrote:> System provide near realtime event notification to thousands of clients, > sending out many hundred messages per second. > > There is writer thread which send messages to all client and multiple reader > thread which poll from different source and run delta algorithm to generate > event message to different client.Don''t do it that way. Have your readers exist as one or more separate processes. They pass the event notifications to one or more writer processes, which in turn send those notifications out to the clients. If you create your readers in such a way that each can communicate with multiple writers, then even with the 1024 descriptor limit per process, you can trivially exceed that limitation for the total number of clients.> Client each have stateful event filter and can change filter depending on > event, and also poll different source depending on if client ask. > > Must not hang writer if client is congested, and keep near realtime writeout > to all client.This shouldn''t be a problem.> Ruby is great for read and transform data from many different source, but > writeout is bad with many client.Write a prototype, using EM, with a multiprocess model like I describe. It shouldn''t take long to write enough code to test the concept, and I think you will find that you can manage writing to many clients without any particular problems.> Maybe it better to write C program with libevent that talk over socket and > only use Ruby for read and transform data? > > But communication between C program and Ruby would be complex.Communication between C and Ruby is pretty simple. If they are separate processes, just pass mesages over a socket. If your C code is written as a Ruby extension, you can hook into Ruby trivially easily. But give pure Ruby with EM a try. Kirk Haines
On Dec 11, 2007 5:22 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> How do the clients connect to the server? Does the server run a TCP > acceptor on a well-known port that the clients make connections to? >Not on well known port no, but yes TCP connection. I not design protocol but there was discuss of UDP but requirement of in-order event delivery. Even though event there is timestamp client stateful and use history of past event to handle present event so without guarantee delivery in-order could cause problem with client. TCP was simple solution, although hurt latency. Does this application run in the Internet across firewalls, or are all of> the clients and servers within one network? >Main use is on firewalled network, but there is also VPN so people use that way too. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/1d58db02/attachment.html
> Even though event there is timestamp client stateful and use history of past > event to handle present event so without guarantee delivery in-order could > cause problem with client. TCP was simple solution, although hurt latency.Not to be pedantic, but the in-order delivery requirement is what hurt latency. -- --- Thomas H. Ptacek // matasano security read us on the web: http://www.matasano.com/log
On Dec 11, 2007 7:19 AM, Kirk Haines <wyhaines at gmail.com> wrote:> Don''t do it that way. Have your readers exist as one or more separate > processes. They pass the event notifications to one or more writer > processes, which in turn send those notifications out to the clients. >Well why I do that? Isn''t point of reactor framework to run all in single event loop? No thread. One process.> If you create your readers in such a way that each can communicate > with multiple writers, then even with the 1024 descriptor limit per > process, you can trivially exceed that limitation for the total number > of clients. >How I do that? Client connect to single host with single port. I not want write some sort load balancing proxy. Use prefork server or something? Ruby even support that? Why do that when can just write one program support more than 1024 connection?> Write a prototype, using EM, with a multiprocess model like I > describe. It shouldn''t take long to write enough code to test the > concept, and I think you will find that you can manage writing to many > clients without any particular problems. >That not make sense. Why use multiprocess with EM? EM not enough to do this with one process? That very easy in C. Although data transform hard.> Communication between C and Ruby is pretty simple. If they are > separate processes, just pass mesages over a socket.Yes but number of messages high. I can try to describe. There is two case. Either we read from legacy application or competitor application, or current application. Current application use same protocol as write. Also have stateful filter. Current application send very large number of message, so use filter its side reduce number of message much. Filter is union of current client filter. Legacy application and competitor application both polling. No filter. But run delta algorithm. Must stay in sync with state of current writeout. Can be done with per-client buffer, but cost of backup large. Often there is backed up client. Especially on VPN. So for current application client need to both commission and decommission source and change filter. Legacy application and competitor application just need commission and decommission source. If C server for writeout, filter probably best there. But would need message to commission and decommission source and change filter. I guess not so complex. Maybe in comparison to current code.> If your C code is written as a Ruby extension, you can hook into Ruby > trivially easily. But give pure Ruby with EM a try. >There is no C code now. Was talking about separate process using libevent to do writeout, written in C. Right now, sound like best solution. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071211/76c947a3/attachment.html
On Dec 12, 2007 12:11 AM, Franki Rai <frankirai at gmail.com> wrote:> On Dec 11, 2007 7:19 AM, Kirk Haines <wyhaines at gmail.com> wrote: > > > Don''t do it that way. Have your readers exist as one or more separate > > processes. They pass the event notifications to one or more writer > > processes, which in turn send those notifications out to the clients. > > > > Well why I do that? > > Isn''t point of reactor framework to run all in single event loop? > > No thread. One process. > > > > If you create your readers in such a way that each can communicate > > with multiple writers, then even with the 1024 descriptor limit per > > process, you can trivially exceed that limitation for the total number > > of clients. > > > > How I do that? Client connect to single host with single port. > > I not want write some sort load balancing proxy. > > Use prefork server or something? Ruby even support that? > > Why do that when can just write one program support more than 1024 > connection? > > > > Write a prototype, using EM, with a multiprocess model like I > > describe. It shouldn''t take long to write enough code to test the > > concept, and I think you will find that you can manage writing to many > > clients without any particular problems. > > > > That not make sense. > > Why use multiprocess with EM? > > EM not enough to do this with one process? > > That very easy in C. Although data transform hard. >You''re obviously looking for reasons not to try Kirk''s suggestion. We can only lead horses to water. As far as multi-processes are concerned, EM style says nothing against them. EM discourages the use of multiple THREADS, not multiple processes (because threaded programming is leads to very challenging bugs, and is often slower and less scalable, particularly on Ruby). In fact, multiple processes collaborating through message passing is the model that EM was originally designed for. It''s easy enough to write a single server that returns a different port number, which a client can then use to reconnect to a different process (that''s why I asked if your clients have to connect through a firewall). Preforking is possible, but not very portable. You''ll find this more difficult to do. In all candor, the rest of your description makes it sound like there are serious flaws in the architecture of this application to start with. If you''re experiencing client latency, then you need to figure out how your design can adaptively shed load. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/24e331bb/attachment.html
On Dec 11, 2007 10:11 PM, Franki Rai <frankirai at gmail.com> wrote:> On Dec 11, 2007 7:19 AM, Kirk Haines <wyhaines at gmail.com> wrote: > > > Don''t do it that way. Have your readers exist as one or more separate > > processes. They pass the event notifications to one or more writer > > processes, which in turn send those notifications out to the clients. > > > > Well why I do that? > > Isn''t point of reactor framework to run all in single event loop?The point of it is to handle the (network) IO in an event based way.> No thread. One process.Sure, but consider a few things: Right _now_ on BSD platforms, you are limited to select(), which, in addition to having a 1024 descriptor limit, sees performance degredation as the number of descriptors increases. If someone does the work to allow the use of kqueue on BSD platforms, you break the 1024 descriptor limit, which would be great. I have users of my software that could probably benefit from that, too (though more of them would benefit from something that lets Solaris not use select()). However, there are other factors to consider. A chief one is that one process means one CPU. You are talking about reading and writing some "thousands" of messages per second. Depending on just how many thousands that is, at some point on CPU becomes saturated with work, and your system is stuck at a scalability ceiling. Maybe your "thousands" isn''t that high, though, and this will never be a factor? If it might be, designing from the beginning to use multiple processes will save you a lot of work later. What about multiple machines? Will a reader/writer always exist in a 1 to 1 ratio on a given machine, or might it make sense to have readers on multiple machines, all talking to writers on a single machine?> > If you create your readers in such a way that each can communicate > > with multiple writers, then even with the 1024 descriptor limit per > > process, you can trivially exceed that limitation for the total number > > of clients. > > > > > > How I do that? Client connect to single host with single port.I don''t know, since I don''t know the details of your system. Give 1000 clients one port, and the second 1000 clients a different port? Put more than one IP on a system, and just DNS round robin between them? Depending on the real details, there are solutions.> Why do that when can just write one program support more than 1024 > connection?If you are on BSD, you can''t do that today. If you are running on a Linux system, no problem, so long as a single CPU can handle your message load, as I mentioned above.> Yes but number of messages high.You''d be surprised at the number of messages one can push through a Ruby process. Here''s part of the benchmark output in my asynchronous logger''s test output, ran on a 3.0Ghz Intel Xeon CPU. Analogger Speedtest -- short messages Testing 100000 messages of 10 bytes each. user system total real 0.360000 0.040000 0.400000 ( 1.274181) Message rate: 78456.6632851845/second (1.274589) Analogger Speedtest -- larger messages Testing 100000 messages of 100 bytes each. user system total real 0.380000 0.050000 0.430000 ( 1.435402) Message rate: 69661.3621861407/second (1.435516) You can move a LOT of messages through Ruby with EM.>From everything you have described, even if we did have kqueue supportin EventMachine tomorrow, if it were me, I''d still decouple the readers and writers. I feel even more strongly about this after reading your filtering requirements. If the readers and writers are decoupled, you have a scalability path. If they are not, then you don''t. There is elegance, simplicity, and performance to multiple-process-message-passing-event-based-designs that is hard to beat. Kirk Haines
On Dec 12, 2007 7:16 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> You''re obviously looking for reasons not to try Kirk''s suggestion. We can > only lead horses to water. As far as multi-processes are concerned, EM style > says nothing against them. EM discourages the use of multiple THREADS, not > multiple processes (because threaded programming is leads to very > challenging bugs, and is often slower and less scalable, particularly on > Ruby). In fact, multiple processes collaborating through message passing is > the model that EM was originally designed for. >Okay, well fewer process is less complex. Multiple process mean each need supervisor, plus client manage source mean create and stop process and supervisor. Trying to keep software simple.> > It''s easy enough to write a single server that returns a different port > number, which a client can then use to reconnect to a different process > (that''s why I asked if your clients have to connect through a firewall). > Preforking is possible, but not very portable. You''ll find this more > difficult to do. >How am server to return different port? Is no redirect in protocol. Would have to seed client with different port number. But then no load balance.> In all candor, the rest of your description makes it sound like there are > serious flaws in the architecture of this application to start with. If > you''re experiencing client latency, then you need to figure out how your > design can adaptively shed load. >Latency is important concern, but not too bad right now. At least not with proxy to current application. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/32b043f1/attachment.html
On Dec 12, 2007 9:24 AM, Kirk Haines <wyhaines at gmail.com> wrote:> A chief one is that one process means one CPU. You are talking about > reading and writing some "thousands" of messages per second. > Depending on just how many thousands that is, at some point on CPU > becomes saturated with work, and your system is stuck at a scalability > ceiling. Maybe your "thousands" isn''t that high, though, and this > will never be a factor? If it might be, designing from the beginning > to use multiple processes will save you a lot of work later. >Our thousands is not high. Maybe 5000 in highest. Our product have one CPU.> I don''t know, since I don''t know the details of your system. Give > 1000 clients one port, and the second 1000 clients a different port? > Put more than one IP on a system, and just DNS round robin between > them? Depending on the real details, there are solutions. >What if client resolve round robin to machine with already too many client? Then it not work.> From everything you have described, even if we did have kqueue support > in EventMachine tomorrow, if it were me, I''d still decouple the > readers and writers. I feel even more strongly about this after > reading your filtering requirements. If the readers and writers are > decoupled, you have a scalability path. If they are not, then you > don''t. There is elegance, simplicity, and performance to > multiple-process-message-passing-event-based-designs that is hard to > beat. >Decouple is no problem, but I think I just do separate C write process with libevent. ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/d042f8be/attachment-0001.html
On Dec 12, 2007 9:24 AM, Kirk Haines <wyhaines at gmail.com> wrote:> You''d be surprised at the number of messages one can push through a > Ruby process. Here''s part of the benchmark output in my asynchronous > logger''s test output, ran on a 3.0Ghz Intel Xeon CPU. > > Analogger Speedtest -- short messages > Testing 100000 messages of 10 bytes each. > user system total real > 0.360000 0.040000 0.400000 ( 1.274181) > Message rate: 78456.6632851845/second (1.274589) > > Analogger Speedtest -- larger messages > Testing 100000 messages of 100 bytes each. > user system total real > 0.380000 0.050000 0.430000 ( 1.435402) > Message rate: 69661.3621861407/second (1.435516) > > You can move a LOT of messages through Ruby with EM. >Wow. Something''s certainly slow with DistribuStream... On a 2GHz Core 2 Duo... ~/src/distribustream/tools$ ./messaging_throughput 13:10:30 -- Sending 50000 JSON asynchronous messages 13:10:38 -- Finished Duration: 7.98 seconds Throughput: 6262.14 messages per second You''re an entire order of magnitude above me. Removing JSON parsing from the equation: 13:12:39 -- Sending 50000 asynchronous messages 13:12:45 -- Finished Duration: 5.96 seconds Throughput: 8390.55 messages per second -- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/bf9eb93e/attachment.html
On Dec 12, 2007 3:14 PM, Tony Arcieri <tony at clickcaster.com> wrote:> On Dec 12, 2007 9:24 AM, Kirk Haines <wyhaines at gmail.com> wrote: > > > You''d be surprised at the number of messages one can push through a > > Ruby process. Here''s part of the benchmark output in my asynchronous > > logger''s test output, ran on a 3.0Ghz Intel Xeon CPU. > > > > Analogger Speedtest -- short messages > > Testing 100000 messages of 10 bytes each. > > user system total real > > 0.360000 0.040000 0.400000 ( 1.274181) > > Message rate: 78456.6632851845/second (1.274589) > > > > Analogger Speedtest -- larger messages > > Testing 100000 messages of 100 bytes each. > > user system total real > > 0.380000 0.050000 0.430000 ( 1.435402) > > Message rate: 69661.3621861407/second (1.435516) > > > > You can move a LOT of messages through Ruby with EM. > > > > Wow. Something''s certainly slow with DistribuStream... > > On a 2GHz Core 2 Duo... > > ~/src/distribustream/tools$ ./messaging_throughput > 13:10:30 -- Sending 50000 JSON asynchronous messages > 13:10:38 -- Finished > Duration: 7.98 seconds > Throughput: 6262.14 messages per second > > You''re an entire order of magnitude above me. >Kirk''s code is rocket-propelled. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071212/a2636719/attachment-0001.html
Hello all, Zed Shaw of Ruby/Event show me this: http://software.schmorp.de/pkg/libev.html Say he like better than libevent. Is like libevent, only faster and simple. Better document. Has libevent layer too. But probably not use that if faster not to. Maybe write wrap for that with ruby1.9? ~ Franki -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071213/be2baba7/attachment.html
That looks pretty interesting On Dec 13, 2007 10:22 PM, Franki Rai <frankirai at gmail.com> wrote:> Hello all, > > Zed Shaw of Ruby/Event show me this: > > http://software.schmorp.de/pkg/libev.html > > Say he like better than libevent. > > Is like libevent, only faster and simple. Better document. > > Has libevent layer too. But probably not use that if faster not to. > > Maybe write wrap for that with ruby1.9? > > ~ Franki > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- Tony Arcieri ClickCaster, Inc. tony at clickcaster.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071214/6f3aab3a/attachment-0001.html
On Dec 14, 2007 12:22 AM, Franki Rai <frankirai at gmail.com> wrote:> Hello all, > > Zed Shaw of Ruby/Event show me this: > > http://software.schmorp.de/pkg/libev.html > > Say he like better than libevent. > > Is like libevent, only faster and simple. Better document. > > Has libevent layer too. But probably not use that if faster not to. > > Maybe write wrap for that with ruby1.9? >It sounds interesting, but I''d like to be clear on why this is interesting for EventMachine. I know that you want to see support for kqueue. That would be a clear benefit but it''s not clear whether it would be easier to rewrite the reactor or simply to support kqueue in the existing framework. As far as increasing performance is concerned, it''s worth finding out whether the current reactor core is in fact much slower than libev. Keep in mind that we need to support pure Ruby and pure Java versions of the reactor, so adding yet another dependency on compiled code is not an unalloyed positive. A final point: I should warn you that Zed Shaw is not a friend of EventMachine. Based on public statements he''s made in the past about us, we shouldn''t assume that he is thinking in terms of what is good for our project. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071214/5b481c61/attachment.html
> > > Analogger Speedtest -- short messages > > > Testing 100000 messages of 10 bytes each. > > > user system total real > > > 0.360000 0.040000 0.400000 ( 1.274181) > > > Message rate: 78456.6632851845/second (1.274589) > > > > > > Analogger Speedtest -- larger messages > > > Testing 100000 messages of 100 bytes each. > > > user system total real > > > 0.380000 0.050000 0.430000 ( 1.435402) > > > Message rate: 69661.3621861407/second (1.435516) > > > > > > You can move a LOT of messages through Ruby with EM. > Kirk''s code is rocket-propelled.I have also been impressed with EM''s throughput, though I don''t have numbers--it just ''feels'' fast, especially compared to threaded network apps. I''d say try it out and see if it is fast enough. Pre-mature optimization is at times easy :) Take care. -Roger
On Dec 14, 2007 12:22 AM, Franki Rai <frankirai at gmail.com> wrote:> Hello all, > > Zed Shaw of Ruby/Event show me this: > > http://software.schmorp.de/pkg/libev.html > > Say he like better than libevent. > > Is like libevent, only faster and simple. Better document. > > Has libevent layer too. But probably not use that if faster not to. > > Maybe write wrap for that with ruby1.9? >I''ll be happy to see the wrapper that you''re planning to write for this library. I couldn''t get the current release to compile (it gives undefined symbol errors). Given that the library was first released only a month ago, that may not be too surprising. libevent may be a better choice for you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071214/e7a05a54/attachment.html