Hello all- Forgive me if this has been discussed before- I searched the archives and didn''t see anything. Anyway, here it goes- There is a lot of talk lately about Comet-based applications- that is, web apps that hold connections to the server open for a very long time, to effectively achieve realtime data push from the server to the browser. Currently, there are a few web servers out there that are specially architected to support this- Twisted and Jetty 6 come to mind. The problem with those solutions is that neither of them is natively Ruby/Rails- you have to step into Python-land or Java-land there. What would really be nice is if Mongrel supported long-lived connections by separating the notion of a serviceable request from a thread or process. That is, be able to suspend a request until it is resumed by some other request, or by timeout, a-la Jetty 6''s Continuations (they''re really nice!). Has there been any thought or discussion to adding such a feature to Mongrel? I realize it probably seems a bit pie-in-the-sky at present, but it could really enable some exciting new types of in- browser applications. Combined with all the innovation that is going into Rails apps now, it could produce some really thrilling applications. I know that there is some talk around about something called "Armageddon", which I believe achieves basically the same end, but requires Flash on the client side to manage the backchannel socket. In any case, the requirements on the server side for tolerating huge numbers of idle connections are similar. Any thoughts or comments are appreciated. Best Regards, Danny Burkes
> Has there been any thought or discussion to adding such a feature to > Mongrel? I realize it probably seems a bit pie-in-the-sky at > present, but it could really enable some exciting new types of in- > browser applications. Combined with all the innovation that is going > into Rails apps now, it could produce some really thrilling > applications.I think a Mongrel Gem Plugin would be a much better fit. Zed has done a great job of making it easily extensible. Josh Ferguson released a secure download plugin (http://rubyforge.org/projects/msecuredownload/) for mongrel. I also added you to the mongrel upload progress backpack page (http://technoweenie.backpackit.com/pub/602283) that I''ve set up. The code isn''t *quite* ready so I have not set up a rubyforge project yet. -- Rick Olson http://techno-weenie.net
> I think a Mongrel Gem Plugin would be a much better fit. Zed has done > a great job of making it easily extensible. Josh Ferguson released a > secure download plugin > (http://rubyforge.org/projects/msecuredownload/) for mongrel. I also > added you to the mongrel upload progress backpack page > (http://technoweenie.backpackit.com/pub/602283) that I''ve set up. The > code isn''t *quite* ready so I have not set up a rubyforge project yet. >Thanks- I''ll take a look at the Mongrel Gem Plugin APIs and see if they would be sufficient to serve Comet apps. Essentially what we need to be able to do is hold a socket open while releasing it''s associated thread back to the pool handling incoming connections. Based on your knowledge of Mongrel Gem Plugins et al, do you know if this is possible? Do you have access to the worker threads and sockets at THAT low of a level? Best Regards, Danny
On Tue, 2006-06-06 at 15:17 -0700, Daniel Burkes wrote:> Hello all- > > Forgive me if this has been discussed before- I searched the archives > and didn''t see anything. Anyway, here it goes- >It was brought up in the past, but nothing came of it.> There is a lot of talk lately about Comet-based applications- that > is, web apps that hold connections to the server open for a very long > time, to effectively achieve realtime data push from the server to > the browser. Currently, there are a few web servers out there that > are specially architected to support this- Twisted and Jetty 6 come > to mind. The problem with those solutions is that neither of them is > natively Ruby/Rails- you have to step into Python-land or Java-land > there. What would really be nice is if Mongrel supported long-lived > connections by separating the notion of a serviceable request from a > thread or process. That is, be able to suspend a request until it is > resumed by some other request, or by timeout, a-la Jetty 6''s > Continuations (they''re really nice!).I would now like to point you at: http://en.wikipedia.org/wiki/Rube_Goldberg And ask you to read about Ruby Goldberg machines. The most important quote being: "A Rube Goldberg machine or device is any exceedingly complex apparatus that performs a very simple task in a very indirect and convoluted way."> Has there been any thought or discussion to adding such a feature to > Mongrel? I realize it probably seems a bit pie-in-the-sky at > present, but it could really enable some exciting new types of in- > browser applications. Combined with all the innovation that is going > into Rails apps now, it could produce some really thrilling > applications. >Well, Ruby has serious serious problems dealing with IO. Number one problem is that it uses select so you only get 1024 open files at once. And that''s *all* files, not just sockets. Throw in lacking performance, poor IO processing, and only having Strings to do data storage and you''re screwed. I tried to bring this up in my interview with Pat Eyler at oreillynet.com and it failed miserably. Nobody in the Ruby community seems interested in improving Ruby''s speed or scalability. Rather than face facts and listen to people like me who are in the trenches dealing with Ruby''s problems, they would rather rant about how nobody really needs performance. I''d say, don''t even bother until Ruby reaches 2.0 and I''ve ported over to Rite/YARV/Whatever. Until then, trying to do anything that can handle more than a few persistent connections is just not worth it. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/
> Well, Ruby has serious serious problems dealing with IO. Number one > problem is that it uses select so you only get 1024 open files at > once. > And that''s *all* files, not just sockets. Throw in lacking > performance, > poor IO processing, and only having Strings to do data storage and > you''re screwed. >Yeah that is a problem. On the comet app I''m working on now, we expect thousands of suspended connections per server to be a nominal situation.> I tried to bring this up in my interview with Pat Eyler at > oreillynet.com and it failed miserably. Nobody in the Ruby community > seems interested in improving Ruby''s speed or scalability. Rather > than > face facts and listen to people like me who are in the trenches > dealing > with Ruby''s problems, they would rather rant about how nobody really > needs performance. > > I''d say, don''t even bother until Ruby reaches 2.0 and I''ve ported over > to Rite/YARV/Whatever. Until then, trying to do anything that can > handle more than a few persistent connections is just not worth it. >While I respect greatly the fact that you have intimate knowledge of Ruby''s scalability issues, I think you''re missing a bit of the point here. I really don''t care whether the connection suspend/resume functionality is itself implemented in Ruby, only that it is implemented in a web server which supports calling the suspend/resume APIs from Ruby handlers. I was hoping that maybe mongrel could help out here, since part of it was already built in C, but it appears that all the socket handling and thread scheduling is in Ruby- is that correct? In any case, I appreciate your comments and thank you for your work on Mongrel so far- I''m seriously considering using it on the Rails side of the comet site I mentioned above, rather than lightty/ fastcgi. Anyway, it''s hard to complain about being involved in such interesting decisions :-) Best Regards, Danny