I heard about a new mongrel plugin or version that''s single threaded and uses non-blocking IO with events. It''s called something like "swifty fly". Anyone have a link for it? Thanks.
Swiftiply? http://swiftiply.swiftcore.org/index.html On May 21, 2007, at 6:52 PM, Andrew Arrow wrote:> I heard about a new mongrel plugin or version that''s single > threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it? > > Thanks. > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users
On 5/21/07, Andrew Arrow <andrew at geni.com> wrote:> I heard about a new mongrel plugin or version that''s single threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it?Swiftiply. http://swiftiply.swiftcore.org Swiftiply itself is a clustering proxy server for web apps with a twist; it runs as a server, but the backends are clients to it, instead of standalone servers. It is bundled with two hotpatches to mongrel to make two different flavors of mongrel. One is swiftiplied_mongrel. It is an event based mongrel that is intended to be used with Swiftiply. The other is evented_mongrel. This was sort of a step on the way to implementing swiftiplied_mongrel, and is just a mongrel that runs in an event loop instead of using threads. The intention with both is that the patches are transparent to any Mongrel handlers, so the Rails or Camping or Merb or Ramaze or whatever handlers just work. Kirk Haines
On 5/21/07, Andrew Arrow <andrew at geni.com> wrote:> I heard about a new mongrel plugin or version that''s single threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it? >http://swiftiply.swiftcore.org/ Post form Ezra: http://brainspl.at/articles/2007/05/12/event-driven-mongrel-and-swiftiply-proxy -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
I''m very interested in this thing but am not going to touch it until Zed blesses it. I''ll probably also wind up benchmarking it against everything else (nginx, lighty 1.5, apache), but if someone else does it first please post here and all that. -faisal On May 21, 2007, at 6:52 PM, Andrew Arrow wrote:> I heard about a new mongrel plugin or version that''s single > threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it? > > Thanks. > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users
On 5/21/07, Faisal N Jawdat <faisal at faisal.com> wrote:> I''m very interested in this thing but am not going to touch it until > Zed blesses it.I don''t know if Zed will "bless it", (the evented_mongrel patch). Event-Driven socket programming is a good thing, but ruby support for it isn''t. As far as my tests goes, a few things float into the "nebulossa" regarding re-entrant points in Ruby VM (the whole event-machine). I use Events as programming model in lots of projects, including ones with sockets, but none of them in ruby. Guess Zed evaluated the same problem when creating Mongrel.> I''ll probably also wind up benchmarking it against > everything else (nginx, lighty 1.5, apache), but if someone else does > it first please post here and all that. >Benchmarks? statistics? If anyone do that please: - Run the test across the network, not locally (to actually use the network card). - Don''t run these a few minutes, but a week will provide a good result of application behavior across time. - Measure everything, at least 5 times prior calculating "means" ;-) -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
On 5/22/07, Andrew Arrow <andrew at geni.com> wrote:> > I heard about a new mongrel plugin or version that''s single threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it?Well, with the caveats from Faisal and Luis, enjoy: http://swiftiply.swiftcore.org/index.html (by way of http://www.rubyinside.com/swiftiply-a-step-forward-in-rails-serving-489.html ) /David -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070522/93f097ad/attachment.html
On May 21, 2007, at 7:40 PM, Luis Lavena wrote:> I don''t know if Zed will "bless it", (the evented_mongrel patch).If Zed thinks it''s probably a reasonable thing to use even if he doesn''t want to incorporate it, it''s probably a reasonable thing to use. If Zed looks at it, gags, and points out that the entire system will break as of Mongrel 1.02, it''s probably a bad idea to go near. There''s a range in between, obviously... -faisal
On Mon, 21 May 2007 15:52:03 -0700 Andrew Arrow <andrew at geni.com> wrote:> I heard about a new mongrel plugin or version that''s single threaded and > uses non-blocking IO with events. It''s called something like "swifty > fly". Anyone have a link for it?There''s was quite a few announced "mongrel needs some help" kind of proxies: * Swiftiply by Kirk Haines (as he mentioned) http://swiftiply.swiftcore.org/index.html * Fuzed by Dave Fayram and Tom Preston-Warner (which is actually F U Zed if Rumor is right) http://code.google.com/p/fuzed/ * drproxy by Lucas Carlson which will have a project page soon, but you can grab super alpha code at http://rufy.com/drproxy.tgz The interesting thing is all three have different approaches to the same problem, but Fuzed and drproxy use Erlang rather than Ruby. Fuzed is using Yaws (an Erlang web server) so it can do smarter HTTP level load balancing and other goodies for a web app. drproxy is doing just TCP level routing and clustering so it could work for just about any protocol, but might not be so smart. A big thing about all three solutions is they are auto-configurable in that you fire up mongrel nodes and they tell the proxy about their existence. In a shared hosting environment this is a total disaster for security (since anyone would just join your proxy without proper encryption and security involved). BUT, for people doing deployed apps like most of the above authors, this is a really useful feature. Finally, I''m holding judgement and not going to "bless" any of them. None of the solutions will be put into Mongrel directly since I don''t think that''s really needed. The whole point of Mongrel is people can modify it very easily to do what they need, rather than waiting for me to be some kind of gate keeper. That let''s people use GPL code all they want, write commercial solutions, do their own coding style, and use any enhancers they feel like without having to be "blessed". Another point I''d like to make is there''s some great work coming from the virtual machine camp. I''m not going to try using EventMachine stuff until I see how well or poorly Mongrel does on some of these new VMs. I''m hoping that the JRuby and Rubinius folks just fix threads, IO, and GC once and for all and then everyone can live in a happy loving land without relying on tons of nasty C++ to fix things. I''ll be posting a few more highlights and thoughts from railsconf2007 soon. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/
On Mon, 21 May 2007 20:17:27 -0400 Faisal N Jawdat <faisal at faisal.com> wrote:> On May 21, 2007, at 7:40 PM, Luis Lavena wrote: > > I don''t know if Zed will "bless it", (the evented_mongrel patch). > > If Zed thinks it''s probably a reasonable thing to use even if he > doesn''t want to incorporate it, it''s probably a reasonable thing to > use. If Zed looks at it, gags, and points out that the entire system > will break as of Mongrel 1.02, it''s probably a bad idea to go near. > There''s a range in between, obviously...That''s not really my attitude or approach. Mongrel will stay core and usable in the way it is now. The thing I like about Mongrel is it''s simple and flexible enough that people can *finally* take charge and just write what they need for their situation rather than rely on me (or bitch and whine that I won''t implement their great features :-) My approach is, if it works, and you''re comfortable with the code, then go for it. The more solutions the better, as long as there''s good information so people can make great decisions. In fact, I try to just be upfront about my opinions and research on these solutions as they adapt so that people can make educated decisions. But, I also tell people never believe the hype. Always, always, always, go test things people tell you and double check what they say. Then, don''t believe your own tests in the same way you don''t believe what people tell you. Really make an effort to validate claims every time you are doing something new. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/
On May 21, 2007, at 5:38 PM, Zed A. Shaw wrote:> On Mon, 21 May 2007 20:17:27 -0400 > Faisal N Jawdat <faisal at faisal.com> wrote: > >> On May 21, 2007, at 7:40 PM, Luis Lavena wrote: >>> I don''t know if Zed will "bless it", (the evented_mongrel patch). >> >> If Zed thinks it''s probably a reasonable thing to use even if he >> doesn''t want to incorporate it, it''s probably a reasonable thing to >> use. If Zed looks at it, gags, and points out that the entire system >> will break as of Mongrel 1.02, it''s probably a bad idea to go near. >> There''s a range in between, obviously... > > That''s not really my attitude or approach. Mongrel will stay core and > usable in the way it is now. The thing I like about Mongrel is it''s > simple and flexible enough that people can *finally* take charge and > just write what they need for their situation rather than rely on me > (or bitch and whine that I won''t implement their great features :-) > > My approach is, if it works, and you''re comfortable with the code, > then > go for it. The more solutions the better, as long as there''s good > information so people can make great decisions. In fact, I try to > just > be upfront about my opinions and research on these solutions as they > adapt so that people can make educated decisions. > > But, I also tell people never believe the hype. Always, always, > always, go test things people tell you and double check what they say. > Then, don''t believe your own tests in the same way you don''t believe > what people tell you. Really make an effort to validate claims every > time you are doing something new. > > -- > Zed A. ShawI agree that this all does not belong in Mongrel core. Mongrel core is already tight as hell. That''s what makes it possible to add features you want by building on top of mongrel. Merb gets really good perf because it''s tightly integrated with mongrel. The event driven stuff does bench really well but like Zed said, don''t just blindly go using shit you read of teh intarwebs ;) Test it for yourself and decide. Cheers- -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273)
On 5/21/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> There''s was quite a few announced "mongrel needs some help" kind of > proxies: > > * Swiftiply by Kirk Haines (as he mentioned) > http://swiftiply.swiftcore.org/index.htmlReally, when I started writing this code, it had nothing at all to do with Mongrel needing anything. It was simply about building a fast clustering proxy that would work well with IOWA''s implicit session capability and would be available before Swiftcore IOWA 1.0 is released. I only came back and added the swiftiplied_mongrel because Mongrel is a target for all of the current existing Ruby frameworks, so it was a path of least resistance for making Swiftiply available to those frameworks, too, and I actually included evented_mongrel kind of as an afterthought. I had to implement it, anyway, to get to swiftiplied_mongrel, and there had been talk elsewhere about what a Mongrel that ran in an event loop might look like and work like. Turns out it was pretty simple, and it works pretty well.> A big thing about all three solutions is they are auto-configurable in > that you fire up mongrel nodes and they tell the proxy about their > existence. In a shared hosting environment this is a total disaster > for security (since anyone would just join your proxy without proper > encryption and security involved). BUT, for people doing deployed apps > like most of the above authors, this is a really useful feature.Analogger (async logging service) handles this by requiring a client to authenticate itself immediately after connecting. If there''s any demand for it, that''s an easy thing to add to Swiftiply, too.> to be some kind of gate keeper. That let''s people use GPL code all > they want, write commercial solutions, do their own coding style, and > use any enhancers they feel like without having to be "blessed".EM''s license is actually the same as Ruby''s license.> Another point I''d like to make is there''s some great work coming from > the virtual machine camp. I''m not going to try using EventMachine > stuff until I see how well or poorly Mongrel does on some of these new > VMs. I''m hoping that the JRuby and Rubinius folks just fix threads, > IO, and GC once and for all and then everyone can live in a happy > loving land without relying on tons of nasty C++ to fix things.Agreed. The upcoming crop of vms have potential. I''m building apps today, though, and jruby just isn''t a viable option right now. The RAM usage alone is a crippling factor. The C++ issue is a red herring. Factor the comments out of the code, and it''s not particularly large. Somewhat larger than the C code in Mongrel, but not out of line for the work that it does, and it is a mature, stable body of code in use in a variety of places for a lot more applications than just HTTP traffic, on Solaris, Windows, OSX, and Linux (at least). All of that aside, though, the central thing for me with regard to Mongrel is that since I have chosen to support a patched Mongrel as a mechanism for using Ruby frameworks with Swiftiply, and I have chosen to release the evented_mongrel, my intention with those two pieces of the package is for them to be as transparent as possible to any Mongrel handlers. They should just work. If they don''t, I have bugs to fix. Swiftiply and Mongrel are not married to each other, though. Mongrel is just the one of the first of the supported platforms, and the evented_mongrel is a nice extra component that we get basically for free out of the deal. I have setup a mailing list to use for discussion of anything specific to Swiftiply. It is swiftiply-users at rubyforge.org and can be reached here: http://rubyforge.org/mailman/listinfo/swiftiply-users Questions that are specific to Swiftiply or Swiftiply''s Mongrel support are better off over there, I think. I don''t want to pollute Zed''s list here with things that aren''t really about Mongrel itself. I hope this clears up some misimpressions that people may have. Thanks, Kirk Haines
On Tue, 22 May 2007 10:25:32 -0700 "Kirk Haines" <wyhaines at gmail.com> wrote:> On 5/21/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> > to be some kind of gate keeper. That let''s people use GPL code all > > they want, write commercial solutions, do their own coding style, and > > use any enhancers they feel like without having to be "blessed". > > EM''s license is actually the same as Ruby''s license.I wasn''t referring to EM but now that you bring it up, if that''s true then why do I get this: eventmachine-0.7.0> grep -lr "GPL" * lib/em/deferrable.rb lib/em/eventable.rb lib/pr_eventmachine.rb lib/eventmachine_version.rb lib/eventmachine.rb lib/protocols/header_and_content.rb lib/protocols/tcptest.rb lib/protocols/httpclient.rb lib/protocols/line_and_text.rb README tests/test_hc.rb tests/test_ud.rb tests/test_ltp.rb tests/test_httpclient.rb tests/test_basic.rb tests/test_eventables.rb As well as the giant COPYING file with the whole GPL license in it. Until every last one of those statements is gone, it''s GPL licensed. Even worse is the README says LGPL, the license statements on these files says GPLv2, and then everyone thinks it''s Ruby licensed. You folks should do more homework before you use this stuff. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/
On 5/22/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> As well as the giant COPYING file with the whole GPL license in it. > > Until every last one of those statements is gone, it''s GPL licensed. > Even worse is the README says LGPL, the license statements on these > files says GPLv2, and then everyone thinks it''s Ruby licensed. > > You folks should do more homework before you use this stuff.Thanks for the advice. "It will be a disjunctive Ruby/LGPL license. The various source files which indicate pure GPL are in error." I already spoke with Francis Cianfrocca about making sure that an update is done, ASAP, which clears the issue up. Kirk Haines