Hi Zed,
I checked in a pure-Ruby URI classifier to Mongrel trunk. Ola''s Java
port of the TST had some bug, and I don''t think it''s necessary
in the
first place. The Ruby classifier is around 25 lines instead of the
400-odd lines for the C extension and the 200-odd for the Java
extension. It uses a Regexp which is perhaps shady:
@matcher = Regexp.new(routes.map do |uri|
Regexp.new(''^'' + Regexp.escape(uri))
end.join(''|''))
One surprising thing is how fast it is. I wrote a benchmark which
should favor the TST because it''s relatively balanced, and has a high
number of nodes for real-world usage. Rails only installs one route by
default; Camping usually less than five (from what I''ve seen), etc.
--
Pure Ruby, MRI:
19 URIs / 170000 requests
user system total real
1.240000 0.010000 1.250000 ( 1.251764)
.
Finished in 1.255602 seconds.
C/TST, MRI:
19 URIs / 170000 requests
user system total real
0.450000 0.000000 0.450000 ( 0.463202)
.
Finished in 0.464033 seconds.
--
Pure Ruby, JRuby:
19 URIs / 170000 requests
user system total real
3.327000 0.000000 3.327000 ( 3.328000)
.
Finished in 3.403 seconds.
J/TST, JRuby:
19 URIs / 170000 requests
user system total real
2.091000 0.000000 2.091000 ( 2.091000)
.
Finished in 2.194 seconds.
--
The benchmark is at the bottom of test/test_uriclassifier.rb, if you
want to look at it. Another surprise is that the speedup is pretty
much linear around these node counts or less. It looks like the
savings were in the static typing more than the TST.
Do you know anyone who''s running a shitload of Mongrel routes in a
single server? Is there any reason not to keep this change? I know
that TST dude put some patch in his library just for Mongrel, but
still.
Evan
--
Evan Weaver
Cloudburst, LLC
> It uses a Regexp which is perhaps shady:Why not just use Ruby''s Hash function? Is it much slower?> Is there any reason not to keep this change? I know > that TST dude put some patch in his library just for Mongrel, but > still.Are you proposing dumping the C extension? It''s faster and it works. And it''s not 400 lines, it''s probably less than 100. The trie library, ext/http11/tst_*, should be viewed as external to mongrel (included in the distribution only for convenience) . BTW: There has been some talk of making a more general Ruby binding to Zed''s (?) Trei library. Perhaps having both a Ruby and Cext would be good? ry
Hash doesn''t do partial matches. I am proposing dumping the C extension. The Trie library is not external, because I still have to debug the JRuby port, which is broken. There was a maintenance cost before even for the pure C version; we just didn''t have to deal with it ourselves. It could certainly become its own gem, but I don''t think maintaining it in Mongrel is worth the 5 microseconds per request that it saves (less on real hardware). Evan On 10/22/07, ry dahl <ry at tinyclouds.org> wrote:> > It uses a Regexp which is perhaps shady: > > Why not just use Ruby''s Hash function? Is it much slower? > > > Is there any reason not to keep this change? I know > > that TST dude put some patch in his library just for Mongrel, but > > still. > > Are you proposing dumping the C extension? > It''s faster and it works. And it''s not 400 lines, it''s probably less > than 100. The trie library, ext/http11/tst_*, should be viewed as > external to mongrel (included in the distribution only for > convenience) . > > BTW: There has been some talk of making a more general Ruby binding to > Zed''s (?) Trei library. > > Perhaps having both a Ruby and Cext would be good? > > ry >-- Evan Weaver Cloudburst, LLC
It means that pure Ruby is slower in this particular extremely fast section of the request stack. I''ll benchmark the entire stack and report back on that. Yeah, I understand that''s its a big change. On the other hand, I want to get the JRuby gem out soon. We could do a 1.0.2 release for C only and then a 1.1 release right away with the parser changes for both; then people could easily choose which they wanted. Either there are no bugs in the RCs or no one is really using them (more likely), because I haven''t heard anything. I dunno; I''m open to suggestions. Evan On 10/22/07, Luis Lavena <luislavena at gmail.com> wrote:> On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > Hi Zed, > > > > I checked in a pure-Ruby URI classifier to Mongrel trunk. Ola''s Java > > port of the TST had some bug, and I don''t think it''s necessary in the > > first place. The Ruby classifier is around 25 lines instead of the > > 400-odd lines for the C extension and the 200-odd for the Java > > extension. It uses a Regexp which is perhaps shady: > > > > @matcher = Regexp.new(routes.map do |uri| > > Regexp.new(''^'' + Regexp.escape(uri)) > > end.join(''|'')) > > > > One surprising thing is how fast it is. I wrote a benchmark which > > should favor the TST because it''s relatively balanced, and has a high > > number of nodes for real-world usage. Rails only installs one route by > > default; Camping usually less than five (from what I''ve seen), etc. > > > > -- > > > > Pure Ruby, MRI: > > > > 19 URIs / 170000 requests > > user system total real > > 1.240000 0.010000 1.250000 ( 1.251764) > > . > > Finished in 1.255602 seconds. > > > > C/TST, MRI: > > > > 19 URIs / 170000 requests > > user system total real > > 0.450000 0.000000 0.450000 ( 0.463202) > > . > > Finished in 0.464033 seconds. > > > > Ok, that means the Pure Ruby thing is 270% slower? > > All the changes introduced in trunk are too radical to be performed > between Release Candidate and Release. > > This is not the right time to drop something that was working, maybe > doing the work on a branch to later be merged... > > JRuby could wait, maybe we could focus on it *after* 1.0.2 final release. > > But, as I said, maybe is not the right time, and not the right decision. > > -- > Luis Lavena > Multimedia systems > - > Leaders are made, they are not born. They are made by hard effort, > which is the price which all of us must pay to achieve any goal that > is worthwhile. > Vince Lombardi >-- Evan Weaver Cloudburst, LLC
BTW, r721 is the revision before the change, if you want to compare. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> It means that pure Ruby is slower in this particular extremely fast > section of the request stack. I''ll benchmark the entire stack and > report back on that. > > Yeah, I understand that''s its a big change. On the other hand, I want > to get the JRuby gem out soon. We could do a 1.0.2 release for C only > and then a 1.1 release right away with the parser changes for both; > then people could easily choose which they wanted. > > Either there are no bugs in the RCs or no one is really using them > (more likely), because I haven''t heard anything. > > I dunno; I''m open to suggestions. > > Evan > > On 10/22/07, Luis Lavena <luislavena at gmail.com> wrote: > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > Hi Zed, > > > > > > I checked in a pure-Ruby URI classifier to Mongrel trunk. Ola''s Java > > > port of the TST had some bug, and I don''t think it''s necessary in the > > > first place. The Ruby classifier is around 25 lines instead of the > > > 400-odd lines for the C extension and the 200-odd for the Java > > > extension. It uses a Regexp which is perhaps shady: > > > > > > @matcher = Regexp.new(routes.map do |uri| > > > Regexp.new(''^'' + Regexp.escape(uri)) > > > end.join(''|'')) > > > > > > One surprising thing is how fast it is. I wrote a benchmark which > > > should favor the TST because it''s relatively balanced, and has a high > > > number of nodes for real-world usage. Rails only installs one route by > > > default; Camping usually less than five (from what I''ve seen), etc. > > > > > > -- > > > > > > Pure Ruby, MRI: > > > > > > 19 URIs / 170000 requests > > > user system total real > > > 1.240000 0.010000 1.250000 ( 1.251764) > > > . > > > Finished in 1.255602 seconds. > > > > > > C/TST, MRI: > > > > > > 19 URIs / 170000 requests > > > user system total real > > > 0.450000 0.000000 0.450000 ( 0.463202) > > > . > > > Finished in 0.464033 seconds. > > > > > > > Ok, that means the Pure Ruby thing is 270% slower? > > > > All the changes introduced in trunk are too radical to be performed > > between Release Candidate and Release. > > > > This is not the right time to drop something that was working, maybe > > doing the work on a branch to later be merged... > > > > JRuby could wait, maybe we could focus on it *after* 1.0.2 final release. > > > > But, as I said, maybe is not the right time, and not the right decision. > > > > -- > > Luis Lavena > > Multimedia systems > > - > > Leaders are made, they are not born. They are made by hard effort, > > which is the price which all of us must pay to achieve any goal that > > is worthwhile. > > Vince Lombardi > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
The 1.0.2/1.1 split is starting to seem like a good solution to me. It would be nice to be able to say "Mongrel gets faster, not slower", so I''ll see if I can squeeze out some unrelated performance increases to make up for the slower URIClassifier--although I realize that you guys aren''t convinced it''s necessary yet. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> BTW, r721 is the revision before the change, if you want to compare. > > Evan > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > It means that pure Ruby is slower in this particular extremely fast > > section of the request stack. I''ll benchmark the entire stack and > > report back on that. > > > > Yeah, I understand that''s its a big change. On the other hand, I want > > to get the JRuby gem out soon. We could do a 1.0.2 release for C only > > and then a 1.1 release right away with the parser changes for both; > > then people could easily choose which they wanted. > > > > Either there are no bugs in the RCs or no one is really using them > > (more likely), because I haven''t heard anything. > > > > I dunno; I''m open to suggestions. > > > > Evan > > > > On 10/22/07, Luis Lavena <luislavena at gmail.com> wrote: > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > > Hi Zed, > > > > > > > > I checked in a pure-Ruby URI classifier to Mongrel trunk. Ola''s Java > > > > port of the TST had some bug, and I don''t think it''s necessary in the > > > > first place. The Ruby classifier is around 25 lines instead of the > > > > 400-odd lines for the C extension and the 200-odd for the Java > > > > extension. It uses a Regexp which is perhaps shady: > > > > > > > > @matcher = Regexp.new(routes.map do |uri| > > > > Regexp.new(''^'' + Regexp.escape(uri)) > > > > end.join(''|'')) > > > > > > > > One surprising thing is how fast it is. I wrote a benchmark which > > > > should favor the TST because it''s relatively balanced, and has a high > > > > number of nodes for real-world usage. Rails only installs one route by > > > > default; Camping usually less than five (from what I''ve seen), etc. > > > > > > > > -- > > > > > > > > Pure Ruby, MRI: > > > > > > > > 19 URIs / 170000 requests > > > > user system total real > > > > 1.240000 0.010000 1.250000 ( 1.251764) > > > > . > > > > Finished in 1.255602 seconds. > > > > > > > > C/TST, MRI: > > > > > > > > 19 URIs / 170000 requests > > > > user system total real > > > > 0.450000 0.000000 0.450000 ( 0.463202) > > > > . > > > > Finished in 0.464033 seconds. > > > > > > > > > > Ok, that means the Pure Ruby thing is 270% slower? > > > > > > All the changes introduced in trunk are too radical to be performed > > > between Release Candidate and Release. > > > > > > This is not the right time to drop something that was working, maybe > > > doing the work on a branch to later be merged... > > > > > > JRuby could wait, maybe we could focus on it *after* 1.0.2 final release. > > > > > > But, as I said, maybe is not the right time, and not the right decision. > > > > > > -- > > > Luis Lavena > > > Multimedia systems > > > - > > > Leaders are made, they are not born. They are made by hard effort, > > > which is the price which all of us must pay to achieve any goal that > > > is worthwhile. > > > Vince Lombardi > > > > > > > > > -- > > Evan Weaver > > Cloudburst, LLC > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
We could, but that''s more complexity yet. If I optimize the classifier for the pathological case of only one route (Rails), it might be faster than the original C extension for most people. Would that be acceptable? Evan On 10/22/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:> On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > The 1.0.2/1.1 split is starting to seem like a good solution to me. It > > would be nice to be able to say "Mongrel gets faster, not slower", so > > I''ll see if I can squeeze out some unrelated performance increases to > > make up for the slower URIClassifier--although I realize that you guys > > aren''t convinced it''s necessary yet. > > > > Evan > > > Not quite yet, no :) Is this really only necessary to support JRuby? If so, > perhaps we can build in to the installation the ability to compile and use > the extension if able? > > ~Wayne-- Evan Weaver Cloudburst, LLC
> I am proposing dumping the C extension. The Trie library is not > external, because I still have to debug the JRuby port, which is > broken. There was a maintenance cost before even for the pure C > version; we just didn''t have to deal with it ourselvesI don''t know much about java or jruby. Do you have to port the trie library to Java? If so, how will you handle the http parser? ry
The parser is ported. Ola did most of that work although I integrated the two Ragel branches and updated some accessors. Ragel can generate Java now. Ola ported the Trie a while back, but it''s buggy. Evan On 10/22/07, ry dahl <ry at tinyclouds.org> wrote:> > I am proposing dumping the C extension. The Trie library is not > > external, because I still have to debug the JRuby port, which is > > broken. There was a maintenance cost before even for the pure C > > version; we just didn''t have to deal with it ourselves > > I don''t know much about java or jruby. > > Do you have to port the trie library to Java? If so, how will you > handle the http parser? > > ry >-- Evan Weaver Cloudburst, LLC
Optimization is effective, but it adds 7 lines of code.
Ruby, MRI:
1 URIs / 170000 requests
user system total real
0.300000 0.010000 0.310000 ( 0.308417)
Finished in 0.309255 seconds.
--
C/TST, MRI:
1 URIs / 170000 requests
user system total real
0.370000 0.000000 0.370000 ( 0.375385)
Finished in 0.376807 seconds.
Evan
On 10/22/07, Evan Weaver <evan at cloudbur.st>
wrote:> We could, but that''s more complexity yet.
>
> If I optimize the classifier for the pathological case of only one
> route (Rails), it might be faster than the original C extension for
> most people. Would that be acceptable?
>
> Evan
>
> On 10/22/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote:
> > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:
> > > The 1.0.2/1.1 split is starting to seem like a good solution to
me. It
> > > would be nice to be able to say "Mongrel gets faster, not
slower", so
> > > I''ll see if I can squeeze out some unrelated performance
increases to
> > > make up for the slower URIClassifier--although I realize that you
guys
> > > aren''t convinced it''s necessary yet.
> > >
> > > Evan
> >
> >
> > Not quite yet, no :) Is this really only necessary to support JRuby?
If so,
> > perhaps we can build in to the installation the ability to compile and
use
> > the extension if able?
> >
> > ~Wayne
>
>
> --
> Evan Weaver
> Cloudburst, LLC
>
--
Evan Weaver
Cloudburst, LLC
Ry helped come up with a good idea. Because the new Ruby classifier builds a Regexp out of the URI strings, it''s a small step to just supporting Regexp''s directly, which means a framework could forgo its own router in most situations. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> Optimization is effective, but it adds 7 lines of code. > > Ruby, MRI: > > 1 URIs / 170000 requests > user system total real > 0.300000 0.010000 0.310000 ( 0.308417) > Finished in 0.309255 seconds. > > -- > > C/TST, MRI: > > 1 URIs / 170000 requests > user system total real > 0.370000 0.000000 0.370000 ( 0.375385) > Finished in 0.376807 seconds. > > Evan > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > We could, but that''s more complexity yet. > > > > If I optimize the classifier for the pathological case of only one > > route (Rails), it might be faster than the original C extension for > > most people. Would that be acceptable? > > > > Evan > > > > On 10/22/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote: > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > > The 1.0.2/1.1 split is starting to seem like a good solution to me. It > > > > would be nice to be able to say "Mongrel gets faster, not slower", so > > > > I''ll see if I can squeeze out some unrelated performance increases to > > > > make up for the slower URIClassifier--although I realize that you guys > > > > aren''t convinced it''s necessary yet. > > > > > > > > Evan > > > > > > > > > Not quite yet, no :) Is this really only necessary to support JRuby? If so, > > > perhaps we can build in to the installation the ability to compile and use > > > the extension if able? > > > > > > ~Wayne > > > > > > -- > > Evan Weaver > > Cloudburst, LLC > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
More tests show that the benefits of the TST start to kick in around 14 registered URIs. So that would be an argument in favor of the C version if people are often registering that many. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> Ry helped come up with a good idea. Because the new Ruby classifier > builds a Regexp out of the URI strings, it''s a small step to just > supporting Regexp''s directly, which means a framework could forgo its > own router in most situations. > > Evan > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > Optimization is effective, but it adds 7 lines of code. > > > > Ruby, MRI: > > > > 1 URIs / 170000 requests > > user system total real > > 0.300000 0.010000 0.310000 ( 0.308417) > > Finished in 0.309255 seconds. > > > > -- > > > > C/TST, MRI: > > > > 1 URIs / 170000 requests > > user system total real > > 0.370000 0.000000 0.370000 ( 0.375385) > > Finished in 0.376807 seconds. > > > > Evan > > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > We could, but that''s more complexity yet. > > > > > > If I optimize the classifier for the pathological case of only one > > > route (Rails), it might be faster than the original C extension for > > > most people. Would that be acceptable? > > > > > > Evan > > > > > > On 10/22/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote: > > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > > > The 1.0.2/1.1 split is starting to seem like a good solution to me. It > > > > > would be nice to be able to say "Mongrel gets faster, not slower", so > > > > > I''ll see if I can squeeze out some unrelated performance increases to > > > > > make up for the slower URIClassifier--although I realize that you guys > > > > > aren''t convinced it''s necessary yet. > > > > > > > > > > Evan > > > > > > > > > > > > Not quite yet, no :) Is this really only necessary to support JRuby? If so, > > > > perhaps we can build in to the installation the ability to compile and use > > > > the extension if able? > > > > > > > > ~Wayne > > > > > > > > > -- > > > Evan Weaver > > > Cloudburst, LLC > > > > > > > > > -- > > Evan Weaver > > Cloudburst, LLC > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> More tests show that the benefits of the TST start to kick in around > 14 registered URIs. So that would be an argument in favor of the C > version if people are often registering that many.Expect tens to hundreds of routes per Rails app as folks move over to resources. jeremy
> Expect tens to hundreds of routes per Rails app as folks move over to resources.Do rails resources register themselves as handlers with mongrel? or does Rails use Mongrel::URIClassifier? ry
On Mon, 22 Oct 2007, Evan Weaver wrote:> Optimization is effective, but it adds 7 lines of code. >Hey, if you can make the ruby version faster then the c one, I see no problems in changing this for a next version (1.1 or 1.0.3 or whatever we can come with). The C version is some kind of complicated, and when I needed to close a bug in this I asked the upstream maintainer to do this. I also asked because I wanted to keep mongrel and upstream versions in sync. Upstream was kind and closed the bug, but if we had this under our control and written in ruby it would be a lot easier to maintain. filipe { @ icewall.org GPG 1024D/A6BA423E Jabber lautert at jabber.ru }
Can you elaborate? On 10/22/07, Jeremy Kemper <jeremy at bitsweat.net> wrote:> On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > More tests show that the benefits of the TST start to kick in around > > 14 registered URIs. So that would be an argument in favor of the C > > version if people are often registering that many. > > Expect tens to hundreds of routes per Rails app as folks move over to resources. > > jeremy >-- Evan Weaver Cloudburst, LLC
That post was sent just to evan, we need to correct the Reply-To fields for the list... ---------- Forwarded message ---------- From: Luis Lavena <luislavena at gmail.com> Date: Oct 22, 2007 2:30 PM Subject: Re: [Mongrel-development] TST is right out To: Evan Weaver <evan at cloudbur.st> On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> More tests show that the benefits of the TST start to kick in around > 14 registered URIs. So that would be an argument in favor of the C > version if people are often registering that many. >Forget just a second about Rails... the other frameworks... (a) are registering URIs in Mongrel, or (b) just their own dispatcher? If the answer was (a), we should keep the C part, or drop it if it''s (b). -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> Can you elaborate? > > On 10/22/07, Jeremy Kemper <jeremy at bitsweat.net> wrote: > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > More tests show that the benefits of the TST start to kick in around > > > 14 registered URIs. So that would be an argument in favor of the C > > > version if people are often registering that many. > > > > Expect tens to hundreds of routes per Rails app as folks move over to resources. > > > > jeremy > >We should differentiate the dispatcher rails use and the URI mongrel uses... URIs registered with Mongrel are the handlers, and unless you hook upload_progress, merb in parasite mode and "/" for rails handler, there is no other hook. Rails has it''s own URI classifier and dispatcher, the same goes for Merb... So... -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
Right... for current Rails. I thought Jeremy was saying they were splitting up the internal dispatcher. But that wouldn''t make sense for Webrick, Glassfish et al, I suppose. Evan On 10/22/07, Luis Lavena <luislavena at gmail.com> wrote:> On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > Can you elaborate? > > > > On 10/22/07, Jeremy Kemper <jeremy at bitsweat.net> wrote: > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > > More tests show that the benefits of the TST start to kick in around > > > > 14 registered URIs. So that would be an argument in favor of the C > > > > version if people are often registering that many. > > > > > > Expect tens to hundreds of routes per Rails app as folks move over to resources. > > > > > > jeremy > > > > > We should differentiate the dispatcher rails use and the URI mongrel > uses... URIs registered with Mongrel are the handlers, and unless you > hook upload_progress, merb in parasite mode and "/" for rails handler, > there is no other hook. > > Rails has it''s own URI classifier and dispatcher, the same goes for > Merb... So... > > -- > Luis Lavena > Multimedia systems > - > Leaders are made, they are not born. They are made by hard effort, > which is the price which all of us must pay to achieve any goal that > is worthwhile. > Vince Lombardi > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development >-- Evan Weaver Cloudburst, LLC
Current plan then as discussed on IRC and elsewhere: Rollback trunk, make a 1.0.2 release, and then once JRuby fixes their socket bugs, make a 1.1 release with the new classifier and a separate ''mongrel_performance'' gem that includes the C extension. Then people can install the C extension if they need lots of routes, but it won''t be required for JRuby and the features in the mongrel_performance gem won''t have to march in lockstep with the reference implementation. Ry''s query parser will also go in mongrel_performance once it is ready, again, with a reference pure Ruby implementation for Windows, MRI and JRuby in core. This lets us do things like test out features in pure Ruby first before committing to implementing them in C and Java, as well as include more experimental or platform-specific hacks in the mongrel_performance gem. If some enterprising person wants to write a mongrel_performance-jruby gem, they can do it, but we don''t necessarily have to maintain it. Thoughts? I make it sound more complicated than it really is. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> Right... for current Rails. I thought Jeremy was saying they were > splitting up the internal dispatcher. But that wouldn''t make sense for > Webrick, Glassfish et al, I suppose. > > Evan > > > On 10/22/07, Luis Lavena <luislavena at gmail.com> wrote: > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > Can you elaborate? > > > > > > On 10/22/07, Jeremy Kemper <jeremy at bitsweat.net> wrote: > > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > > > More tests show that the benefits of the TST start to kick in around > > > > > 14 registered URIs. So that would be an argument in favor of the C > > > > > version if people are often registering that many. > > > > > > > > Expect tens to hundreds of routes per Rails app as folks move over to resources. > > > > > > > > jeremy > > > > > > > > We should differentiate the dispatcher rails use and the URI mongrel > > uses... URIs registered with Mongrel are the handlers, and unless you > > hook upload_progress, merb in parasite mode and "/" for rails handler, > > there is no other hook. > > > > Rails has it''s own URI classifier and dispatcher, the same goes for > > Merb... So... > > > > -- > > Luis Lavena > > Multimedia systems > > - > > Leaders are made, they are not born. They are made by hard effort, > > which is the price which all of us must pay to achieve any goal that > > is worthwhile. > > Vince Lombardi > > _______________________________________________ > > Mongrel-development mailing list > > Mongrel-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-development > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> > Current plan then as discussed on IRC and elsewhere: > > Rollback trunk, make a 1.0.2 release, and then once JRuby fixes their > socket bugs, make a 1.1 release with the new classifier and a separate > ''mongrel_performance'' gem that includes the C extension. Then people > can install the C extension if they need lots of routes, but it won''t > be required for JRuby and the features in the mongrel_performance gem > won''t have to march in lockstep with the reference implementation. > > Ry''s query parser will also go in mongrel_performance once it is > ready, again, with a reference pure Ruby implementation for Windows, > MRI and JRuby in core. > > This lets us do things like test out features in pure Ruby first > before committing to implementing them in C and Java, as well as > include more experimental or platform-specific hacks in the > mongrel_performance gem. > > If some enterprising person wants to write a mongrel_performance-jruby > gem, they can do it, but we don''t necessarily have to maintain it. > > Thoughts? I make it sound more complicated than it really is. > > Evan >So we really want a separate gem rather than having mongrel attempt to use the high-performance parts first and rescue to Ruby? Just asking one last time ;) ~Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-development/attachments/20071022/86bcff10/attachment.html
I would prefer it. Otherwise the C extension always has to exactly match the feature set of the Ruby reference implementation, whereas with separate gems people will understand if the C extension lags behind or takes shortcuts for performance. I''d like some other people''s opinions though. Evan On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote:> I would prefer it. Otherwise the C extension always has to exactly > match the feature set of the Ruby reference implementation, whereas > with separate gems people will understand if the C extension lags > behind or takes shortcuts for performance. > > I''d like some other people''s opinions though. > > Evan > > On 10/22/07, Wayne E. Seguin <wayneeseguin at gmail.com> wrote: > > > > > > On 10/22/07, Evan Weaver <evan at cloudbur.st> wrote: > > > Current plan then as discussed on IRC and elsewhere: > > > > > > Rollback trunk, make a 1.0.2 release, and then once JRuby fixes their > > > socket bugs, make a 1.1 release with the new classifier and a separate > > > ''mongrel_performance'' gem that includes the C extension. Then people > > > can install the C extension if they need lots of routes, but it won''t > > > be required for JRuby and the features in the mongrel_performance gem > > > won''t have to march in lockstep with the reference implementation. > > > > > > Ry''s query parser will also go in mongrel_performance once it is > > > ready, again, with a reference pure Ruby implementation for Windows, > > > MRI and JRuby in core. > > > > > > This lets us do things like test out features in pure Ruby first > > > before committing to implementing them in C and Java, as well as > > > include more experimental or platform-specific hacks in the > > > mongrel_performance gem. > > > > > > If some enterprising person wants to write a mongrel_performance-jruby > > > gem, they can do it, but we don''t necessarily have to maintain it. > > > > > > Thoughts? I make it sound more complicated than it really is. > > > > > > Evan > > > > > > > So we really want a separate gem rather than having mongrel attempt to use > > the high-performance parts first and rescue to Ruby? Just asking one last > > time ;) > > > > ~Wayne > > _______________________________________________ > > Mongrel-development mailing list > > Mongrel-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-development > > > > > > > -- > Evan Weaver > Cloudburst, LLC >-- Evan Weaver Cloudburst, LLC
On Mon, 22 Oct 2007, Evan Weaver wrote:> I would prefer it. Otherwise the C extension always has to exactly > match the feature set of the Ruby reference implementation, whereas > with separate gems people will understand if the C extension lags > behind or takes shortcuts for performance. > > I''d like some other people''s opinions though. >other person here ;) For me it''s cool to have it in a different gem - also if we found a bug in it we can only release this specific gem and do not need to release the entire mongrel. filipe { @ icewall.org GPG 1024D/A6BA423E Jabber lautert at jabber.ru }
> I don''t know about you, but my experience with people tells me that > when they see a gem called mongrel_performance they''d go "ooooh, my > awsum blawg can has needz hi-perfurmance",You''re right. What if we give it a boring or scary name instead? ''mongrel_experimental'' or something. Evan On 10/23/07, Filipe <filipe at icewall.org> wrote:> On Mon, 22 Oct 2007, Evan Weaver wrote: > > > I would prefer it. Otherwise the C extension always has to exactly > > match the feature set of the Ruby reference implementation, whereas > > with separate gems people will understand if the C extension lags > > behind or takes shortcuts for performance. > > > > I''d like some other people''s opinions though. > > > > other person here ;) > > For me it''s cool to have it in a different gem - also if we found a bug > in it we can only release this specific gem and do not need to release the > entire mongrel. > > > filipe { > @ icewall.org > GPG 1024D/A6BA423E > Jabber lautert at jabber.ru > } >-- Evan Weaver Cloudburst, LLC
On 10/23/07, Evan Weaver <evan at cloudbur.st> wrote:> > > I don''t know about you, but my experience with people tells me that > > when they see a gem called mongrel_performance they''d go "ooooh, my > > awsum blawg can has needz hi-perfurmance", > > You''re right. What if we give it a boring or scary name instead? > ''mongrel_experimental'' or something. >I was just making sure that''s the route we wanted to take. Now that we''re taking that route, I''m thinking that we should create two new gems: mongrel_treats - Known performance boosting C extensions mongrel_chocolate - The dog likes it but it may not be good for him at all (aka "experimental") What do you all think? ~Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-development/attachments/20071023/9c8bb63d/attachment.html
> mongrel_treats - Known performance boosting C extensions > > mongrel_chocolate - The dog likes it but it may not be good for him at all > (aka "experimental")I love the nomenclature - what do you have in mind for these? We don''t have soo many performance things. ALSO: Evan, the escape/unescape functions written in C are very simple and fast, and probably should be in the http11 core. It shouldn''t be a problem to port them to java, I think. ry
And again, just sent this to Evan! Zed: you''re admin, please correct this! (I always forget to change the To field!) On 10/23/07, Evan Weaver <evan at cloudbur.st> wrote:> > I don''t know about you, but my experience with people tells me that > > when they see a gem called mongrel_performance they''d go "ooooh, my > > awsum blawg can has needz hi-perfurmance", > > You''re right. What if we give it a boring or scary name instead? > ''mongrel_experimental'' or something. >That sounds better than "cgi_multipart_eof_fix" :-P Try type that in the terminal fast enough and your finger will feel the pain ;-) So the "mongrel_experimental_extension" should also depend on gem_plugin, or we could leave the whole mambo about it off the table? # if mongrel_experimental_extension is present, use it begin require ''mongrel_experimental_extension/something'' rescue LoadError require ''pure_ruby_classiffier'' end Or something like that... (don''t have a ruby handy ;-) -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
Perhaps we should call it "bone". Evan On 10/23/07, ry dahl <ry at tinyclouds.org> wrote:> > mongrel_treats - Known performance boosting C extensions > > > > mongrel_chocolate - The dog likes it but it may not be good for him at all > > (aka "experimental") > > I love the nomenclature - what do you have in mind for these? We don''t > have soo many performance things. > > ALSO: Evan, the escape/unescape functions written in C are very simple > and fast, and probably should be in the http11 core. It shouldn''t be a > problem to port them to java, I think. > > ry > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development >-- Evan Weaver Cloudburst, LLC
I''ll try both at some point (classifier into CRagel, parser into Ruby Ragel). The Ruby URIClassifier is down in the realm of 6 dispatches though, so that is definitely a theoretical exercise rather than a practical one, but that''s ok. Thanks for the admin access. Wayne, I think we need only one extra gem. Don''t want to get carried away. Anyway... I think this discussion is basically resolved. I''ll try to grind out 1.0.2 and 1.1 releases by RubyConf. Luis, what do I have to do to package win32 builds? Then I can stop breaking your packagin scripts. I have Parallels and all that; I can get Visual Studio 6 if necessary. Have fun in Rome, Zed. Evan On 10/23/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> On Mon, 22 Oct 2007 04:21:55 -0400 > "Evan Weaver" <evan at cloudbur.st> wrote: > > > Hi Zed, > > > > I checked in a pure-Ruby URI classifier to Mongrel trunk. Ola''s Java > > port of the TST had some bug, and I don''t think it''s necessary in the > > first place. The Ruby classifier is around 25 lines instead of the > > 400-odd lines for the C extension and the 200-odd for the Java > > extension. It uses a Regexp which is perhaps shady: > > Sorry, haven''t had internet for a loooong time. Rome blows for internet access. > > If this works and you can get rid of more .c code then go for it. It was originally because the Rails routing sucked so bad that I wanted to make sure mongrel could route reliably. In the end it was overkill most likely. > > Only thing is, Ragel produces a regex, is verifiable, and has a Ruby generator. You could probably use that to parse the URL for recognition using the existing grammar for URIs in the C parser. > > In addition, why not play with Ragel''s Ruby output and see if you can kill C entirely? That''d make a bunch of folks happy, and you can include the .c extension version for the paranoid. > > -- > Zed A. Shaw > - Hate: http://savingtheinternetwithhate.com/ > - Good: http://www.zedshaw.com/ > - Evil: http://yearofevil.com/ >-- Evan Weaver Cloudburst, LLC
On 10/23/07, Evan Weaver <evan at cloudbur.st> wrote:> I''ll try both at some point (classifier into CRagel, parser into Ruby > Ragel). The Ruby URIClassifier is down in the realm of 6 dispatches > though, so that is definitely a theoretical exercise rather than a > practical one, but that''s ok. > > Thanks for the admin access. > > Wayne, I think we need only one extra gem. Don''t want to get carried away. > > Anyway... I think this discussion is basically resolved. I''ll try to > grind out 1.0.2 and 1.1 releases by RubyConf. > > Luis, what do I have to do to package win32 builds? Then I can stop > breaking your packagin scripts. I have Parallels and all that; I can > get Visual Studio 6 if necessary. >Hehehe, I''ll mail you off the list later today.> Have fun in Rome, Zed. >I didn''t know he was in Italy! :-D Regards and have fun everybody (specially Zed). PS: Please, someone with ML password could fix the reply-to thing?) Thanks! :-D -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi
On 10/23/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> In addition, why not play with Ragel''s Ruby output and see if you can kill C entirely? > That''d make a bunch of folks happy, and you can include the .c extension version for the paranoid.That''s a great idea. I''ll try it tomorrow. I''m interested what sort of speed increase ragel has when compiling into Ruby. ry