Hi Mongrels, I''m working on trunk and would like to change the following things and release it as a 2.0rc. - Timestamped logging - 1.9.1rc compatibility - Early PID drop - Error code instead of connection-close, as discussed earlier - Option to fork and run from a shared socket on Linux instead of queueing - Move to a Rack handler interface, and include backwards-compatible versions of the current ones - Eliminate gem_plugins - Other random bugfixes from off the trac and github I''m unsure of what to do about mongrel_cluster. Do people still use that? Only for the --clean option? It would be nice if it could go away. At a later date, I would like to include optional ESI parsing to help people''s dev environments. Please fuss at me if these are bad. Evan -- Evan Weaver
Also, I will document the release process. Evan On Mon, Nov 17, 2008 at 3:32 PM, Evan Weaver <evan at cloudbur.st> wrote:> Hi Mongrels, > > I''m working on trunk and would like to change the following things and > release it as a 2.0rc. > > - Timestamped logging > - 1.9.1rc compatibility > - Early PID drop > - Error code instead of connection-close, as discussed earlier > - Option to fork and run from a shared socket on Linux instead of queueing > - Move to a Rack handler interface, and include backwards-compatible > versions of the current ones > - Eliminate gem_plugins > - Other random bugfixes from off the trac and github > > I''m unsure of what to do about mongrel_cluster. Do people still use > that? Only for the --clean option? It would be nice if it could go > away. > > At a later date, I would like to include optional ESI parsing to help > people''s dev environments. > > Please fuss at me if these are bad. > > Evan > > -- > Evan Weaver >-- Evan Weaver
These all look good to me. If you move the --clean option into mongrel proper then we no longer need mongrel_cluster afaik. Cheers- -Ezra On Nov 17, 2008, at 12:33 PM, Evan Weaver wrote:> Also, I will document the release process. > > Evan > > On Mon, Nov 17, 2008 at 3:32 PM, Evan Weaver <evan at cloudbur.st> wrote: >> Hi Mongrels, >> >> I''m working on trunk and would like to change the following things >> and >> release it as a 2.0rc. >> >> - Timestamped logging >> - 1.9.1rc compatibility >> - Early PID drop >> - Error code instead of connection-close, as discussed earlier >> - Option to fork and run from a shared socket on Linux instead of >> queueing >> - Move to a Rack handler interface, and include backwards-compatible >> versions of the current ones >> - Eliminate gem_plugins >> - Other random bugfixes from off the trac and github >> >> I''m unsure of what to do about mongrel_cluster. Do people still use >> that? Only for the --clean option? It would be nice if it could go >> away. >> >> At a later date, I would like to include optional ESI parsing to help >> people''s dev environments. >> >> Please fuss at me if these are bad. >> >> Evan >> >> -- >> Evan Weaver >> > > > > -- > Evan Weaver > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-developmentEzra Zygmuntowicz ez at engineyard.com
Someone is doing dev work? whow Nice mate, go ahead! I can help after the first rc to find bugs/close them. Cheers filipe Ezra Zygmuntowicz wrote:> > These all look good to me. If you move the --clean option into > mongrel proper then we no longer need mongrel_cluster afaik. > > Cheers- > -Ezra > > > On Nov 17, 2008, at 12:33 PM, Evan Weaver wrote: > >> Also, I will document the release process. >> >> Evan >> >> On Mon, Nov 17, 2008 at 3:32 PM, Evan Weaver <evan at cloudbur.st> wrote: >>> Hi Mongrels, >>> >>> I''m working on trunk and would like to change the following things and >>> release it as a 2.0rc. >>> >>> - Timestamped logging >>> - 1.9.1rc compatibility >>> - Early PID drop >>> - Error code instead of connection-close, as discussed earlier >>> - Option to fork and run from a shared socket on Linux instead of >>> queueing >>> - Move to a Rack handler interface, and include backwards-compatible >>> versions of the current ones >>> - Eliminate gem_plugins >>> - Other random bugfixes from off the trac and github >>> >>> I''m unsure of what to do about mongrel_cluster. Do people still use >>> that? Only for the --clean option? It would be nice if it could go >>> away. >>> >>> At a later date, I would like to include optional ESI parsing to help >>> people''s dev environments. >>> >>> Please fuss at me if these are bad. >>> >>> Evan >>> >>> -- >>> Evan Weaver >>> >> >> >> >> -- >> Evan Weaver >> _______________________________________________ >> Mongrel-development mailing list >> Mongrel-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-development > > Ezra Zygmuntowicz > ez at engineyard.com > > > > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development
Very cool, nice work. One small comment... On Mon, Nov 17, 2008 at 03:32:33PM -0500, Evan Weaver wrote:> Hi Mongrels, > > - Error code instead of connection-close, as discussed earlierWith this do you mean returning one of the many error codes in an HTTP response when the server is overloaded? I would recommend against that if that''s the case. In theory it''s nice to return an error code to clients that could be waiting. In practice, your queue is already overloaded, so taking more time to send a response to clients only adds to the problem. In a modern system this isn''t such a big deal, but in Ruby it becomes a mess because of the file id limits in the select loop. I''d say, if you add that, make it optional to turn it off and be prepared with a FAQ about it in case people have problems. Have fun. -- Zed A. Shaw http://freehackersunion.org/
Optional is a good compromise. Evan On Tue, Nov 18, 2008 at 12:36 AM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> Very cool, nice work. One small comment... > > On Mon, Nov 17, 2008 at 03:32:33PM -0500, Evan Weaver wrote: >> Hi Mongrels, >> >> - Error code instead of connection-close, as discussed earlier > > With this do you mean returning one of the many error codes in an HTTP > response when the server is overloaded? > > I would recommend against that if that''s the case. In theory it''s nice > to return an error code to clients that could be waiting. In practice, > your queue is already overloaded, so taking more time to send a response > to clients only adds to the problem. In a modern system this isn''t such > a big deal, but in Ruby it becomes a mess because of the file id limits > in the select loop. > > I''d say, if you add that, make it optional to turn it off and be > prepared with a FAQ about it in case people have problems. > > Have fun. > > -- > Zed A. Shaw > http://freehackersunion.org/ > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development >-- Evan Weaver
On Mon, Nov 17, 2008 at 10:36 PM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> Very cool, nice work. One small comment... > > On Mon, Nov 17, 2008 at 03:32:33PM -0500, Evan Weaver wrote: >> Hi Mongrels, >> >> - Error code instead of connection-close, as discussed earlier > > With this do you mean returning one of the many error codes in an HTTP > response when the server is overloaded? > > I would recommend against that if that''s the case. In theory it''s nice > to return an error code to clients that could be waiting. In practice, > your queue is already overloaded, so taking more time to send a response > to clients only adds to the problem. In a modern system this isn''t such > a big deal, but in Ruby it becomes a mess because of the file id limits > in the select loop.In an overload situation, I think that just dropping the connection is an acceptable thing to do, though. The mongrel is overloaded, so it is a correct and beneficial response on the part of the proxy to assume something is wrong with it, and to stop sending traffic to it for a while. The use case for sending a response instead of just dropping the connection is when the HTTP parser throws an exception because of malformed HTTP. Currently that results in an immediate closed connection. The problem there is that upstream proxies can assume that getting the connection cut unexpectedly means that the mongrel itself has a problem. If that proxy responds to this by removing that mongrel from the proxy rotation for a period of time, one misbehaving client, whether intentionally or unintentionally, can DOS a whole cluster. The fix is to just return a canned 400 response before closing. Kirk Haines
Whether the mongrel is overloaded such that responding further degrades service depends on whether you are blocked on local CPU or on a shared resource. In my experience, shared resource problems are more common. Evan On 11/18/08, Kirk Haines <wyhaines at gmail.com> wrote:> On Mon, Nov 17, 2008 at 10:36 PM, Zed A. Shaw <zedshaw at zedshaw.com> wrote: >> Very cool, nice work. One small comment... >> >> On Mon, Nov 17, 2008 at 03:32:33PM -0500, Evan Weaver wrote: >>> Hi Mongrels, >>> >>> - Error code instead of connection-close, as discussed earlier >> >> With this do you mean returning one of the many error codes in an HTTP >> response when the server is overloaded? >> >> I would recommend against that if that''s the case. In theory it''s nice >> to return an error code to clients that could be waiting. In practice, >> your queue is already overloaded, so taking more time to send a response >> to clients only adds to the problem. In a modern system this isn''t such >> a big deal, but in Ruby it becomes a mess because of the file id limits >> in the select loop. > > In an overload situation, I think that just dropping the connection is > an acceptable thing to do, though. The mongrel is overloaded, so it > is a correct and beneficial response on the part of the proxy to > assume something is wrong with it, and to stop sending traffic to it > for a while. > > The use case for sending a response instead of just dropping the > connection is when the HTTP parser throws an exception because of > malformed HTTP. > > Currently that results in an immediate closed connection. The problem > there is that upstream proxies can assume that getting the connection > cut unexpectedly means that the mongrel itself has a problem. If that > proxy responds to this by removing that mongrel from the proxy > rotation for a period of time, one misbehaving client, whether > intentionally or unintentionally, can DOS a whole cluster. > > The fix is to just return a canned 400 response before closing. > > > Kirk Haines > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development >-- Evan Weaver
Evan, I believe that optional with default being to drop the connection would be best in this case. It will serve the 98% case just fine and for those who are more demanding they can opt in. The rest look like a good set for a 2.0 release to me. ~Wayne On Nov 18, 2008, at 00:49 , Evan Weaver wrote:> Optional is a good compromise. > > Evan > > On Tue, Nov 18, 2008 at 12:36 AM, Zed A. Shaw <zedshaw at zedshaw.com> > wrote: >> Very cool, nice work. One small comment... >> >> On Mon, Nov 17, 2008 at 03:32:33PM -0500, Evan Weaver wrote: >>> Hi Mongrels, >>> >>> - Error code instead of connection-close, as discussed earlier >> >> With this do you mean returning one of the many error codes in an >> HTTP >> response when the server is overloaded? >> >> I would recommend against that if that''s the case. In theory it''s >> nice >> to return an error code to clients that could be waiting. In >> practice, >> your queue is already overloaded, so taking more time to send a >> response >> to clients only adds to the problem. In a modern system this isn''t >> such >> a big deal, but in Ruby it becomes a mess because of the file id >> limits >> in the select loop. >> >> I''d say, if you add that, make it optional to turn it off and be >> prepared with a FAQ about it in case people have problems. >> >> Have fun. >> >> -- >> Zed A. Shaw >> http://freehackersunion.org/ >> _______________________________________________ >> Mongrel-development mailing list >> Mongrel-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-development >> > > > > -- > Evan Weaver > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development
On Mon, Nov 17, 2008 at 6:32 PM, Evan Weaver <evan at cloudbur.st> wrote:> Hi Mongrels, > > I''m working on trunk and would like to change the following things and > release it as a 2.0rc. > > - Timestamped logging > - 1.9.1rc compatibility > - Early PID drop > - Error code instead of connection-close, as discussed earlier > - Option to fork and run from a shared socket on Linux instead of queueingCan you give more details on this?> - Move to a Rack handler interface, and include backwards-compatible > versions of the current onesHow this will overlap / affect / interfere Rack and it''s own handler for mongrel?> - Eliminate gem_plugins > - Other random bugfixes from off the trac and github >I''ll really liek to have mongrel repository moved to git prior the release, I lost track of all the branches and what is current the past months...> I''m unsure of what to do about mongrel_cluster. Do people still use > that? Only for the --clean option? It would be nice if it could go > away. > > At a later date, I would like to include optional ESI parsing to help > people''s dev environments. > > Please fuss at me if these are bad. >Most of thos looks good, but hearing more the details of some points details will help me a bit. For the record, I''m working on some rake helpers to ease the building of all the gems and also cross compilation, with the goal that noone needs me or Windows to be able to release gems for the platform. -- Luis Lavena AREA 17 - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
>> - Option to fork and run from a shared socket on Linux instead of queueing > > Can you give more details on this?On Linux (and some other *nixes), there''s a funky little thing that happens when a process that is bound to a socket is forked. All of the forked processes are also bound to the socket, and the kernel will distribute connections between them. It''s a low cost way to load balance between workers on a machine that is running an OS which will support it. Kirk Haines
On Wed, Nov 19, 2008 at 9:39 PM, Kirk Haines <wyhaines at gmail.com> wrote:> On Linux (and some other *nixes), there''s a funky little thing that > happens when a process that is bound to a socket is forked. > > All of the forked processes are also bound to the socket, and the > kernel will distribute connections between them. It''s a low cost way > to load balance between workers on a machine that is running an OS > which will support it. >Prefork servers should be fairly portable. Any reason why you aren''t trying this on other platforms? -- Tony Arcieri medioh.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/mongrel-development/attachments/20081119/ce426e41/attachment-0001.html>
I was under the impression that it wasn''t portable. If it is, that would be great. Evan On Wed, Nov 19, 2008 at 9:54 PM, Tony Arcieri <tony at medioh.com> wrote:> On Wed, Nov 19, 2008 at 9:39 PM, Kirk Haines <wyhaines at gmail.com> wrote: >> >> On Linux (and some other *nixes), there''s a funky little thing that >> happens when a process that is bound to a socket is forked. >> >> All of the forked processes are also bound to the socket, and the >> kernel will distribute connections between them. It''s a low cost way >> to load balance between workers on a machine that is running an OS >> which will support it. > > Prefork servers should be fairly portable. Any reason why you aren''t trying > this on other platforms? > > -- > Tony Arcieri > medioh.com > > _______________________________________________ > Mongrel-development mailing list > Mongrel-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-development > >-- Evan Weaver
Evan Weaver <evan at cloudbur.st> wrote:> Hi Mongrels,Hi Evan,> I''m working on trunk and would like to change the following things and > release it as a 2.0rc.Is this still happening in SVN? I haven''t seen anything (of course, you may be busy, as I haven''t checked this list in nearly a month, either :x)> - 1.9.1rc compatibilityWill we be trying different connection/thread models to (possibly) work better with the 1.9 and its use of native threading? Fibers, maybe? or just use async sockets in our code explicitly... -- Eric Wong