On 10/23/07, Robert Mela <rob at robmela.com> wrote: > Several years back I accidentally discovered that multiple processes can > listen on the same TCP/IP socket. The trick of course is that all the > processes are in the same process group, and the socket is opened by a > shared parent. The OS somehow was managing queuing up the various > calls to accept() on that socket. Since the watchdog parent / multiple > child servers is a common, this was a workable solution on the versions > of Linux we were using. [snip] At 05:09 AM 10/24/2007, Wayne wrote:>I believe that the main issue here is on the win32 platform, Luis? > >We do have something similar in the works for a future release, >however I am >unsure as to how your suggestion ties in at the moment. It appears to >be >well worth investigation for what we have planned. > >Thank you kindly for this, > > ~WayneHi, I thought I''d chime in as a Windows developer. I''m running Windows in development mode and deploying to Linux (Ezra''s rig at EngineYard actually). If this idea actually works, it''s super appealing for me as a simple solution to the "slow returning mongrel" load balancing challenge. (i.e. the problems where mongrels are loaded up with requests irrespective of their availability, leaving some mongrels idle and some overloaded.) I''ll give you a simple example: we have to build admin tools periodically. It''s possible to make these tools spin off background processes but that''s more time consuming to build and debug. Since these tools are just basic admin utilities that we want to invest as little time as possible in them: their initial functionality is 95% of their overall value. So for example one of these tools lets us bulk manage photos and can take 10 seconds to return. Because of mongrel/nginx/load balance architecture, we can run into performance trouble if we aren''t careful at what time of day we use the tool. You can complain about my sloppy software architecture but I''ll complain that I''m running a business and need to invest most in what returns the most value to our customers. But I''d prefer to just agree not to start that discussion. [smile] If I understand what''s being discussed (not a given) the system Robert is proposing would mean that a mongrel would only get requests from port 80 when it was ready to call "receive()" again from the IP stack. So mongrels would consume what they can off a common stack. This would mean that I don''t have to balance all my Rails processes to keep them returning results at roughly equal intervals. If my servers are heavily loaded now, it seems like a slow returning mongrel can cause havoc even if it''s just slow by a couple of seconds.. So given the mongrels a common pool of requests to consume from and using such a low level pool such as an IP port seems great, and I''d vote for it. The fact that my development box doesn''t run against the same code is totally ok. I''m sure Windows people who run in production will have other opinions but this idea, if it''s practical, seems very elegant and useful. +1 Steve
Steve Midgley wrote:> > Hi, > > I thought I''d chime in as a Windows developer. I''m running Windows in > development mode and deploying to Linux (Ezra''s rig at EngineYard > actually). > > If this idea actually works, it''s super appealing for me as a simple > solution to the "slow returning mongrel" load balancing challenge. > (i.e. the problems where mongrels are loaded up with requests > irrespective of their availability, leaving some mongrels idle and some > overloaded.) > >If you are using mod_proxy_balancer then I''m fairly sure that "the slow returning mongrel" challenge has been solved using "max" and "acquire" parameters to BalancerMember. It''s working for me, and I tested using a pool of Nitro servers using controller method that sleeps. I was hoping that Rafael G. would report back with his results on this with is issue ( though with "conn refused" I think his issue may be a little different ). Here''s the product of my devinations and experiments: <Proxy balancer://myserverpool> # no trailing slash BalancerMember http://192.168.10.10:10000 keepalive=on max=1 lbset=0 acquire=1 timeout=1 BalancerMember http://192.168.10.10:10001 keepalive=on max=1 lbset=1 acquire=1 timeout=1 BalancerMember http://192.168.10.10:10002 keepalive=on max=1 lbset=0 acquire=1 timeout=1 </Proxy>> If I understand what''s being discussed (not a given) the system Robert > is proposing would mean that a mongrel would only get requests from > port 80 when it was ready to call "receive()" again from the IP stack. > So mongrels would consume what they can off a common stack. > >Yes. Minor nit -- each process is in an accept() loop. Accept a connection, process the request (it''s in processing that the read occurs). After processing and then closing the socket to the client, enter accept() to wait for another request.> This would mean that I don''t have to balance all my Rails processes to > keep them returning results at roughly equal intervals. If my servers > are heavily loaded now, it seems like a slow returning mongrel can > cause havoc even if it''s just slow by a couple of seconds.. > >Servers within the pool would auto-balance. You''d still have to balance across pools if they''re running on multiple hosts. But even so, yes, this is an improvement. Swiftiply as I understand it also addresses the Mongrel/Rails request queuing problem -- I haven''t researched it yet. The max/acquire settings are working for us, and I experimented with a sleeping controller method to verify that it works.> So given the mongrels a common pool of requests to consume from and > using such a low level pool such as an IP port seems great, and I''d > vote for it. The fact that my development box doesn''t run against the > same code is totally ok. > >Is there a cheap or demo version of Visual Studio I could try this on? I''ve got XP Home Edition running under VMWare, and could code a simple test to see if this works there. While I''m at it, I could verify that this works on Mac OSX as well..> I''m sure Windows people who run in production will have other opinions > but this idea, if it''s practical, seems very elegant and useful. +1 > > Steve > >It may be a horrendous hack for reasons I''m unaware of. But it worked, and in a pretty demanding scenario, essentially unchanged for eight years and counting ( ZDNet, then CNET ad servers ) so I didn''t research it too deeply. I looked for problems like immediate error returns from accept(), deadlocks, delays with growing queues of accept() callers, but didn''t find any. I keep meaning to read through Linux'' TCP/IP stack to find out why it was so. Maybe there''s a non-standard quirk in there that makes it all just dumb luck... darn... now I need to know...> _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > > >-------------- next part -------------- A non-text attachment was scrubbed... Name: rob.vcf Type: text/x-vcard Size: 116 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20071024/cc998c48/attachment.vcf
Steve Midgley wrote:> Hi, > > I thought I''d chime in as a Windows developer. I''m running Windows in > development mode and deploying to Linux (Ezra''s rig at EngineYard > actually). > > If this idea actually works, it''s super appealing for me as a simple > solution to the "slow returning mongrel" load balancing challenge. > (i.e. the problems where mongrels are loaded up with requests > irrespective of their availability, leaving some mongrels idle and some > overloaded.) >I found out why it works on Linux 2.6.20. In net/ipv4/tcp_ipv4.c accept is mapped to inet_csk_accept in net/ipv4/inet_connection_sock.c. That csk accept calls inet_csk_wait_for_connect to wait for a connection. That wait for connect function locks the socket so that if multiple processes are calling accept() on that socket, only one process at a time will be awaken when a connection request arrives. This wil require some change in Mongrel''s request loop. Mongrel currently reenters accept even if it''s got a worker locking Rails. If Mongrels in the pool are calling accept while still locking Rals then we''re right back to square one, request queing. Mongrel would need to either poll for worker list length == 0 before calling accept, or have a "single-worker" mode in which it joins a worker thread as soon as it''s started, and only calls accept() after returning from that join. -------------- next part -------------- A non-text attachment was scrubbed... Name: rob.vcf Type: text/x-vcard Size: 116 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20071024/489dfca1/attachment.vcf