Streaming works with Unicorn + Apache. Both with and without deflating. My understanding is that Unicorn + Apache is not a good combination though because Apache does not buffer, and thus Unicorn has no fast client in front. (I don''t know which is the ultimate technical reason Unicorn puts such an emphasis on fast clients, but will do some research about it.) I have seen in http://unicorn.bogomips.org/examples/nginx.conf the comment "You normally want nginx to buffer responses to slow clients, even with Rails 3.1 streaming because otherwise a slow client can become a bottleneck of Unicorn." If I understand how this works correctly, nginx buffers the entire response from Unicorn. First filling what''s configured in proxy_buffer_size and proxy_buffers, and then going to disk if needed as a last resort. Thus, even if the application streams, I believe the client will receive the chunked response, but only after it has been generated by the application and fully buffered by nginx. Which defeats the purpose of streaming in the use case we have in mind in Rails 3.1, which is to serve HEAD as soon as possible. Is that comment in the example configuration file actually saying that Unicorn with nginx buffering is not broken? I mean, that if your application has some actions with stream enabled and you put it behind this setup, the content will be delivered albeit not streamed? If that is correct. Is it reasonable to send to nginx the header X-Accel-Buffering to disable buffering only for streamed responses? Or is it a very bad idea? If it is a real bad idea, is the recommendation to Unicorn users that they should just ignore this new feature?
Xavier Noria <fxn at hashref.com> wrote:> Streaming works with Unicorn + Apache. Both with and without deflating. > > My understanding is that Unicorn + Apache is not a good combination > though because Apache does not buffer, and thus Unicorn has no fast > client in front. (I don''t know which is the ultimate technical reason > Unicorn puts such an emphasis on fast clients, but will do some > research about it.)Basically the per-connection overhead of Unicorn is huge, an entire Ruby process (tens to several hundreds of megabytes). The per-connection overhead of nginx is tiny: maybe a few KB in userspace (including buffers), and a few KB in in the kernel. You don''t want to maintain connections to Unicorn for a long time because of that cost. OK, if you have any specific questions that aren''t answered in the website, please ask.> I have seen in > > http://unicorn.bogomips.org/examples/nginx.conf > > the comment > > "You normally want nginx to buffer responses to slow > clients, even with Rails 3.1 streaming because otherwise a slow > client can become a bottleneck of Unicorn." > > If I understand how this works correctly, nginx buffers the entire > response from Unicorn. First filling what''s configured in > proxy_buffer_size and proxy_buffers, and then going to disk if needed > as a last resort. Thus, even if the application streams, I believe the > client will receive the chunked response, but only after it has been > generated by the application and fully buffered by nginx. Which > defeats the purpose of streamingYes.> in the use case we have in mind in > Rails 3.1, which is to serve HEAD as soon as possible.Small nit: s/HEAD/the response header/ "HEAD" is a /request/ that only expects to receive the response header. nginx only sends HTTP/1.0 requests to unicorn, so Rack::Chunked won''t actually send a chunked/streamed response. Rails 3.1 /could/ enable streaming without chunking for HTTP/1.0, but only if the client didn''t set a non-standard HTTP/1.0 header to enable keepalive. This is because HTTP/1.0 (w/o keepalive) relies on the server to close the connection to signal the end of a response.> Is that comment in the example configuration file actually saying that > Unicorn with nginx buffering is not broken? I mean, that if your > application has some actions with stream enabled and you put it behind > this setup, the content will be delivered albeit not streamed?Correct.> If that is correct. Is it reasonable to send to nginx the header > X-Accel-Buffering to disable buffering only for streamed responses? Or > is it a very bad idea? If it is a real bad idea, is the recommendation > to Unicorn users that they should just ignore this new feature?You can use "X-Accel-Buffering: no" if you know your responses are small enough to fit into the kernel socket buffers. There''s two kernel buffers (Unicorn + nginx), you can get a little more space there. nginx shouldn''t make another request to Unicorn if it''s blocked writing a response to the client already, so an evil pipelining client should not hurt unicorn in this case: require "socket" host = "example.com" s = TCPSocket.new(host, 80) req = "GET /something/big HTTP/1.1\r\nHost: #{host}\r\n\r\n" # pipeline a large number of requests, nginx won''t send another # request to an upstream if it''s still writing one 30.times { s.write(req) } # don''t read the response, or read it slowly, just keep the socket # open here... sleep -- Eric Wong
Xavier Noria <fxn at hashref.com> wrote:> If it is a real bad idea, is the recommendation > to Unicorn users that they should just ignore this new feature?Another thing, Rainbows! + ThreadSpawn/ThreadPool concurrency may do the trick (without needing nginx at all). The per-client overhead of Rainbows! + threads (several hundred KB) is higher than nginx, but still much lower than Unicorn. All your Rails code must be thread-safe, though. If you use Linux, XEpollThreadPool/XEpollThreadSpawn can be worth a try, too. The cost of completely idle keepalive clients should be roughly inline with nginx. If you want to forgo thread-safety, Rainbows! + StreamResponseEpoll[1] + ForceStreaming middleware[2] may also be an option, too (needs nginx). Keep in mind that I don''t know of anybody using Rainbows! for any serious sites, so there could still be major bugs :) Rainbows! http://rainbows.rubyforge.org/ [1] currently in rainbows.git, will probably be released this weekend... [2] I''ll probably move this to Rainbows! instead of a Unicorn branch: http://bogomips.org/unicorn.git/tree/lib/unicorn/force_streaming.rb?h=force_streaming This is 100% untested, I''ve never run it. -- Eric Wong
On Sat, Jun 25, 2011 at 10:16 PM, Eric Wong <normalperson at yhbt.net> wrote:> Basically the per-connection overhead of Unicorn is huge, an entire Ruby > process (tens to several hundreds of megabytes). ?The per-connection > overhead of nginx is tiny: maybe a few KB in userspace (including > buffers), and a few KB in in the kernel. ?You don''t want to maintain > connections to Unicorn for a long time because of that cost.I see. I''ve read also the docs about design and philosophy in the website. So if I understand it correctly, as far as memory consumption is concerned the situation seems to be similar to the old days when mongrel cluster was the standard for production, except perhaps for setups with copy-on-write friendly interpreters, which weren''t available then. So you configure only a few processes because of memory consumption, and since there aren''t many you want them to be ready to serve a new request as soon as possible to handle some normal level of concurrency. Hence the convenience of buffering in Nginx.>> in the use case we have in mind in >> Rails 3.1, which is to serve HEAD as soon as possible. > > Small nit: s/HEAD/the response header/ ? "HEAD" is a /request/ that only > expects to receive the response header.Oh yes, that was ambiguous. I actually meant the HEAD element of HTML documents. The main use case in mind for adding streaming to Rails is to be able to send the top of your layout (typically everything before yielding to the view) so that the browser may issue requests for CSS and JavaScript assets while the application builds an hypothetically costly dynamic response.> nginx only sends HTTP/1.0 requests to unicorn, so Rack::Chunked won''t > actually send a chunked/streamed response. ?Rails 3.1 /could/ enable > streaming without chunking for HTTP/1.0, but only if the client > didn''t set a non-standard HTTP/1.0 header to enable keepalive. ?This > is because HTTP/1.0 (w/o keepalive) relies on the server to close > the connection to signal the end of a response.It''s clear then. Also, Rails has code that prevents streaming from being triggered if the request is HTTP/1.0: https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L243-244> You can use "X-Accel-Buffering: no" if you know your responses are small > enough to fit into the kernel socket buffers. ?There''s two kernel > buffers (Unicorn + nginx), you can get a little more space there. ?nginx > shouldn''t make another request to Unicorn if it''s blocked writing a > response to the client already, so an evil pipelining client should not > hurt unicorn in this case:Excellent. Thanks Eric!