Rails core - Aug 2012 - Should chunked encoding (by Rails) be optional for HTTP streaming?

Hello,

I have been using the HTTP streaming feature for my company''s website
for a
few months now.  It has significantly decreased the page load times for our 
asset-heavy pages.

I have encountered a few problems with HTTP streaming which can be traced 
to Rails always doing the chunked encoding itself:

 - Incompatible middleware such as newrelic (granted it''s easy to solve
by
putting the newrelic header and footer in manually but there are probably 
other incompatible middleware).

 - Problems with running assertions against a response body in integration 
tests.  The response has chunk markers in it making it difficult to parse 
the body.

 - Incompatible servers that chunk encode again resulting in a double 
chunked encoding.  I had to monkey patch Rails to not chunk encode in order 
to get streaming working properly with JRuby / Tomcat.  See 
https://github.com/jruby/jruby-rack/issues/117

I understand the chunked encoding is required in the end in order to 
stream.  But I wonder if instead of having Rails doing it at the core, it 
can instead be moved to a Rack middleware that is placed at the end of the 
middleware stack when needed and can be left out when the server is capable 
of doing the chunked encoding.  This could be controlled by a configuration 
option.  

As an intermediate step towards moving it to a Rack middleware, chunked 
encoding could first be made optional.  This would at least solve the JRuby 
/ Tomcat issue.

Thoughts?

-Tim

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/vlZ-xXEV5gsJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Hi Tim!

On Fri, Aug 24, 2012 at 06:44:02AM -0700, Tim Olsen
wrote:
> Hello,
> 
> I have been using the HTTP streaming feature for my company''s
website for a
> few months now.  It has significantly decreased the page load times for our
> asset-heavy pages.
> 
> I have encountered a few problems with HTTP streaming which can be traced 
> to Rails always doing the chunked encoding itself:
> 
>  - Incompatible middleware such as newrelic (granted it''s easy to
solve by
> putting the newrelic header and footer in manually but there are probably 
> other incompatible middleware).
I''m not familiar with this, but it seems like something newrelic should
provide?
>  - Problems with running assertions against a response body in integration 
> tests.  The response has chunk markers in it making it difficult to parse 
> the body.
Are these *rails* integration tests?  If so, we should consider this a
bug.  I would expect chunks to be assembled in to normal bodies in the
integration tests.
>  - Incompatible servers that chunk encode again resulting in a double 
> chunked encoding.  I had to monkey patch Rails to not chunk encode in order
> to get streaming working properly with JRuby / Tomcat.  See 
> https://github.com/jruby/jruby-rack/issues/117
Not fun. :(
> I understand the chunked encoding is required in the end in order to 
> stream.  But I wonder if instead of having Rails doing it at the core, it 
> can instead be moved to a Rack middleware that is placed at the end of the 
> middleware stack when needed and can be left out when the server is capable
> of doing the chunked encoding.  This could be controlled by a configuration
> option.  
> 
> As an intermediate step towards moving it to a Rack middleware, chunked 
> encoding could first be made optional.  This would at least solve the JRuby
> / Tomcat issue.
It seems entirely possible to do this.  Rack has a chunking middleware
that can be inserted in the stack.  I think the main problem is that by
the time the framework learns that you want a streaming response (the
point when you call render with stream), the middleware stack is already
assembled.  In order to make the streaming Just Work, you''d have to add
a chunked body right at that point.

We could eliminate the chunked body [from here][1], but it means two
things:

1. You''d have to manually add the chunking middleware for *that*
   controller.

2. It would break backwards compatibility

We could add the chunking middleware by default, but I''m not sure
everyone would want that.

I have another idea to fix this, but it''s a bit more effort.  AC::Live
doesn''t chunk responses.  I think it''s possible to get the ERB
to write
to a buffer that AC::Live uses.  I''ve been looking in to this, but it
hasn''t been my highest priority lately. :(

[1]:
https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L219

-- 
Aaron Patterson
http://tenderlovemaking.com/

On Friday, August 24, 2012 6:58:08 PM UTC-4, Aaron Patterson
wrote:
>
> On Fri, Aug 24, 2012 at 06:44:02AM -0700, Tim Olsen wrote: 
> > I have encountered a few problems with HTTP streaming which can be 
> traced 
> > to Rails always doing the chunked encoding itself: 
> > 
> >  - Incompatible middleware such as newrelic (granted it''s
easy to solve
> by 
> > putting the newrelic header and footer in manually but there are 
> probably 
> > other incompatible middleware). 
>
> I''m not familiar with this, but it seems like something newrelic
should
> provide? 
>
Maybe, but it seems a little redundant for everything along the middleware 
stack to have to dechunk if it wants to read the body and rechunk if it 
wants to modify the body.
 
>
> >  - Problems with running assertions against a response body in 
> integration 
> > tests.  The response has chunk markers in it making it difficult to 
> parse 
> > the body. 
>
> Are these *rails* integration tests?  If so, we should consider this a 
> bug.  I would expect chunks to be assembled in to normal bodies in the 
> integration tests. 
>
It''s happening for me in rails request specs, which I assume means it
is
happening in rails integration tests.  The problem went away when I monkey 
patched Rails to not chunk.

> > I understand the chunked encoding is required in the end in order to 
> > stream.  But I wonder if instead of having Rails doing it at the core,
> it 
> > can instead be moved to a Rack middleware that is placed at the end of
> the 
> > middleware stack when needed and can be left out when the server is 
> capable 
> > of doing the chunked encoding.  This could be controlled by a 
> configuration 
> > option.   
> > 
> > As an intermediate step towards moving it to a Rack middleware,
chunked
> > encoding could first be made optional.  This would at least solve the 
> JRuby 
> > / Tomcat issue. 
>
> It seems entirely possible to do this.  Rack has a chunking middleware 
> that can be inserted in the stack.  I think the main problem is that by 
> the time the framework learns that you want a streaming response (the 
> point when you call render with stream), the middleware stack is already 
> assembled.  In order to make the streaming Just Work, you''d have
to add
> a chunked body right at that point. 
>
How about a chunking middleware that only chunk encodes if the 
Content-Length header is not set?  Otherwise it allows the response to pass 
through as is.
 
>
> We could eliminate the chunked body [from here][1], but it means two 
> things: 
>
> 1. You''d have to manually add the chunking middleware for *that* 
>    controller. 
>
> 2. It would break backwards compatibility 
>
The only thing I can think of breaking is middleware that expects a chunked 
encoding when Content-Length is not set.  I would hope they would detect 
chunked encoding by the Transfer-Encoding header, but who knows.  Is there 
anything else you think can break?
 
>
> We could add the chunking middleware by default, but I''m not sure 
> everyone would want that. 
>
If chunking middleware proves distasteful for too many, maybe just having a 
configuration option to turn off the chunking (for servers such as Tomcat) 
is the way to go.
 
>
> I have another idea to fix this, but it''s a bit more effort. 
AC::Live
> doesn''t chunk responses.  I think it''s possible to get
the ERB to write
> to a buffer that AC::Live uses.  I''ve been looking in to this, but
it
> hasn''t been my highest priority lately. :( 

Interesting.  Does AC::Live just rely on the webserver to do the chunking?

Tim

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/Xy-Lc-1-zF0J.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Saturday 25 August 2012, Tim Olsen wrote:
> On Friday, August 24, 2012 6:58:08 PM UTC-4, Aaron Patterson wrote:
> > We could add the chunking middleware by default, but I''m not
sure
> > everyone would want that.
> 
> If chunking middleware proves distasteful for too many, maybe just
> having a configuration option to turn off the chunking (for servers
> such as Tomcat) is the way to go.
Is there a problem with chunking and Tomcat? jruby-rack doesn''t support
it directly, but it doesn''t prohibit it either. It appears to work if 
you set a response_body whose #each yields properly formatted chunks and 
a terminating empty chunk. I''ve just started to use this to pass
through
(already chunked) responses from another services.

Michael

-- 
Michael Schuerig
mailto:michael@schuerig.de
http://www.schuerig.de/michael/

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Sat, Aug 25, 2012 at 2:55 AM, Michael Schuerig
<michael.lists@schuerig.de
> wrote:
>
> Is there a problem with chunking and Tomcat? jruby-rack doesn''t
support
> it directly, but it doesn''t prohibit it either. It appears to work
if
> you set a response_body whose #each yields properly formatted chunks and
> a terminating empty chunk. I''ve just started to use this to pass
through
> (already chunked) responses from another services.

I and others have seen Tomcat double chunk streamed responses coming from
Rails.  It may be the case, however, that this only happens for responses
over a certain size.  I''ve filed an issue with more details at
https://github.com/jruby/jruby-rack/issues/117 .

How large are your responses?  I''ve only tried so far to get one of our
streamed pages to work in a jruby setup.  It''s response is 68,019
bytes.

Tim

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Saturday 25 August 2012, Tim Olsen wrote:
> On Sat, Aug 25, 2012 at 2:55 AM, Michael Schuerig
> <michael.lists@schuerig.de
> 
> > wrote:
> > 
> > Is there a problem with chunking and Tomcat? jruby-rack
doesn''t
> > support it directly, but it doesn''t prohibit it either. It
appears
> > to work if you set a response_body whose #each yields properly
> > formatted chunks and a terminating empty chunk. I''ve just
started
> > to use this to pass through (already chunked) responses from
> > another services.
> 
> I and others have seen Tomcat double chunk streamed responses coming
> from Rails.  It may be the case, however, that this only happens for
> responses over a certain size.  I''ve filed an issue with more
> details at https://github.com/jruby/jruby-rack/issues/117 .
I think our cases are different. I''m not using the Rails streaming 
support at all. Instead, I''m setting the necessary headers and the 
response_body myself. In particular, I''m setting response_body to an 
object implementing #each in such a way that it yields properly 
formatted chunks. jruby-rack writes these chunks individually to the 
servlet''s output stream. I haven''t yet tested this
exhaustively, but as
far as I can tell, Tomcat (or rather Winstone in my tests) simply passes 
the chunks through to its browser client in turn. That is exactly the 
behavior I want, as my Rails app in this case only proxies attachments 
stored by another, internal webservice.
> How large are your responses?  I''ve only tried so far to get one
of
> our streamed pages to work in a jruby setup.  It''s response is
> 68,019 bytes.
I don''t think our cases are comparable. I''ve tried my
mechanism on cases
where the document body is at most 1MB; but I haven''t seen any 
indication that it would work differently for much larger or smaller 
responses.

Michael

-- 
Michael Schuerig
mailto:michael@schuerig.de
http://www.schuerig.de/michael/

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Sat, Aug 25, 2012 at 10:14 AM, Michael Schuerig <
michael.lists@schuerig.de> wrote:
> I think our cases are different. I''m not using the Rails streaming
> support at all. Instead, I''m setting the necessary headers and the
> response_body myself. In particular, I''m setting response_body to
an
> object implementing #each in such a way that it yields properly
> formatted chunks. jruby-rack writes these chunks individually to the
> servlet''s output stream. I haven''t yet tested this
exhaustively, but as
> far as I can tell, Tomcat (or rather Winstone in my tests) simply passes
> the chunks through to its browser client in turn. That is exactly the
> behavior I want, as my Rails app in this case only proxies attachments
> stored by another, internal webservice.
>
By chunks do you mean HTTP/1.1 chunks [1]?  In other words, do you set the
Transfer-Encoding to "chunked" and are your chunks prefixed by the
size of
the chunk in hex and a CRLF?  If not, then that is not HTTP chunked
encoding.  It''s by removing the chunked encoding (the header, the size
in
hex and CRLF) that I was able to get things working properly with Tomcat.

As for Winstone, I have tried that server and it works with the chunked
encoding.

> I don''t think our cases are comparable. I''ve tried my
mechanism on cases
> where the document body is at most 1MB; but I haven''t seen any
> indication that it would work differently for much larger or smaller
> responses.
>
With that size, I would expect you to see double chunked encoding.  Double
chunked encoding results in the hex showing through, that is if you''re
not
using a browser such as Chrome which refuses to display a page with two
Transfer-Encoding headers.

Tim

[1] http://greenbytes.de/tech/webdav/rfc2616.html#chunked.transfer.encoding

>
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Sunday 26 August 2012, Tim Olsen wrote:
> On Sat, Aug 25, 2012 at 10:14 AM, Michael Schuerig <
> 
> michael.lists@schuerig.de> wrote:
> > I think our cases are different. I''m not using the Rails
streaming
> > support at all. Instead, I''m setting the necessary headers
and the
> > response_body myself. In particular, I''m setting
response_body to
> > an object implementing #each in such a way that it yields properly
> > formatted chunks. jruby-rack writes these chunks individually to
> > the servlet''s output stream. I haven''t yet tested
this
> > exhaustively, but as far as I can tell, Tomcat (or rather Winstone
> > in my tests) simply passes the chunks through to its browser
> > client in turn. That is exactly the behavior I want, as my Rails
> > app in this case only proxies attachments stored by another,
> > internal webservice.
> 
> By chunks do you mean HTTP/1.1 chunks [1]?  In other words, do you
> set the Transfer-Encoding to "chunked" and are your chunks
prefixed
> by the size of the chunk in hex and a CRLF?
Yes. I do this myself in the #each method of the object to which I set 
response_body.
> If not, then that is
> not HTTP chunked encoding.  It''s by removing the chunked encoding
> (the header, the size in hex and CRLF) that I was able to get things
> working properly with Tomcat.
> 
> As for Winstone, I have tried that server and it works with the
> chunked encoding.
Ouch. I mistakenly thought that Winstone was an embedded variant of 
Tomcat. I was probably confusing it with Trinidad. Anyway, for 
Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just 
yield the data then Tomcat does the right thing. It does not use the 
exact blocks of data for chunks that I yield, but it generates properly 
formatted chunks and a trailing empty chunk.

Michael

-- 
Michael Schuerig
mailto:michael@schuerig.de
http://www.schuerig.de/michael/

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Sun, Aug 26, 2012 at 4:01 AM, Michael Schuerig
<michael.lists@schuerig.de
> wrote:
> On Sunday 26 August 2012, Tim Olsen wrote:
> > As for Winstone, I have tried that server and it works with the
> > chunked encoding.
>
> Ouch. I mistakenly thought that Winstone was an embedded variant of
> Tomcat. I was probably confusing it with Trinidad. Anyway, for
> Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just
> yield the data then Tomcat does the right thing. It does not use the
> exact blocks of data for chunks that I yield, but it generates properly
> formatted chunks and a trailing empty chunk.
>
>
If I am understanding you correctly, you are not sending chunked data to
Tomcat and Tomcat does the chunking.  The problem I''m having is that
Tomcat
chunks again the already chunked data coming from Rails streaming.  So my
solution is similar to yours (if I''m understanding you correctly): I
monkey
patched Rails to not send chunked data to Tomcat and Tomcat takes care of
the chunking.

Tim

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

OK, just to sum up things a bit. This whole thing started as a report by 
Tim on JRuby-Rack https://github.com/jruby/jruby-rack/issues/117

It''s a shame but JRuby-Rack had no support for `render stream: true` up
till now (should arrive with the next version). 
Now to implement this on top of the Servlet API is a bit cumbersome since 
all containers I''ve checked handle chunking automatically for us
whenever
the output stream (body) gets flushed early on (and there''s likely no 
Content-Length specified). Thus receiving an already chunked response from 
Rack is unfortunate and there''s no other "safe" way around as
to de-chunk
it, send the raw data down the container and let it do it''s magic.

This would have been easier if Rails instead of wrapping the body in a 
Rack::Chunked::Body decided to setup the Rack::Chunked middleware.
The middleware, I guess, might have been also removed in test env since it 
kind of makes sense to not test a chunked response ...

But there''s likely a reason for that I''m missing, maybe
someone could
clarify why Rails is not using the Rack::Chunked middleware.
Seems it all started somewhere around here 
https://github.com/rack/rack/commit/4d93f90b58b09329a227fac975228eeef76f2952

Thanks, K.

On Sunday, August 26, 2012 1:57:41 PM UTC+2, Tim Olsen
wrote:
>
>
>
> On Sun, Aug 26, 2012 at 4:01 AM, Michael Schuerig
<michae...@schuerig.de<javascript:>
> > wrote:
>
>> On Sunday 26 August 2012, Tim Olsen wrote:
>> > As for Winstone, I have tried that server and it works with the
>> > chunked encoding.
>>
>> Ouch. I mistakenly thought that Winstone was an embedded variant of
>> Tomcat. I was probably confusing it with Trinidad. Anyway, for
>> Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just
>> yield the data then Tomcat does the right thing. It does not use the
>> exact blocks of data for chunks that I yield, but it generates properly
>> formatted chunks and a trailing empty chunk.
>>
>>
> If I am understanding you correctly, you are not sending chunked data to 
> Tomcat and Tomcat does the chunking.  The problem I''m having is
that Tomcat
> chunks again the already chunked data coming from Rails streaming.  So my 
> solution is similar to yours (if I''m understanding you correctly):
I monkey
> patched Rails to not send chunked data to Tomcat and Tomcat takes care of 
> the chunking.
>
> Tim
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/0iL61eRyaikJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.