mongrel unicorn - Jun 2011 - problem setting multiple cookies

Hey guys,

When I try to set multiple cookies in one request, the
''Set-Cookies''
header in the http response doesn''t look right -- all the cookies are
set without any line breaks.

Looks like this:

Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
expires=Sun, 20-Jun-2021 23:34:30 GMT

Should look more like this, which is what I got from a fresh Rails project test:

Set-Cookie:first=33; path=/
second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
third=69; path=/

In the first example, neither cookies (first/second) are set. The
second example, all 3 cookies are set.

I think it''s happening somewhere here, but I can''t figure out
what''s
going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb

Any help appreciated, thanks in advance!

jason

Jason Su <jason at lookbook.nu> wrote:
> Hey guys,
> 
> When I try to set multiple cookies in one request, the
''Set-Cookies''
> header in the http response doesn''t look right -- all the cookies
are
> set without any line breaks.
> 
> Looks like this:
> 
> Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
> 01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
> expires=Sun, 20-Jun-2021 23:34:30 GMT
Which versions of Rails and Rack are you using?  How is your app
setting cookies?

Older versions of Rack may have specified a different delimiter (or an
Array).  Current versions only use "\n" inside a string, so it should
be
something like:

  headers["Set-Cookie"] = "first=33; path=/\n" \
                          "second=; path=/\n" \
                          "third=; path=/"

Using Rack::Utils.set_cookie_header! (or a Rails-level wrapper) is
recommended, though.
> Should look more like this, which is what I got from a fresh Rails project
test:
> 
> Set-Cookie:first=33; path=/
> second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
> third=69; path=/
The response should have 3 individual Set-Cookie lines:

  Set-Cookie: first=33; path=/
  Set-Cookie: second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
  Set-Cookie: third=69; path=/
> In the first example, neither cookies (first/second) are set. The
> second example, all 3 cookies are set.
> 
> I think it''s happening somewhere here, but I can''t figure
out what''s
> going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb
The code that does this transformation of the Rack header to the
client socket is here:

http://bogomips.org/unicorn.git/tree/lib/unicorn/http_response.rb

cgi_wrapper.rb is only for Rails <= 2.2, where it is called before
http_response.rb

-- 
Eric Wong

Hey Eric,

I''m using Rails 2.3.8, and Rack 1.1.2

I''m setting cookies like this:

cookies[:user_id] = { :domain => ".#{domain}", :value =>
"#{user.id}",
:expires => 10.years.from_now }

How can I get the response to show up correctly, with individual
Set-Cookie lines for each?

Thanks again,

Jason



On Mon, Jun 20, 2011 at 5:06 PM, Eric Wong <normalperson at yhbt.net>
wrote:
> Jason Su <jason at lookbook.nu> wrote:
>> Hey guys,
>>
>> When I try to set multiple cookies in one request, the
''Set-Cookies''
>> header in the http response doesn''t look right -- all the
cookies are
>> set without any line breaks.
>>
>> Looks like this:
>>
>> Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
>> 01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
>> expires=Sun, 20-Jun-2021 23:34:30 GMT
>
> Which versions of Rails and Rack are you using? ?How is your app
> setting cookies?
>
> Older versions of Rack may have specified a different delimiter (or an
> Array). ?Current versions only use "\n" inside a string, so it
should be
> something like:
>
> ?headers["Set-Cookie"] = "first=33; path=/\n" \
> ? ? ? ? ? ? ? ? ? ? ? ? ?"second=; path=/\n" \
> ? ? ? ? ? ? ? ? ? ? ? ? ?"third=; path=/"
>
> Using Rack::Utils.set_cookie_header! (or a Rails-level wrapper) is
> recommended, though.
>
>> Should look more like this, which is what I got from a fresh Rails
project test:
>>
>> Set-Cookie:first=33; path=/
>> second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
>> third=69; path=/
>
> The response should have 3 individual Set-Cookie lines:
>
> ?Set-Cookie: first=33; path=/
> ?Set-Cookie: second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
> ?Set-Cookie: third=69; path=/
>
>> In the first example, neither cookies (first/second) are set. The
>> second example, all 3 cookies are set.
>>
>> I think it''s happening somewhere here, but I can''t
figure out what''s
>> going on...
http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb
>
> The code that does this transformation of the Rack header to the
> client socket is here:
>
> http://bogomips.org/unicorn.git/tree/lib/unicorn/http_response.rb
>
> cgi_wrapper.rb is only for Rails <= 2.2, where it is called before
> http_response.rb
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

Jason Su <jason at lookbook.nu> wrote:
> Hey Eric,
> 
> I''m using Rails 2.3.8, and Rack 1.1.2
Please don''t top post, if you''re just replying to the message
in general
I''d rather you not quote at all.
> I''m setting cookies like this:
> 
> cookies[:user_id] = { :domain => ".#{domain}", :value =>
"#{user.id}",
> :expires => 10.years.from_now }
> How can I get the response to show up correctly, with individual
> Set-Cookie lines for each?
I wonder if it''s a bug in older versions of Rails or Rack, there was
some confusion back in the day about how to handle multiple values
for a single header...

Can you try loading the following middleware in your app?

If it doesn''t work, uncomment the ''p'' statement and
show us what data
structure is used for your headers.

---------------------- join_cookie.rb ---------------------------------
# Totally untested:
# usage (in config.ru):
#   use JoinCookie
#   run YourApp.new
class JoinCookie < Struct.new(:app)
  def call(env)
    status, headers, body = app.call(env)

    ## uncomment and show me the output of the next line if this
doesn''t work
    # p headers

    headers = Rack::Utils::HeaderHash.new(headers)
    case set_cookie = headers["Set-Cookie"]
    when Array
      headers["Set-Cookie"] = set_cookie.join("\n")
    end

    [ status, headers, body ]
  end
end
-----------------------------------------------------------------

-- 
Eric Wong

I don''t think it worked, but here''s the output:

{"X-Runtime"=>"174",
"Content-Type"=>"text/html",
"Content-Length"=>"1",
"Set-Cookie"=>"first=; domain=.test.com;
path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
domain=.test.com; path=/; expires=Mon, 21-Jun-2021 01:22:23
GMT\n_test_session=ef56a1c675e576bd0b922acb0027ebde; domain=.test.com;
path=/; HttpOnly", "Cache-Control"=>"no-cache"}

Here is how the response header looks:

Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
21-Jun-2021 01:32:26 GMT

Thanks,
Jason

Sorry to post again, but the response headers were captured after I
cleared my cookies, so it''s not consistent with the first output.

Jason Su <jason at lookbook.nu> wrote:
> I don''t think it worked, but here''s the output:
> 
> {"X-Runtime"=>"174",
"Content-Type"=>"text/html",
> "Content-Length"=>"1",
"Set-Cookie"=>"first=; domain=.test.com;
> path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
> domain=.test.com; path=/; expires=Mon, 21-Jun-2021 01:22:23
> GMT\n_test_session=ef56a1c675e576bd0b922acb0027ebde; domain=.test.com;
      ^-- this is suspicous, the _test_session= cookie *is* set
correctly...
> path=/; HttpOnly", "Cache-Control"=>"no-cache"}
> 
> Here is how the response header looks:
> 
> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
> 21-Jun-2021 01:32:26 GMT
And you got a second Set-Cookie: line with "_test_session" in there,
right?

Any chance you could upgrade Rails to 2.3.11?  Some rails commits
since 2.3.8 look interesting:
e0eb8e9c65ededce64169948d4dd51b0079cdd10
85b6d79d8a17fdef667770e31b44ac6647f8b584

-- 
Eric Wong

Hey Eric,

I called p headers in JoinCookie before manipulating the headers at
all, so what you saw was probably already set correctly by rails. I''m
not setting the _test_session cookie myself.

Here''s what I get in the response, with the _test_session cookie line:

Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
21-Jun-2021 02:40:27 GMT
_test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
path=/; HttpOnly

It doesn''t add a second Set-Cookie: line, but the line break seems to
work...

2.3.11 is giving me a ton of trouble.. I''ve tried it before.. and I
just tried again with little luck... Is there anything that I can just
grab from the commits to make it work on 2.3.8?

Thanks,
Jason

Jason Su <jason at lookbook.nu> wrote:
> Here''s what I get in the response, with the _test_session cookie
line:
> 
> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
> 21-Jun-2021 02:40:27 GMT
> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
> path=/; HttpOnly
> 
> It doesn''t add a second Set-Cookie: line, but the line break seems
to work...
Huh?  Which version of unicorn are you using?  The /\n/ splitting in
headers has been in the earliest releases of unicorn...
> 2.3.11 is giving me a ton of trouble.. I''ve tried it before.. and
I
> just tried again with little luck... Is there anything that I can just
> grab from the commits to make it work on 2.3.8?
Look at the individual commits in rails.git.  I just used

  git log -p v2.3.8..v2.3.11 -- actionpack

in rails.git to find those for you.   I just looked for things that
would manipulate cookies/headers.

-- 
Eric Wong

I''m using unicorn 3.6.2

What I was trying to say is that the JoinCookie middleware didn''t do
anything. headers["Set-Cookie"] is a string, and the _test_session
cookie was always being set correctly. The problem happens only when I
try to set multiple cookies myself.

Here''s the output of p headers in JoinCookie again

{"X-Runtime"=>"92",
"Content-Type"=>"text/html",
"Content-Length"=>"1",
"Set-Cookie"=>"first=; domain=.test.com;
path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
path=/; HttpOnly", "Cache-Control"=>"no-cache"}

Where is JoinCookie is getting it''s headers from?


On Mon, Jun 20, 2011 at 8:15 PM, Eric Wong <normalperson at yhbt.net>
wrote:
> Jason Su <jason at lookbook.nu> wrote:
>> Here''s what I get in the response, with the _test_session
cookie line:
>>
>> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
>> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
>> 21-Jun-2021 02:40:27 GMT
>> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
>> path=/; HttpOnly
>>
>> It doesn''t add a second Set-Cookie: line, but the line break
seems to work...
>
> Huh? ?Which version of unicorn are you using? ?The /\n/ splitting in
> headers has been in the earliest releases of unicorn...
>
>> 2.3.11 is giving me a ton of trouble.. I''ve tried it before..
and I
>> just tried again with little luck... Is there anything that I can just
>> grab from the commits to make it work on 2.3.8?
>
> Look at the individual commits in rails.git. ?I just used
>
> ?git log -p v2.3.8..v2.3.11 -- actionpack
>
> in rails.git to find those for you. ? I just looked for things that
> would manipulate cookies/headers.
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

Jason Su <jason at lookbook.nu> wrote:
> I''m using unicorn 3.6.2
> 
> What I was trying to say is that the JoinCookie middleware didn''t
do
> anything. headers["Set-Cookie"] is a string, and the
_test_session
> cookie was always being set correctly. The problem happens only when I
> try to set multiple cookies myself.
Yeah, you''re not getting an Array (at least not at that stage) as I
semi-hoped so JoinCookie wouldn''t solve it.
> Here''s the output of p headers in JoinCookie again
> 
> {"X-Runtime"=>"92",
"Content-Type"=>"text/html",
> "Content-Length"=>"1",
"Set-Cookie"=>"first=; domain=.test.com;
> path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
> domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
> GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
> path=/; HttpOnly", "Cache-Control"=>"no-cache"}
> 
> Where is JoinCookie is getting it''s headers from?
app.call(env) is what dispatches your Rails app, so the headers are what
Rails (and middlewares further down the stack) sets.

Any luck with looking at Rails changes that might impact this?

At this point it''s clear it''s a Rails/Rack issue (and for an
old
version, at that) and not something that can be done with Unicorn.

Maybe you can fix whatever the Rails v2.3.8..v2.3.11 upgrade broke
in your app, but whatever''s broken is broken at the application layer
and not in Unicorn.

-- 
Eric Wong

That''s strange, doesn''t really explain why cookies work fine
on a
fresh Rails 2.3.8 app, nor why cookies stopped working for me after I
switched to Unicorn.

I''ll keep looking..

Jason Su <jason at lookbook.nu> wrote:
> That''s strange, doesn''t really explain why cookies work
fine on a
> fresh Rails 2.3.8 app, nor why cookies stopped working for me after I
> switched to Unicorn.
Actually, another option you could try is to use OldRails (designed for
Rails <= 2.2) and try to get that working with Rails 2.3.  This should
cause Rails 2.3 to use the cgi_wrapper.rb and not the native Rack
interface:

--------------- config.ru ------------------
require "unicorn/app/old_rails"
use Unicorn::App::OldRails::Static
run Unicorn::App::OldRails.new
--------------------------------------------

-- 
Eric Wong

Hey Eric,

Actually I think I am using OldRails -- don''t know what it is or why
I''m doing it, but this is in my config.ru:

map ''/'' do
  if RAILS_ENV != ''production''
    begin
      require ''rack/bug''
      use Rack::Bug, :password => nil
    rescue LoadError
    end

    use Rails::Rack::Static
  end

  run Unicorn::App::OldRails.new
end

Should I not be using OldRails?



I''m also trying to upgrade to Rails 2.3.11 again. Here''s the
error
that''s stopping me:

You have a nil object when you didn''t expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split You have a nil object
when you didn''t expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/cgi_process.rb:52:in
`dispatch_cgi''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:101:in
`dispatch_cgi''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:27:in
`dispatch''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails.rb:22:in
`call''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails/static.rb:56:in
`call''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:47:in
`call''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`each''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`call''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:545:in
`process_client''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/oob_gc.rb:60:in
`process_client''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:623:in
`worker_loop''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`each''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`worker_loop''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:509:in
`spawn_missing_workers''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`fork''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`spawn_missing_workers''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`each''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`spawn_missing_workers''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:516:in
`maintain_worker_count''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:166:in
`start''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn.rb:30:in
`run''
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/bin/unicorn_rails:208
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19:in
`load''
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19

I found this, which I think is somehow related to the fix:
https://gist.github.com/826692

Do you see anything that looks familiar?

Thanks,
jason

FYI I found that gist on this blog:
http://rcaguilar.wordpress.com/2011/02/14/broken-mongrels-after-rails-2-3-11-upgrade

Jason Su <jason at lookbook.nu> wrote:
> Hey Eric,
> 
> Actually I think I am using OldRails -- don''t know what it is or
why
> I''m doing it, but this is in my config.ru:
It might be because you were on Rails 2.2 or earlier and upgraded?
> map ''/'' do
>   if RAILS_ENV != ''production''
>     begin
>       require ''rack/bug''
>       use Rack::Bug, :password => nil
>     rescue LoadError
>     end
> 
>     use Rails::Rack::Static
>   end
> 
>   run Unicorn::App::OldRails.new
> end
> 
> Should I not be using OldRails?
OldRails was for Rails <= 2.2, so you should probably try this instead:

  run ActionController::Dispatcher.new

This should use the pure Rack backend that avoids the old CGI
and CGIWrapper code which Rails <=2.2 relied on.
> I''m also trying to upgrade to Rails 2.3.11 again. Here''s
the error
> that''s stopping me:
> 
> You have a nil object when you didn''t expect it!
> You might have expected an instance of Array.
> The error occurred while evaluating nil.split You have a nil object
> when you didn''t expect it!
<snip>
> I found this, which I think is somehow related to the fix:
> https://gist.github.com/826692
> 
> Do you see anything that looks familiar?
I haven''t seen it before, but it looks to be on the right track.

OldRails uses the CGIWrapper code which is based on the original
code in Mongrel, too.

If you avoid CGIWrapper, you should be able to avoid the
workarounds/issues with it like the above gist.

-- 
Eric Wong

Upgrading to 2.3.11 and not using OldRails did it!

Thanks so much Eric

Jason