mongrel unicorn - Oct 2013 - [PATCH] support SO_REUSEPORT on new listeners (:reuseport)

This allows users to start an independent instance of unicorn on
a the same port as a running unicorn (as long as both instances
use :reuseport).

ref: https://lwn.net/Articles/542629/
---
 lib/unicorn/configurator.rb     | 19 +++++++++++++++++++
 lib/unicorn/socket_helper.rb    | 30 ++++++++++++++++++++++--------
 test/unit/test_socket_helper.rb |  8 ++++++++
 3 files changed, 49 insertions(+), 8 deletions(-)

diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index 0d0eac7..fc3405a 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -319,6 +319,25 @@ class Unicorn::Configurator
   #
   #   Default: Operating-system dependent
   #
+  # [:reuseport => true or false]
+  #
+  #   This enables multiple, independently-started unicorn instances to
+  #   bind to the same port (as long as all the processes enable this).
+  #
+  #   This option must be used when unicorn first binds the listen socket.
+  #   It cannot be enabled when a socket is inherited via SIGUSR2
+  #   (but it will remain on if inherited), and it cannot be enabled
+  #   directly via SIGHUP.
+  #
+  #   Note: there is a chance of connections being dropped if
+  #   one of the unicorn instances is stopped while using this.
+  #
+  #   This is supported on *BSD systems and Linux 3.9 or later.
+  #
+  #   ref: https://lwn.net/Articles/542629/
+  #
+  #   Default: false (unset)
+  #
   # [:tries => Integer]
   #
   #   Times to retry binding a socket if it is already in use
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index 18b0be7..2701d58 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -41,6 +41,15 @@ module Unicorn
 
       # do not send out partial frames (Linux)
       TCP_CORK = 3 unless defined?(TCP_CORK)
+
+      # Linux got SO_REUSEPORT in 3.9, BSDs have had it for ages
+      unless defined?(SO_REUSEPORT)
+        if RUBY_PLATFORM =~ /(?:alpha|mips|parisc|sparc)/
+          SO_REUSEPORT = 0x0200 # untested
+        else
+          SO_REUSEPORT = 15 # only tested on x86_64 and i686
+        end
+      end
     when /freebsd/
       # do not send out partial frames (FreeBSD)
       TCP_NOPUSH = 4 unless defined?(TCP_NOPUSH)
@@ -142,9 +151,9 @@ module Unicorn
           File.umask(old_umask)
         end
       elsif /\A\[([a-fA-F0-9:]+)\]:(\d+)\z/ =~ address
-        new_ipv6_server($1, $2.to_i, opt)
+        new_tcp_server($1, $2.to_i, opt.merge(:ipv6=>true))
       elsif /\A(\d+\.\d+\.\d+\.\d+):(\d+)\z/ =~ address
-        Kgio::TCPServer.new($1, $2.to_i)
+        new_tcp_server($1, $2.to_i, opt)
       else
         raise ArgumentError, "Don''t know how to bind:
#{address}"
       end
@@ -152,13 +161,18 @@ module Unicorn
       sock
     end
 
-    def new_ipv6_server(addr, port, opt)
-      opt.key?(:ipv6only) or return Kgio::TCPServer.new(addr, port)
-      defined?(IPV6_V6ONLY) or
-        abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your
OS"
-      sock = Socket.new(AF_INET6, SOCK_STREAM, 0)
-      sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0)
+    def new_tcp_server(addr, port, opt)
+      # n.b. we set FD_CLOEXEC in the workers
+      sock = Socket.new(opt[:ipv6] ? AF_INET6 : AF_INET, SOCK_STREAM, 0)
+      if opt.key?(:ipv6only)
+        defined?(IPV6_V6ONLY) or
+          abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your
OS"
+        sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0)
+      end
       sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
+      if defined?(SO_REUSEPORT) && opt[:reuseport]
+        sock.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1)
+      end
       sock.bind(Socket.pack_sockaddr_in(port, addr))
       IO_PURGATORY << sock
       Kgio::TCPServer.for_fd(sock.fileno)
diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb
index a38082c..abc177b 100644
--- a/test/unit/test_socket_helper.rb
+++ b/test/unit/test_socket_helper.rb
@@ -184,4 +184,12 @@ class TestSocketHelper < Test::Unit::TestCase
     assert_equal 1, cur
     rescue Errno::EAFNOSUPPORT
   end if RUBY_VERSION >= "1.9.2"
+
+  def test_reuseport
+    port = unused_port @test_addr
+    name = "#@test_addr:#{port}"
+    sock = bind_listen(name, :reuseport => true)
+    cur = sock.getsockopt(Socket::SOL_SOCKET,
SO_REUSEPORT).unpack(''i'')[0]
+    assert_equal 1, cur
+  end if defined?(SO_REUSEPORT)
 end
-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Eric Wong <normalperson@yhbt.net> wrote:
> This allows users to start an independent instance of unicorn on
> a the same port as a running unicorn (as long as both instances
> use :reuseport).
> 
> ref: https://lwn.net/Articles/542629/
> ---
Also pushed out a couple of trivial IO_PURGATORY-related fixes to
support this.  We''ll drop 1.8 support one of these days.

commit e025cd99beee500f175a3bcc302a1307b39ffb77
Author: Eric Wong <e@80x24.org>
Date:   Fri Oct 25 19:45:15 2013 +0000

    avoid IO_PURGATORY on Ruby 1.9+
    
    Ruby 1.9 and later includes IO#autoclose=, so we can use it
    and prevent some dead IO objects from hanging around.

commit 7c125886b5862bf20711bae22e6697ad46141434
Author: Eric Wong <e@80x24.org>
Date:   Fri Oct 25 19:27:05 2013 +0000

    support SO_REUSEPORT on new listeners (:reuseport)
    
    This allows users to start an independent instance of unicorn on
    a the same port as a running unicorn (as long as both instances
    use :reuseport).
    
    ref: https://lwn.net/Articles/542629/

commit 1dc099228ee0f59c13385a3e7346a2cb37d85153
Author: Eric Wong <e@80x24.org>
Date:   Fri Oct 25 19:54:39 2013 +0000

    tests: limit oobgc check to accepted sockets
    
    Otherwise these tests fail if we start using IO#autoclose=true
    on Ruby 1.9 (and also if we use IPv6 sockets for tests).
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

In case we (and Linux) supports other values in the future,
we can update it then.  Until now, ensure users only set
true or false for this option.
---
 lib/unicorn/configurator.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index fc3405a..9406223 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -415,7 +415,7 @@ class Unicorn::Configurator
         Integer === value or
           raise ArgumentError, "not an integer:
#{key}=#{value.inspect}"
       end
-      [ :tcp_nodelay, :tcp_nopush, :ipv6only ].each do |key|
+      [ :tcp_nodelay, :tcp_nopush, :ipv6only, :reuseport ].each do |key|
         (value = options[key]).nil? and next
         TrueClass === value || FalseClass === value or
           raise ArgumentError, "not boolean: #{key}=#{value.inspect}"
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying