Dan Herrera <dan at revelationglobal.com> wrote:> Hi Eric,
>
> Thanks for getting in touch. We were also pleasantly surprised by the
> performance of unicorn!
>
> As to the blog post, we misspoke and will revise our language on the post
to
> reflect that what we encountered weren''t bugs, but issues with our
own
> setup.
Thanks for the clarification, I was worried :x
> Our app has a memory leak, and we haven''t yet settled on a monit
recipe to
> restart unicorn when the sub processes that are started run over a certain
> memory threshold. There were also some changes we needed to make with our
> nginx configuration, but again, they didn''t reflect any problems
with
> unicorn.
You can probably have monit just SIGQUIT the workers that run over the
memory threshold and just let the unicorn master restart them (note:
I''m
not a monit user).
If your OS supports it, you can try setrlimit in your after_fork hook.
Modern Linux 2.6 doesn''t support Process::RLIMIT_RSS, but you can still
do Process::RLIMIT_AS which is still somewhat useful[1].
after_fork do |server, worker|
Process.setrlimit(Process::RLIMIT_AS, size_in_kilobytes)
end
Then have a simple Rack middleware that traps NoMemoryError:
# XXX totally untested!, I''m not sure what to do if Process.kill
# here hits NoMemoryError, either...
class ExitOnOOM < Struct.new(:app)
def call(env)
begin
app.call(env)
rescue NoMemoryError
Process.kill(:QUIT, 0) # graceful exit signal
# or maybe just: exit!
end
end
end
[1] On modern GNU/Linux systems that don''t have RLIMIT_RSS, the VM size
taken by the mmap()''ed shared libraries usually doesn''t
change over the
lifetime of the process, so you can use that as a baseline and figure
out how far you want to go from there.... Of course, your Unicorns
shouldn''t be swapping so swap usage won''t have to be
accounted for.
Of course the proper solution to all this is to fix your memory leaks :)
I posted some notes here a few months back on the Mongrel list:
http://thread.gmane.org/gmane.comp.lang.ruby.mongrel.general/5760/focus=5766
> Engine Yard doesn''t currently support unicorn on their slices, but
once we
> have a satisfactory monit recipe and no other issues with our setup, we
plan
> to work with them so that they''ll hopefully change their mind. I
think
> this, and the monit recipe is what James meant when he said we
wouldn''t be
> using unicorn in the near future.
Ah, didn''t know that. I guess github was a special case where they
thought: "oh well, these guys are leaving us for another host anyways,
they can be our guinea pig!" :)
> Anyway, we are all great fans, and would like to thank all of you for all
of
> your hard work.
No problem :)
--
Eric Wong