mongrel unicorn - Jul 2010 - Autokill Workers which have grown too big

Hey,


at Euruko 2010 I saw a presentation where Florian Hanke presented parts 
of a unicorn config which killed workers once they''ve used up too much 
memory.

The snipped looks like this:

RackResponder = lambda do |env|
   possibly_commit_harakiri ...
end

# Commit Harakiri/Seppuku, kill yourself. #
def possibly_commit_harakiri
   return unless commit_harakiri?
   Process.kill ?KILL?, Process.pid
end

Now I couldn''t find anything about that RackResponder part on the web 
and was hoping that Florian himself or somebody else on the list could 
clue me in how that works.

Also I''d like to know if there is a way to get the current memory 
footprint of a worker other than something like:

%x(ps -o rss= -p #{Process.pid}).to_i

Kind regards,

John

hukl <hukl at berlin.ccc.de> wrote:
> Also I''d like to know if there is a way to get the current memory
> footprint of a worker other than something like:
>
> %x(ps -o rss= -p #{Process.pid}).to_i
Hi John,

I know under Linux it''s possible to parse /proc/$$/stat and also
(the bigger, but more human-readable) /proc/$$/status to get
current memory usage.

There''s also the sys-proctable gem, too, which I haven''t used.

-- 
Eric Wong

hukl <hukl <at> berlin.ccc.de> writes:
> at Euruko 2010 I saw a presentation where Florian Hanke presented parts 
> of a unicorn config which killed workers once they''ve used up too
much
> memory.
>
> The snipped looks like this:
> 
> RackResponder = lambda do |env|
>    possibly_commit_harakiri ...
> end
> 
> # Commit Harakiri/Seppuku, kill yourself. #
> def possibly_commit_harakiri
>    return unless commit_harakiri?
>    Process.kill ?KILL?, Process.pid
> end
> 
> Now I couldn''t find anything about that RackResponder part on the
web
> and was hoping that Florian himself or somebody else on the list could 
> clue me in how that works.
Hi John

I thought I''d answer your question here too (not just on Github) for
others to see. Perhaps someone else has similar questions. 

Sure. Using the RackResponder constant was just my way of naming the
application. It could be any old lambda.

I didn''t want the GC to run and slow down any queries (in the search
engine). The way the whole thing works is this:
1. The forked Unicorn child switches off the GC just after forking.

unicorn.rb:
  after_fork do |_, _|
    GC.disable
  end

2. On each request, the child takes a look at how many requests
it has already answered (or how much memory it uses, or any metric). If a
threshold has been passed, it will honorably kill itself, by sending
itself the QUIT signal. It will then finish responding the request and die.

A Rack middleware for the request counting metric can be found here: 
http://bit.ly/bjkbRG

3. The Unicorn master process notices that a child has gone and
   forks a new one.

4. The OS ? rather than the GC ? cleans up the memory mess left
   by the dead child ;)
> Also I''d like to know if there is a way to get the current memory 
> footprint of a worker other than something like:
> 
> %x(ps -o rss= -p #{Process.pid}).to_i
On Linux, the /proc folder holds a lot of information on the currently
running processes.
One can for example use
cat /proc/some_proc_id/statm
to get Memory information.

Cheers and have fun,
   Florian