Hello, I''m quite consistently seeing an issue where Unicorn leaves its PID files in place after a clean shut down, which is causing some issues when deploying releases that update Unicorn config. We''re doing hot restarts with USR2+QUIT, and using Upstart for process supervision based on the presence of a PID file (this admittedly might be a mistake, but I''ve not seen any better solutions). This works fine in the case of a hot restart since Unicorn correctly updates the PID file, however when we change the configuration for Unicorn shutting down the master process doesn''t remove the PID file, so the supervising Upstart script doesn''t detect that the master has been shut down and bring up a new one. I''ve created a gist[1] of our Upstart and Unicorn configuration files. As I say, we might just be doing something stupid here, but Google doesn''t seem to have shed any light on the issue. Thanks for any help, Jon [1] https://gist.github.com/jellybob/3789c3f3a00989b63d74 _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying
Jon Wood <jon@hubbub.co.uk> wrote:> Hello, > > I''m quite consistently seeing an issue where Unicorn leaves its PID > files in place after a clean shut down, which is causing some issues > when deploying releases that update Unicorn config. > > We''re doing hot restarts with USR2+QUIT, and using Upstart for process > supervision based on the presence of a PID file (this admittedly might > be a mistake, but I''ve not seen any better solutions).It''s probably better to test the listening port unicorn uses (perhaps setup a generic health check endpoint) via socat/curl to detect if unicorn is down.> This works fine in the case of a hot restart since Unicorn correctly > updates the PID file, however when we change the configuration for > Unicorn shutting down the master process doesn''t remove the PID file, > so the supervising Upstart script doesn''t detect that the master has > been shut down and bring up a new one. > > I''ve created a gist[1] of our Upstart and Unicorn configuration files. > As I say, we might just be doing something stupid here, but Google > doesn''t seem to have shed any light on the issue.> [1] https://gist.github.com/jellybob/3789c3f3a00989b63d74Do you have anything logging to stderr_path? That might give you a better idea of what''s wrong. Unfortunately, PID files are always bound to be racy so perhaps supervisor scripts should test and endpoint of an app using socat or curl. _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying