We are hosting multiple Rails applications on same server and using Passenger + Apache. I can see Unicorn be useful for seamless deploys, few questions running it on Rails platform with 50+ rails apps: a) How does unicorn scale if app is known to be slow with long running quieres to database? Is there a way to a have master worker spin up new workers aka dynamic workers? b) Can Unicorn be configured to start with zero worker and master spawning first worker as first request comes in? Very useful to stack multiple apps that are not used all the time.? From (a) and (b) I guess I am asking if worker_processes can scale from 0 to X if database is slow or if there is no need to have workers running if the application is not gonna be used (in Freetrails and Sandboxes) c) Unicorn for zero lost connection (no downtime during deploy and changes) needs new workers spinned up before old workers are sent USR2 signal, does this not mean that one has to provision double memory for servers? --Amit
On Tue, Feb 26, 2013 at 4:08 PM, Amol Dev <devamol at yahoo.com> wrote:> We are hosting multiple Rails applications on same server and using Passenger + Apache. I can see Unicorn be useful for seamless deploys, few questions running it on Rails platform with 50+ rails apps: > > a) How does unicorn scale if app is known to be slow with long running quieres to database? Is there a way to a have master worker spin up new workers aka dynamic workers?Unicorn is designed for short-running requests. Long-running requests are best taken care of with app servers designed for that purpose, for example Rainbows! or Phusion Passenger Enterprise 4. Both app servers can be used in combination with Apache or Nginx. Rainbows! is designed for a reverse proxy setup while Phusion Passenger Enterprise 4 can also integrate with the web server. As far as I know only Phusion Passenger supports dynamic workers. Unicorn and Rainbows! are both designed for a static number of workers. Unicorn and Rainbows! allow modifying the number of workers during runtime through signals, but not automatically according to traffic as is done by Phusion Passenger.> b) Can Unicorn be configured to start with zero worker and master spawning first worker as first request comes in? Very useful to stack multiple apps that are not used all the time.No, because it does not support spawning processes based on traffic. As far as I know only Phusion Passenger does this.> c) Unicorn for zero lost connection (no downtime during deploy and changes) needs new workers spinned up before old workers are sent USR2 signal, does this not mean that one has to provision double memory for servers?Yes it does. Although it''s possible to change this behavior through a script that sends signals in a certain order. -- Phusion | Ruby & Rails deployment, scaling and tuning solutions Web: http://www.phusion.nl/ E-mail: info at phusion.nl Chamber of commerce no: 08173483 (The Netherlands)
Thanks. Did went through Phusion Passenger Enterprise 4 features, you guys did neat work Hongli. However the cost of it is driving us to check on alternate solutions. Can one signal unicorn to increase workers or just spin new set of workers.?Any one tried alicorn? https://github.com/bensomers/alicorn. Provisioning double memory on deploy can be solved differently by spinning new server that has upgrade and moving load balancer to divert trafic, the netflix way.? ----- Original Message ----- From: Hongli Lai <hongli at phusion.nl> To: unicorn list <mongrel-unicorn at rubyforge.org> Cc: Sent: Tuesday, February 26, 2013 7:36 AM Subject: Re: Unicorn on shared apps platform On Tue, Feb 26, 2013 at 4:08 PM, Amol Dev <devamol at yahoo.com> wrote:> We are hosting multiple Rails applications on same server and using Passenger + Apache. I can see Unicorn be useful for seamless deploys, few questions running it on Rails platform with 50+ rails apps: > > a) How does unicorn scale if app is known to be slow with long running quieres to database? Is there a way to a have master worker spin up new workers aka dynamic workers?Unicorn is designed for short-running requests. Long-running requests are best taken care of with app servers designed for that purpose, for example Rainbows! or Phusion Passenger Enterprise 4. Both app servers can be used in combination with Apache or Nginx. Rainbows! is designed for a reverse proxy setup while Phusion Passenger Enterprise 4 can also integrate with the web server. As far as I know only Phusion Passenger supports dynamic workers. Unicorn and Rainbows! are both designed for a static number of workers. Unicorn and Rainbows! allow modifying the number of workers during runtime through signals, but not automatically according to traffic as is done by Phusion Passenger.> b) Can Unicorn be configured to start with zero worker and master spawning first worker as first request comes in? Very useful to stack multiple apps that are not used all the time.No, because it does not support spawning processes based on traffic. As far as I know only Phusion Passenger does this.> c) Unicorn for zero lost connection (no downtime during deploy and changes) needs new workers spinned up before old workers are sent USR2 signal, does this not mean that one has to provision double memory for servers?Yes it does. Although it''s possible to change this behavior through a script that sends signals in a certain order. -- Phusion | Ruby & Rails deployment, scaling and tuning solutions Web: http://www.phusion.nl/ E-mail: info at phusion.nl Chamber of commerce no: 08173483 (The Netherlands) _______________________________________________ Unicorn mailing list - mongrel-unicorn at rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying
> Can one signal unicorn to increase workers or just spin new set of workers. Any one tried alicorn? https://github.com/bensomers/alicorn.I''m the author of alicorn; I''ve tried it :). I only know of one team that''s currently running it in production, but they''ve been using it with no difficulties for about six months now. It''s meant to solve exactly the problem you have, dynamically scaling unicorn workers, though it can''t idle an app at zero workers (I think it technically could; but the results would be bad). Happy to help you with it if needed.> Provisioning double memory on deploy can be solved differently by spinning new server that has upgrade and moving load balancer to divert trafic, the netflix way.I don''t like doing that just because of the time it takes to spin up new servers (even on EC2, it''s not instant). If you''re running multiple apps on one box, then you can cheat a little bit, because you won''t need enough memory for running all the apps twice, just enough to run everything once + one app twice. -ben
Hongli Lai <hongli at phusion.nl> wrote:> On Tue, Feb 26, 2013 at 4:08 PM, Amol Dev <devamol at yahoo.com> wrote: > > We are hosting multiple Rails applications on same server and using > > Passenger + Apache. I can see Unicorn be useful for seamless > > deploys, few questions running it on Rails platform with 50+ rails > > apps:> Rainbows! is designed for a reverse proxy setupThat''s not true, Rainbows! was designed to serve clients directly. On the other hand, I do not know if anybody uses Rainbows! that way (or at all in production). Anyways everything else is accurate. unicorn isn''t for cases where you''re cramming many apps on one box and trying to squeak by on memory usage.
On Tue, Feb 26, 2013 at 6:26 PM, Eric Wong <normalperson at yhbt.net> wrote:> That''s not true, Rainbows! was designed to serve clients directly. > On the other hand, I do not know if anybody uses Rainbows! that way > (or at all in production).I didn''t know that. Is it also the recommended setup though? Are you fairly sure there are no exploits in the request parser and that kind of stuff, that web servers usually shield for in reverse proxy setups? -- Phusion | Ruby & Rails deployment, scaling and tuning solutions Web: http://www.phusion.nl/ E-mail: info at phusion.nl Chamber of commerce no: 08173483 (The Netherlands)
Hongli Lai <hongli at phusion.nl> wrote:> On Tue, Feb 26, 2013 at 6:26 PM, Eric Wong <normalperson at yhbt.net> wrote: > > That''s not true, Rainbows! was designed to serve clients directly. > > On the other hand, I do not know if anybody uses Rainbows! that way > > (or at all in production). > > I didn''t know that. Is it also the recommended setup though? Are you > fairly sure there are no exploits in the request parser and that kind > of stuff, that web servers usually shield for in reverse proxy setups?Yes, it is recommended to expose Rainbows! directly to clients. I''ve tested the parser extensively over many years. As far as I can tell, there are no exploits... but also no users, nor warranty :>
> I''m the author of alicorn; I''ve tried it :). I only know of one team > that''s currently running it in production, but they''ve been using it > with no difficulties for about six months now. It''s meant to solve > exactly the problem you have, dynamically scaling unicorn workers, > though it can''t idle an app at zero workers (I think it technically > could; but the results would be bad). Happy to help you with it if > needed.Thanks. Looked at?https://github.com/bensomers/alicorn/blob/master/lib/alicorn/scaler.rb nice use of TTIN and TTOU. It is good out of band scaling option.?Will surely need your help for zero worker if I end up using alicorn.?> won''t need enough memory for running all the apps twice, just enough > to run everything once + one app twice.Agree. --Amit ----- Original Message -----> From: Ben Somers <somers.ben at gmail.com> > To: unicorn list <mongrel-unicorn at rubyforge.org> > Cc: > Sent: Tuesday, February 26, 2013 8:55 AM > Subject: Re: Unicorn on shared apps platform > >> Can one signal unicorn to increase workers or just spin new set of workers. > Any one tried alicorn? https://github.com/bensomers/alicorn. > > I''m the author of alicorn; I''ve tried it :). I only know of one team > that''s currently running it in production, but they''ve been using it > with no difficulties for about six months now. It''s meant to solve > exactly the problem you have, dynamically scaling unicorn workers, > though it can''t idle an app at zero workers (I think it technically > could; but the results would be bad). Happy to help you with it if > needed. > >> Provisioning double memory on deploy can be solved differently by spinning > new server that has upgrade and moving load balancer to divert trafic, the > netflix way. > > I don''t like doing that just because of the time it takes to spin up > new servers (even on EC2, it''s not instant). If you''re running > multiple apps on one box, then you can cheat a little bit, because you > won''t need enough memory for running all the apps twice, just enough > to run everything once + one app twice. > > -ben > _______________________________________________ > Unicorn mailing list - mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Do not quote signatures (like this one) or top post when replying >
> unicorn isn''t for cases where > you''re cramming many apps on one box and trying to squeak by on memory > usage.Thanks. The discussion helped a lot. ----- Original Message -----> From: Eric Wong <normalperson at yhbt.net> > To: unicorn list <mongrel-unicorn at rubyforge.org> > Cc: > Sent: Tuesday, February 26, 2013 9:26 AM > Subject: Re: Unicorn on shared apps platform > > Hongli Lai <hongli at phusion.nl> wrote: >> On Tue, Feb 26, 2013 at 4:08 PM, Amol Dev <devamol at yahoo.com> wrote: >> > We are hosting multiple Rails applications on same server and using >> > Passenger + Apache. I can see Unicorn be useful for seamless >> > deploys, few questions running it on Rails platform with 50+ rails >> > apps: > >> Rainbows! is designed for a reverse proxy setup > > That''s not true, Rainbows! was designed to serve clients directly. > On the other hand, I do not know if anybody uses Rainbows! that way > (or at all in production). > > Anyways everything else is accurate.? unicorn isn''t for cases where > you''re cramming many apps on one box and trying to squeak by on memory > usage. > _______________________________________________ > Unicorn mailing list - mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Do not quote signatures (like this one) or top post when replying >