mongrel unicorn - Jan 2010 - 'before_fork' is executed in parallel by two processes ¿?

Hi, by adding `echo $$ >> /tmp/kk` into ''before_fork''
I''ve realized that two
pids are printed to the file when starting Unicorn.
This means that ''before_fork'' is being runned by two different
processes in
parallel. Is it the expected behaviour? perhaps something very wrong in my 
configuration?

Thanks.

-- 
I?aki Baz Castillo <ibc at aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo
escribi?:
> Hi, by adding `echo $$ >> /tmp/kk` into
''before_fork'' I''ve realized that
>  two pids are printed to the file when starting Unicorn.
> This means that ''before_fork'' is being runned by two
different processes in
> parallel. Is it the expected behaviour? perhaps something very wrong in my
> configuration?
I''ve totally lost. before_fork is executed in parallel by the number of
workers set in the "worker_processes". I''ve tested the
following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
  end

  after_fork do |server, worker|
    `echo "after_fork  $$" >> /tmp/kk`
  end

And this is the file output:

  before_fork 3447
  after_fork  3456
  before_fork 3454
  before_fork 3463
  after_fork  3464
  after_fork  3472


Could I know what is happening please? This really breaks the concept I had.
In fact I use before_fork to check database connections and so, but
I''ve
realized that such check if done N (number of workers).

Thanks for any help.




-- 
I?aki Baz Castillo <ibc at aliax.net>

El Martes, 5 de Enero de 2010, I?aki Baz Castillo
escribi?:
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>   end
> And this is the file output:
> 
>   before_fork 3447
>   after_fork  3456
>   before_fork 3454
>   before_fork 3463
>   after_fork  3464
>   after_fork  3472

Hummm, this could be something strange related to how `` works...

If I do the following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
     File.open("/tmp/kk", "a") do |f|
       f.puts "$$=#{$$}"
     end
  end

then I get:

  before_fork 11989
  $$=11977
  before_fork 11994
  $$=11977
  before_fork 12000
  $$=11977

So I''m totally lost. However if I test my databases in
"before_fork" then I
see N number of DB connections (N = workers num).





-- 
I?aki Baz Castillo <ibc at aliax.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, I?aki Baz Castillo escribi?:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >   end
> 
> 
> > And this is the file output:
> > 
> >   before_fork 3447
> >   after_fork  3456
> >   before_fork 3454
> >   before_fork 3463
> >   after_fork  3464
> >   after_fork  3472
> 
> 
> Hummm, this could be something strange related to how `` works...
> 
> If I do the following:
> 
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>      File.open("/tmp/kk", "a") do |f|
>        f.puts "$$=#{$$}"
>      end
>   end
Odd, I''m confused by the way `` is working there too...  I wonder if
this is a Ruby bug...  Anyhow, in your use case system() or your
File.open block should be the way to go.  Don''t use `` when system()
will do.
> then I get:
> 
>   before_fork 11989
>   $$=11977
>   before_fork 11994
>   $$=11977
>   before_fork 12000
>   $$=11977
> 
> So I''m totally lost. However if I test my databases in
"before_fork" then I
> see N number of DB connections (N = workers num).
How are you testing the database?  `` or via Ruby code?

-- 
Eric Wong

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > So I''m totally lost. However if I test my databases in
"before_fork" then
> > I  see N number of DB connections (N = workers num).
> 
> How are you testing the database?  `` or via Ruby code?
I use Ruby Sequel ORM.
I''ve also added the following code to before_fork:

  File.open("/tmp/lalala", "ab") do |f|
    f.puts "My PID is #{$$}"
  end

I confirm that there are two lines printed in that file when Unicorn starts 
(being 2 the number of workers). Is it the expected behavior?

Regards.

-- 
I?aki Baz Castillo <ibc at aliax.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > So I''m totally lost. However if I test my databases in
"before_fork" then
> > > I  see N number of DB connections (N = workers num).
> > 
> > How are you testing the database?  `` or via Ruby code?
> 
> I use Ruby Sequel ORM.
> I''ve also added the following code to before_fork:
> 
>   File.open("/tmp/lalala", "ab") do |f|
>     f.puts "My PID is #{$$}"
>   end
> 
> I confirm that there are two lines printed in that file when Unicorn starts
> (being 2 the number of workers). Is it the expected behavior?
Yes, before_fork and after_fork are both called for every worker forked.

-- 
Eric Wong

El Martes, 5 de Enero de 2010, I?aki Baz Castillo
escribi?:
> I confirm that there are two lines printed in that file when Unicorn starts
> (being 2 the number of workers). Is it the expected behavior?
Simpler example:

-----------------
worker_processes 3

before_fork do |server, worker|
  puts "*** I''my PID #{$$}"
end
----------------


Then run Unicorn in foreground and I see:

  *** I''m PID 8791
  *** I''m PID 8791
  *** I''m PID 8791

8791 is the PID of the master process, so yes, it seems that workers
don''t
exist yet in before_fork (as obviously expected). However the block into 
before_fork is executed "worker_processes" times by master process.

Do I miss something?

-- 
I?aki Baz Castillo <ibc at aliax.net>

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> Yes, before_fork and after_fork are both called for every worker forked.
So if I just want to test a DB connection then I do better wrtitting such code 
out of before_fork in the config file, right? (at least it''s the
workaround
that works for me).

Thanks. 


-- 
I?aki Baz Castillo <ibc at aliax.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> 
> > Yes, before_fork and after_fork are both called for every worker
forked.
> 
> So if I just want to test a DB connection then I do better wrtitting such
code
> out of before_fork in the config file, right? (at least it''s the
workaround
> that works for me).
You can also do it on a certain worker only:

  before_fork do |server, worker|
    if worker.nr == 0
      ...
    end
  end

-- 
Eric Wong

El Mi?rcoles, 6 de Enero de 2010, Eric Wong escribi?:
> I?aki Baz Castillo <ibc at aliax.net> wrote:
> > El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > Yes, before_fork and after_fork are both called for every worker
> > > forked.
> >
> > So if I just want to test a DB connection then I do better wrtitting
such
> > code out of before_fork in the config file, right? (at least
it''s the
> > workaround that works for me).
> 
> You can also do it on a certain worker only:
> 
>   before_fork do |server, worker|
>     if worker.nr == 0
>       ...
>     end
>   end
So if I''m not wrong when before_fork block is runned the $stderr is
already
redirected to the IO set in "stderr_path", rigth?
Then any error in the config file into before_fork would not vi raised to the 
terminal screen but to the stderr (if it has been redirected).

Then I see no advantage on using what you suggest ("if worker.nr == 0
...")
over adding such code at the top of the config file. Do I miss something?

Thanks a lot.



-- 
I?aki Baz Castillo <ibc at aliax.net>

I?aki Baz Castillo <ibc at aliax.net> wrote:
> El Mi?rcoles, 6 de Enero de 2010, Eric Wong escribi?:
> > I?aki Baz Castillo <ibc at aliax.net> wrote:
> > > El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> > > > Yes, before_fork and after_fork are both called for every
worker
> > > > forked.
> > >
> > > So if I just want to test a DB connection then I do better
wrtitting such
> > > code out of before_fork in the config file, right? (at least
it''s the
> > > workaround that works for me).
> > 
> > You can also do it on a certain worker only:
> > 
> >   before_fork do |server, worker|
> >     if worker.nr == 0
> >       ...
> >     end
> >   end
> 
> So if I''m not wrong when before_fork block is runned the $stderr
is already
> redirected to the IO set in "stderr_path", rigth?
Yes, stderr is redirected before any before_fork hooks run.
> Then any error in the config file into before_fork would not vi raised to
the
> terminal screen but to the stderr (if it has been redirected).
If it''s a Ruby syntax error, then it would''ve been sent to the
terminal at startup.  Otherwise it''ll be redirected.
> Then I see no advantage on using what you suggest ("if worker.nr == 0
...")
> over adding such code at the top of the config file. Do I miss something?
I depends on whether you use preload_app or not, if you use preload_app,
then you''ll get to have access to any objects your app creates at
initialization.

-- 
Eric Wong

El Martes, 5 de Enero de 2010, Eric Wong escribi?:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >      File.open("/tmp/kk", "a") do |f|
> >        f.puts "$$=#{$$}"
> >      end
> >   end
> 
> Odd, I''m confused by the way `` is working there too...  I wonder
if
> this is a Ruby bug...  Anyhow, in your use case system() or your
> File.open block should be the way to go.  Don''t use `` when
system()
> will do.
I think that the above makse sense since `` creates a new process (with its 
own pid). So each time it''s called (once per worker) it will be a
different
process. 


-- 
I?aki Baz Castillo <ibc at aliax.net>