Rails - Mar 2005 - Can I fork() in a model?

Hi Everyone,

  I have a long running background process that I would like to do with the
DB but I want the UI to return immediately.

  Can I fork within model class method?  I am running under FastCGI.

  How should the child process terminate itself at the end of the function?
is Kernel.exit! appropriate? or is plain old Kernel.exit better?  Should I
return a status?

  My code thus far:

  def self.sortBG(spec)
   
    pid=fork
    
    # in the parent, return now
    if (pid)
      Process.detach(pid)
      return
    end
    
    # in the child, go ahead with the sort.
   
    # ... DO SOME LENGTHY PROCESSING ...

    Kernel.exit! 
  end

  I am experimenting now but hoped someone out there may have
tried this already.

  Thanks so much,

   Lee


-- 
Naxos Technology

On Sat, 12 Mar 2005, LN wrote:
> Hi Everyone,
>
>  I have a long running background process that I would like to do with the
>  DB but I want the UI to return immediately.
>
>  Can I fork within model class method?  I am running under FastCGI.
>
>  How should the child process terminate itself at the end of the function?
>  is Kernel.exit! appropriate? or is plain old Kernel.exit better?  Should I
>  return a status?
>
>  My code thus far:
>
>  def self.sortBG(spec)
>
>    pid=fork
>
>    # in the parent, return now
>    if (pid)
>      Process.detach(pid)
>      return
>    end
this simply starts a thread that periodically waits for your child if that
what you want...
>
>    # in the child, go ahead with the sort.
>
>    # ... DO SOME LENGTHY PROCESSING ...
>
>    Kernel.exit!
>  end
>
>  I am experimenting now but hoped someone out there may have
> tried this already.
>
>  Thanks so much,

the biggest issue with the above is that your system will accumulate zombies
if (when - remeber the process manager) your fastcgi process exits.  i *think*
what you want to do is become a child of init: being careful not not to
generate any out/err/put... something like:

   jib:~/eg/ruby > cat dp.rb

     require ''fileutils''

     class DetachedProcess
       def initialize(*a, &b)
         @b = b
       end
       def run
         pid = fork do
           [ STDIN, STDOUT, STDERR ].each{|io| io.reopen
''/dev/null'', ''r+''}
           fork{ @b.call(*@a) }
           exit!
         end
         Process::waitpid pid
       end
     end

     dp        DetachedProcess::new do
         sleep 5
         FileUtils::touch ''when_grandchild_died''
       end

     dp.run

     puts "#{ Process::pid } exited @ #{ Time::now }"


   jib:~/eg/ruby > ruby dp.rb && sleep 5 && stat
when_grandchild_died

     10021 exited @ Sat Mar 12 19:02:28 MST 2005

       File: `when_grandchild_died''
       Size: 0               Blocks: 0          IO Block: 8192   Regular File
     Device: bh/11d  Inode: 16449677    Links: 1
     Access: (0664/-rw-rw-r--)  Uid: (  447/ ahoward)   Gid: (  447/ ahoward)
     Access: 2005-03-12 19:02:33.000000000 -0700
     Modify: 2005-03-12 19:02:33.000000000 -0700
     Change: 2005-03-12 19:02:33.000000000 -0700


this won''t leave behind any zombies AND you don''t have to
collect the exit
status directly in your app.

HTH.

kind regards.

-a
-- 
==============================================================================|
EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself.  --Shunryu Suzuki
===============================================================================

I tried it and it seems to work.  But now I am having a problem that
the shared database connection between parent and child gets 
corrupted and I get "packets out of order" errors from the MySQL
library.

Does anyone know how to drop the rails connection and open a
new one (in the child process, of course), I couldn''t find it in
the API docs?
> 
> this simply starts a thread that periodically waits for your child 
> if that what you want...
Actually I think it starts a a child while the parent returns.
> the biggest issue with the above is that your system will accumulate 
> zombies if (when - remeber the process manager) your fastcgi process 
> exits.  i *think* what you want to do is become a child of init: 
I didn''t try becoming a child of init, but calling detach prevents the
zombie problem.

You bring up a good point about redirecting the IO, I''m not generating
any so it hasn''t been a problem.  But I need to study your example a
bit more (I''m a ruby newbie).

Thanks,

Lee


-- 
Naxos Technology

On Sat, 12 Mar 2005, LN wrote:
> I tried it and it seems to work.  But now I am having a problem that the
> shared database connection between parent and child gets corrupted and I
get
> "packets out of order" errors from the MySQL library.
i have had the same issue with forking with open sqlite handles and created a
library to handle it.  the basic notion is that you setup a co-process that
does all the forking and waiting for you as a drb server.  it uses my slave
library from the RAA.  you can read about that here

   http://www.codeforpeople.com/lib/ruby/slave/

and check out sample usage (which is essentially exactly what you are doing)
in the rq clustering lib i have on the raa.  closing the mysql handle first
would be a MUCH easier solution...
> Does anyone know how to drop the rails connection and open a new one (in
the
> child process, of course), I couldn''t find it in the API docs?
dunno.
>> this simply starts a thread that periodically waits for your child
>> if that what you want...
>
> Actually I think it starts a a child while the parent returns.
nope.  from the c source:

/*
  *  call-seq:
  *     Process.detach(pid)   => thread
  *
  *  Some operating systems retain the status of terminated child
  *  processes until the parent collects that status (normally using
  *  some variant of <code>wait()</code>. If the parent never
collects
  *  this status, the child stays around as a <em>zombie</em>
process.
  *  <code>Process::detach</code> prevents this by setting up a
  *  separate Ruby thread whose sole job is to reap the status of the
  *  process _pid_ when it terminates. Use <code>detach</code>
  *  only when you do not intent to explicitly wait for the child to
  *  terminate.  <code>detach</code> only checks the status
  *  periodically (currently once each second).
...
...
...
>> the biggest issue with the above is that your system will accumulate
>> zombies if (when - remeber the process manager) your fastcgi process
>> exits.  i *think* what you want to do is become a child of init:
>
> I didn''t try becoming a child of init, but calling detach prevents
the
> zombie problem.
again, not quite - it''s a race condition :

   jib:~ > cat zombie.rb
   pid = fork{ sleep }
   Process::detach pid; exit

   jib:~ > ruby zombie.rb

   jib:~ > ps
     PID TTY          TIME CMD
    9887 pts/20   00:00:00 bash
   10372 pts/20   00:00:00 ruby
   10373 pts/20   00:00:00 ps

you WILL end up with zombies if you use that approach.
> You bring up a good point about redirecting the IO, I''m not
generating any
> so it hasn''t been a problem.  But I need to study your example a
bit more
> (I''m a ruby newbie).
it''s basically straight out of the stephen''s book if c is
easier to read for
you.

cheers.

-a
-- 
==============================================================================|
EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself.  --Shunryu Suzuki
===============================================================================