mongrel unicorn - Jan 2013 - preload_app = true causing - ActiveModel::MissingAttributeError: missing attribute: some_attr

Greetings,

I''m getting this  - ActiveModel::MissingAttributeError: missing
attribute: some_attr  - on a random basis under a unicorn server,
running rails 3.2 and ruby 1.9.3
As the case of the last poster in the following thread -
https://github.com/rails/rails/issues/1906 -  I too am able to resolve
this issue by settingpreload_app to false.

However, this is not the behaviour I want to have.

Any inputs on that, or data I can provide around this issue?


Best Regards,

Avner Cohen

Avner Cohen <avner.cohen at fiverr.com> wrote:
> Greetings,
> 
> I''m getting this  - ActiveModel::MissingAttributeError: missing
> attribute: some_attr  - on a random basis under a unicorn server,
> running rails 3.2 and ruby 1.9.3
> As the case of the last poster in the following thread -
> https://github.com/rails/rails/issues/1906 -  I too am able to resolve
> this issue by settingpreload_app to false.
> 
> However, this is not the behaviour I want to have.
> 
> Any inputs on that, or data I can provide around this issue?
Did you try disconnecting in your before_fork hook and reconnecting
after_fork?

before_fork do |server, worker|
  # the following is highly recomended for Rails + "preload_app true"
  # as there''s no need for the master process to hold a connection
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!
end

after_fork do |server, worker|
  # the following is *required* for Rails + "preload_app true",
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
end

Open database handles must not be carried across fork.

Eric,
Thanks for the quick reply, and aplogies for not providing full info.
I do have these set up, here is my full configuration:

# -*- encoding : utf-8 -*-
worker_processes 4
working_directory "."
listen 3000
timeout 120

preload_app true

before_fork do |server, worker|
  defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!
end


after_fork do |server, worker|
  defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
end

Best Regards,

Avner Cohen

On Sun, Jan 20, 2013 at 10:04 AM, Eric Wong <normalperson at yhbt.net>
wrote:
> Avner Cohen <avner.cohen at fiverr.com> wrote:
>> Greetings,
>>
>> I''m getting this  - ActiveModel::MissingAttributeError:
missing
>> attribute: some_attr  - on a random basis under a unicorn server,
>> running rails 3.2 and ruby 1.9.3
>> As the case of the last poster in the following thread -
>> https://github.com/rails/rails/issues/1906 -  I too am able to resolve
>> this issue by settingpreload_app to false.
>>
>> However, this is not the behaviour I want to have.
>>
>> Any inputs on that, or data I can provide around this issue?
>
> Did you try disconnecting in your before_fork hook and reconnecting
> after_fork?
>
> before_fork do |server, worker|
>   # the following is highly recomended for Rails + "preload_app
true"
>   # as there''s no need for the master process to hold a connection
>   defined?(ActiveRecord::Base) and
>     ActiveRecord::Base.connection.disconnect!
> end
>
> after_fork do |server, worker|
>   # the following is *required* for Rails + "preload_app true",
>   defined?(ActiveRecord::Base) and
>     ActiveRecord::Base.establish_connection
> end
>
> Open database handles must not be carried across fork.

Avner Cohen <avner.cohen at fiverr.com> wrote:
> Eric,
> Thanks for the quick reply, and aplogies for not providing full info.
> I do have these set up, here is my full configuration:
> 
> # -*- encoding : utf-8 -*-
> worker_processes 4
> working_directory "."
> listen 3000
> timeout 120
> 
> preload_app true
> 
> before_fork do |server, worker|
>   defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
> end
> 
> 
> after_fork do |server, worker|
>   defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
> end
Interesting.  I haven''t kept up with changes in Rails over the
past few years.  Hopefully somebody else on the list, has...

Which database and adapter are you using?

Do you have anything else that opens a socket at application startup?
(e.g. memcache, redis, ..., especially anything that would interact
with ActiveRecord/Model)

Can you try just using one worker_process + preload_app=true and
doing "lsof -p" on both the PID of the master and single worker
to show open sockets (and any other FDs which may be inadvertantly
shared)?

The only stream socket which should be shared are the listeners.

Hi,
>> after_fork do |server, worker|
>>  defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
>> end
Not sure it would make a difference, but I always use this in the after_fork:

   defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.reconnect!



Cheers,
Lawrence

Thanks Eric, I''ll use this to explore the issue.

I suspect the issue is indeed around sockets, I found a reply from you
from a while back that seems to point on memcache as another thing I
need to manage at the worker level:
http://rubyforge.org/pipermail/mongrel-unicorn/2010-January/000309.html

That being said, I''m extremely surprised there is no published unicorn
startup scripts that consider the common use case of a deployment
stack that includes:
Rails + Active Record
Redis
Memcache

I''ll post my final script once I conclude the investigation.

Best Regards,
Avner Cohen

On Sun, Jan 20, 2013 at 11:17 AM, Eric Wong <normalperson at yhbt.net>
wrote:
> Avner Cohen <avner.cohen at fiverr.com> wrote:
>> Eric,
>> Thanks for the quick reply, and aplogies for not providing full info.
>> I do have these set up, here is my full configuration:
>>
>> # -*- encoding : utf-8 -*-
>> worker_processes 4
>> working_directory "."
>> listen 3000
>> timeout 120
>>
>> preload_app true
>>
>> before_fork do |server, worker|
>>   defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
>> end
>>
>>
>> after_fork do |server, worker|
>>   defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
>> end
>
> Interesting.  I haven''t kept up with changes in Rails over the
> past few years.  Hopefully somebody else on the list, has...
>
> Which database and adapter are you using?
>
> Do you have anything else that opens a socket at application startup?
> (e.g. memcache, redis, ..., especially anything that would interact
> with ActiveRecord/Model)
>
> Can you try just using one worker_process + preload_app=true and
> doing "lsof -p" on both the PID of the master and single worker
> to show open sockets (and any other FDs which may be inadvertantly
> shared)?
>
> The only stream socket which should be shared are the listeners.

> That being said, I''m extremely surprised there is no published
unicorn
> startup scripts that consider the common use case of a deployment
> stack that includes:
> Rails + Active Record
> Redis
> Memcache
http://unicorn.bogomips.org/examples/unicorn.conf.rb

In the after_fork you want:

    Rails.cache.reset

or if you''re using an older version of the memcache gem (i.e.
you''re not
using the dalli gem which I highly recommend) you may need this hack:

   if Rails.cache.is_a?(ActiveSupport::Cache::MemCacheStore)
     Rails.cache.instance_variable_get(:@data).reset
   end

If you use the UUID gem you need this in your after_fork:

   UUID.generator.next_sequence

As for redis:

   
https://github.com/redis/redis-rb/blob/master/examples/unicorn/unicorn.rb


Cheers,
Lawrence