Rails core - Jul 2010 - Rake tasks in production, application models, and Rails 2.3.8-3.0

Consider this simple rake task:

  task :heavy_process => :environment do
    MyModel.process
  end

The problem I''m seeing here is that this will work in development mode,
in
which autoloading kicks in when the "MyModel" constant is referenced
for the
first time, but will *fail* in production mode because of the behavior
described below. This issue has been bugging me for a while. I can''t
recall
it being a problem with Rails 2.3.5. Not sure if this is expected behavior.

If someone is writing a rake task that needs the application to be loaded,
he or she will make it depend on the ":environment" task which will
boot the
app in the respective environment. It will also set the `$rails_rake_task`
global to true. This has been around for a while.

However, since the 2.3.8 and 3.0 releases, this global prevents preloading
of application classes in production (or any environment in which
"cache_classes" setting is enabled). This is probably so the app boots
faster; users are expected to require what they need in the rake task. Here
is this logic in Rails 3:

      initializer :eager_load! do
        if config.cache_classes && !$rails_rake_task
          ActiveSupport.run_load_hooks(:before_eager_load, self)
          eager_load!
        end
      end

Going back to our initial rake task, our initial reaction is that we should
put `require my_model` before trying to reference MyModel. However, if
MyModel is linked to other models via associations, we also need to require
all of the associated models in case the `MyModel.process` method uses these
associations. In the end, wherever we put the require (or
`require_dependency`) statements, it just feels wrong and clumsy.

We''ve also been seeing migrations fail in production mode because they
reference a model to do some processing. Again, the biggest problem here is
that all this works like a breeze in development, making us developers feel
relaxed and secure, but then surprises us when we push to production.

It occurred to me that an easy solution might be to turn off
"cache_classes"
setting in production if $rails_rake_task is true. Comments on that?

Thanks

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

> . In the end, wherever we put the require (or
> `require_dependency`) statements, it just feels wrong and clumsy.
> We''ve also been seeing migrations fail in production mode because
they
> reference a model to do some processing. Again, the biggest problem here is
> that all this works like a breeze in development, making us developers feel
> relaxed and secure, but then surprises us when we push to production.
> It occurred to me that an easy solution might be to turn off
"cache_classes"
> setting in production if $rails_rake_task is true. Comments on that?
eager loading is a relatively new piece of functionality, and
disabling it for rake tasks should *ideally* simply fall back on the
const_missing hook.  I''m a little confused why it''s not doing
that.
Do you have config.threadsafe! set or something?

Either way, unless you have config.threadsafe! set, this is *clearly*
a bug.  And if you do, we should probably force the eager loading to
take place irrespective of whether it''s a rake task.

> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Core" group.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> To unsubscribe from this group, send email to
> rubyonrails-core+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/rubyonrails-core?hl=en.
>


-- 
Cheers

Koz

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Mon, Jul 12, 2010 at 02:15, Michael Koziarski
<michael@koziarski.com>wrote:
>
> Either way, unless you have config.threadsafe! set, this is *clearly*
> a bug.  And if you do, we should probably force the eager loading to
> take place irrespective of whether it''s a rake task.

You''re right, we had threadsafe on. I still think it''s a bug,
because rake
tasks are not threaded web server runtime and therefore shouldn''t guard
against race conditions and such.

For now I''ll just turn off threadsafe (I guess it''s not needed
for
Passenger?). Thanks Koz

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Passenger should be able to make use of threadsafe mode. Either way, the
problem here seems to be that Rake tasks have a kludge in this one area of
the system, but not in others. We either need to decide that Rake tasks are
never considered threadsafe (even if the threadsafe! flag is on), or eager
load for Rake tasks.

I''m surprised nobody else hit this before.

Yehuda Katz
Architect | Engine Yard
(ph) 718.877.1325


On Tue, Jul 13, 2010 at 2:58 AM, Mislav Marohnić
<mislav.marohnic@gmail.com>wrote:
> On Mon, Jul 12, 2010 at 02:15, Michael Koziarski
<michael@koziarski.com>wrote:
>
>>
>> Either way, unless you have config.threadsafe! set, this is *clearly*
>> a bug.  And if you do, we should probably force the eager loading to
>> take place irrespective of whether it''s a rake task.
>
>
> You''re right, we had threadsafe on. I still think it''s a
bug, because rake
> tasks are not threaded web server runtime and therefore shouldn''t
guard
> against race conditions and such.
>
> For now I''ll just turn off threadsafe (I guess it''s not
needed for
> Passenger?). Thanks Koz
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Core" group.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> To unsubscribe from this group, send email to
>
rubyonrails-core+unsubscribe@googlegroups.com<rubyonrails-core%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/rubyonrails-core?hl=en.
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

Actually, this is happening in my project since the beggining. I just 
didn''t have enough time to debug this behavior. I required the
necessary
files from inside the task to solve my needs...

Rodrigo.

Em 13-07-2010 13:49, Yehuda Katz escreveu:
> Passenger should be able to make use of threadsafe mode. Either way, 
> the problem here seems to be that Rake tasks have a kludge in this one 
> area of the system, but not in others. We either need to decide that 
> Rake tasks are never considered threadsafe (even if the threadsafe! 
> flag is on), or eager load for Rake tasks.
>
> I''m surprised nobody else hit this before.
>
> Yehuda Katz
> Architect | Engine Yard
> (ph) 718.877.1325
>
>
> On Tue, Jul 13, 2010 at 2:58 AM, Mislav Marohnić 
> <mislav.marohnic@gmail.com <mailto:mislav.marohnic@gmail.com>>
wrote:
>
>     On Mon, Jul 12, 2010 at 02:15, Michael Koziarski
>     <michael@koziarski.com <mailto:michael@koziarski.com>>
wrote:
>
>
>         Either way, unless you have config.threadsafe! set, this is
>         *clearly*
>         a bug.  And if you do, we should probably force the eager
>         loading to
>         take place irrespective of whether it''s a rake task.
>
>
>     You''re right, we had threadsafe on. I still think
it''s a bug,
>     because rake tasks are not threaded web server runtime and
>     therefore shouldn''t guard against race conditions and such.
>
>     For now I''ll just turn off threadsafe (I guess it''s
not needed for
>     Passenger?). Thanks Koz
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

> You''re right, we had threadsafe on. I still think it''s a
bug, because rake
> tasks are not threaded web server runtime and therefore shouldn''t
guard
> against race conditions and such.
> For now I''ll just turn off threadsafe (I guess it''s not
needed for
> Passenger?). Thanks Koz
config.threadsafe! just does:

    # Enable threaded mode. Allows concurrent requests to controller actions and
    # multiple database connections. Also disables automatic dependency loading
    # after boot, and disables reloading code on every request, as these are
    # fundamentally incompatible with thread safety.
    def threadsafe!
      self.preload_frameworks = true
      self.cache_classes = true
      self.dependency_loading = false
      self.action_controller.allow_concurrency = true
      self
    end

It''s the dependency_loading = false that''s messing you up. 
The
initializer should probably be checking that, not cache_classes.



> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Core" group.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> To unsubscribe from this group, send email to
> rubyonrails-core+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/rubyonrails-core?hl=en.
>


-- 
Cheers

Koz

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Wed, Jul 14, 2010 at 05:48, Michael Koziarski
<michael@koziarski.com>wrote:
>      self.dependency_loading = false
>
How about changing that line to:

  self.dependency_loading = defined?($rails_rake_task) &&
!!$rails_rake_task

Good idea?

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

> How about changing that line to:
>   self.dependency_loading = defined?($rails_rake_task) &&
!!$rails_rake_task
> Good idea?
That has a few potential side effects, but how does this look for the
2-3-stable change:

http://gist.github.com/476526




-- 
Cheers

Koz

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Thu, Jul 15, 2010 at 07:03, Michael Koziarski
<michael@koziarski.com>wrote:
>
> That has a few potential side effects, but how does this look for the
> 2-3-stable change: http://gist.github.com/476526

Looks good. Why just 2.3-stable?

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

> Looks good. Why just 2.3-stable?
2.3 needs a ''smallest possible bugfix'' change,  master can
''do the
right thing'' which probably *isn''t* adding another
conditional.


-- 
Cheers

Koz

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.