rspec users - Jul 2009 - Loading (or stubbing) data before model loading

Hi,

A Rails project I''m working on has, for better or worse, some data- 
driven structure in its models: a few model classes need certain  
values (e.g. some enumerations) to be available in the database at  
load time in order for certain bits of metaprogramming to kick off  
correctly. It would be possible to do this all lazily instead of when  
the classes load, but in reality the database values are always there  
and everything already works beautifully so it''s hard to justify  
putting in the effort required to be that lazy.

The problem is that RSpec starts with an empty test database and I  
can''t see the best way or place to prepopulate it before the models  
get loaded (i.e. when Rails bootstraps, since config.cache_classes =  
true), ideally so that both rake spec and script/spec do the correct  
setup in time. It would be incredibly convenient to be able to use  
Fixtures.create_fixtures to do the loading so that the required data  
can live in YAML fixtures, but maybe it''s optimistic to try to do that
before Rails has loaded.

Or maybe I''m just going about this completely the wrong way, and  
someone has a clever idea about what the right way is? That would be  
brilliant.

Cheers,
-Tom

On 21 Jul 2009, at 20:58, Tom Stuart wrote:
> Hi,
>
> A Rails project I''m working on has, for better or worse, some
data-
> driven structure in its models: a few model classes need certain  
> values (e.g. some enumerations) to be available in the database at  
> load time in order for certain bits of metaprogramming to kick off  
> correctly. It would be possible to do this all lazily instead of  
> when the classes load, but in reality the database values are always  
> there and everything already works beautifully so it''s hard to  
> justify putting in the effort required to be that lazy.
>
> The problem is that RSpec starts with an empty test database and I  
> can''t see the best way or place to prepopulate it before the
models
> get loaded (i.e. when Rails bootstraps, since config.cache_classes =  
> true), ideally so that both rake spec and script/spec do the correct  
> setup in time. It would be incredibly convenient to be able to use  
> Fixtures.create_fixtures to do the loading so that the required data  
> can live in YAML fixtures, but maybe it''s optimistic to try to do
> that before Rails has loaded.
>
> Or maybe I''m just going about this completely the wrong way, and  
> someone has a clever idea about what the right way is? That would be  
> brilliant.
One challenge for you before we dig into how to solve it the way  
you''ve suggested: do you really need to keep these enumerables in the  
database, or could you keep them in code? That is, is there a valid  
use case where they will be altered at run-time, or do they change so  
seldom that it would be OK to have to release a new version of the  
code when they need to be changed?

I ask because these things tend to get a lot easier if you''re not  
having to mess around with databases, so it''s always worth pushing  
back on that one first.

cheers,
Matt Wynne

http://mattwynne.net
+447974 430184

On Tue, Jul 21, 2009 at 2:58 PM, Tom Stuart<tom at experthuman.com>
wrote:
> Hi,
>
> A Rails project I''m working on has, for better or worse, some
data-driven
> structure in its models: a few model classes need certain values (e.g. some
> enumerations) to be available in the database at load time in order for
> certain bits of metaprogramming to kick off correctly. It would be possible
> to do this all lazily instead of when the classes load, but in reality the
> database values are always there and everything already works beautifully
so
> it''s hard to justify putting in the effort required to be that
lazy.
>
> The problem is that RSpec starts with an empty test database
Not exactly. RSpec starts with whatever database you have and rolls
back to that state after each example provided you''re only using
before(:each) to set up state before each example.

Any pre-existing database state will remain, and any state that you
set up in before(:suite) or before(:all) will not be rolled back
unless you do so explicitly.

HTH,
David
> and I can''t see
> the best way or place to prepopulate it before the models get loaded (i.e.
> when Rails bootstraps, since config.cache_classes = true), ideally so that
> both rake spec and script/spec do the correct setup in time. It would be
> incredibly convenient to be able to use Fixtures.create_fixtures to do the
> loading so that the required data can live in YAML fixtures, but maybe
it''s
> optimistic to try to do that before Rails has loaded.
>
> Or maybe I''m just going about this completely the wrong way, and
someone has
> a clever idea about what the right way is? That would be brilliant.
>
> Cheers,
> -Tom
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On 22 Jul 2009, at 13:57, David Chelimsky wrote:
>> The problem is that RSpec starts with an empty test database
> Not exactly. RSpec starts with whatever database you have and rolls
> back to that state after each example provided you''re only using
> before(:each) to set up state before each example.
Quite right, and I should''ve been more specific: rake spec sets up an  
empty database and then runs RSpec. In the end I solved my problem by  
extending the db:test:prepare task to load the fixtures after cloning  
the development database structure, which in hindsight is obviously  
the right way to fix it.
> Any pre-existing database state will remain, and any state that you
> set up in before(:suite) or before(:all) will not be rolled back
> unless you do so explicitly.
On an unrelated note, why do I occasionally see leakage of fixture  
data between specs? I''d expect RSpec''s explicit state
rollback, plus
Rails'' transactional fixtures, to mean that fixtures loaded in one  
example would always be invisible to another, but that doesn''t seem to
be the case; I''ve seen many instances of mysteriously-appearing spec  
failures which turned out to be because of a change in running order  
(touching files + --loadby mtime --reverse) causing fixture leakage to  
move somewhere else, always easily remedied by explicitly loading  
whatever fixture had been forgotten. Is this a bug somewhere in RSpec  
or Rails (or MySQL), or something I''m just doing wrong?

Cheers,
-Tom

On Wed, Jul 22, 2009 at 8:27 AM, Tom Stuart<tom at experthuman.com>
wrote:
> On 22 Jul 2009, at 13:57, David Chelimsky wrote:
>>>
>>> The problem is that RSpec starts with an empty test database
>>
>> Not exactly. RSpec starts with whatever database you have and rolls
>> back to that state after each example provided you''re only
using
>> before(:each) to set up state before each example.
>
> Quite right, and I should''ve been more specific: rake spec sets up
an empty
> database and then runs RSpec. In the end I solved my problem by extending
> the db:test:prepare task to load the fixtures after cloning the development
> database structure, which in hindsight is obviously the right way to fix
it.
>
>> Any pre-existing database state will remain, and any state that you
>> set up in before(:suite) or before(:all) will not be rolled back
>> unless you do so explicitly.
>
> On an unrelated note, why do I occasionally see leakage of fixture data
> between specs? I''d expect RSpec''s explicit state rollback
There is no such thing as RSpec''s explicit state rollback, thought
that does sound quite fancy.

All RSpec does is create new instances of each example group to run
each example in. That means that any local state does not get copied
from one example to another. You are responsible for managing any
global state. The same is true for any other testing framework btw (at
least those that I know of).
> , plus Rails''
> transactional fixtures, to mean that fixtures loaded in one example would
> always be invisible to another, but that doesn''t seem to be the
case; I''ve
> seen many instances of mysteriously-appearing spec failures which turned
out
> to be because of a change in running order (touching files + --loadby mtime
> --reverse) causing fixture leakage to move somewhere else, always easily
> remedied by explicitly loading whatever fixture had been forgotten. Is this
> a bug somewhere in RSpec or Rails (or MySQL), or something I''m
just doing
> wrong?
I can''t speak for your situation in specific, but here are some
possible causes:

* setting up state in a before(:all)
This data is not rolled back after the example group is run, so you
have to explicitly roll it back in an after(:all). If other specs were
passing due to state set up in a before(:all) that wasn''t properly
matched with an after(:all) cleanup and happened to run before those
passing specs, then running those specs in isolation, or before the
offending group (with the before(:all)) may cause them to fail.

* data-backed global variables
This may seem counter-intuitive, since the database is rolled back
after each example - but keep in mind that _only_ the database is
rollled back. If a data-backed model is loaded into memory in a global
variable, that object is still there, even if the record in the
database no longer exists.

HTH,
David
> Cheers,
> -Tom