rspec users - Feb 2008 - spec_server requiring absolute paths (and autotest breaking)

rspec and rspec_on_rails rev 3316, ZenTest 3.9.1

When run through spec_server, running spec fails if the spec file is  
not specified with an absolute path:

Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts  
`pwd`/spec/models/address_spec.rb
......

Finished in 0.166251 seconds

6 examples, 0 failures
Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts  
spec/models/address_spec.rb
(druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/runner/ 
options.rb:197:in `files_to_load'': File or directory not found: spec/ 
models/address_spec.rb (RuntimeError)
	from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ 
runner/options.rb:189:in `each''
	from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ 
runner/options.rb:189:in `files_to_load''


In the same way, autotest fails with spec_server:

Probutanol:koulutusweb jarkko$ autotest
loading autotest/rails_rspec
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -S  
script/spec -O spec/spec.opts  spec/controllers/ 
activations_controller_spec.rb spec/helpers/clients_helper_spec.rb
[a bunch of files omitted]
spec/models/enrollment_notifier_spec.rb spec/views/clients/ 
email_form.html.erb_spec.rb
(druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/runner/ 
options.rb:5:in `mtime'': Not a directory - spec/models/address_spec.rb
(Errno::ENOTDIR)
	from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ 
runner/options.rb:5
	from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ 
runner/options.rb:251:in `sort''
	from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ 
runner/options.rb:251:in `sorted_files''


Any ideas why spec_server has lost its marbles? Without the --drb  
option autotest runs fine.

Cheers,
//jarkko

--
Jarkko Laine
http://jlaine.net
http://dotherightthing.com
http://www.railsecommerce.com
http://odesign.fi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2417 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/rspec-users/attachments/20080219/14d880fc/attachment.bin

On Feb 19, 2008 10:42 AM, Jarkko Laine <jarkko at jlaine.net>
wrote:
> rspec and rspec_on_rails rev 3316, ZenTest 3.9.1
>
> When run through spec_server, running spec fails if the spec file is
> not specified with an absolute path:
>
> Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts
> `pwd`/spec/models/address_spec.rb
> ......
>
> Finished in 0.166251 seconds
>
> 6 examples, 0 failures
> Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts
> spec/models/address_spec.rb
> (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/runner/
> options.rb:197:in `files_to_load'': File or directory not found:
spec/
> models/address_spec.rb (RuntimeError)
>         from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/
> runner/options.rb:189:in `each''
>         from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/
> runner/options.rb:189:in `files_to_load''
>
>
> In the same way, autotest fails with spec_server:
>
> Probutanol:koulutusweb jarkko$ autotest
> loading autotest/rails_rspec
> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -S
> script/spec -O spec/spec.opts  spec/controllers/
> activations_controller_spec.rb spec/helpers/clients_helper_spec.rb
> [a bunch of files omitted]
> spec/models/enrollment_notifier_spec.rb spec/views/clients/
> email_form.html.erb_spec.rb
> (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/runner/
> options.rb:5:in `mtime'': Not a directory -
spec/models/address_spec.rb
> (Errno::ENOTDIR)
>         from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/
> runner/options.rb:5
>         from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/
> runner/options.rb:251:in `sort''
>         from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/
> runner/options.rb:251:in `sorted_files''
>
>
> Any ideas why spec_server has lost its marbles? Without the --drb
> option autotest runs fine.
Did this just start happening for you? If so, it could be the patch I
applied in r3310. Would you mind checking out 3309 and seeing if you
still have this problem?
>
> Cheers,
> //jarkko
>
> --
> Jarkko Laine
> http://jlaine.net
> http://dotherightthing.com
> http://www.railsecommerce.com
> http://odesign.fi
>
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On 19.2.2008, at 17.57, David Chelimsky wrote:
> Did this just start happening for you? If so, it could be the patch I
> applied in r3310. Would you mind checking out 3309 and seeing if you
> still have this problem?
That''s it. However, with 3309 it seems that autotest doesn''t
really
run through drb at all since it runs the same whether or not the  
spec_server is running. Yeah, I know that was the reason for the patch.

//jarkko

--
Jarkko Laine
http://jlaine.net
http://dotherightthing.com
http://www.railsecommerce.com
http://odesign.fi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2417 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/rspec-users/attachments/20080219/881f102a/attachment.bin

On 19.2.2008, at 17.57, David Chelimsky wrote:
> Did this just start happening for you? If so, it could be the patch I
> applied in r3310. Would you mind checking out 3309 and seeing if you
> still have this problem?

Ok, found out the reason. I started spec_server from within the script  
directory and it only works correctly if you start it from the root of  
your Rails app. Gotcha.

That said, now that autotest runs via drb, it''s vastly (like 50%)  
slower than without it, doesn''t respect the --colour option in  
spec.opts and causes failures that don''t happen when running just  
plain autotest. All of them seem to be related to methods being run  
more times than they are supposed to. Examples:

Mock ''Location_18994'' expected :valid? with (any args) once,
but
received it twice

expected: "Barney Hops doesn''t have the required competency (MBA)
to
teach this class",
      got: ["Barney Hops doesn''t have the required competency
(MBA) to
teach this class", "Barney Hops doesn''t have the required
competency
(MBA) to teach this class"]

Mock ''Enrollment'' expected :on_waiting_list? with (any args)
twice,
but received it 3 times

//jarkko

--
Jarkko Laine
http://jlaine.net
http://dotherightthing.com
http://www.railsecommerce.com
http://odesign.fi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2417 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/rspec-users/attachments/20080219/d0305d7a/attachment-0001.bin

On Feb 19, 2008 12:08 PM, Jarkko Laine <jarkko at jlaine.net>
wrote:
> On 19.2.2008, at 17.57, David Chelimsky wrote:
> > Did this just start happening for you? If so, it could be the patch I
> > applied in r3310. Would you mind checking out 3309 and seeing if you
> > still have this problem?
>
>
> Ok, found out the reason. I started spec_server from within the script
> directory and it only works correctly if you start it from the root of
> your Rails app. Gotcha.
>
> That said, now that autotest runs via drb, it''s vastly (like 50%)
> slower than without it, doesn''t respect the --colour option in
> spec.opts and causes failures that don''t happen when running just
> plain autotest. All of them seem to be related to methods being run
> more times than they are supposed to. Examples:
>
> Mock ''Location_18994'' expected :valid? with (any args)
once, but
> received it twice
>
> expected: "Barney Hops doesn''t have the required competency
(MBA) to
> teach this class",
>       got: ["Barney Hops doesn''t have the required competency
(MBA) to
> teach this class", "Barney Hops doesn''t have the
required competency
> (MBA) to teach this class"]
>
> Mock ''Enrollment'' expected :on_waiting_list? with (any
args) twice,
> but received it 3 times
There are a few drb related tickets in lighthouse. Please add this if
information if/where appropriate or add a new ticket.

Thanks,
David

>
>
> //jarkko
>
> --
> Jarkko Laine
> http://jlaine.net
> http://dotherightthing.com
> http://www.railsecommerce.com
> http://odesign.fi
>
>
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Feb 19, 2008, at 12:08 PM, Jarkko Laine wrote:
> On 19.2.2008, at 17.57, David Chelimsky wrote:
>> Did this just start happening for you? If so, it could be the patch I
>> applied in r3310. Would you mind checking out 3309 and seeing if you
>> still have this problem?
>
>
> Ok, found out the reason. I started spec_server from within the  
> script directory and it only works correctly if you start it from  
> the root of your Rails app. Gotcha.
>
> That said, now that autotest runs via drb, it''s vastly (like 50%)
> slower than without it,
That''s the idea.
> doesn''t respect the --colour option in spec.opts
That''s been a bug since 0.6 or 0.7 when I started with rspec.  I  
guess not enough people have used the spec server for it to be  
already patched.

I''ll take a look at fixing it this weekend.
> and causes failures that don''t happen when running just plain  
> autotest.
> All of them seem to be related to methods being run more times than  
> they are supposed to. Examples:
>
> Mock ''Location_18994'' expected :valid? with (any args)
once, but
> received it twice
I saw this happen recently when the spec server was started twice:  
once with the rake task (rake spec:server:start), the other with a  
straight script/spec_sever.  My hunch is that both servers are  
loading the files at the same time.
>
> expected: "Barney Hops doesn''t have the required competency
(MBA)
> to teach this class",
>      got: ["Barney Hops doesn''t have the required competency
(MBA)
> to teach this class", "Barney Hops doesn''t have the
required
> competency (MBA) to teach this class"]
>
Yeah - this is because rails load''s instead of requires. 
You''ll also
see this sort of thing if you just call load directly in your specs.   
I''ve never dug deeper into how all of the rails magic happens  
regarding loading and constants (and the method generation with the  
association macros (has_* belongs_to), so if you have any insight,  
I''d be happy to hear it.

Scott

On 20.2.2008, at 6.55, Scott Taylor wrote:
>> That said, now that autotest runs via drb, it''s vastly (like
50%)
>> slower than without it,
>
> That''s the idea.
Isn''t the idea that specs run faster through drb, rather than slower?  
However, in general my specs seem to take a lot longer when run  
through autotest in the first place (compared to running just rake  
spec:models, e.g.). I haven''t so far been able to track down what  
causes this.
>> Mock ''Location_18994'' expected :valid? with (any
args) once, but
>> received it twice
>
> I saw this happen recently when the spec server was started twice:
> once with the rake task (rake spec:server:start), the other with a
> straight script/spec_sever.  My hunch is that both servers are
> loading the files at the same time.
That would seem logical. However, I just checked that I only have one  
spec_server process running and am still able to reproduce the problem.
>> expected: "Barney Hops doesn''t have the required
competency (MBA)
>> to teach this class",
>>     got: ["Barney Hops doesn''t have the required
competency (MBA)
>> to teach this class", "Barney Hops doesn''t have the
required
>> competency (MBA) to teach this class"]
>>
>
> Yeah - this is because rails load''s instead of requires. 
You''ll also
> see this sort of thing if you just call load directly in your specs.
> I''ve never dug deeper into how all of the rails magic happens
> regarding loading and constants (and the method generation with the
> association macros (has_* belongs_to), so if you have any insight,
> I''d be happy to hear it.
Yeah, all these failures are certainly related and probably caused by  
something like what you describe. Got to try to figure out what  
spec_server does differently than normal spec. Or maybe we should just  
bite the bullet and make the deeptest spec fork work :-)

Cheers,
//jarkko

--
Jarkko Laine
http://jlaine.net
http://dotherightthing.com
http://www.railsecommerce.com
http://odesign.fi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2417 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/rspec-users/attachments/20080220/6c07b70e/attachment.bin

On Feb 20, 2008, at 1:52 AM, Jarkko Laine wrote:
> On 20.2.2008, at 6.55, Scott Taylor wrote:
>>> That said, now that autotest runs via drb, it''s vastly
(like 50%)
>>> slower than without it,
>>
>> That''s the idea.
>
> Isn''t the idea that specs run faster through drb, rather than  
> slower? However, in general my specs seem to take a lot longer when  
> run through autotest in the first place (compared to running just  
> rake spec:models, e.g.). I haven''t so far been able to track down
> what causes this.
Oops.  Misread your post.

Is this true running a single spec (or a single file)?  I''ve noticed  
that a test suite which takes about 5 seconds normally takes about 7  
seconds under drb (according to the numbers reported by rspec).  Then  
again, running a single file is blazingly fast, as their is no wait  
(as though I''m testing against a non-rails project).

>
>>> Mock ''Location_18994'' expected :valid? with (any
args) once, but
>>> received it twice
>>
>> I saw this happen recently when the spec server was started twice:
>> once with the rake task (rake spec:server:start), the other with a
>> straight script/spec_sever.  My hunch is that both servers are
>> loading the files at the same time.
>
> That would seem logical. However, I just checked that I only have  
> one spec_server process running and am still able to reproduce the  
> problem.
Hmm.  Are you reloading classes or using anonymous classes derived  
from AR::Base, or anything crazy like that?  Are all your models in  
app/models?  Do you have pending migrations?  Are your running with  
rake, or with autotest? How about running the file alone?
>
>>> expected: "Barney Hops doesn''t have the required
competency (MBA)
>>> to teach this class",
>>>     got: ["Barney Hops doesn''t have the required
competency (MBA)
>>> to teach this class", "Barney Hops doesn''t have
the required
>>> competency (MBA) to teach this class"]
>>>
>>
>> Yeah - this is because rails load''s instead of requires. 
You''ll also
>> see this sort of thing if you just call load directly in your specs.
>> I''ve never dug deeper into how all of the rails magic happens
>> regarding loading and constants (and the method generation with the
>> association macros (has_* belongs_to), so if you have any insight,
>> I''d be happy to hear it.
>
> Yeah, all these failures are certainly related and probably caused  
> by something like what you describe. Got to try to figure out what  
> spec_server does differently than normal spec. Or maybe we should  
> just bite the bullet and make the deeptest spec fork work :-)
>
Yeah - deeptest is great for slow suites, but now that I have a fast  
suite (take 5-7 seconds), deeptest probably wouldn''t be much of a  
performance gain for me.

Scott

On 21.2.2008, at 4.14, Scott Taylor wrote:
>
> On Feb 20, 2008, at 1:52 AM, Jarkko Laine wrote:
>
>> On 20.2.2008, at 6.55, Scott Taylor wrote:
>>>> That said, now that autotest runs via drb, it''s vastly
(like 50%)
>>>> slower than without it,
>>>
>>> That''s the idea.
>>
>> Isn''t the idea that specs run faster through drb, rather than
>> slower? However, in general my specs seem to take a lot longer when
>> run through autotest in the first place (compared to running just
>> rake spec:models, e.g.). I haven''t so far been able to track
down
>> what causes this.
>
> Oops.  Misread your post.
>
> Is this true running a single spec (or a single file)?  I''ve
noticed
> that a test suite which takes about 5 seconds normally takes about 7
> seconds under drb (according to the numbers reported by rspec).  Then
> again, running a single file is blazingly fast, as their is no wait
> (as though I''m testing against a non-rails project).
If I run a single, small file (like a helper spec), it''s faster in an  
absolute sense (using time):

Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/ 
helpers/date_extension_spec.rb
....

Finished in 2.061605 seconds

4 examples, 0 failures

real	0m2.212s
user	0m0.060s
sys	0m0.024s
Probutanol:koulutusweb jarkko$ time script/spec spec/helpers/ 
date_extension_spec.rb
....

Finished in 0.102506 seconds

4 examples, 0 failures

real	0m4.682s
user	0m3.738s
sys	0m0.688s


However, as you can see even there the actual running of the specs is  
painfully slow (2 vs 0.1 secs) and the only reason drb is faster is  
because there is no startup time. This is on 3316. Also, there really  
seems to be something wonky going on with the loading. When I run a  
model spec right after starting the spec server, it works, but running  
it the second time already gives some weird stuff back:

Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/ 
models/account_spec.rb
..........

Finished in 0.371612 seconds

10 examples, 0 failures

real	0m1.647s
user	0m0.068s
sys	0m0.024s
Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/ 
models/account_spec.rb
.FFF.FFFFF

1)
ActiveRecord::AssociationTypeMismatch in ''Account users should return  
user''
Address expected, got Address
>
>>
>>>> Mock ''Location_18994'' expected :valid? with
(any args) once, but
>>>> received it twice
>>>
>>> I saw this happen recently when the spec server was started twice:
>>> once with the rake task (rake spec:server:start), the other with a
>>> straight script/spec_sever.  My hunch is that both servers are
>>> loading the files at the same time.
>>
>> That would seem logical. However, I just checked that I only have
>> one spec_server process running and am still able to reproduce the
>> problem.
>
> Hmm.  Are you reloading classes or using anonymous classes derived
> from AR::Base, or anything crazy like that?  Are all your models in
> app/models?  Do you have pending migrations?  Are your running with
> rake, or with autotest? How about running the file alone?
Doesn''t matter. Shouldn''t be doing anything super special with
models
but got to think about it.
>>>> expected: "Barney Hops doesn''t have the required
competency (MBA)
>>>> to teach this class",
>>>>    got: ["Barney Hops doesn''t have the required
competency (MBA)
>>>> to teach this class", "Barney Hops doesn''t
have the required
>>>> competency (MBA) to teach this class"]
>>>>
>>>
>>> Yeah - this is because rails load''s instead of requires. 
You''ll
>>> also
>>> see this sort of thing if you just call load directly in your
specs.
>>> I''ve never dug deeper into how all of the rails magic
happens
>>> regarding loading and constants (and the method generation with the
>>> association macros (has_* belongs_to), so if you have any insight,
>>> I''d be happy to hear it.
>>
>> Yeah, all these failures are certainly related and probably caused
>> by something like what you describe. Got to try to figure out what
>> spec_server does differently than normal spec. Or maybe we should
>> just bite the bullet and make the deeptest spec fork work :-)
>>
> Yeah - deeptest is great for slow suites, but now that I have a fast
> suite (take 5-7 seconds), deeptest probably wouldn''t be much of a
> performance gain for me.
Yeah, the funny thing is, model specs that sometimes need to go to the  
db are pretty fast but controller specs where most of the db traffic  
is stubbed out seem to become a drag, and I would think deeptest would  
help there.

--
Jarkko Laine
http://jlaine.net
http://dotherightthing.com
http://www.railsecommerce.com
http://odesign.fi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2417 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/rspec-users/attachments/20080221/5338a226/attachment.bin