Hey! I have implemented a quick solution for the should != .. , should ! ~ ... ''problem''. It uses source code inspection (I think it''s the only way) and i''ve done some benchmarking to see if it''s really that slow. A direct comparison of running ''1.should == 1'' with the unmodified rspec source against the rspec-version with source inspect shows that the latter is 7 times slower ... :( But benchmarks with ''1.should == 1'' inside a real example file are showing ''only'' an overall speed decrease of 15% to 50%. That is still bad but having lots of people consider the passing of ''1.should != 1'' an rspec bug is bad too. Here is the benchmarking script: http://pastie.caboo.se/206853 And the code is at branch ''inspect'' in: git://github.com/mhennemeyer/rspec There are examples for should and should_not != and !~ and i will definitely work on performance if you not totally reject this whole idea. Matthias.
Very cool. Maybe we could print a warning when people use !=. That would let us get the ugly != bug fixed, but not have the perf issues. Pat On 6/1/08, Matthias Hennemeyer <mhennemeyer at googlemail.com> wrote:> Hey! > > I have implemented a quick solution for the should != .. , should ! > ~ ... ''problem''. > It uses source code inspection (I think it''s the only way) and i''ve > done some > benchmarking to see if it''s really that slow. > > A direct comparison of > running ''1.should == 1'' > with the unmodified rspec source against the rspec-version with > source inspect > shows that the latter is 7 times slower ... :( > > But benchmarks with ''1.should == 1'' inside a real example file are > showing ''only'' > an overall speed decrease of 15% to 50%. That is still bad but having > lots of people > consider the passing of ''1.should != 1'' an rspec bug is bad too. > > Here is the benchmarking script: > http://pastie.caboo.se/206853 > > And the code is at branch ''inspect'' in: > git://github.com/mhennemeyer/rspec > > There are examples for should and should_not != and !~ and i will > definitely work on performance if you not totally reject this whole > idea. > > Matthias. > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On Jun 1, 2008, at 12:17 PM, Matthias Hennemeyer wrote:> Hey! > > I have implemented a quick solution for the should != .. , should ! > ~ ... ''problem''. > It uses source code inspection (I think it''s the only way) and i''ve > done some > benchmarking to see if it''s really that slow. > > A direct comparison of > running ''1.should == 1'' > with the unmodified rspec source against the rspec-version with > source inspect > shows that the latter is 7 times slower ... :( > > But benchmarks with ''1.should == 1'' inside a real example file are > showing ''only'' > an overall speed decrease of 15% to 50%. That is still bad but > having lots of people > consider the passing of ''1.should != 1'' an rspec bug is bad too.RSpec is already getting dinged for being slower than test/unit. Making it run any slower than it already does is a deal breaker for me.> Here is the benchmarking script: > http://pastie.caboo.se/206853 > > And the code is at branch ''inspect'' in: > git://github.com/mhennemeyer/rspec > > There are examples for should and should_not != and !~ and i will > definitely work on performance if you not totally reject this whole > idea.If you can get the performance up to or better than current, I''m all for it. Cheers, David
On Jun 1, 2008, at 4:55 PM, David Chelimsky wrote:> On Jun 1, 2008, at 12:17 PM, Matthias Hennemeyer wrote: > >> Hey! >> >> I have implemented a quick solution for the should != .. , should ! >> ~ ... ''problem''. >> It uses source code inspection (I think it''s the only way) and i''ve >> done some >> benchmarking to see if it''s really that slow. >> >> A direct comparison of >> running ''1.should == 1'' >> with the unmodified rspec source against the rspec-version with >> source inspect >> shows that the latter is 7 times slower ... :( >> >> But benchmarks with ''1.should == 1'' inside a real example file are >> showing ''only'' >> an overall speed decrease of 15% to 50%. That is still bad but >> having lots of people >> consider the passing of ''1.should != 1'' an rspec bug is bad too. > > RSpec is already getting dinged for being slower than test/unit. > Making it run any slower than it already does is a deal breaker for > me.Maybe using autoloading instead of requiring would help the situation? It seems perfectly reasonable that rspec is bigger than test/unit: Test::Unit doesn''t have a mocking framework, and so on. On the other hand, if no one uses (rspec''s) mocking framework, should rspec have the code in place anyway? Anyway - I''ve never found it to be too slow. Has anyone done benchmarks on it? Scott> > >> Here is the benchmarking script: >> http://pastie.caboo.se/206853 >> >> And the code is at branch ''inspect'' in: >> git://github.com/mhennemeyer/rspec >> >> There are examples for should and should_not != and !~ and i will >> definitely work on performance if you not totally reject this whole >> idea. > > If you can get the performance up to or better than current, I''m all > for it. > > Cheers, > David > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On Sun, Jun 1, 2008 at 10:13 PM, Scott Taylor <mailing_lists at railsnewbie.com> wrote:> > On Jun 1, 2008, at 4:55 PM, David Chelimsky wrote: > >> >> RSpec is already getting dinged for being slower than test/unit. Making it >> run any slower than it already does is a deal breaker for me. >> > > It seems perfectly reasonable that rspec is bigger than test/unit: > Test::Unit doesn''t have a mocking framework, and so on. On the other hand, > if no one uses (rspec''s) mocking framework, should rspec have the code in > place anyway?I spoke up on the code size issue some months ago: http://talklikeaduck.denhaven2.com/articles/2008/01/29/why-i-dont-mind-using-rspec-in-fact-ive-come-to-love-it As for whether or not the rspec mocking framework should be separately loadable, that''s a separate question.> > > Anyway - I''ve never found it to be too slow. Has anyone done benchmarks on > it?Like most other things, how slow or fast depends on how you use it. Here at @work, we have a really large rails app with a mixture of legacy tests and new specs. The slowest part of the overall test/spec suite, by far, is the legacy functional tests, which we hope to gradually convert to controller and view specs, using mocking as possible to avoid the db overhead, which is the real culprit. -- Rick DeNatale My blog on Ruby http://talklikeaduck.denhaven2.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080602/bcbc3008/attachment-0001.html>
On Sun, Jun 1, 2008 at 1:55 PM, David Chelimsky <dchelimsky at gmail.com> wrote:> On Jun 1, 2008, at 12:17 PM, Matthias Hennemeyer wrote: > >> Hey! >> >> I have implemented a quick solution for the should != .. , should !~ ... >> ''problem''. >> It uses source code inspection (I think it''s the only way) and i''ve done >> some >> benchmarking to see if it''s really that slow. >> >> A direct comparison of >> running ''1.should == 1'' >> with the unmodified rspec source against the rspec-version with source >> inspect >> shows that the latter is 7 times slower ... :( >> >> But benchmarks with ''1.should == 1'' inside a real example file are >> showing ''only'' >> an overall speed decrease of 15% to 50%. That is still bad but having lots >> of people >> consider the passing of ''1.should != 1'' an rspec bug is bad too. > > RSpec is already getting dinged for being slower than test/unit. Making it > run any slower than it already does is a deal breaker for me.What about showing an ugly warning in the test runner? != is a funky edge case that can really bite people. It''d be nice to have it fixed, but then tell the programmer to change it to == because of performance. Pat
Am 02.06.2008 um 20:01 schrieb Pat Maddox:>> > > What about showing an ugly warning in the test runner? != is a funky > edge case that can really bite people. It''d be nice to have it fixed, > but then tell the programmer to change it to == because of > performance. > > Pat > ______________________________________________The performance is much better now - but that warning thing is interesting. I think people should be able to decide if they use != or not and there should be an option that, when applied, turns out warnings and enables ''negative operator expectations''. One more word on performance: It is possible to see the difference in performance of both versions (with source inspect and without). But only if you run benchmarks that are exclusivley designed for this purpose. Running the rspec spec suite for both versions doesn''t show any difference. Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080604/f248c5fd/attachment.html>