Hey list, RSpec is great, I''ve been using it at work for a couple of months now. One gripe I have is that I find it hard to know exactly to what extent my code is covered by the specs. I am aware of RCov and Heckle; they''re great, but I''d like to demonstrate another perspective of coverage analysis I''ve hacked up. The perspective is that of what is mocked and what isn''t, with RCov execution analysis mixed in for better viewing. My itch was "To what extent is my code mocked?" -- hence the name MockCov. It produces a report for each file with stubbed methods and executed lines highlighted. From that I can easily see what my specs failed to mock and therefore most probably didn''t specify any kind of behavior for. Here is a sample report: http://zioko.com/review_drop.rb.html (please forgive the small size of this sample, my employer understandably didn''t want me showing too much of our code) You''ll notice that there isn''t any kind of statistical coverage summary, I''m not sure how to quantity the results and indeed haven''t spent any time thinking about it; that can come later. For now I''d just like to gauge the response to this. I haven''t packaged the code up yet, it''s a horrible hack and needs to be reworked from scratch; I need to talk with someone on how best to integrate it with RSpec before that happens though. Let me know your thoughts. Cheers Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070725/b4b657f7/attachment-0001.html
Really cool - I have asked myself that question several times. /Marcus On 7/25/07, Ian Leitch <port001 at gmail.com> wrote:> Hey list, > > RSpec is great, I''ve been using it at work for a couple of months now. One > gripe I have is that I find it hard to know exactly to what extent my code > is covered by the specs. I am aware of RCov and Heckle; they''re great, but > I''d like to demonstrate another perspective of coverage analysis I''ve hacked > up. > > The perspective is that of what is mocked and what isn''t, with RCov > execution analysis mixed in for better viewing. My itch was "To what extent > is my code mocked?" -- hence the name MockCov. > It produces a report for each file with stubbed methods and executed lines > highlighted. From that I can easily see what my specs failed to mock and > therefore most probably didn''t specify any kind of behavior for. > > Here is a sample report: > http://zioko.com/review_drop.rb.html > (please forgive the small size of this sample, my employer understandably > didn''t want me showing too much of our code) > > You''ll notice that there isn''t any kind of statistical coverage summary, I''m > not sure how to quantity the results and indeed haven''t spent any time > thinking about it; that can come later. For now I''d just like to gauge the > response to this. > > I haven''t packaged the code up yet, it''s a horrible hack and needs to be > reworked from scratch; I need to talk with someone on how best to integrate > it with RSpec before that happens though. > > Let me know your thoughts. > > Cheers > Ian > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-- http://marcus.ahnve.net
Yeah - I think that''s a neat idea too. Does the code only work with RSpec''s mocking framework? Maybe it could be integrated into Rcov somehow. Scott On Jul 25, 2007, at 12:27 PM, Marcus Ahnve wrote:> Really cool - I have asked myself that question several times. > > /Marcus > > On 7/25/07, Ian Leitch <port001 at gmail.com> wrote: >> Hey list, >> >> RSpec is great, I''ve been using it at work for a couple of months >> now. One >> gripe I have is that I find it hard to know exactly to what extent >> my code >> is covered by the specs. I am aware of RCov and Heckle; they''re >> great, but >> I''d like to demonstrate another perspective of coverage analysis >> I''ve hacked >> up. >> >> The perspective is that of what is mocked and what isn''t, with RCov >> execution analysis mixed in for better viewing. My itch was "To >> what extent >> is my code mocked?" -- hence the name MockCov. >> It produces a report for each file with stubbed methods and >> executed lines >> highlighted. From that I can easily see what my specs failed to >> mock and >> therefore most probably didn''t specify any kind of behavior for. >> >> Here is a sample report: >> http://zioko.com/review_drop.rb.html >> (please forgive the small size of this sample, my employer >> understandably >> didn''t want me showing too much of our code) >> >> You''ll notice that there isn''t any kind of statistical coverage >> summary, I''m >> not sure how to quantity the results and indeed haven''t spent any >> time >> thinking about it; that can come later. For now I''d just like to >> gauge the >> response to this. >> >> I haven''t packaged the code up yet, it''s a horrible hack and needs >> to be >> reworked from scratch; I need to talk with someone on how best to >> integrate >> it with RSpec before that happens though. >> >> Let me know your thoughts. >> >> Cheers >> Ian >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users >> > > > -- > http://marcus.ahnve.net > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On 7/25/07, Ian Leitch <port001 at gmail.com> wrote:> Hey list, > > RSpec is great, I''ve been using it at work for a couple of months now. One > gripe I have is that I find it hard to know exactly to what extent my code > is covered by the specs. I am aware of RCov and Heckle; they''re great, but > I''d like to demonstrate another perspective of coverage analysis I''ve hacked > up. > > The perspective is that of what is mocked and what isn''t, with RCov > execution analysis mixed in for better viewing. My itch was "To what extent > is my code mocked?" -- hence the name MockCov. > It produces a report for each file with stubbed methods and executed lines > highlighted. From that I can easily see what my specs failed to mock and > therefore most probably didn''t specify any kind of behavior for.I''m not sure I understand how you get from this report to some action you need to take. Would you elaborate with a specific example?> > Here is a sample report: > http://zioko.com/review_drop.rb.html > (please forgive the small size of this sample, my employer understandably > didn''t want me showing too much of our code) > > You''ll notice that there isn''t any kind of statistical coverage summary, I''m > not sure how to quantity the results and indeed haven''t spent any time > thinking about it; that can come later. For now I''d just like to gauge the > response to this. > > I haven''t packaged the code up yet, it''s a horrible hack and needs to be > reworked from scratch; I need to talk with someone on how best to integrate > it with RSpec before that happens though. > > Let me know your thoughts. > > Cheers > Ian > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Yeah, only RSpec at the moment, I haven''t looked into integrating it with any of the others. The way it does its magic is tightly coupled with RSpecs design, specifically the metaclass proxies. Other frameworks may use the same approach, I don''t know. It can''t really be integrated with RCov for the reason above, indeed it wouldn''t be able to produce any meaningful output if RCov were not used in combination with RSpec. On 26/07/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:> > Yeah - I think that''s a neat idea too. > > Does the code only work with RSpec''s mocking framework? Maybe it > could be integrated into Rcov somehow. > > Scott > > On Jul 25, 2007, at 12:27 PM, Marcus Ahnve wrote: > > > Really cool - I have asked myself that question several times. > > > > /Marcus > > > > On 7/25/07, Ian Leitch <port001 at gmail.com> wrote: > >> Hey list, > >> > >> RSpec is great, I''ve been using it at work for a couple of months > >> now. One > >> gripe I have is that I find it hard to know exactly to what extent > >> my code > >> is covered by the specs. I am aware of RCov and Heckle; they''re > >> great, but > >> I''d like to demonstrate another perspective of coverage analysis > >> I''ve hacked > >> up. > >> > >> The perspective is that of what is mocked and what isn''t, with RCov > >> execution analysis mixed in for better viewing. My itch was "To > >> what extent > >> is my code mocked?" -- hence the name MockCov. > >> It produces a report for each file with stubbed methods and > >> executed lines > >> highlighted. From that I can easily see what my specs failed to > >> mock and > >> therefore most probably didn''t specify any kind of behavior for. > >> > >> Here is a sample report: > >> http://zioko.com/review_drop.rb.html > >> (please forgive the small size of this sample, my employer > >> understandably > >> didn''t want me showing too much of our code) > >> > >> You''ll notice that there isn''t any kind of statistical coverage > >> summary, I''m > >> not sure how to quantity the results and indeed haven''t spent any > >> time > >> thinking about it; that can come later. For now I''d just like to > >> gauge the > >> response to this. > >> > >> I haven''t packaged the code up yet, it''s a horrible hack and needs > >> to be > >> reworked from scratch; I need to talk with someone on how best to > >> integrate > >> it with RSpec before that happens though. > >> > >> Let me know your thoughts. > >> > >> Cheers > >> Ian > >> _______________________________________________ > >> rspec-users mailing list > >> rspec-users at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/rspec-users > >> > > > > > > -- > > http://marcus.ahnve.net > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070727/2747c7fe/attachment-0001.html
Sure. Say I have Class A and to test it, Spec B. From a quick look at Spec B I''m able to read the ''when'' and ''should'' descriptions and get a rough idea what the spec covers, yet nothing detailed, to do that I''d have to read the Class and Spec in conjunction, even still, it may not be obvious if a specific method or object is mocked. For example: my_important_method(meth1, meth2) Say we have spec''ed this to test the call with multiple values for meth1, but not for meth2. MockCov allows you to easily see that meth2 isn''t stubbed and therefore that the behavior of my_important_method isn''t sufficiently covered. Whereas Heckle lets you know about insufficient specs by performing mutations, MockCov should allow you to instead _see_ where the specs are lacking. I''m not saying this is a replacement for Heckle; they should compliment each other quite well. The example I posted in the initial email is very poor. I''m working on an new feature that will highlight variables in the code that were returned by mocked calls (it does so using ParseTree). I''ll produce a better example to demonstrate that in the next few days. On 27/07/07, David Chelimsky <dchelimsky at gmail.com> wrote:> > On 7/25/07, Ian Leitch <port001 at gmail.com> wrote: > > Hey list, > > > > RSpec is great, I''ve been using it at work for a couple of months now. > One > > gripe I have is that I find it hard to know exactly to what extent my > code > > is covered by the specs. I am aware of RCov and Heckle; they''re great, > but > > I''d like to demonstrate another perspective of coverage analysis I''ve > hacked > > up. > > > > The perspective is that of what is mocked and what isn''t, with RCov > > execution analysis mixed in for better viewing. My itch was "To what > extent > > is my code mocked?" -- hence the name MockCov. > > It produces a report for each file with stubbed methods and executed > lines > > highlighted. From that I can easily see what my specs failed to mock and > > therefore most probably didn''t specify any kind of behavior for. > > I''m not sure I understand how you get from this report to some action > you need to take. Would you elaborate with a specific example? > > > > > Here is a sample report: > > http://zioko.com/review_drop.rb.html > > (please forgive the small size of this sample, my employer > understandably > > didn''t want me showing too much of our code) > > > > You''ll notice that there isn''t any kind of statistical coverage summary, > I''m > > not sure how to quantity the results and indeed haven''t spent any time > > thinking about it; that can come later. For now I''d just like to gauge > the > > response to this. > > > > I haven''t packaged the code up yet, it''s a horrible hack and needs to be > > reworked from scratch; I need to talk with someone on how best to > integrate > > it with RSpec before that happens though. > > > > Let me know your thoughts. > > > > Cheers > > Ian > > _______________________________________________ > > rspec-users mailing list > > rspec-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/rspec-users > > > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/rspec-users/attachments/20070727/0626a023/attachment.html