llvm dev - Nov 2013 - [LLVMdev] Any objections to my importing GoogleMock to go with GoogleTest in LLVM?

I have some concrete use cases in testing the pass manager where it will
allow the tests of this API to be more thorough, less verbose, and easier
to maintain. I'm not claiming to be the biggest fan of some features in
GoogleMock, but on the whole, I think it's better than the alternative and
will allow more careful testing of C++ APIs where the interesting part is
the API itself.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/3efdbca0/attachment.html>

Could you maybe give an example or two to whet our testing appetite?


On Tue, Nov 12, 2013 at 10:04 PM, Chandler Carruth <chandlerc at
google.com>wrote:
> I have some concrete use cases in testing the pass manager where it will
> allow the tests of this API to be more thorough, less verbose, and easier
> to maintain. I'm not claiming to be the biggest fan of some features in
> GoogleMock, but on the whole, I think it's better than the alternative
and
> will allow more careful testing of C++ APIs where the interesting part is
> the API itself.
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/5e094ee0/attachment.html>

On Tue, Nov 12, 2013 at 7:04 PM, Chandler Carruth <chandlerc at
google.com>wrote:
> I have some concrete use cases in testing the pass manager where it will
> allow the tests of this API to be more thorough, less verbose, and easier
> to maintain. I'm not claiming to be the biggest fan of some features in
> GoogleMock, but on the whole, I think it's better than the alternative
and
> will allow more careful testing of C++ APIs where the interesting part is
> the API itself.
>
Emphatic +1

One of the major problems with unit testing (in general and in LLVM in
particular) is that it's difficult to isolate coupled components from each
other. A good mocking framework makes this much easier and thus can help us
create more unit tests in the long run. GoogleMock is popular,
battle-proven and is the natural companion for GTest.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/48cb4613/attachment.html>

I don't really have a strong opinion, but it ended up being fairly
controversial in Chromium:
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/-KH_IP0rIWQ

It might occupy a space like GTest does for us today, which is used for
things like Support where we often can't write a good lit test.


On Tue, Nov 12, 2013 at 7:42 PM, Eli Bendersky <eliben at google.com>
wrote:
> On Tue, Nov 12, 2013 at 7:04 PM, Chandler Carruth <chandlerc at
google.com>wrote:
>
>> I have some concrete use cases in testing the pass manager where it
will
>> allow the tests of this API to be more thorough, less verbose, and
easier
>> to maintain. I'm not claiming to be the biggest fan of some
features in
>> GoogleMock, but on the whole, I think it's better than the
alternative and
>> will allow more careful testing of C++ APIs where the interesting part
is
>> the API itself.
>>
>
> Emphatic +1
>
> One of the major problems with unit testing (in general and in LLVM in
> particular) is that it's difficult to isolate coupled components from
each
> other. A good mocking framework makes this much easier and thus can help us
> create more unit tests in the long run. GoogleMock is popular,
> battle-proven and is the natural companion for GTest.
>
> Eli
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/8d4680c0/attachment.html>

On Nov 12, 2013, at 7:04 PM, Chandler Carruth <chandlerc at google.com>
wrote:
> I have some concrete use cases in testing the pass manager where it will
allow the tests of this API to be more thorough, less verbose, and easier to
maintain. I'm not claiming to be the biggest fan of some features in
GoogleMock, but on the whole, I think it's better than the alternative and
will allow more careful testing of C++ APIs where the interesting part is the
API itself.
Personally, I rather not do this, without very clear and compelling reasons.

I understand that this could be very useful for your bringup (and so could be
very useful locally), but once the passmanager is the default, it will get lost
of in-tree testing by just about everything in the compiler.

I'm not really excited about dragging another out of tree project in unless
there is a compelling reason to do so.

-Chris

Keep in mind that I am a maintainer for gmock so this would not change the
external project decencies of LLVM.
On Nov 12, 2013 9:16 PM, "Chris Lattner" <clattner at apple.com>
wrote:
>
> On Nov 12, 2013, at 7:04 PM, Chandler Carruth <chandlerc at
google.com>
> wrote:
>
> > I have some concrete use cases in testing the pass manager where it
will
> allow the tests of this API to be more thorough, less verbose, and easier
> to maintain. I'm not claiming to be the biggest fan of some features in
> GoogleMock, but on the whole, I think it's better than the alternative
and
> will allow more careful testing of C++ APIs where the interesting part is
> the API itself.
>
> Personally, I rather not do this, without very clear and compelling
> reasons.
>
> I understand that this could be very useful for your bringup (and so could
> be very useful locally), but once the passmanager is the default, it will
> get lost of in-tree testing by just about everything in the compiler.
>
> I'm not really excited about dragging another out of tree project in
> unless there is a compelling reason to do so.
>
> -Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131112/5ff04739/attachment.html>

On Tue, Nov 12, 2013 at 7:24 PM, Sean Silva <silvas at purdue.edu> wrote:
> Could you maybe give an example or two to whet our testing appetite?

It would honestly be simpler for me to write the tests after pulling it in
and point at them. The GoogleMock project has some good examples as well.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/e97774d0/attachment.html>

Writing a more thought-out reply....

On Tue, Nov 12, 2013 at 9:16 PM, Chris Lattner <clattner at apple.com>
wrote:
>
> On Nov 12, 2013, at 7:04 PM, Chandler Carruth <chandlerc at
google.com>
> wrote:
>
> > I have some concrete use cases in testing the pass manager where it
will
> allow the tests of this API to be more thorough, less verbose, and easier
> to maintain. I'm not claiming to be the biggest fan of some features in
> GoogleMock, but on the whole, I think it's better than the alternative
and
> will allow more careful testing of C++ APIs where the interesting part is
> the API itself.
>
> Personally, I rather not do this, without very clear and compelling
> reasons.
>
> I understand that this could be very useful for your bringup (and so could
> be very useful locally), but once the passmanager is the default, it will
> get lost of in-tree testing by just about everything in the compiler.
>
I would much rather have it in the tree than just use it locally. I think
it will also make subsequent iterations much easier to test and show are
correct. I think it would also allow significantly more precise regression
testing in the future.

This also isn't the first time I've wanted it in LLVM and in Clang.
It's
just the first time I've been working on something large enoguh to feel
like importing it would be worth the cost.

My feeling is that both gtest and gmock suffer from the same flaw: they can
easily be overused or misused in circumstances where there are clearly
better ways to go about things. However, I feel like within LLVM we have
been really good at pushing back against that and using integration tests
with excellent tool support (how I love FileCheck) much more prevalently.
As long as we continue to code review unittests with an eye toward
skepticism, I think there is very little risk of things getting out of
hand. I think adding gmock to gtest doesn't shift that risk in any
significant way.

However, when we are adding interfaces or generic utilities to LLVM
(admittedly, not the common case) I don't think we do ourselves any favors
by using only half of the available tools to write unit tests for them.

>
> I'm not really excited about dragging another out of tree project in
> unless there is a compelling reason to do so.
>
It helps that gmock is a sibling of gtest. It doesn't really pull in very
much new stuff and like gtest it has strictly managed its dependencies down
to zero. I'm happy to do the importing, the fixing, and to even police
unittests for misuses. I actually don't expect it to be widely used, but I
expect it to make tests significantly more comprehensible and brief in the
limited cases where it applies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/37e7fc86/attachment.html>