llvm dev - Nov 2013 - [LLVMdev] Any objections to my importing GoogleMock to go with GoogleTest in LLVM?

On 15 November 2013 04:38, Chris Lattner <clattner at apple.com> wrote:
> Right, validating my assertion that while TDD and unit testing are good in
> general, they may not be right for LLVM.  In LLVM, we have mature tests of
> other sorts, as well as a strong process of review.
>
While I understand the value of TDD, I have to agree with Chris, here.

I have been bitten by having to write silly boundary checks tests that the
code would never allow, and getting 100% test coverage (of lines AND
branches), only to realize that a test engineer could break my new feature
in many different ways by misusing it on a command-line level.

I personally think we'd have a much better use of time by making the MI
layer dumpable and re-readable, so we could create lots of very specific
low-level tests, than having yet-another unit-test infrastructure.

The only place I think that unit-tests are worthy is on base libraries
(APFLoat, APInt, containers, basic algorithms), and for that, you need
nothing special.


> Further, even if you guys didn't notice the bug immediately, I would
have
> noticed it in my (planned) next commit.  That commit was the code that used
> it.  You don't need a unit test to catch obvious bozo bugs like the one
you
> are referring to.
>
That is the point. Most of us don't commit before a check-all anyway, so
looking back on the commits list won't give you a clear picture.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/3c623c4b/attachment.html>

I'm sorry I even sent the original email.

To be clear, I am trying to write some specific code, and gmock would make
my life significantly easier. I'm not really trying to start or win a
debate about how to write tests in LLVM. I think that debate should be held
around tests, not around abstract libraries if it is even worth having at
all. I am a big believer in giving people a powerful set of tools and
asking that they use only those appropriate for a particular situation, and
using code review to help uphold those standards.

Despite the increasingly polarized discussion here, I am not actually in
serious disagreement with any of the principles or high level sentiments
expressed by folks like Chris, Renato, and to a limited extent Alp.
(Limited because he got off on a tangent that isn't even relevant about
some kind of corporate interest in freezing APIs. That is total bunk.)
Mostly, I think that these are all valid concerns and that there is a
tradeoff that has to be assessed on a case-by-case basis. Others clearly
see it as a bit more black and white. Totally OK, as reasonable folks can
disagree here.

Ultimately, I'm really uninterested in a time wasting philosophical debate
at this stage. I didn't want to start one.

Clearly people are afraid of the tests that might be written with gmock.
OK, I will keep significant amounts of testing out of tree while doing
development. I find this silly and a waste of everyone's time, but I really
see these as a tool to aid my development and maybe my code reviewers'
understanding of the code. If there is some long-term maintenance burden
perceived by having such tests around, I'll happily never commit them.

Let's get back to writing and reviewing code rather than trying to lecture
each other on how to write better tests (and I include my own emails in
that set, I'm completely guilty of this as well).

On Fri, Nov 15, 2013 at 1:47 AM, Renato Golin <renato.golin at
linaro.org>wrote:
> On 15 November 2013 04:38, Chris Lattner <clattner at apple.com>
wrote:
>
>> Right, validating my assertion that while TDD and unit testing are good
>> in general, they may not be right for LLVM.  In LLVM, we have mature
tests
>> of other sorts, as well as a strong process of review.
>>
>
> While I understand the value of TDD, I have to agree with Chris, here.
>
> I have been bitten by having to write silly boundary checks tests that the
> code would never allow, and getting 100% test coverage (of lines AND
> branches), only to realize that a test engineer could break my new feature
> in many different ways by misusing it on a command-line level.
>
> I personally think we'd have a much better use of time by making the MI
> layer dumpable and re-readable, so we could create lots of very specific
> low-level tests, than having yet-another unit-test infrastructure.
>
> The only place I think that unit-tests are worthy is on base libraries
> (APFLoat, APInt, containers, basic algorithms), and for that, you need
> nothing special.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/6a4ce50f/attachment.html>

On 15/11/2013 10:01, Chandler Carruth wrote:
> Despite the increasingly polarized discussion here, I am not actually
> in serious disagreement with any of the principles or high level
> sentiments expressed by folks like Chris, Renato, and to a limited
> extent Alp. (Limited because he got off on a tangent that isn't even
> relevant about some kind of corporate interest in freezing APIs. That
> is total bunk.)
Good to hear there is no conspiracy ;-)

Can I suggest winning back confidence by pruning the unit test sprawl
covering high-level interfaces like tooling, and not adding further to it?

I seriously don't think anyone would argue vehemently against unit /
mock testing of fundamental data types and algorithms once this has been
addressed.

Alp.

-- 
http://www.nuanti.com
the browser experts