llvm dev - Jun 2019 - [CMake] External File Dependencies for Unit Tests

Hi,

I want to write unit tests for a project I am working on. I need to create
object files, preferably I want to have yaml files in the source tree and
have cmake generate object files using yaml2obj. Does anyone know what I
would put in the CMakeLists.txt to get this behavior?

For a concrete example if I didn’t explain clearly:

In llvm/unittests/Object I would maybe have an Inputs directory, with a
basic-obj.yaml. In llvm/unittests/Object I would want a test case which can
open Inputs/basic-obj.o.

Thanks,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190622/997e18ab/attachment.html>

Hi Alex,

Not answering your question directly, but I'd highly recommend against
using CMake to bake the test outputs at the cmake step for several
different reasons:

1) If yaml2obj changes, you'd have to rerun cmake to see whether the new
version would cause the tests to fail. This is not the default, so could
easily lead to build-bot breakages.
2) It's not clear from your statement whether you're talking about lit
tests or gtest unit tests. If the former, there's no real precedence for
doing this. If the latter, you'd need to have your C++ code able to detect
the build output directory somehow, which feels like it would make the code
at least unclear, and at worst unusable.
3) In either case, if you wanted to change the test input, you'd need to
rerun CMake again, rather than it just working. Runnning CMake is not
cheap, especially on Windows for Visual Studio generators.

As Jordan has mentioned offline, and assuming you're talking about gtest,
you're probably better off just calling the corresponding library functions
to generate the object from YAML. Otherwise, just use yaml2obj directly.

If you want to explore your original suggestion further, and need help
doing so, you probably need to give a bit more context as to what you've
tried and what didn't work about it.

Regards,

James


On Sat, 22 Jun 2019 at 21:28, Alex Brachet-Mialot via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> I want to write unit tests for a project I am working on. I need to create
> object files, preferably I want to have yaml files in the source tree and
> have cmake generate object files using yaml2obj. Does anyone know what I
> would put in the CMakeLists.txt to get this behavior?
>
> For a concrete example if I didn’t explain clearly:
>
> In llvm/unittests/Object I would maybe have an Inputs directory, with a
> basic-obj.yaml. In llvm/unittests/Object I would want a test case which can
> open Inputs/basic-obj.o.
>
> Thanks,
> Alex
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190624/5fdcab5e/attachment.html>

Thanks James!

Thanks for the help. I certainly see your point.
> In either case, if you wanted to change the test input, you'd need to
rerun CMake again
This is unfortunate, I would have hoped I could have CMake create
$(wildcard *.yaml) type of output in the make files. But it sounds like
this doesn't exist on make alternatives, so this wont work.

There is actually a function to get the Inputs directory,
http://llvm.org/doxygen/namespacellvm_1_1unittest.html. I think it is only
used in one test, though. Would you recommend that I do what is done in
unittest/ObjectYAML/MinidumpYAMLTest.cpp which just uses a string inside
the source file, or use an Inputs directory? Both are not great, as you
point out about using an Inputs directory.

Also, I'm not sure that I can create an object without invoking yaml2obj
itself, I can't seem to find a function to do this exposed by
libObjectYAML. If I do need to invoke yaml2obj, would you happen to know
how to tell CMake to build yaml2obj first, and also where would I find that
executable? Thanks so much, James.


On Mon, Jun 24, 2019 at 5:19 AM James Henderson <
jh7370.2008 at my.bristol.ac.uk> wrote:
> Hi Alex,
>
> Not answering your question directly, but I'd highly recommend against
> using CMake to bake the test outputs at the cmake step for several
> different reasons:
>
> 1) If yaml2obj changes, you'd have to rerun cmake to see whether the
new
> version would cause the tests to fail. This is not the default, so could
> easily lead to build-bot breakages.
> 2) It's not clear from your statement whether you're talking about
lit
> tests or gtest unit tests. If the former, there's no real precedence
for
> doing this. If the latter, you'd need to have your C++ code able to
detect
> the build output directory somehow, which feels like it would make the code
> at least unclear, and at worst unusable.
> 3) In either case, if you wanted to change the test input, you'd need
to
> rerun CMake again, rather than it just working. Runnning CMake is not
> cheap, especially on Windows for Visual Studio generators.
>
> As Jordan has mentioned offline, and assuming you're talking about
gtest,
> you're probably better off just calling the corresponding library
functions
> to generate the object from YAML. Otherwise, just use yaml2obj directly.
>
> If you want to explore your original suggestion further, and need help
> doing so, you probably need to give a bit more context as to what
you've
> tried and what didn't work about it.
>
> Regards,
>
> James
>
>
> On Sat, 22 Jun 2019 at 21:28, Alex Brachet-Mialot via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> I want to write unit tests for a project I am working on. I need to
>> create object files, preferably I want to have yaml files in the source
>> tree and have cmake generate object files using yaml2obj. Does anyone
know
>> what I would put in the CMakeLists.txt to get this behavior?
>>
>> For a concrete example if I didn’t explain clearly:
>>
>> In llvm/unittests/Object I would maybe have an Inputs directory, with a
>> basic-obj.yaml. In llvm/unittests/Object I would want a test case which
can
>> open Inputs/basic-obj.o.
>>
>> Thanks,
>> Alex
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190625/de2fa8ab/attachment.html>