llvm dev - Jun 2021 - [debuginfo-test][cross-project-test] New cross-project lit directory/debuginfo-test suite has moved

Hi all,

TL;DR: there's a new top-level project called cross-project-test for tests
which require multiple LLVM projects, which replaces debuginfo-test. You
may need to update LLVM_ENABLE_PROJECT accordingly.

---

A short while ago, I pushed a series of commits that created a new
top-level LLVM directory called `cross-project-tests`, and moved the
`debuginfo-test` directory inside. I also renamed the debuginfo-test
project in LLVM_ENABLE_PROJECTS in to `cross-project-tests`.
check-debuginfo still exists as a target, and will run just the tests
inside the debuginfo-test directory. A new check-cross-project-tests target
also exists, to run all tests (including debuginfo-test tests) within the
cross-project-tests directory.

What is the new directory for?

The new directory is intended for tests that have dependencies on multiple
LLVM sub-projects, e.g. both clang and lld, or clang and lldb etc, where
the thing under test is some interaction between these components. It is
also for tests that test main LLVM components that require some non-main
LLVM component (e.g. LLD) to generate a test input, where generating that
test input is otherwise not possible.

What is this new directory NOT for?

The new directory is not intended for tests that test a main LLVM component
where it is possible to generate the test input at runtime using other main
LLVM components (such tests should use the main LLVM tools, such as
yaml2obj/llvm-mc etc to generate test inputs). It is also not a place for
testing individual LLVM sub-projects, if they do not rely on things other
than main LLVM components to generate inputs (such tests belong in the
corresponding sub-project test directory).

Importantly, although this new directory does allow a location for
end-to-end tests, individual parts of LLVM should still be tested in
isolation in their respective test directories. The existence of this test
directory does not in itself constitute a go-ahead for end-to-end tests in
general. That discussion is a separate topic.

New tests within this directory should make use of REQUIRES: <X> tags
where
X is one (or more) of "clang", "lldb", "ld.lld",
"lld-link", "ld64.lld" or
"wasm-ld" to indicate that clang/lldb/the respective lld port is
required
for the test. These will be marked as UNSUPPORTED if the respective project
is not enabled via LLVM_ENABLE_PROJECT.

Actions you may need to take:

1) If you have `debuginfo-test` in your list of projects specified to
LLVM_ENABLE_PROJECTS, you need to change it to `cross-project-tests`. If
you don't, you will stop generating the relevant check targets.
2) If you are a build-bot owner the same applies - you likely need to
update your CMake command-line.
3) If you are currently using check-debuginfo-test, consider switching to
check-cross-project-tests.

The original RFC thread for this can be found at
https://lists.llvm.org/pipermail/llvm-dev/2021-January/148048.html. This
includes further discussion about some of the above points. The patch
series begins with https://reviews.llvm.org/D95339 (aka
https://github.com/llvm/llvm-project/commit/1364750dadbb56032ef73b4d0d8cbc88a51392da)
if you'd like to see the details.

Thanks to all those who contributed to the original RFC and reviews!

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210628/c8c7b290/attachment.html>

Just one slight correction to my previous email, caused by a faulty memory!
In the "Actions you may need to take:" block, point 3 refers to
"check-debuginfo-tests" when it should be simply
"check-debuginfo".

On Mon, 28 Jun 2021 at 12:42, James Henderson <jh7370.2008 at
my.bristol.ac.uk>
wrote:
> Hi all,
>
> TL;DR: there's a new top-level project called cross-project-test for
tests
> which require multiple LLVM projects, which replaces debuginfo-test. You
> may need to update LLVM_ENABLE_PROJECT accordingly.
>
> ---
>
> A short while ago, I pushed a series of commits that created a new
> top-level LLVM directory called `cross-project-tests`, and moved the
> `debuginfo-test` directory inside. I also renamed the debuginfo-test
> project in LLVM_ENABLE_PROJECTS in to `cross-project-tests`.
> check-debuginfo still exists as a target, and will run just the tests
> inside the debuginfo-test directory. A new check-cross-project-tests target
> also exists, to run all tests (including debuginfo-test tests) within the
> cross-project-tests directory.
>
> What is the new directory for?
>
> The new directory is intended for tests that have dependencies on multiple
> LLVM sub-projects, e.g. both clang and lld, or clang and lldb etc, where
> the thing under test is some interaction between these components. It is
> also for tests that test main LLVM components that require some non-main
> LLVM component (e.g. LLD) to generate a test input, where generating that
> test input is otherwise not possible.
>
> What is this new directory NOT for?
>
> The new directory is not intended for tests that test a main LLVM
> component where it is possible to generate the test input at runtime using
> other main LLVM components (such tests should use the main LLVM tools, such
> as yaml2obj/llvm-mc etc to generate test inputs). It is also not a place
> for testing individual LLVM sub-projects, if they do not rely on things
> other than main LLVM components to generate inputs (such tests belong in
> the corresponding sub-project test directory).
>
> Importantly, although this new directory does allow a location for
> end-to-end tests, individual parts of LLVM should still be tested in
> isolation in their respective test directories. The existence of this test
> directory does not in itself constitute a go-ahead for end-to-end tests in
> general. That discussion is a separate topic.
>
> New tests within this directory should make use of REQUIRES: <X> tags
> where X is one (or more) of "clang", "lldb",
"ld.lld", "lld-link",
> "ld64.lld" or "wasm-ld" to indicate that clang/lldb/the
respective lld port
> is required for the test. These will be marked as UNSUPPORTED if the
> respective project is not enabled via LLVM_ENABLE_PROJECT.
>
> Actions you may need to take:
>
> 1) If you have `debuginfo-test` in your list of projects specified to
> LLVM_ENABLE_PROJECTS, you need to change it to `cross-project-tests`. If
> you don't, you will stop generating the relevant check targets.
> 2) If you are a build-bot owner the same applies - you likely need to
> update your CMake command-line.
> 3) If you are currently using check-debuginfo-test, consider switching to
> check-cross-project-tests.
>
> The original RFC thread for this can be found at
> https://lists.llvm.org/pipermail/llvm-dev/2021-January/148048.html. This
> includes further discussion about some of the above points. The patch
> series begins with https://reviews.llvm.org/D95339 (aka
>
https://github.com/llvm/llvm-project/commit/1364750dadbb56032ef73b4d0d8cbc88a51392da)
> if you'd like to see the details.
>
> Thanks to all those who contributed to the original RFC and reviews!
>
> James
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210630/42e101a8/attachment.html>