llvm dev - Oct 2021 - Proposal: introduce dependency on abseil when building benchmarks

On Thu, 30 Sept 2021 at 18:08, Mircea Trofin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Abseil has some requirements
>
<https://abseil.io/docs/cpp/platforms/platforms#:~:text=Abseil%20requires%20a%20code%20base,14%20through%20C%2B%2B20).>
> that may not perfectly match those of the impacted projects. For example,
> abseil stopped supporting Ubuntu 14.04 before its TLS.
>
The list of supported platforms is *definitely* too small for LLVM users.
Half of their support is "best effort", which really isn't going
to cut it
once we forcefully depend on it.

We definitely run benchmarks on X86_64, Arm32/64, MIPS, PowerPC (Linux, Mac
and Windows on a mix of those), and there are probably people running on
SystemZ, RISCV and other less known architectures.

What is your plan for all the other platforms where abseil isn't supported?

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/201f5f72/attachment.html>

On Thu, Oct 7, 2021 at 8:40 AM Renato Golin <rengolin at gmail.com> wrote:
> On Thu, 30 Sept 2021 at 18:08, Mircea Trofin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Abseil has some requirements
>>
<https://abseil.io/docs/cpp/platforms/platforms#:~:text=Abseil%20requires%20a%20code%20base,14%20through%20C%2B%2B20).>
>> that may not perfectly match those of the impacted projects. For
example,
>> abseil stopped supporting Ubuntu 14.04 before its TLS.
>>
>
> The list of supported platforms is *definitely* too small for LLVM users.
> Half of their support is "best effort", which really isn't
going to cut it
> once we forcefully depend on it.
>
> We definitely run benchmarks on X86_64, Arm32/64, MIPS, PowerPC (Linux,
> Mac and Windows on a mix of those), and there are probably people running
> on SystemZ, RISCV and other less known architectures.
>
Making sure we're talking about the same thing: this is strictly about
benchmarks that require flipping LLVM_BUILD_BENCHMARKS
or LIBCXX_INCLUDE_BENCHMARKS; and it's about exploring how those specific
benchmarks relate to the 'benchmark' project which we currently have as
a
source copy in the llvm tree (i.e. we're sometimes manually cherry-picking
changes over, but we're not updating it through any automated means)

I looked under llvm-zorg, and can't find any bot configuration that flips
those flags in the first place, so perhaps these specific benchmarks are
run elsewhere (or my search was naive - I just grep-ed for '_BENCHMARKS'
-
and only got one entry turning off TEST_SUITE_RUN_BENCHMARKS, which appears
to be unrelated)?
>From the OS x CPU matrix above, it seems the main gap is in the CPU column
- since abseil supports Linux, MacOS, and Windows, but doesn't appear to
specifically support mips, powerpc, systemz, or riscv. So if anyone runs
these kinds of benchmarks (i.e. the ones that require flipping those cmake
flags) on those CPUs, *and* we wanted, in llvm, to periodically auto-update
the source copy of 'benchmark' project, only then we would have a
breaking
change.

I think there are 2 topics:
- I think the main one is: was there ever any intent to automatically
update the 2 copies of 'benchmarks' in the llvm tree? what about the one
in
llvm-test-suite?

- the second is getting a better understanding of where folks build and run
those specific benchmarks (it'd help the upstream 'benchmark'
project with
data points).

Renato, would you mind detailing your scenario (i.e. which of those 2 types
of benchmarks, on which OSxCPU)

If there are other folks that can chime in with their scenario, it'd be
much appreciated.

Thanks!


> What is your plan for all the other platforms where abseil isn't
supported?
>
> cheers,
> --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/1e31f93b/attachment.html>

On Thu, Oct 7, 2021 at 11:40 AM Renato Golin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Thu, 30 Sept 2021 at 18:08, Mircea Trofin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Abseil has some requirements
>>
<https://abseil.io/docs/cpp/platforms/platforms#:~:text=Abseil%20requires%20a%20code%20base,14%20through%20C%2B%2B20).>
>> that may not perfectly match those of the impacted projects. For
example,
>> abseil stopped supporting Ubuntu 14.04 before its TLS.
>>
>
> The list of supported platforms is *definitely* too small for LLVM users.
> Half of their support is "best effort", which really isn't
going to cut it
> once we forcefully depend on it.
>
> We definitely run benchmarks on X86_64, Arm32/64, MIPS, PowerPC (Linux,
> Mac and Windows on a mix of those), and there are probably people running
> on SystemZ, RISCV and other less known architectures.
>
I think this isn't actually as big a deal as it sounds. The primary
difference between "supported" and "best-effort" is whether
they have
working CI for that platform. Furthermore, many unlisted platforms do work.
In practice, abseil has a history of accepting patches to add portability
for "unsupported" platforms.

E.g. to take two examples:
https://github.com/abseil/abseil-cpp/pull/1021 adding support for AIX
https://github.com/abseil/abseil-cpp/pull/904 adding support for Linux on
HPPA, Alpha, IA64, and SH.

> What is your plan for all the other platforms where abseil isn't
supported?
>
The plan should simply be: submit patches if it's broken on an architecture
or OS that someone cares about.

Old versions of the toolchains is a different matter, however -- Abseil's
general promise is to support an old release of a C++ compiler for 5 years
after it has been superseded by a newer version. And supporting old
compilers tends to be a significant burden, so unlike porting to a new CPU
or OS, *that* isn't simply a "patches welcome" situation -- there
would
have to be a compelling case made that preserving support for an compiler
older than that was important enough to be worth the hassle.

See also: https://abseil.io/blog/20201001-platforms
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/ec410a5b/attachment.html>