llvm dev - Oct 2021 - Proposal: introduce dependency on abseil when building benchmarks

On Tue, Oct 5, 2021 at 9:37 PM Mehdi AMINI <joker.eph at gmail.com> wrote:
>
>
> On Tue, Oct 5, 2021 at 9:26 PM Mircea Trofin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>>
>> On Tue, Oct 5, 2021 at 8:57 PM Reid Kleckner <rnk at google.com>
wrote:
>>
>>> On Thu, Sep 30, 2021 at 10:08 AM Mircea Trofin via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> TL;DR; When either of LLVM_BUILD_BENCHMARKS
>>>> or LIBCXX_INCLUDE_BENCHMARKS are enabled, as well as for
llvm-test-suite, a
>>>> dependency to abseil would either be auto-downloaded by the
build system,
>>>> or need to be user-specifiable, or provided in the source tree.
>>>>
>>>
>>> Could you please elaborate on which of these approaches will be
used for
>>> LLVM? How will this affect regular LLVM developers?
>>>
>> Assuming regular LLVM developers means developers that don't enable
>> LLVM_BUILD_BENCHMARKS, nor LIBCXX_INCLUDE_BENCHMARKS, then they are not
>> affected.
>>
>> The current PR in "benchmarks" upstream is set up so that it
will either
>> download the abseil dependency at build time, or, if the location of
abseil
>> is specified via a cmake flag, then it uses that one (which covers the
last
>> 2 options). I don't know if the first option is acceptable by those
that
>> enable LLVM_BUILD_BENCHMARKS / LIBCXX_INCLUDE_BENCHMARKS, and this is
>> something I'm hoping to discover with this thread. If it is
acceptable,
>> it's a transparent option, so it won't directly impact those
users either.
>>
>
> What should we expect in terms of ability to benchmark LLVM on various
> platforms / OSes? It seems like we will tie ourselves to Abseil, and I
> don't know anything about how their compatibility matrix compares to
LLVM
> one? (and if it will continue being true in the foreseeable future?)
>
The goal isn't for benchmark to change its dependency matrix due to the new
dependency ( +dominic hamon <dominic at google.com> , please correct me).
So
from the perspective of LLVM, this doesn't change anything. Granted, there
is now a strictly higher probability for breaking changes, if abseil
decides to change its support matrix and break benchmark, and thus affect
the latter's LLVM users.


> Thanks,
>
> --
> Mehdi
>
>
>
>>
>>> ----
>>>
>>> This is truly unrelated, but I have a lot of feelings about this,
and I
>>> will use this opportunity to inappropriately complain that the
benchmarks
>>> library has been spamming me with cmake warnings about std::regex
for
>>> years: https://bugs.llvm.org/show_bug.cgi?id=38874 The CMake step
>>> really ought to be warning-clean.
>>>
>> Ack. Added an issue on the project side:
>> https://github.com/google/benchmark/issues/1236 (maybe it has better
>> visibility)
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/c4d46f8f/attachment.html>

On Thu, Oct 7, 2021 at 4:06 PM Mircea Trofin <mtrofin at google.com>
wrote:
>
>
> On Tue, Oct 5, 2021 at 9:37 PM Mehdi AMINI <joker.eph at gmail.com>
wrote:
>
>>
>>
>> On Tue, Oct 5, 2021 at 9:26 PM Mircea Trofin via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>>
>>>
>>> On Tue, Oct 5, 2021 at 8:57 PM Reid Kleckner <rnk at
google.com> wrote:
>>>
>>>> On Thu, Sep 30, 2021 at 10:08 AM Mircea Trofin via llvm-dev
<
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> TL;DR; When either of LLVM_BUILD_BENCHMARKS
>>>>> or LIBCXX_INCLUDE_BENCHMARKS are enabled, as well as for
llvm-test-suite, a
>>>>> dependency to abseil would either be auto-downloaded by the
build system,
>>>>> or need to be user-specifiable, or provided in the source
tree.
>>>>>
>>>>
>>>> Could you please elaborate on which of these approaches will be
used
>>>> for LLVM? How will this affect regular LLVM developers?
>>>>
>>> Assuming regular LLVM developers means developers that don't
enable
>>> LLVM_BUILD_BENCHMARKS, nor LIBCXX_INCLUDE_BENCHMARKS, then they are
not
>>> affected.
>>>
>>> The current PR in "benchmarks" upstream is set up so that
it will either
>>> download the abseil dependency at build time, or, if the location
of abseil
>>> is specified via a cmake flag, then it uses that one (which covers
the last
>>> 2 options). I don't know if the first option is acceptable by
those that
>>> enable LLVM_BUILD_BENCHMARKS / LIBCXX_INCLUDE_BENCHMARKS, and this
is
>>> something I'm hoping to discover with this thread. If it is
acceptable,
>>> it's a transparent option, so it won't directly impact
those users either.
>>>
>>
>> What should we expect in terms of ability to benchmark LLVM on various
>> platforms / OSes? It seems like we will tie ourselves to Abseil, and I
>> don't know anything about how their compatibility matrix compares
to LLVM
>> one? (and if it will continue being true in the foreseeable future?)
>>
>
> The goal isn't for benchmark to change its dependency matrix due to the
> new dependency ( +dominic hamon <dominic at google.com> , please
correct
> me). So from the perspective of LLVM, this doesn't change anything.
> Granted, there is now a strictly higher probability for breaking changes,
> if abseil decides to change its support matrix and break benchmark, and
> thus affect the latter's LLVM users.
>
>
We're reducing our dependency matrix (dropping support for older
compilers/OSs) as part of the adoption of abseil. This may impact llvm if
there was an expectation to continue to benchmark against those older
compilers/OSs.

https://google.github.io/benchmark/dependencies.html is the current policy
but with abseil that "best effort" support will no longer be viable.

>
>
>> Thanks,
>>
>> --
>> Mehdi
>>
>>
>>
>>>
>>>> ----
>>>>
>>>> This is truly unrelated, but I have a lot of feelings about
this, and I
>>>> will use this opportunity to inappropriately complain that the
benchmarks
>>>> library has been spamming me with cmake warnings about
std::regex for
>>>> years: https://bugs.llvm.org/show_bug.cgi?id=38874 The CMake
step
>>>> really ought to be warning-clean.
>>>>
>>> Ack. Added an issue on the project side:
>>> https://github.com/google/benchmark/issues/1236 (maybe it has
better
>>> visibility)
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/59aa1bd9/attachment.html>

On Thu, Oct 7, 2021 at 8:10 AM dominic hamon <dominic at google.com>
wrote:
>
>
> On Thu, Oct 7, 2021 at 4:06 PM Mircea Trofin <mtrofin at google.com>
wrote:
>
>>
>>
>> On Tue, Oct 5, 2021 at 9:37 PM Mehdi AMINI <joker.eph at
gmail.com> wrote:
>>
>>>
>>>
>>> On Tue, Oct 5, 2021 at 9:26 PM Mircea Trofin via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Oct 5, 2021 at 8:57 PM Reid Kleckner <rnk at
google.com> wrote:
>>>>
>>>>> On Thu, Sep 30, 2021 at 10:08 AM Mircea Trofin via llvm-dev
<
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> TL;DR; When either of LLVM_BUILD_BENCHMARKS
>>>>>> or LIBCXX_INCLUDE_BENCHMARKS are enabled, as well as
for llvm-test-suite, a
>>>>>> dependency to abseil would either be auto-downloaded by
the build system,
>>>>>> or need to be user-specifiable, or provided in the
source tree.
>>>>>>
>>>>>
>>>>> Could you please elaborate on which of these approaches
will be used
>>>>> for LLVM? How will this affect regular LLVM developers?
>>>>>
>>>> Assuming regular LLVM developers means developers that
don't enable
>>>> LLVM_BUILD_BENCHMARKS, nor LIBCXX_INCLUDE_BENCHMARKS, then they
are not
>>>> affected.
>>>>
>>>> The current PR in "benchmarks" upstream is set up so
that it will
>>>> either download the abseil dependency at build time, or, if the
location of
>>>> abseil is specified via a cmake flag, then it uses that one
(which covers
>>>> the last 2 options). I don't know if the first option is
acceptable by
>>>> those that enable LLVM_BUILD_BENCHMARKS /
LIBCXX_INCLUDE_BENCHMARKS, and
>>>> this is something I'm hoping to discover with this thread.
If it is
>>>> acceptable, it's a transparent option, so it won't
directly impact those
>>>> users either.
>>>>
>>>
>>> What should we expect in terms of ability to benchmark LLVM on
various
>>> platforms / OSes? It seems like we will tie ourselves to Abseil,
and I
>>> don't know anything about how their compatibility matrix
compares to LLVM
>>> one? (and if it will continue being true in the foreseeable
future?)
>>>
>>
>> The goal isn't for benchmark to change its dependency matrix due to
the
>> new dependency ( +dominic hamon <dominic at google.com> , please
correct
>> me). So from the perspective of LLVM, this doesn't change anything.
>> Granted, there is now a strictly higher probability for breaking
changes,
>> if abseil decides to change its support matrix and break benchmark, and
>> thus affect the latter's LLVM users.
>>
>>
> We're reducing our dependency matrix (dropping support for older
> compilers/OSs) as part of the adoption of abseil. This may impact llvm if
> there was an expectation to continue to benchmark against those older
> compilers/OSs.
>
Isn't that support drop part of usual business, though (i.e. drop after
TLS)?
>
> https://google.github.io/benchmark/dependencies.html is the current
> policy but with abseil that "best effort" support will no longer
be viable.
>
>
>>
>>
>>> Thanks,
>>>
>>> --
>>> Mehdi
>>>
>>>
>>>
>>>>
>>>>> ----
>>>>>
>>>>> This is truly unrelated, but I have a lot of feelings about
this, and
>>>>> I will use this opportunity to inappropriately complain
that the benchmarks
>>>>> library has been spamming me with cmake warnings about
std::regex for
>>>>> years: https://bugs.llvm.org/show_bug.cgi?id=38874 The
CMake step
>>>>> really ought to be warning-clean.
>>>>>
>>>> Ack. Added an issue on the project side:
>>>> https://github.com/google/benchmark/issues/1236 (maybe it has
better
>>>> visibility)
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211007/14e1294d/attachment.html>