llvm dev - Aug 2019 - Switching to the New Pass Manager by Default

I believe a good amount of them (if not most of them) have already been
ported! Off the top of my head, I remember that asan, tsan, msan, hwasan,
the kernel santizers, and sancov have been ported. I don't think ubsan has
been ported yet though.

You can also check if other passes you need run under the new PM by
checking PassRegistry.def.

On Fri, Aug 2, 2019, 14:20 Jordan Rupprecht <rupprecht at google.com>
wrote:
> Hi Leonard,
> Is the new pass manager expected to work with sanitizers now?
>
> On Fri, Aug 2, 2019 at 1:25 PM Leonard Chan via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I think so far most individual projects have their own statistics on
>> build time/performance impact on switching to the new PM, but I agree
that
>> there should at least be one place that people can reference on the
impact.
>>
>> On Fri, Aug 2, 2019 at 1:14 PM Philip Reames <listmail at
philipreames.com>
>> wrote:
>>
>>> Thanks for all the hard work that went into getting it here.
>>>
>>> +1
>>>
>>> Can you spell out what performance validation has been done?
>>> Informally, I know there's been quite a bit, but getting a
summary in one
>>> place for later reference would be super helpful.
>>>
>>> Philip
>>> On 8/2/19 10:25 AM, Leonard Chan via llvm-dev wrote:
>>>
>>> Hello all,
>>>
>>> As of now, all LLVM and Clang tests have been updated/addressed to
run
>>> under the new/experimental pass manager (at least the ones that
failed when
>>> using the new PM).
>>>
>>> For those who aren't aware of what the new pass manager (PM)
is, the
>>> tl;dr is that this will serve as a replacement for the legacy PM,
and
>>> promises faster build times by restructuring how passes are run
over IR
>>> units. For some time, when someone wants to create a new pass, they
may
>>> need to implement it for both pass managers since the legacy one is
what is
>>> enabled by default. Passes that were initially made under the new
PM have
>>> also slowly been ported over time.
>>>
>>> Now that LLVM 9.0.0 has branched, we have about 6 months before the
next
>>> release. We think it would be a good idea to take this opportunity
and make
>>> the new PM the default one which gives us enough time to work out
any kinks
>>> that might come out of this switch before LLVM 10.0.0 branch point.
We
>>> suspect that there may be other LLVM projects that will be affected
by
>>> this, probably from unported passes.
>>>
>>> Does anyone have any opinions on this?
>>>
>>> Off the top of my head, the next immediate work would be to update
the
>>> docs with instructions on how to write or port a new PM pass, and
address
>>> any breakages for other LLVM projects.
>>>
>>> Thanks,
>>> Leonard
>>>
>>> _______________________________________________
>>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190802/f1c5b5a7/attachment.html>

OK. I have a tsan crash to report, will file shortly :)

FWIW, we've been using the new pass manager for a long time now in our
non-sanitizer builds, and haven't noticed issues other than a few compiler
crashes, which mostly seem due to people testing only with the legacy pass
manager.

On Fri, Aug 2, 2019 at 2:29 PM Leonard Chan <leonardchan at google.com>
wrote:
> I believe a good amount of them (if not most of them) have already been
> ported! Off the top of my head, I remember that asan, tsan, msan, hwasan,
> the kernel santizers, and sancov have been ported. I don't think ubsan
has
> been ported yet though.
>
> You can also check if other passes you need run under the new PM by
> checking PassRegistry.def.
>
> On Fri, Aug 2, 2019, 14:20 Jordan Rupprecht <rupprecht at google.com>
wrote:
>
>> Hi Leonard,
>> Is the new pass manager expected to work with sanitizers now?
>>
>> On Fri, Aug 2, 2019 at 1:25 PM Leonard Chan via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> I think so far most individual projects have their own statistics
on
>>> build time/performance impact on switching to the new PM, but I
agree that
>>> there should at least be one place that people can reference on the
impact.
>>>
>>> On Fri, Aug 2, 2019 at 1:14 PM Philip Reames <listmail at
philipreames.com>
>>> wrote:
>>>
>>>> Thanks for all the hard work that went into getting it here.
>>>>
>>>> +1
>>>>
>>>> Can you spell out what performance validation has been done?
>>>> Informally, I know there's been quite a bit, but getting a
summary in one
>>>> place for later reference would be super helpful.
>>>>
>>>> Philip
>>>> On 8/2/19 10:25 AM, Leonard Chan via llvm-dev wrote:
>>>>
>>>> Hello all,
>>>>
>>>> As of now, all LLVM and Clang tests have been updated/addressed
to run
>>>> under the new/experimental pass manager (at least the ones that
failed when
>>>> using the new PM).
>>>>
>>>> For those who aren't aware of what the new pass manager
(PM) is, the
>>>> tl;dr is that this will serve as a replacement for the legacy
PM, and
>>>> promises faster build times by restructuring how passes are run
over IR
>>>> units. For some time, when someone wants to create a new pass,
they may
>>>> need to implement it for both pass managers since the legacy
one is what is
>>>> enabled by default. Passes that were initially made under the
new PM have
>>>> also slowly been ported over time.
>>>>
>>>> Now that LLVM 9.0.0 has branched, we have about 6 months before
the
>>>> next release. We think it would be a good idea to take this
opportunity and
>>>> make the new PM the default one which gives us enough time to
work out any
>>>> kinks that might come out of this switch before LLVM 10.0.0
branch point.
>>>> We suspect that there may be other LLVM projects that will be
affected by
>>>> this, probably from unported passes.
>>>>
>>>> Does anyone have any opinions on this?
>>>>
>>>> Off the top of my head, the next immediate work would be to
update the
>>>> docs with instructions on how to write or port a new PM pass,
and address
>>>> any breakages for other LLVM projects.
>>>>
>>>> Thanks,
>>>> Leonard
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttps://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190802/a4d83ced/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4849 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190802/a4d83ced/attachment-0001.bin>

Getting ubsan ported would be good if it hasn't. Do you happen to know
what's needed there?

-eric

On Fri, Aug 2, 2019 at 2:29 PM Leonard Chan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> I believe a good amount of them (if not most of them) have already been
ported! Off the top of my head, I remember that asan, tsan, msan, hwasan, the
kernel santizers, and sancov have been ported. I don't think ubsan has been
ported yet though.
>
> You can also check if other passes you need run under the new PM by
checking PassRegistry.def.
>
> On Fri, Aug 2, 2019, 14:20 Jordan Rupprecht <rupprecht at google.com>
wrote:
>>
>> Hi Leonard,
>> Is the new pass manager expected to work with sanitizers now?
>>
>> On Fri, Aug 2, 2019 at 1:25 PM Leonard Chan via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>>
>>> I think so far most individual projects have their own statistics
on build time/performance impact on switching to the new PM, but I agree that
there should at least be one place that people can reference on the impact.
>>>
>>> On Fri, Aug 2, 2019 at 1:14 PM Philip Reames <listmail at
philipreames.com> wrote:
>>>>
>>>> Thanks for all the hard work that went into getting it here.
>>>>
>>>> +1
>>>>
>>>> Can you spell out what performance validation has been done? 
Informally, I know there's been quite a bit, but getting a summary in one
place for later reference would be super helpful.
>>>>
>>>> Philip
>>>>
>>>> On 8/2/19 10:25 AM, Leonard Chan via llvm-dev wrote:
>>>>
>>>> Hello all,
>>>>
>>>> As of now, all LLVM and Clang tests have been updated/addressed
to run under the new/experimental pass manager (at least the ones that failed
when using the new PM).
>>>>
>>>> For those who aren't aware of what the new pass manager
(PM) is, the tl;dr is that this will serve as a replacement for the legacy PM,
and promises faster build times by restructuring how passes are run over IR
units. For some time, when someone wants to create a new pass, they may need to
implement it for both pass managers since the legacy one is what is enabled by
default. Passes that were initially made under the new PM have also slowly been
ported over time.
>>>>
>>>> Now that LLVM 9.0.0 has branched, we have about 6 months before
the next release. We think it would be a good idea to take this opportunity and
make the new PM the default one which gives us enough time to work out any kinks
that might come out of this switch before LLVM 10.0.0 branch point. We suspect
that there may be other LLVM projects that will be affected by this, probably
from unported passes.
>>>>
>>>> Does anyone have any opinions on this?
>>>>
>>>> Off the top of my head, the next immediate work would be to
update the docs with instructions on how to write or port a new PM pass, and
address any breakages for other LLVM projects.
>>>>
>>>> Thanks,
>>>> Leonard
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 8/2/19 7:20 PM, Eric Christopher via llvm-dev wrote:
> Getting ubsan ported would be good if it hasn't. Do you happen to know
> what's needed there?

Are we sure that there's anything to port? I thought that ubsan's 
instrumentation is added in Clang's codegen, not by an instrumentation 
pass. I don't see anything in BackendUtil.cpp that jumps out at me as 
ubsan (I might be missing something because ubsan is really a sanitizer 
group in Clang, not an individual sanitizer, so it's broken up into a 
large number of flags).

  -Hal

>
> -eric
>
> On Fri, Aug 2, 2019 at 2:29 PM Leonard Chan via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> I believe a good amount of them (if not most of them) have already been
ported! Off the top of my head, I remember that asan, tsan, msan, hwasan, the
kernel santizers, and sancov have been ported. I don't think ubsan has been
ported yet though.
>>
>> You can also check if other passes you need run under the new PM by
checking PassRegistry.def.
>>
>> On Fri, Aug 2, 2019, 14:20 Jordan Rupprecht <rupprecht at
google.com> wrote:
>>> Hi Leonard,
>>> Is the new pass manager expected to work with sanitizers now?
>>>
>>> On Fri, Aug 2, 2019 at 1:25 PM Leonard Chan via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>>> I think so far most individual projects have their own
statistics on build time/performance impact on switching to the new PM, but I
agree that there should at least be one place that people can reference on the
impact.
>>>>
>>>> On Fri, Aug 2, 2019 at 1:14 PM Philip Reames <listmail at
philipreames.com> wrote:
>>>>> Thanks for all the hard work that went into getting it
here.
>>>>>
>>>>> +1
>>>>>
>>>>> Can you spell out what performance validation has been
done?  Informally, I know there's been quite a bit, but getting a summary in
one place for later reference would be super helpful.
>>>>>
>>>>> Philip
>>>>>
>>>>> On 8/2/19 10:25 AM, Leonard Chan via llvm-dev wrote:
>>>>>
>>>>> Hello all,
>>>>>
>>>>> As of now, all LLVM and Clang tests have been
updated/addressed to run under the new/experimental pass manager (at least the
ones that failed when using the new PM).
>>>>>
>>>>> For those who aren't aware of what the new pass manager
(PM) is, the tl;dr is that this will serve as a replacement for the legacy PM,
and promises faster build times by restructuring how passes are run over IR
units. For some time, when someone wants to create a new pass, they may need to
implement it for both pass managers since the legacy one is what is enabled by
default. Passes that were initially made under the new PM have also slowly been
ported over time.
>>>>>
>>>>> Now that LLVM 9.0.0 has branched, we have about 6 months
before the next release. We think it would be a good idea to take this
opportunity and make the new PM the default one which gives us enough time to
work out any kinks that might come out of this switch before LLVM 10.0.0 branch
point. We suspect that there may be other LLVM projects that will be affected by
this, probably from unported passes.
>>>>>
>>>>> Does anyone have any opinions on this?
>>>>>
>>>>> Off the top of my head, the next immediate work would be to
update the docs with instructions on how to write or port a new PM pass, and
address any breakages for other LLVM projects.
>>>>>
>>>>> Thanks,
>>>>> Leonard
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory