llvm dev - Oct 2017 - IR Pass Ordering Sensitivity

On Sat, Oct 14, 2017 at 11:05 AM, John Regehr via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> These are definitely LLVM bugs. It would be best to report reduced test
> cases against top of tree.
>
> We should have some automated infrastructure for finding these too...
>
> John
>
Zhendong & friends generally do that (and reported many bugs :) I
tried that myself, but never got to automate the whole procedure.
While the problem is combinatorial in nature, evidence shows that you
don't need to enumerate all the possible combination of passes to
break LLVM. I think an `opt` option would be a good start. You can
then take the testsuite and run with that, or generate random IR with
llvm-stress and pipe that to opt. The swift folks have a slightly more
sophisticated infrastructure for doing this (you can dump the SIL
pipeline in a YAML file, and then IIRC, feed that back to `sil-opt`),
but something simpler will do, IMHO. Happy to discuss this further if
folks are in California next week :)

Thanks,

--
Davide

> something simpler will do, IMHO. Happy to discuss this further if
> folks are in California next week :)
Yes, I'll be in California next week, let's chat!

We could make use of the autotuner I'm currently building:

https://github.com/kavon/autotune

It tries to find an optimal sequence of IR passes, and I've run into a bunch
of different bugs with it so far (e.g., it seems structurizecfg interacts poorly
with 'invoke'). I'll start submitting bug reports for them soon.

~kavon
> On Oct 14, 2017, at 5:52 PM, Davide Italiano via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On Sat, Oct 14, 2017 at 11:05 AM, John Regehr via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> These are definitely LLVM bugs. It would be best to report reduced test
>> cases against top of tree.
>> 
>> We should have some automated infrastructure for finding these too...
>> 
>> John
>> 
> 
> Zhendong & friends generally do that (and reported many bugs :) I
> tried that myself, but never got to automate the whole procedure.
> While the problem is combinatorial in nature, evidence shows that you
> don't need to enumerate all the possible combination of passes to
> break LLVM. I think an `opt` option would be a good start. You can
> then take the testsuite and run with that, or generate random IR with
> llvm-stress and pipe that to opt. The swift folks have a slightly more
> sophisticated infrastructure for doing this (you can dump the SIL
> pipeline in a YAML file, and then IIRC, feed that back to `sil-opt`),
> but something simpler will do, IMHO. Happy to discuss this further if
> folks are in California next week :)
> 
> Thanks,
> 
> --
> Davide
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Sat, Oct 14, 2017 at 10:58:17PM -0500, Kavon Farvardin via llvm-dev
wrote:
> > something simpler will do, IMHO. Happy to discuss this further if
> > folks are in California next week :)
> 
> Yes, I'll be in California next week, let's chat!
> 
> We could make use of the autotuner I'm currently building:
> 
> https://github.com/kavon/autotune
> 
> It tries to find an optimal sequence of IR passes, and I've run into a
bunch of different bugs with it so far (e.g., it seems structurizecfg interacts
poorly with 'invoke'). I'll start submitting bug reports for them
soon.
As part of the stress-suite of the LLVM obfuscator we're going to
present next Thursday, we ran into similar problems. Well, when you
s/optimization/obfuscation/ many things go wrong anyway.

I'm quite interested in the pass scheduling problem, because interaction
between optimization and obfuscation are complex, let's chat!