llvm dev - May 2018 - llvm-dev Digest, Vol 167, Issue 3

Hi all,
We have enabled Thin LTO and LTO for a specific target architecture. What
can be the possible scopes of improvement depending on the target  after we
enable the basic LTO and thin LTO.?  Wanted to know the possible approach
we can think to see how it performs compared to a not LTO enabled case and
also what all possible directions can we think to improve upon it depending
on the arch. I am looking for a general approach of improvement. Any
suggestion will be valuable.
Thanks,
Siddharth

On Tue, May 1, 2018 at 11:15 PM, via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> Send llvm-dev mailing list submissions to
>         llvm-dev at lists.llvm.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> or, via email, send a message with subject or body 'help' to
>         llvm-dev-request at lists.llvm.org
>
> You can reach the person managing the list at
>         llvm-dev-owner at lists.llvm.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of llvm-dev digest..."
>
>
> Today's Topics:
>
>    1. Re: Need guidance to work on NEW PASS managers bugs
>       (Kaylor, Andrew via llvm-dev)
>    2. Re: Disabling Exception in LLVM (Chris Bieneman via llvm-dev)
>    3. Re: Disabling Exception in LLVM
>       (Siddharth Shankar Swain via llvm-dev)
>    4. Re: ThinLTO + CFI (via llvm-dev)
>    5. Re: Disabling Exception in LLVM (Chris Bieneman via llvm-dev)
>    6. Re: RFC: LLVM Assembly format for ThinLTO Summary
>       (Teresa Johnson via llvm-dev)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 1 May 2018 17:22:26 +0000
> From: "Kaylor, Andrew via llvm-dev" <llvm-dev at
lists.llvm.org>
> To: vivek pandya <vivekvpandya at gmail.com>
> Cc: "llvm-dev at lists.llvm.org" <llvm-dev at
lists.llvm.org>
> Subject: Re: [llvm-dev] Need guidance to work on NEW PASS managers
>         bugs
> Message-ID:
>         <0983E6C011D2DC4188F8761B533492DE825CFA11 at ORSMSX115.amr.
> corp.intel.com>
>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Vivek,
>
> Have you read the mailing list threads on this topic? I don’t believe
> we’re quite ready to make the switch yet. There was a discussion last
> October about what was left to be done. I’m sure it has been discussed
> since then too. Here’s a link to the start of the October discussion.
>
> http://lists.llvm.org/pipermail/llvm-dev/2017-October/118280.html
>
> If you’d like to get involved, one possible area you could contribute is
> adding optbisect/optnone support as mentioned in this bug:
>
> https://bugs.llvm.org/show_bug.cgi?id=28316
>
> If that looks like something you’re interested in I can offer some
> guidance with it.
>
> Thanks,
> Andy
>
> From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of
> vivek pandya via llvm-dev
> Sent: Saturday, April 28, 2018 9:23 AM
> To: llvm-dev <llvm-dev at lists.llvm.org>
> Subject: [llvm-dev] Need guidance to work on NEW PASS managers bugs
>
> Hello LLVM-Devs,
>
> I am trying to get some starting points for working on following new pass
> manager related bugs:
>
> https://bugs.llvm.org/show_bug.cgi?id=28322 [PM] Remove use of old PM in
> the middle-end.
>
> https://bugs.llvm.org/show_bug.cgi?id=28323 [PM] Use new PM in production
> for Clang, LLD, libLTO, etc. middle-end
> https://bugs.llvm.org/show_bug.cgi?id=28321 [PM] Remove use of old PM in
> the backend
>
> I read related code but did not get a good starting point.
> Can someone guide me through this? Can we add more details to these bugs?
> Or can we further divide these bugs to smaller workable items?
>
> Any help will be appreciated.
>
> Thanks,
> Vivek
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/llvm-dev/
> attachments/20180501/1fe4dac9/attachment-0001.html>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 01 May 2018 10:32:14 -0700
> From: Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org>
> To: Siddharth Shankar Swain <h2015096 at pilani.bits-pilani.ac.in>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] Disabling Exception in LLVM
> Message-ID: <4C53C08A-8B60-443C-9731-A139760B4C21 at me.com>
> Content-Type: text/plain; charset="utf-8"
>
> Siddharth,
>
> I'm not sure what coding standards you refer to when you say "some
C++
> coding standard". This question is answered in the LLVM Coding
Standards
> document here:
>
> https://www.llvm.org/docs/CodingStandards.html#do-not-
> use-rtti-or-exceptions <https://www.llvm.org/docs/
> CodingStandards.html#do-not-use-rtti-or-exceptions>
>
> As such LLVM's coding standards prohibit the use of exceptions and
RTTI.
>
> -Chris
>
> > On May 1, 2018, at 10:02 AM, Siddharth Shankar Swain via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi all,
> > Can anyone explain why exceptions are disabled in LLVM, even if some
C++
> coding standard tells to use exceptions ?
> > Thanks,
> > Siddharth
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/llvm-dev/
> attachments/20180501/49f2c367/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 1 May 2018 23:08:17 +0530
> From: Siddharth Shankar Swain via llvm-dev <llvm-dev at
lists.llvm.org>
> To: Chris Bieneman <chris.bieneman at me.com>, llvm-dev
>         <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] Disabling Exception in LLVM
> Message-ID:
>         <CAMkbrzKtMUrYVSA_Ke3YJA=jkZNms1N9Ao9_2DWVfwD3qcqn8g at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Chris,
> Thanks for answering, Can u clarify on this comment mentioned in
> https://github.com/Z3Prover/z3/issues/861 .
>
>   cplusplus no exception support · Issue #861 · Z3Prover/z3 · GitHub
>
>    - LLVM's *source code* does not use exceptions for performance
reasons
>    and so is compiled by default with -fno-exceptions. When using
LLVM's
>    libraries via it's C++ interface it is important to match how LLVM
was
>    built (i.e. do not mix code built with and without exception support).
>    - The Clang compiler which is built on top of LLVM is also compiled
>    without exceptions by default. However the built Clang can compile C++
> code
>    with or without exceptions (using the -fno-exceptions). flag. Said
>    another way Clang's implementation doesn't use exceptions but
Clang
> itself
>    can compile C++ code with or without exception support.
>
>
> Can anyone clarify on this comment ?
>
> Thanks,
> Siddharth
>
> On Tue, May 1, 2018 at 11:02 PM, Chris Bieneman <chris.bieneman at
me.com>
> wrote:
>
> > Siddharth,
> >
> > I'm not sure what coding standards you refer to when you say
"some C++
> > coding standard". This question is answered in the LLVM Coding
Standards
> > document here:
> >
> > https://www.llvm.org/docs/CodingStandards.html#do-not-
> > use-rtti-or-exceptions
> >
> > As such LLVM's coding standards prohibit the use of exceptions and
RTTI.
> >
> > -Chris
> >
> > On May 1, 2018, at 10:02 AM, Siddharth Shankar Swain via llvm-dev <
> > llvm-dev at lists.llvm.org> wrote:
> >
> > Hi all,
> > Can anyone explain why exceptions are disabled in LLVM, even if some
C++
> > coding standard tells to use exceptions ?
> > Thanks,
> > Siddharth
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/llvm-dev/
> attachments/20180501/be88b48c/attachment-0001.html>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 1 May 2018 17:41:00 +0000
> From: via llvm-dev <llvm-dev at lists.llvm.org>
> To: <vlad at tsyrklevich.net>
> Cc: llvm-dev at lists.llvm.org, pcc at google.com
> Subject: Re: [llvm-dev] ThinLTO + CFI
> Message-ID: <0C0C8796-2DCB-403B-8756-E8FD258EDD6E at sony.com>
> Content-Type: text/plain; charset="utf-8"
>
>
>
> > On Apr 30, 2018, at 5:43 PM, Vlad Tsyrklevich <vlad at
tsyrklevich.net>
> wrote:
> >
> > Hi Dmitry, the direct call patch seems like a good start; however, by
> falling through to the F->replaceUsesExceptBlockAddr(FDecl) at the
bottom
> of LowerTypeTestsModule::importFunction() I believe it will replace all
> uses of a function (for a definition outside the TU it's defined in)
with
> the direct function reference instead of just direct calls.
>
> You’re right, it was meant to bail after direct call replacement.
>
> > Also, I believe the logic for replaceDirectCalls() should be more
> restrictive, currently it will also rewrite calls where the function use is
> as a function pointer argument instead of the call target.
>
> I’m checking if the user is a call instruction and that it’s
> getCalledFunction() returns non-null. I thought this would only give me
> direct calls. If not, what would be a good check?
>
> I will create a Phab review to make it easier to comment.
>
>
> >
> > On Mon, Apr 30, 2018 at 1:05 PM <dmitry.mikulin at sony.com>
wrote:
> > Replacing direct calls to jump table entries with calls to real
targets
> recovers most of the performance loss in my benchmark. Dealing with link
> order files is a bit cumbersome though: I can’t keep both, say “foo” and
> “foo.cfi” next to each other in link order, because if both exist, the
> linker reorders the jump table next to the real function. This is not what
> we want when the goal is to get rid of calls through jump tables. I had to
> manually pick the functions which were renamed to “foo.cfi” and replace
> them in the link order file. The process can be automated, but it’s a bit
> flaky if we have to do it to make CFI work.
> >
> > I attached 2 patches here: one is the direct call replacement and the
> other is moving type test lowering pass to run later in the pipeline.
> Interestingly, running the pass later seems to help with performance as
> well. Though the comment in PassManagerBuilder implies that this pass needs
> to run early. Why is it different from full LTO?
> >
> >
> >
> >
> >
> > > On Apr 27, 2018, at 11:04 AM, via llvm-dev <llvm-dev at
lists.llvm.org>
> wrote:
> > >
> > > For the test case below, I get the following IR for main() on
entry to
> ThinLTO backend invocation of LowerTypeTestsModule::lower(). This is after
> I moved this pass down in the pipeline so it’s invoked after inlining.
> > >
> > > The declarations for foo() and bar() are read in at the time of
module
> import, Importer.importFunctions() in lto::thinBackend(). They do not have
> type metadata attached to them.
> > > In lowerTypeTestCall() we check if the pointer in the type test
is of
> a known type, so we look at bitcast and then select operands. foo and bar
> in select are global objects with no type metadata, so the type check
> cannot be guaranteed to be true and it can’t be eliminated. In full LTO
> case this works as expected: both foo and bar are the same known type and
> type check is gone.
> > >
> > > Maybe the problem is not with renaming but with the missing type
> metadata in this particular case.
> > > Though having so many direct calls routed through the jump table
still
> seems problematic. Is there a feasible solution?
> > >
> > > --------------------------------
> > > define hidden i32 @main(i32, i8** nocapture readnone)
> local_unnamed_addr #0 !type !3 !type !4 {
> > >  %3 = add nsw i32 %0, -2
> > >  store i32 %3, i32* @i, align 4, !tbaa !5
> > >  %4 = icmp sgt i32 %0, 1
> > >  %5 = select i1 %4, i32 ()* @foo, i32 ()* @bar
> > >  %6 = bitcast i32 ()* %5 to i8*, !nosanitize !9
> > >  %7 = tail call i1 @llvm.type.test(i8* nonnull %6, metadata
> !"_ZTSFivE"), !nosanitize !9
> > >  br i1 %7, label %10, label %8, !prof !10, !nosanitize !9
> > >
> > > ; <label>:8:                                      ; preds =
%2
> > >  %9 = ptrtoint i32 ()* %5 to i64
> > >  tail call void @__ubsan_handle_cfi_check_fail_abort(i8*
> getelementptr inbounds ({ i8, { [4 x i8]*, i32, i32 }, { i16, i16, [13 x
> i8] }* }, { i8, { [4 x i8]*, i32, i32 }, { i16, i16, [13 x i8] }* }* @anon.
> fad58de7366495db4650cfefac2fcd61.1, i64 0, i32 0), i64 %9, i64 undef) #4,
> !nosanitize !9
> > >  unreachable, !nosanitize !9
> > >
> > > ; <label>:10:                                     ; preds =
%2
> > >  %11 = tail call i32 %5() #5
> > >  ret i32 %11
> > > }
> > >
> > > . . .
> > > declare hidden i32 @foo() #3
> > > declare hidden i32 @bar() #3
> > >
> > >
> > > —————————test case---------------
> > > b.c
> > > ================> > > typedef int (*fptr_t) (void);
> > > fptr_t get_fptr();
> > > extern int i;
> > >
> > > int main(int argc, char *argv[])
> > > {
> > >  i = argc - 2;
> > >  fptr_t fp = get_fptr();
> > >  return fp();
> > > }
> > >
> > > v.c
> > > ===============> > > int i;
> > > typedef int (*fptr_t) (void);
> > > int foo(void) {  return 11; }
> > > int bar(void) {  return 22; }
> > > fptr_t get_fptr(void) {  return (i >= 0) ? foo : bar; }
> > >
> > >
> > >> On Apr 26, 2018, at 5:29 PM, Peter Collingbourne <peter at
pcc.me.uk>
> wrote:
> > >>
> > >>> We could probably tolerate a certain amount of unused
jump table
> entries. However, I just realized that all non-inline imported calls end up
> going through a jump table entry. Is that correct?
> > >>
> > >> In fact it is all calls that go through a function pointer
type that
> is used anywhere in the program for an indirect call, but depending on your
> program that could be very close to "yes".
> > >>
> > >>> Moving the type check lowering pass further down the
pipeline (after
> inlining) still does not solve the problem because CFI renaming happens
> early and symbols attached to the jump table do not have a matching type.
> > >>
> > >> As far as I know, renaming happens during the LowerTypeTests
pass,
> after the type checks are lowered.
> > >> Lowering:
http://llvm-cs.pcc.me.uk/lib/Transforms/IPO/LowerTypeTests.
> cpp#1620
> > >> Renaming:
http://llvm-cs.pcc.me.uk/lib/Transforms/IPO/LowerTypeTests.
> cpp#1642
> > >> Do you have an example of what you are seeing?
> > >>
> > >> Peter
> > >>
> > >> On Thu, Apr 26, 2018 at 4:54 PM,  <dmitry.mikulin at
sony.com> wrote:
> > >> Hi Peter,
> > >>
> > >> We could probably tolerate a certain amount of unused jump
table
> entries. However, I just realized that all non-inline imported calls end up
> going through a jump table entry. Is that correct? Initially I thought you
> meant calls promoted from indirect. While this can be fixed by replacing
> direct calls to jump tables with direct calls to real targets, I found
> other cases where ThinLTO+CFI has issues.
> > >>
> > >> In ThinLTO backend, type test lowering happens very early in
the
> pipeline, before inlining. When the type check after the call to get_fptr()
> is lowered (in my original example, below), the compiler cannot see that
> both targets belong to the same type and that the type check will always
> return ‘true’ and can be eliminated. Moving the type check lowering pass
> further down the pipeline (after inlining) still does not solve the problem
> because CFI renaming happens early and symbols attached to the jump table
> do not have a matching type.
> > >>
> > >> I’m trying to think if there’s a way to delay renaming until
ThinLTO
> backend type check lowering pass. It would help with solving both problems.
> > >>
> > >> Thanks.
> > >> Dmitry.
> > >>
> > >>
> > >>
> > >>>>>>
> > >>>>>> a.c
> > >>>>>> ============================> >
>>>>>> typedef int (*fptr_t) (void);
> > >>>>>> fptr_t get_fptr();
> > >>>>>> int main(int argc, char *argv[])
> > >>>>>> {
> > >>>>>> fptr_t fp = get_fptr();
> > >>>>>> return fp();
> > >>>>>> }
> > >>>>>>
> > >>>>>>
> > >>>>>> b.c
> > >>>>>> ============================> >
>>>>>> typedef int (*fptr_t) (void);
> > >>>>>> int foo(void) { return 11; }
> > >>>>>> int bar(void) { return 22; }
> > >>>>>>
> > >>>>>> static fptr_t fptr = bar;
> > >>>>>> static int i = 53;
> > >>>>>>
> > >>>>>> fptr_t get_fptr(void)
> > >>>>>> {
> > >>>>>> if (i >= 0)
> > >>>>>>   fptr = foo;
> > >>>>>> else
> > >>>>>>   fptr = bar;
> > >>>>>>
> > >>>>>> return fptr;
> > >>>>>> }
> > >>>>>>
> > >>
> > >>
> > >>
> > >>
> > >>> On Apr 19, 2018, at 6:18 PM, Peter Collingbourne
<peter at pcc.me.uk>
> wrote:
> > >>>
> > >>> Regarding the orderfile, yes, I was thinking more about
ordering the
> real functions.
> > >>>
> > >>> In that case it sounds like your best option may be to
implement the
> optimization pass to make direct calls go directly to the real function.
> From a performance perspective I don't think it would make much
difference
> if there are unused jump table entries.
> > >>>
> > >>> Peter
> > >>>
> > >>> On Thu, Apr 19, 2018 at 6:09 PM, via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> > >>> Teresa, Peter,
> > >>>
> > >>> Thanks for your help!
> > >>> I need to re-run my experiments as the compiler I used
did not have
> the latest changes like r327254.
> > >>> The fact that the decision about routing calls through
jump table
> entries is made early may be problematic. In my experiments with FreeBSD
> kernel, ThinLTO produced thousands jump table entries compared to only
> dozens with full LTO. As for re-ordering jump table entries, I don’t think
> it’s going to work as they are placed in the same section. Including *.cfi
> names into a link order file will take care of re-ordering real functions
> routed through jump table entries, but in our case we need to force some
> functions to be on the same page. So not having jump table entries for the
> functions that don't really need them would be ideal.
> > >>>
> > >>> Thanks.
> > >>> Dmitry.
> > >>>
> > >>>
> > >>>> On Apr 18, 2018, at 6:11 PM, Teresa Johnson
<tejohnson at google.com>
> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wed, Apr 18, 2018 at 4:49 PM,  <dmitry.mikulin
at sony.com> wrote:
> > >>>> Hi Teresa,
> > >>>>
> > >>>> Thanks for the info!
> > >>>> This example is my attempt to reduce FreeBSD kernel
to something
> more manageable :)
> > >>>>
> > >>>> I will take a look at why globals are not being
imported in this
> case. What’s the best tool to look into ThinLTO objects and their
> summaries? Most dumping tools don’t seem to like ThinLTO bitcode files…
> > >>>>
> > >>>> Sadly there isn't a really great way to dump the
summaries. =(
> There was a patch awhile back by a GSOC student to dump in YAML format, but
> there was resistance from some who preferred dumping to llvm assembly via
> llvm-dis and support reading in the summary from llvm assembly. It's
been
> on my list of things to do, hasn't yet risen high enough in priority to
> work on that. For now, you have to use llvm-bcanalyzer -dump and look at
> the raw format.
> > >>>>
> > >>>> Teresa
> > >>>>
> > >>>>
> > >>>> Hopefully Peter can chime in regarding CFI related
issues.
> > >>>>
> > >>>> Thanks.
> > >>>> Dmitry.
> > >>>>
> > >>>>
> > >>>>> On Apr 17, 2018, at 9:37 AM, Teresa Johnson
<tejohnson at google.com>
> wrote:
> > >>>>>
> > >>>>> Hi Dmitry,
> > >>>>>
> > >>>>> Sorry for the late reply. For CFI specific code
generation, pcc is
> a better person to answer. But on the issue of global variables being
> optimized, that hasn't happened yet. That would be great if you wanted
to
> pick that up!
> > >>>>>
> > >>>>> In your original email example, it seems like the
file static i=53
> could be constant propagated since there are no other defs, and the code in
> get_fptr simplified during the compile step, but I assume this is part of a
> more complex example where it is not possible to do this? Also note that
> with r327254 we started importing global variables. Do you know why we
> don't import in your case? I wonder if it has to do with it being CFI
> inserted code?
> > >>>>>
> > >>>>> Teresa
> > >>>>>
> > >>>>> On Tue, Apr 17, 2018 at 9:17 AM
<dmitry.mikulin at sony.com> wrote:
> > >>>>> I watched  Teresa’s talk on ThinLTO from last
year’s CppCon, and
> it sounded like adding global variable information to the summaries was in
> the works, or at least in planning. Can someone (Teresa?) please share the
> current status? If it’s part of future plans, are there any specific
> proposals that can be picked up and worked on?
> > >>>>>
> > >>>>> Thanks!
> > >>>>>
> > >>>>>
> > >>>>>> On Apr 9, 2018, at 6:51 PM, via llvm-dev
<llvm-dev at lists.llvm.org>
> wrote:
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I’m working on setting up ThinLTO+CFI for a C
application which
> uses a lot of function pointers. While functionally it appears stable, it’s
> performance is significantly degraded, to the tune of double digit
> percentage points compared to regular LTO+CFI.
> > >>>>>>
> > >>>>>> Looking into possible causes I see that under
ThinLTO+CFI iCall
> type checks almost always generate jump table entries for indirect calls,
> which creates another level of indirection for every such call. On top of
> that it breaks the link order layout because real function names point to
> jump table entries. It appears that I’m hitting a limitation in ThinLTO on
> how much information it can propagate across modules, particularly
> information about constants. In the example below, the fact that “i” is
> effectively a constant, is lost under ThinLTO, and the inlined copy of
> b.c:get_fptr() in a.c does not eliminate the conditional, which, for CFI
> purposes requires to generate a type check/jump table.
> > >>>>>>
> > >>>>>> I was wondering if there was a way to
mitigate this limitation.
> > >>>>>>
> > >>>>>> a.c
> > >>>>>> ============================> >
>>>>>> typedef int (*fptr_t) (void);
> > >>>>>> fptr_t get_fptr();
> > >>>>>> int main(int argc, char *argv[])
> > >>>>>> {
> > >>>>>> fptr_t fp = get_fptr();
> > >>>>>> return fp();
> > >>>>>> }
> > >>>>>>
> > >>>>>>
> > >>>>>> b.c
> > >>>>>> ============================> >
>>>>>> typedef int (*fptr_t) (void);
> > >>>>>> int foo(void) { return 11; }
> > >>>>>> int bar(void) { return 22; }
> > >>>>>>
> > >>>>>> static fptr_t fptr = bar;
> > >>>>>> static int i = 53;
> > >>>>>>
> > >>>>>> fptr_t get_fptr(void)
> > >>>>>> {
> > >>>>>> if (i >= 0)
> > >>>>>>   fptr = foo;
> > >>>>>> else
> > >>>>>>   fptr = bar;
> > >>>>>>
> > >>>>>> return fptr;
> > >>>>>> }
> > >>>>>>
> > >>>>>>
_______________________________________________
> > >>>>>> LLVM Developers mailing list
> > >>>>>> llvm-dev at lists.llvm.org
> > >>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Teresa Johnson |   Software Engineer |     
tejohnson at google.com
> |   408-460-2413
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Teresa Johnson |    Software Engineer |     
tejohnson at google.com
> |   408-460-2413
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> LLVM Developers mailing list
> > >>> llvm-dev at lists.llvm.org
> > >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> --
> > >>> Peter
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> --
> > >> Peter
> > >
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > llvm-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 01 May 2018 10:46:33 -0700
> From: Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org>
> To: Siddharth Shankar Swain <h2015096 at pilani.bits-pilani.ac.in>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>
> Subject: Re: [llvm-dev] Disabling Exception in LLVM
> Message-ID: <BE0014BE-0434-42B6-B6D1-827F3E84F775 at me.com>
> Content-Type: text/plain; charset="utf-8"
>
> LLVM does not allow the use of exceptions in our code. We do not allow
> throwing or catching them. That does not mean you cannot compile the code
> with exceptions enabled, it just means we don't use them.
>
> Clang is a full C++ compiler. Even though LLVM & Clang do not use
> exceptions in their implementation, Clang does support compiling C++ code
> that uses exceptions.
>
> Does this answer your question?
> -Chris
>
> > On May 1, 2018, at 10:38 AM, Siddharth Shankar Swain <
> h2015096 at pilani.bits-pilani.ac.in> wrote:
> >
> > Hi Chris,
> > Thanks for answering, Can u clarify on this comment mentioned in
> https://github.com/Z3Prover/z3/issues/861 <https://github.com/Z3Prover/
> z3/issues/861> .
> >
> >   cplusplus no exception support · Issue #861 · Z3Prover/z3 · GitHub
> > LLVM's source code does not use exceptions for performance reasons
and
> so is compiled by default with -fno-exceptions. When using LLVM's
libraries
> via it's C++ interface it is important to match how LLVM was built
(i.e. do
> not mix code built with and without exception support).
> > The Clang compiler which is built on top of LLVM is also compiled
> without exceptions by default. However the built Clang can compile C++ code
> with or without exceptions (using the -fno-exceptions). flag. Said another
> way Clang's implementation doesn't use exceptions but Clang itself
can
> compile C++ code with or without exception support.
> >
> > Can anyone clarify on this comment ?
> >
> > Thanks,
> > Siddharth
> >
> > On Tue, May 1, 2018 at 11:02 PM, Chris Bieneman <chris.bieneman at
me.com
> <mailto:chris.bieneman at me.com>> wrote:
> > Siddharth,
> >
> > I'm not sure what coding standards you refer to when you say
"some C++
> coding standard". This question is answered in the LLVM Coding
Standards
> document here:
> >
> > https://www.llvm.org/docs/CodingStandards.html#do-not-
> use-rtti-or-exceptions <https://www.llvm.org/docs/
> CodingStandards.html#do-not-use-rtti-or-exceptions>
> >
> > As such LLVM's coding standards prohibit the use of exceptions and
RTTI.
> >
> > -Chris
> >
> >> On May 1, 2018, at 10:02 AM, Siddharth Shankar Swain via llvm-dev
<
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
> >>
> >> Hi all,
> >> Can anyone explain why exceptions are disabled in LLVM, even if
some
> C++ coding standard tells to use exceptions ?
> >> Thanks,
> >> Siddharth
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> >
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/llvm-dev/
> attachments/20180501/04e8f840/attachment-0001.html>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 01 May 2018 17:48:57 +0000
> From: Teresa Johnson via llvm-dev <llvm-dev at lists.llvm.org>
> To: David Blaikie <dblaikie at gmail.com>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>, Davide Italiano
>         <dccitaliano at gmail.com>, David Li <davidxl at
google.com>
> Subject: Re: [llvm-dev] RFC: LLVM Assembly format for ThinLTO Summary
> Message-ID:
>         <CAAe5K+Vm3cpY+VqNVU_rC6Jtms=4HzMbgJvxraNF=Gh8bn0XUQ at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi David,
> Thanks for the comments, replies below.
> Teresa
>
> On Mon, Apr 30, 2018 at 11:52 AM David Blaikie <dblaikie at
gmail.com> wrote:
>
> > Hi Teresa,
> >
> > Awesome to see - looking forward to it!
> >
> > On Tue, Apr 24, 2018 at 7:44 AM Teresa Johnson via llvm-dev <
> > llvm-dev at lists.llvm.org> wrote:
> >
> >> Hi everyone,
> >>
> >> I started working on a long-standing request to have the summary
dumped
> >> in a readable format to text, and specifically to emit to LLVM
assembly.
> >> Proposal below, please let me know your thoughts.
> >>
> >> Thanks,
> >> Teresa
> >>
> >>
> >>
> >>
> >>
> >>
> >> *RFC: LLVM Assembly format for ThinLTO
> >> Summary========================================Background---
> --------------ThinLTO
> >> operates on small summaries computed during the compile step (i.e.
with
> “-c
> >> -flto=thin”), which are then analyzed and updated during the Thin
Link
> >> stage, and utilized to perform IR updates during the post-link
ThinLTO
> >> backends. The summaries are emitted as LLVM Bitcode, however, not
> currently
> >> in the LLVM assembly.There are two ways to generate a bitcode file
> >> containing summary records for a module: 1. Compile with “clang -c
> >> -flto=thin”*
> >>
> >
> > As an aside - I seem to recall that at least internally at Google some
> > kind of summary-only bitcode files are used (so that the whole bitcode
> file
> > (especially in builds with debug info) doesn't have to be shipped
to the
> > node doing the summary merging). How are those summary-only files
> > produced? Is that upstream? Or done in a more low-level way (like an
> > objcopy, llvm-* tool invocation done as a post-processing step, etc)?
> >
>
> This is done upstream, under a special clang option that can be given in
> addition to -flto=thin, so that the compile step emits both the full
> IR+summary (for the distributed backends) as well as a minimized bitcode
> file with summary (for the thin link). Note that the distributed backends
> don't actually need the summary with the IR (as it gets all the info it
> needs from the combined summary index written out by the thin link), so we
> could theoretically improve this to suppress the summary write for that
> first file under that option.
>
>
> >
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> * 1. Build from LLVM assembly using “opt -module-summary”Either of
these
> >> will result in the ModuleSummaryIndex analysis pass (which builds
the
> >> summary index in memory for a module) to be added to the pipeline
just
> >> before bitcode emission.Additionally, a combined index is created
by
> >> merging all the per-module indexes during the Thin Link, which is
> >> optionally emitted as a bitcode file.Currently, the only way to
view
> these
> >> records is via “llvm-bcanalyzer -dump”, then manually decoding the
raw
> >> bitcode dumps.Relatedly, there is YAML reader/writer support for
CFI
> >> related summary fields (-wholeprogramdevirt-read-summary and
> >> -wholeprogramdevirt-write-summary). Last summer, GSOC student
Charles
> >> Saternos implemented support to dump the summary in YAML from
llvm-lto2
> >> (D34080), including the rest of the summary fields (D34063),
however,
> there
> >> was pushback on the related RFC for dumping via YAML or another
format
> >> rather than emitting as LLVM assembly.Goals: 1. Define LLVM
assembly
> format
> >> for summary index2. Define interaction between parsing of summary
from
> LLVM
> >> assembly and synthesis of new summary index from IR.3. Implement
> printing
> >> and parsing of summary index LLVM assemblyProposed LLVM Assembly
> >> Format----------------------------------------------There are
several
> top
> >> level data structures within the ModuleSummaryIndex: 1.
> >> ModulePathStringTable: Holds the paths to the modules summarized
in the
> >> index (only one entry for per-module indexes and multiple in the
> combined
> >> index), along with their hashes (for incremental builds and global
> >> promotion).2. GlobalValueMap: A map from global value GUIDs to the
> >> corresponding function/variable/alias summary (or summaries for
the
> >> combined index and weak linkage).3. CFI-related data structures
> (TypeIdMap,
> >> CfiFunctionDefs, and CfiFunctionDecls)I have a WIP patch to
> AsmWriter.cpp
> >> to print the ModuleSummaryIndex that I was using to play with the
> format.
> >> It currently prints 1 and 2 above. I’ve left the CFI related
summary
> data
> >> structures as a TODO for now, until the format is at least
conceptually
> >> agreed, but from looking at those I don’t see an issue with using
the
> same
> >> format (with a note/question for Peter on CFI type test
representation
> >> below).I modeled the proposed format on metadata, with a few key
> >> differences noted below. Like metadata, I propose enumerating the
> entries
> >> with the SlotTracker, and prefixing them with a special character.
> Avoiding
> >> characters already used in some fashion (i.e. “!” for metadata and
“#”
> for
> >> attributes), I initially have chosen “^”. Open to suggestions
> >> though.Consider the following example:extern void foo();int X;int
bar()
> {
> >>  foo();  return X;}void barAlias() __attribute__ ((alias
("bar")));int
> >> main() {  barAlias();  return bar();}The proposed format has one
entry
> per
> >> ModulePathStringTable entry and one per GlobalValueMap/GUID, and
looks
> >> like:^0 = module: {path: testA.o, hash: 5487197307045666224}^1 =
gv:
> {guid:
> >> 1881667236089500162, name: X, summaries: {variable: {module: ^0,
flags:
> >> {linkage: common, notEligibleToImport: 0, live: 0, dsoLocal:
1}}}}^2 > gv:
> >> {guid: 6699318081062747564, name: foo}^3 = gv: {guid:
> 15822663052811949562,
> >> name: main, summaries: {function: {module: ^0, flags: {linkage:
extern,
> >> notEligibleToImport: 1, live: 0, dsoLocal: 1}, insts: 5,
funcFlags:
> >> {readNone: 0, readOnly: 0, noRecurse: 0, returnDoesNotAlias: 0},
calls:
> >> {{callee: ^5, hotness: unknown}, {callee: ^4, hotness:
unknown}}}}}^4 > gv:
> >> {guid: 16434608426314478903, name: bar, summaries: {function:
{module:
> ^0,
> >> flags: {linkage: extern, notEligibleToImport: 1, live: 0,
dsoLocal: 1},
> >> insts: 3, funcFlags: {readNone: 0, readOnly: 0, noRecurse: 0,
> >> returnDoesNotAlias: 0}, calls: {{callee: ^2, hotness: unknown}},
refs:
> >> {^1}}}}^5 = gv: {guid: 18040127437030252312, name: barAlias,
summaries:
> >> {alias: {module: ^0, flags: {linkage: extern, notEligibleToImport:
0,
> live:
> >> 0, dsoLocal: 1}, aliasee: ^4}}}*
> >>
> >
> > Syntax seems pretty good to me!
> >
>
> Great!
>
>
> >
> >
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> *Like metadata, the fields are tagged (currently using lower camel
case,
> >> maybe upper camel case would be preferable).The proposed format
has a
> >> structure that reflects the data structures in the summary index.
For
> >> example, consider the entry “^4”. This corresponds to the function
> “bar”.
> >> The entry for that GUID in the GlobalValueMap contains a list of
> summaries.
> >> For per-module summaries such as this, there will be at most one
summary
> >> (with no summary list for an external function like “foo”). In the
> combined
> >> summary there may be multiple, e.g. in the case of linkonce_odr
> functions
> >> which have definitions in multiple modules. The summary list for
bar
> (“^4”)
> >> contains a FunctionSummary, so the summary is tagged “function:”.
The
> >> FunctionSummary contains both a flags structure (inherited from
the base
> >> GlobalValueSummary class), and a funcFlags structure (specific to
> >> FunctionSummary). It therefore contains a brace-enclosed list of
flag
> >> tags/values for each.Where a global value summary references
another
> global
> >> value summary (e.g. via a call list, reference list, or aliasee),
the
> entry
> >> is referenced by its slot. E.g. the alias “barAlias” (“^5”)
references
> its
> >> aliasee “bar” as “^4”.Note that in comparison metadata assembly
entries
> >> tend to be much more decomposed since many metadata fields are
> themselves
> >> metadata (so then entries tend to be shorter with references to
other
> >> metadata nodes).Currently, I am emitting the summary entries at
the end,
> >> after the metadata nodes. Note that the ModuleSummaryIndex is not
> currently
> >> referenced from the Module, and isn’t currently created when
parsing the
> >> Module IR bitcode (there is a separate derived class for reading
the
> >> ModuleSummaryIndex from bitcode). This is because they are not
currently
> >> used at the same time. However, in the future there is no reason
why we
> >> couldn’t tag the global values in the Module’s LLVM assembly with
the
> >> corresponding summary entry if the ModuleSummaryIndex is available
when
> >> printing the Module in the assembly writer. I.e. we could do the
> following
> >> for “main” from the above example when printing the IR definition
(note
> the
> >> “^3” at the end):define  dso_local i32 @main() #0 !dbg !17 ^3 {For
CFI
> data
> >> structures, the format would be similar. It appears that TypeIds
are
> >> referred to by string name in the top level TypeIdMap (std::map
indexed
> by
> >> std::string type identifier), whereas they are referenced by GUID
within
> >> the FunctionSummary class (i.e. the TypeTests vector and the
VFuncId
> >> structure). For the LLVM assembly I think there should be a top
level
> entry
> >> for each TypeIdMap, which lists both the type identifier string
and its
> >> GUID (followed by its associated information stored in the map),
and the
> >> TypeTests/VFuncId references on the FunctionSummary entries can
> reference
> >> it by summary slot number. I.e. something like:^1 = typeid: {guid:
> 12345,
> >> identifier: name_of_type, …^2 = gv: {... {function: {....
typeTests:
> {^1,
> >> …Peter - is that correct and does that sound ok?Issues when
Parsing of
> >> Summaries from
> >> Assembly----------------------------------------------------
> ----------------When
> >> reading an LLVM assembly file containing module summary entries, a
> >> ModuleSummaryIndex will be created from the entries.Things to
consider
> are
> >> the behavior when: - Invoked with “opt -module-summary” (which
currently
> >> builds a new summary index from the IR). Options:*
> >>
> >
> >>
> >> * 1. recompute summary and throw away summary in the assembly
file*
> >>
> >
> > What happens currently if you run `opt -module-summary` on a bitcode
file
> > that already contains a summary? I feel like the behavior should be
the
> > same when run on a textual IR file containing a summary, probably?
> >
>
> We rebuild the summary. Note that this in part is due to the fact mentioned
> above that we have separate readers for the Module IR and the summary. The
> opt tool does not even read the summary if present. We currently only read
> the summary during the thin link (when building the combined index for
> analysis), and in the distributed backends where we read the combined
> summary index file emitted for that file by the distributed thin link.
>
>
> >
> >>
> >>
> >>
> >>
> >>
> >> * 1. ignore -module-summary and build the summary from the LLVM
> >> assembly2. give an error3. compare the two summaries (one created
from
> the
> >> assembly and the new one created by the analysis phase from the
IR), and
> >> error if they are different.My opinion is to do a),  so that the
> behavior
> >> using -module-summary doesn’t change. We also need a way to force
> building
> >> of a fresh module summary for cases where the user has modified
the LLVM
> >> assembly of the IR (see below). - How to handle older LLVM
assembly
> files
> >> that don’t contain new summary fields. Options:*
> >>
> >
> > Same thoughts would apply here for "what do we do in the bitcode
case" -
> > with the option to not support old/difficult textual IR. If there are
> > easy/obvious defaults, I'd say it's probably worth baking
those in (&
> > baking them in even for the existing fields we know about, to make it
> > easier to write more terse test cases that don't have to
> > verbosily/redundantly specify lots of default values?) to the
> > parsing/loading logic?
> >
>
> So we do emit an index version in the bitcode, and auto-upgrade in a
> conservative manner anything that wasn't emitted prior. We could
presumably
> serialize out the version number and handle auto-upgrading from textual
> assembly the same way (as the version is bumped beyond the current version
> at least). If we want to allow omission of some fields for test simplicity,
> we could do a similar thing and apply conservative values where possible
> for omitted fields (e.g. the flags). That seems fine to me, in which case I
> don't think we need a version number. Although this has implications
for
> the validator, see below.
>
>
> >
> >>
> >>
> >>
> >>
> >>
> >> * 1. Force the LLVM assembly file to be recreated with a new
summary.
> >> I.e. “opt -module-summary -o - | llvm-dis”.2. Auto-upgrade, by
silently
> >> creating conservative values for the new summary entries.I lean
towards
> b)
> >> (when possible) for user-friendliness and to reduce required churn
on
> test
> >> inputs. - How to handle partial or incorrect LLVM assembly summary
> entries.
> >> How to handle partial summaries depends in part on how we answer
the
> prior
> >> question about auto-upgrading. I think the best option like there
is to
> >> handle it automatically when possible. However, I do think we
should
> error
> >> on glaring errors like obviously missing information. For example,
when
> >> there is summary data in the LLVM assembly, but summary entries
are
> missing
> >> for some global values. E.g. if the user modified the assembly to
add a
> >> function but forgot to add a corresponding summary entry. We could
still
> >> have subtle issues (e.g. user adds a new call but forgets to
update the
> >> caller’s summary call list), but it will be harder to detect
those.*
> >>
> >
> > I'd be OK with the summary being validated by the IR validator
(same way
> > other properties of IR are validated & even simple things like if
you use
> > the wrong IR type to refer to an IR value, you get a parse error, etc)
-
> > which, I realize, would make it feel like the textual summary was
> entirely
> > redundant
> >
>
> It is redundant when the IR is also available, which relates to Peter and
> others' objections to serializing this back in. An issue with
validation
> would be if we allowed omission of some fields and/or auto-upgrading as
> discussed above. The applied conservative values might very well not match
> the recomputed values. But as I mentioned here we may just want to validate
> for glaring errors like required info - i.e. I think we should require that
> every GV has an associated summary entry.
>
> (except in cases of standalone summaries - which I imagine will be the
> > common case in tests, because the summary processing should be tested
in
> > isolation (except for testing things like this validation logic
itself,
> > etc)).
> >
>
> Yes, I suspect the biggest usage in tests would be a standalone combined
> summary file that we can use to test the application of the thin link
> optimizations on a single IR file in the LTO backend pipeline. I.e the
> input to the test would be one module IR assembly file (no summary) and one
> combined index assembly file, it would run just the ThinLTO backend
> pipeline, and check the resulting IR via llvm-dis to ensure the
> optimization is applied effectively.
>
>
>
> > - Dave
> >
> >
> >>
> >>
> >> --
> >> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
> >> 408-460-2413 <(408)%20460-2413>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
> 408-460-2413
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/llvm-dev/
> attachments/20180501/05e933bd/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> llvm-dev mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> ------------------------------
>
> End of llvm-dev Digest, Vol 167, Issue 3
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180501/fbdf5c24/attachment-0001.html>