llvm dev - May 2012 - [LLVMdev] [RFC] Encoding Compile Flags into the IR

On Apr 30, 2012, at 1:03 AM, Renato Golin wrote:
> On 29 April 2012 23:44, Bill Wendling <wendling at apple.com> wrote:
>> Hi,
>> 
>> Link-Time Optimization has a problem. We need to preserve some of the
flags with which the modules were compiled so that the semantics of the
resulting program are correct. For example, a module compiled with
`-msoft-float' should use library calls for floating point. And that's
only the tip of the proverbial iceberg.
> 
> Hi Bill,
> 
> While it's true that knowing compiler flags will help you with linking
> problems (including optimisations), I don't think they're 1:1 with
> link issues, nor I think storing all compilation options on all
> modules every time is a fair price to pay for something that specific.
> 
> You have a goal to correct link-time optimisations, or as we discussed
> earlier in the fp-math thread, even code generation could be broken
> without the knowledge of the user's intent. That can be accomplished
> now by putting "-msoft-float" as a global metadata, yes, but does
that
> fit a general solution for the general problem? I literally don't
> know.
> 
I'm not familiar with the fp-math thread. Could you summarize it for me?
> What you need to do, if your intent to create a long-lasting framework
> - not just a quick fix for the LTO, is to analyse the biggest problems
> and the information you need. If you have problems in multiple domains
> (I'm guessing fp is not the only one), and could get information from
> multiple sources (again, guessing compile options is not the only
> one), then your solution is lacking.
> 
Nothing about the proposal is meant to be a quick fix. The process of adding new
flags that we care about would be a formal process, just not one that modifies
the IR every time. (Yes, I'm fixated on that one aspect of it. I find it too
heavyweight for the problem at hand.)
> I'm guessing linker scripts could have a lot to say about link-time
> issues, as well as environment, ABI, chipset, ISA and so on. If you
> put all compiler flags in metadata now, we'll end up putting all
> options of all sources in global metadata, and well, that's far from
> desirable.
> 
The information is important for correct code generation and linking. I need
alternatives to putting it in metadata. :)
> I propose a more general scheme of global metadata, similar to yours
> (one global for each big problem, multiple options inside for each
> user intent), but generated from cherry-picked sources and put into
> specific global metadata baskets (duplication could occur, if the
> semantics is different). So each further step reads its own basket
> (LTO reads @llvm.metadata.lto {...}) and so on. Of course LTO could
> read other baskets, but it'll have to be for a precise reason, with a
> precise meaning.
> 
> While merging modules (inlining included) with different metadata, you
> have to have a specific well defined merge rule, with warnings and
> errors in case they mismatch. We were discussing the merge semantics
> for fp models earlier, that kind of analysis should happen for every
> new flag you put in.
> 
Yup! The module-level flags has these abilities. :-)
> Though you have to take my proposal with a pinch of salt, because
> that's remarkably similar to ARM's build attributes, and I'm
not sure
> that's the best idea either. There is probably a smarter way of doing
> this, I just didn't think hard enough to find it... ;)
> 
> But either way, you will need some sort of guidelines on how passes
> should treat metadata with stronger guarantees than today, or your LTO
> will still not see the info it needs...
> 
Could you give an example of how yours would look like in a sample Module?

-bw

On 1 May 2012 00:10, Bill Wendling <wendling at apple.com>
wrote:
> I'm not familiar with the fp-math thread. Could you summarize it for
me?
Too many issues to summarize:

http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-April/048951.html

The important bit to this discussion is that metadata should be used
for correctness and we shouldn't disregard it the way we do today.

> The information is important for correct code generation and linking. I
need alternatives to putting it in metadata. :)
What you need is to present the information to the code generation /
linking phases in a concise, de-duplicated and semantically correct
way.

Just dropping compilation flags on the IR will give you the
information, yes, but their meaning could be inaccurate. Compiling for
generic platforms than linking for specific CPUs could lead to
confusion, or slight changes in compilation options semantics could
make a very subtle corner case blow up (silently) in the linker.

> Could you give an example of how yours would look like in a sample Module?
Something similar to yours:

!llvm.module.flags.lto = !{ !0, !1 }
!llvm.module.flags.cg = !{ !0 }

!0 = metadata !{ i32 1, metadata !"soft-float", i1 true } // forcing
soft float all the way through
!1 = metadata !{ i32 1, metadata !"VFPv2", i1 false } // The user do
not want VFPv2 at all costs


My proposal is more than just what it looks like in the IR, I don't
really care that much about that.

My points:
 1. Dropping general info into metadata is not enough. You need to
make sure it'll be semantically correct for the majority of cases,
 2. There will be more than one producer (compilation options, linker
options, ABIs, ISAs) and more than one consumer (lto, cg, etc), you
need to common up as much as possible, and split when the semantics
differ,
 3. You need rules for merging metadata, and that's not just a
requirement of this topic, but also raised in the FP discussion,
 4. Metadata must have its status raised in IR, and passes should be
aware of it and manipulate it correctly.

Points 3 and 4 go against the original design of Metadata, I know. But
it's being used by debug information and correctness (ARC, Obj-C).
We're about to take the leap towards FP correctness with metadata, I
don't think you can still disregard Metadata as quickly as the current
back-end does.

Metadata semantics is much simpler than IR semantics. Most of the
problems arise from merging code, so if there is a clear
identification of the types of metadata (debug, asm, fp) and there is
a simple module with simple, clear rules of merging, you can just
apply those rules, via the same module. So passes don't need to learn
too much, just blindly applying the general rules will get you far
enough.

Merging metadata is (mostly) a matter of dominance and equivalence.
When inlining a function into a different module, for instance, you'll
check for the debug metadata in the former for equivalent (you define
what's equivalent for each case) metadata and move the pointers. So,
for "int a", you will still point "a" to the type
"int" in the new
module. In the FP case, you merge according to dominance: merging a
less strict model into a more strict can either increase the strength
of the former, or leave it be in a per-instruction model. That also
depends on the architecture, and possibly, compiler flags.

My final point is simple: if you don't think about those issues now,
before start filling IR with unchecked metadata, it'll be harder to
enforce the required merge rules later. And they will be necessary.

-- 
cheers,
--renato

http://systemcall.org/

On May 1, 2012, at 1:09 AM, Renato Golin wrote:
>> Could you give an example of how yours would look like in a sample
Module?
> 
> Something similar to yours:
> 
> !llvm.module.flags.lto = !{ !0, !1 }
> !llvm.module.flags.cg = !{ !0 }
> 
> !0 = metadata !{ i32 1, metadata !"soft-float", i1 true } //
forcing
> soft float all the way through
> !1 = metadata !{ i32 1, metadata !"VFPv2", i1 false } // The user
do
> not want VFPv2 at all costs
> 
> 
> My proposal is more than just what it looks like in the IR, I don't
> really care that much about that.
> 
> My points:
> 1. Dropping general info into metadata is not enough. You need to
> make sure it'll be semantically correct for the majority of cases,
I don't know why you think I'm suggesting otherwise or ignoring this.
> 2. There will be more than one producer (compilation options, linker
> options, ABIs, ISAs) and more than one consumer (lto, cg, etc), you
> need to common up as much as possible, and split when the semantics
> differ,
> 3. You need rules for merging metadata, and that's not just a
> requirement of this topic, but also raised in the FP discussion,
> 4. Metadata must have its status raised in IR, and passes should be
> aware of it and manipulate it correctly.
> 
> Points 3 and 4 go against the original design of Metadata, I know. But
> it's being used by debug information and correctness (ARC, Obj-C).
> We're about to take the leap towards FP correctness with metadata, I
> don't think you can still disregard Metadata as quickly as the current
> back-end does.
> 
> Metadata semantics is much simpler than IR semantics. Most of the
> problems arise from merging code, so if there is a clear
> identification of the types of metadata (debug, asm, fp) and there is
> a simple module with simple, clear rules of merging, you can just
> apply those rules, via the same module. So passes don't need to learn
> too much, just blindly applying the general rules will get you far
> enough.
> 
> Merging metadata is (mostly) a matter of dominance and equivalence.
> When inlining a function into a different module, for instance, you'll
> check for the debug metadata in the former for equivalent (you define
> what's equivalent for each case) metadata and move the pointers. So,
> for "int a", you will still point "a" to the type
"int" in the new
> module. In the FP case, you merge according to dominance: merging a
> less strict model into a more strict can either increase the strength
> of the former, or leave it be in a per-instruction model. That also
> depends on the architecture, and possibly, compiler flags.
> 
> My final point is simple: if you don't think about those issues now,
> before start filling IR with unchecked metadata, it'll be harder to
> enforce the required merge rules later. And they will be necessary.
> 
Okay. For some reason you think I haven't thought of these issues. Please
don't make that assumption. I proposed a simple framework for conveying the
information that needs to be conveyed. I omitted the types of information
that's to be conveyed (except for a couple of examples) because that's a
very different discussion than this, and requires much more detail (as you
mentioned above). I'm concerned with figuring out how to do it at this point
in time. Each flag that's passed down will need to be treated in its own
special manner.

-bw