On Apr 30, 2012, at 1:03 AM, Renato Golin wrote:> On 29 April 2012 23:44, Bill Wendling <wendling at apple.com> wrote: >> Hi, >> >> Link-Time Optimization has a problem. We need to preserve some of the flags with which the modules were compiled so that the semantics of the resulting program are correct. For example, a module compiled with `-msoft-float' should use library calls for floating point. And that's only the tip of the proverbial iceberg. > > Hi Bill, > > While it's true that knowing compiler flags will help you with linking > problems (including optimisations), I don't think they're 1:1 with > link issues, nor I think storing all compilation options on all > modules every time is a fair price to pay for something that specific. > > You have a goal to correct link-time optimisations, or as we discussed > earlier in the fp-math thread, even code generation could be broken > without the knowledge of the user's intent. That can be accomplished > now by putting "-msoft-float" as a global metadata, yes, but does that > fit a general solution for the general problem? I literally don't > know. >I'm not familiar with the fp-math thread. Could you summarize it for me?> What you need to do, if your intent to create a long-lasting framework > - not just a quick fix for the LTO, is to analyse the biggest problems > and the information you need. If you have problems in multiple domains > (I'm guessing fp is not the only one), and could get information from > multiple sources (again, guessing compile options is not the only > one), then your solution is lacking. >Nothing about the proposal is meant to be a quick fix. The process of adding new flags that we care about would be a formal process, just not one that modifies the IR every time. (Yes, I'm fixated on that one aspect of it. I find it too heavyweight for the problem at hand.)> I'm guessing linker scripts could have a lot to say about link-time > issues, as well as environment, ABI, chipset, ISA and so on. If you > put all compiler flags in metadata now, we'll end up putting all > options of all sources in global metadata, and well, that's far from > desirable. >The information is important for correct code generation and linking. I need alternatives to putting it in metadata. :)> I propose a more general scheme of global metadata, similar to yours > (one global for each big problem, multiple options inside for each > user intent), but generated from cherry-picked sources and put into > specific global metadata baskets (duplication could occur, if the > semantics is different). So each further step reads its own basket > (LTO reads @llvm.metadata.lto {...}) and so on. Of course LTO could > read other baskets, but it'll have to be for a precise reason, with a > precise meaning. > > While merging modules (inlining included) with different metadata, you > have to have a specific well defined merge rule, with warnings and > errors in case they mismatch. We were discussing the merge semantics > for fp models earlier, that kind of analysis should happen for every > new flag you put in. >Yup! The module-level flags has these abilities. :-)> Though you have to take my proposal with a pinch of salt, because > that's remarkably similar to ARM's build attributes, and I'm not sure > that's the best idea either. There is probably a smarter way of doing > this, I just didn't think hard enough to find it... ;) > > But either way, you will need some sort of guidelines on how passes > should treat metadata with stronger guarantees than today, or your LTO > will still not see the info it needs... >Could you give an example of how yours would look like in a sample Module? -bw
On 1 May 2012 00:10, Bill Wendling <wendling at apple.com> wrote:> I'm not familiar with the fp-math thread. Could you summarize it for me?Too many issues to summarize: http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-April/048951.html The important bit to this discussion is that metadata should be used for correctness and we shouldn't disregard it the way we do today.> The information is important for correct code generation and linking. I need alternatives to putting it in metadata. :)What you need is to present the information to the code generation / linking phases in a concise, de-duplicated and semantically correct way. Just dropping compilation flags on the IR will give you the information, yes, but their meaning could be inaccurate. Compiling for generic platforms than linking for specific CPUs could lead to confusion, or slight changes in compilation options semantics could make a very subtle corner case blow up (silently) in the linker.> Could you give an example of how yours would look like in a sample Module?Something similar to yours: !llvm.module.flags.lto = !{ !0, !1 } !llvm.module.flags.cg = !{ !0 } !0 = metadata !{ i32 1, metadata !"soft-float", i1 true } // forcing soft float all the way through !1 = metadata !{ i32 1, metadata !"VFPv2", i1 false } // The user do not want VFPv2 at all costs My proposal is more than just what it looks like in the IR, I don't really care that much about that. My points: 1. Dropping general info into metadata is not enough. You need to make sure it'll be semantically correct for the majority of cases, 2. There will be more than one producer (compilation options, linker options, ABIs, ISAs) and more than one consumer (lto, cg, etc), you need to common up as much as possible, and split when the semantics differ, 3. You need rules for merging metadata, and that's not just a requirement of this topic, but also raised in the FP discussion, 4. Metadata must have its status raised in IR, and passes should be aware of it and manipulate it correctly. Points 3 and 4 go against the original design of Metadata, I know. But it's being used by debug information and correctness (ARC, Obj-C). We're about to take the leap towards FP correctness with metadata, I don't think you can still disregard Metadata as quickly as the current back-end does. Metadata semantics is much simpler than IR semantics. Most of the problems arise from merging code, so if there is a clear identification of the types of metadata (debug, asm, fp) and there is a simple module with simple, clear rules of merging, you can just apply those rules, via the same module. So passes don't need to learn too much, just blindly applying the general rules will get you far enough. Merging metadata is (mostly) a matter of dominance and equivalence. When inlining a function into a different module, for instance, you'll check for the debug metadata in the former for equivalent (you define what's equivalent for each case) metadata and move the pointers. So, for "int a", you will still point "a" to the type "int" in the new module. In the FP case, you merge according to dominance: merging a less strict model into a more strict can either increase the strength of the former, or leave it be in a per-instruction model. That also depends on the architecture, and possibly, compiler flags. My final point is simple: if you don't think about those issues now, before start filling IR with unchecked metadata, it'll be harder to enforce the required merge rules later. And they will be necessary. -- cheers, --renato http://systemcall.org/
On May 1, 2012, at 1:09 AM, Renato Golin wrote:>> Could you give an example of how yours would look like in a sample Module? > > Something similar to yours: > > !llvm.module.flags.lto = !{ !0, !1 } > !llvm.module.flags.cg = !{ !0 } > > !0 = metadata !{ i32 1, metadata !"soft-float", i1 true } // forcing > soft float all the way through > !1 = metadata !{ i32 1, metadata !"VFPv2", i1 false } // The user do > not want VFPv2 at all costs > > > My proposal is more than just what it looks like in the IR, I don't > really care that much about that. > > My points: > 1. Dropping general info into metadata is not enough. You need to > make sure it'll be semantically correct for the majority of cases,I don't know why you think I'm suggesting otherwise or ignoring this.> 2. There will be more than one producer (compilation options, linker > options, ABIs, ISAs) and more than one consumer (lto, cg, etc), you > need to common up as much as possible, and split when the semantics > differ, > 3. You need rules for merging metadata, and that's not just a > requirement of this topic, but also raised in the FP discussion, > 4. Metadata must have its status raised in IR, and passes should be > aware of it and manipulate it correctly. > > Points 3 and 4 go against the original design of Metadata, I know. But > it's being used by debug information and correctness (ARC, Obj-C). > We're about to take the leap towards FP correctness with metadata, I > don't think you can still disregard Metadata as quickly as the current > back-end does. > > Metadata semantics is much simpler than IR semantics. Most of the > problems arise from merging code, so if there is a clear > identification of the types of metadata (debug, asm, fp) and there is > a simple module with simple, clear rules of merging, you can just > apply those rules, via the same module. So passes don't need to learn > too much, just blindly applying the general rules will get you far > enough. > > Merging metadata is (mostly) a matter of dominance and equivalence. > When inlining a function into a different module, for instance, you'll > check for the debug metadata in the former for equivalent (you define > what's equivalent for each case) metadata and move the pointers. So, > for "int a", you will still point "a" to the type "int" in the new > module. In the FP case, you merge according to dominance: merging a > less strict model into a more strict can either increase the strength > of the former, or leave it be in a per-instruction model. That also > depends on the architecture, and possibly, compiler flags. > > My final point is simple: if you don't think about those issues now, > before start filling IR with unchecked metadata, it'll be harder to > enforce the required merge rules later. And they will be necessary. >Okay. For some reason you think I haven't thought of these issues. Please don't make that assumption. I proposed a simple framework for conveying the information that needs to be conveyed. I omitted the types of information that's to be conveyed (except for a couple of examples) because that's a very different discussion than this, and requires much more detail (as you mentioned above). I'm concerned with figuring out how to do it at this point in time. Each flag that's passed down will need to be treated in its own special manner. -bw
Apparently Analagous Threads
- [LLVMdev] [RFC] Encoding Compile Flags into the IR
- [LLVMdev] [RFC] Encoding Compile Flags into the IR
- [LLVMdev] [RFC] Encoding Compile Flags into the IR
- [LLVMdev] [cfe-dev] [RFC] Encoding Compile Flags into the IR
- [LLVMdev] [cfe-dev] [RFC] Encoding Compile Flags into the IR