On 16 September 2015 at 21:56, Jim Grosbach <grosbach at apple.com> wrote:> Why do we care about GAS? We have an assembler.It's not that simple. There are a lot of old code out there, including the Linux kernel which we do care a lot, that only compiles with GAS. We're slowly moving the legacy code up to modern standards, and specifically some kernel folks are happy to move up not only the asm syntax, but the C standard and move away from GNU-specific behaviour. But we're not quite there yet, and might not be for a few more years. so, yes, we still care about GAS. But this is not just about GAS. As I said on my previous email, this is about clearing the bloat in target descriptions by both: removing the need for adding numerous CPU names, target features, architecture names (xscale, strongarm, etc), AND making sure all parties (front/middle/back-ends) speak the same language, produced from the same source. The TargetTuple is that common language, and the TargetParser created from the TableGen files is the common source. The Triple becomes a legacy constructor value for the Tuple. All other target information classes are already (or should be) generated from the TableGen files, so the ultimate source becomes the TableGen description, which I think it what you were aiming to on your comment. For simple architectures, like x86, you don't even need a TargetParser. You can easily construct the Tuple from a triple and use the Tuple as you've always used the triple. No harm done. But for the complex ones like ARM and MIPS, having a common interface generated from the same place the other interfaces are is important to avoid more bridges between front and middle and back end interpretations of the same target. Whatever legacy ARM or MIPS carry can be isolated in their own implementation, leaving the rest of the targets with a clean and simple interface. cheers, --renato
Let's take a step back here. It appears that you and Daniel are trying to solve some problems. I think solving problems is good, I just want to make sure that we're solving them in a way that gets us a decent API at the end. I also want to make sure we're solving the right problems. TargetTuple appears to be related to the TargetParser as you bring up in this mail. They're two separate parts of similar problems - people trying to both serialize command line options and communication from the front end to the backend with respect to target information. This leads me to a question: What can't be done to TargetMachine to avoid this serialization? And a followup question: What can't be serialized at the function level in the IR to make certain things clear that aren't global? We already do this for a lot of command line options. And one more: What global options do we need to consider here? The goal of the configuration level of the TargetMachine is that it controls things that don't change at the object level. This is a fairly recently stated goal, but I think it makes sense for LLVM in general. TargetSubtargetInfo takes care of everything that resides under this (as much as possible, some bits are still in transition, e.g. TargetOptions). This is part of my suggestion to Daniel about the problems with MCSubtargetInfo and the assembler. Targets like Mips and ARM were unfortunately designed to change things on the fly during assembly and need to collate or at least change defaults as we're processing code. I definitely had to deal with a lot of the pain you're talking about when I was rewriting some of the handling there during the TargetSubtargetInfo work. Now a bit more on TargetParser + TargetTuple: TargetParser appears to be trying to solve the parsing in Triple in a nice way for ARM and also some of the "what kind of subtarget feature canonicalization can we do in llvm that makes sense to communicate to the front end". I like this particular idea and have often wanted a library of feature handling, but it seems to have stabilized at an ARM specific set of code with no defined interface. I can't even figure out how I'd use it in lib/Basic right now for any target other than ARM. This isn't a condemnation of TargetParser, but I think it's something that needs to be thought through a bit more. It's been hooked up well before I'd expected it to and right now if we moved it to the ARM backend from Support it'd make just as much sense as it does where it is now other than making clang depend on the ARM backend as well as the X86 backend :) Right now I see TargetTuple as trying to take over all of the various arguments to TargetMachine and encapsulate them into a single thing. I also don't see this is bad, but I also don't see it taking all of them right now and I'm not sure how it solves some of the existing problems with data sharing that we've got which is where the push back you're both getting is coming from here. Ultimately library-wise I can agree with some of the directions you're headed - I just don't see the unification and interactions right now. As a suggestion as a way forward here let's see if we can get my questions above answered and also show some of how the interactions between llvm's libraries are going to get fixed, moved to a better place, etc here. Thanks! -eric On Wed, Sep 16, 2015 at 3:02 PM Renato Golin <renato.golin at linaro.org> wrote:> On 16 September 2015 at 21:56, Jim Grosbach <grosbach at apple.com> wrote: > > Why do we care about GAS? We have an assembler. > > It's not that simple. > > There are a lot of old code out there, including the Linux kernel > which we do care a lot, that only compiles with GAS. We're slowly > moving the legacy code up to modern standards, and specifically some > kernel folks are happy to move up not only the asm syntax, but the C > standard and move away from GNU-specific behaviour. But we're not > quite there yet, and might not be for a few more years. so, yes, we > still care about GAS. > > But this is not just about GAS. > > As I said on my previous email, this is about clearing the bloat in > target descriptions by both: removing the need for adding numerous CPU > names, target features, architecture names (xscale, strongarm, etc), > AND making sure all parties (front/middle/back-ends) speak the same > language, produced from the same source. > > The TargetTuple is that common language, and the TargetParser created > from the TableGen files is the common source. The Triple becomes a > legacy constructor value for the Tuple. All other target information > classes are already (or should be) generated from the TableGen files, > so the ultimate source becomes the TableGen description, which I think > it what you were aiming to on your comment. > > For simple architectures, like x86, you don't even need a > TargetParser. You can easily construct the Tuple from a triple and use > the Tuple as you've always used the triple. No harm done. But for the > complex ones like ARM and MIPS, having a common interface generated > from the same place the other interfaces are is important to avoid > more bridges between front and middle and back end interpretations of > the same target. Whatever legacy ARM or MIPS carry can be isolated in > their own implementation, leaving the rest of the targets with a clean > and simple interface. > > cheers, > --renato >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150916/736f6463/attachment.html>
On 16 September 2015 at 23:52, Eric Christopher <echristo at gmail.com> wrote:> TargetTuple appears to be related to the TargetParser as you bring up in > this mail. They're two separate parts of similar problems - people trying to > both serialize command line options and communication from the front end to > the backend with respect to target information.Yes, though the relation between TargetTuple and TargetParser is lighter than I made seem. Tuple doesn't need the parser, but for complicated architectures, the parser simplifies the Tuple considerably. For now, this is just ARM. MIPS problems seem to be more related to how the values relate to each other, not how the string is parsed, so it's quite likely that the TargetParser is *only* relevant to ARM. But for ARM, it is crucial. See below.> This leads me to a question: What can't be done to TargetMachine to avoid > this serialization?The misconception here is that we *want* to do the serialization, or that was the design we came up with to tackle complexity. In the ARM world, there are *two* big architectural problems: 1. ARM licenses every part of the chip and allows licencees to mix&match whatever they want. This is key to ARM's success, but it's a nightmare for tools, that have to cope with the combination of virtually every option possible. This complexity can (and should) be tackled at the TargetMachine level, generated from ARM.td, with the options that we do recognise and can make a difference. 2. Legacy in how to describe a target for decades. Target triples and compiler options have been abused for decades to tackle the complexity above. But since they were done one at a time, no one thought it was a good idea, or even remotely relevant to try and tackle the all combinations in a non-exponential way. If we use a target config file, or a target database, and construct the TargetMachine (& al) from that, there is no need at all for parsing strings. But for all other legacy ways to describe a targets, namely triples and compiler options, we do. Things like Krait as a different CPU+extensions is one example, but others like xscale, or iwmmxt, which can be arch name, cpu name or extension name. The only "complex" parsing is to reduce the arch name into sub-arch name+ISA (armv7a -> arm + v7a), which also has to deal with big-endian, etc. Everything else needs to be parser exactly as defined by the legacy options. But all this parsing should be done *once*, then filling TargetMachine or TargetTuple, then using that. Today it doesn't because we're in the process of making it. The status is... Before: every function would parse the same strings themselves via StringSwitch and complex rules. During: All complex rules are moved into one place, but all users still call them all the time. This makes it look stupid, but it's not by design. After: All users stop calling the parsing routines and call the Tuple/Machine classes to infer about the target. Parsing is only done once at startup. The second step is not worse than "before", and it helps show where the inefficiencies were. Once we know all the users of the triple parsing, we can reduce all to enums and switches, and make it almost zero-cost. But for that, we need the TargetTuple or some other mechanism (that is *not* the Triple) to hold unambiguous information about the target.> And a followup question: What can't be serialized at the function level in > the IR to make certain things clear that aren't global? We already do this > for a lot of command line options.These options will change the triple in the IR. Examples: $ clang -target arm-linux-gnueabi -emit-llvm -S foo.c -o - target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64" target triple = "armv4t--linux-gnueabi" $ clang -target arm-linux-gnueabihf -emit-llvm -S foo.c -o - target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64" target triple = "armv6k--linux-gnueabihf" $ clang -target arm-linux-gnueabi -mcpu=cortex-a8 -emit-llvm -S foo.c -o - target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64" target triple = "armv7--linux-gnueabi" $ clang -target armv7-linux-gnueabi -mcpu=arm7tdmi -emit-llvm -S foo.c -o - target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64" target triple = "armv4t--linux-gnueabi" This has the benefit of reducing the number of flags we need in IR, but also has the problem that the mapping between front-end and back-end target description is not 1-to-1, and the transformation from options to triple to target machine is not always injective. Things work today because we test them and "fix" the known issues by adding more string parsing or more enum values to cope with the legacy. There is no design so far, and that's what we're trying to change.> And one more: What global options do we need to consider here?I'm not sure what you mean, here.> Targets like Mips and ARM were > unfortunately designed to change things on the fly during assembly and need > to collate or at least change defaults as we're processing code.Yes. That's the unfortunate example of the .fpu / .arch_extension / .arch / .cpu tags in asm code.> but it seems to have stabilized at an ARM specific set of > code with no defined interface. I can't even figure out how I'd use it in > lib/Basic right now for any target other than ARM.This is a fair assessment, and a result that only ARM folks have worked on it. But to be honest, not many people outside of the ARM community were interested in it, so I took the decision to let it run its course, and tighten the interface later. That's why Chandler's changes to the parser were so intrusive, because he solidified our current state, that was far from any design we would have wanted. Now, it's going to be harder to refactor into a proper interface, but not impossible. But the fact that this is ARM only makes it really hard for us to identify what is common and what is not. When I proposed the parser earlier this year, I expected others to move their targets to it. I would have done myself if this wasn't such a target-specific sensitive thing. I don't want to assume I know about other architectures' problems more than their own engineers, and I don't like when people assume that they know enough about ours to completely refactor our code.> This isn't a condemnation > of TargetParser, but I think it's something that needs to be thought through > a bit more.Our plan was simple: 1. Move all parsing to a single location, share between front and back end. 2. See what's common between targets, keep that into a lightweight dynamic class structure, where {ARCH}TargetParser derives from TargetParser. 3. Makes TargetParser a part of TargetTuple/Machine/Whatever using LLVM's form of RTTI to pick the right one depending on the arch. 4. Keep all the internal complexity in a target specific table-generated file, which has all the strings to be parsed. 5. Move all the users of the parser to enquire on the Tuple, thus using enum/switch instead of string parsing We got as far as finish the first task. Task 2 was (understandably) ignored by everyone else, so invalidating task 3 in the process. Task 4 would be our next step. Task 5 need some form of unambiguous representation, which Daniel was tackling and I let him to it. But finishing our 4th task would still mean that we're parsing a lot of strings unnecessarily, and we have only achieved one goal: to reduce code duplication.>From the parser's point of view, it makes no difference which classholds the target representation, as long as I can change *all* parsing into enum-switch lookups. So, while I sympathised with Daniel's dilemma, I don't have such a strong position on either merits.> Right now I see TargetTuple as trying to take over all of the various > arguments to TargetMachine and encapsulate them into a single thing. I also > don't see this is bad, but I also don't see it taking all of them right now > and I'm not sure how it solves some of the existing problems with data > sharing that we've got which is where the push back you're both getting is > coming from here.>From my understanding, there are two separate worlds: the precise andunambiguous world of target descriptions, and the chaotic world of command line options. Command line options are often ambiguous, they interact with each other and they can change according to the host-target relationship. Parsing all this knowledge and disambiguating it is not a simple task. It requires a lot of code, a lot of parsing, a lot of switches and mappings between enums. But once this is done, a low-level unique representation can be achieved. On the other hand, the class that is tasked to hold all the information of all targets in an orderly and unique fashion has to be precise and correct above all else. There's no space for doubt or guess work in such a class, or the compiler will start taking random decisions and it'd be a nightmare to understand it. So it is clear to me that these two concepts should never cross wires. In the same way that the Tuple shouldn't be parsing target-specific stuff on its own, I don't think that a precise machine representation should be cross-referencing user options to define its properties. I see a clear succession of ownership: TargetMachine( TargetTuple( TargetParser( Triple, arch options ), host info, target defaults, other options ), other target machine parameters ... ) But that's just my uninformed opinion. I'll let you and Daniel to discuss the merits of each other. cheers, --renato
I think we need to take a step further back and re-enter from the right starting point. The thing that's bothering me about the push back so far is that it's trying to discuss and understand the consequences of resolving the core problem while seemingly ignoring the core problem itself. The reason I've been steering everything back to GNU Triple's being ambiguous and inconsistent is because it's the root of all the problems and the fixes to the various issues fall out naturally once this core point has been addressed. Here's the line of thought that I'd like people to start with: · Triples don't describe the target. They look like they should, but they don't. They're really just arbitrary strings. · LLVM relies on Triple as a description of the target. It defines the backend to use, the binary format to use, OS and Vendor specific quirks to enable/disable, the default CPU, the default ABI, the endian, and countless other details about the target. · If LLVM is built on top of an incorrect concept we should fix that but we can't abandon Triple's at the user level since every toolchain uses them. · But we can't keep using Triples inappropriately either. If the information feeding into LLVM is faulty then the resulting behaviour will be faulty too. · So let's start with a Triple, and convert it to a not-broken equivalent as early as possible. We'll call it TargetTuple. Are there any disagreements on this part of the thinking? If there are, then we should resolve these before proceeding to the rest since everything else depends on accepting this core problem exists and can be fixed in this way. If we have agreement on this, then I think that this by itself is ample reason for phases 1-4, and 6 of the plan. The justification for the IR serialization in phase 5 is simply that we need to deliver the Triple/TargetTuple to LTO for it to operate correctly and we currently do this by serializing Triple in the IR. If Triple has been replaced by TargetTuple then TargetTuple must be serializable in the IR somehow. Hopefully, we are agreed so far. Let's assume for the rest of this explanation that Phases 1-6 are complete and we now have const TargetTuple throughout the API. I'd like to draw particular attention to TargetMachine which, like everything else, has had its Triple member (called TargetTriple) replaced with a TargetTuple member (named TheTargetTuple). This member is used in all the same ways it used to be used when it was a Triple (named TargetTriple). At this point, in the MC layer we have a number of classes that need to know the ABI but lack this information. Our TargetMachine has an accurate TargetTuple object that describes the invariants of the desired target. The desired ABI is an invariant too so why not have it in the TargetTuple which is already plumbed in everywhere we need it? After all, it's a property of the target OS/Environment. If we have the ABI in the TargetTuple, then we don't need any other means to set the ABI, tools can set it up front in the TargetTuple and we don't need any command-line option handling for it in the backend. Meanwhile, in clang we have a number of command line options that change the desired target. Let's say we've constructed a Triple and resolved it to TargetTuple (more on that below). We're now processing the –EL option. At the moment, we substitute our mips-linux-gnu triple for a mipsel-linux-gnu triple, construct a Triple object from it and resolve the new Triple to a TargetTuple. But why do we need to bother with that kind of weird hackery when we can simply do Obj.setEndian(Little)? This is what Phase 7 of the plan is about. We end up with a cleaner way to process target changes that, until now, have required weird triple hacking to handle. I skipped the Triple -> TargetTuple resolution a moment ago and I should address that now. We already know that mapping Triple to TargetTuple is a many to many mapping. One Triple has many possible TargetTuple's depending on the environment. One TargetTuple can be formed from multiple possible Triples. In an ideal world, we'd like to bake in all of these mappings so that one clang binary supports everything. Unfortunately, being a many to many mapping, some of these mappings are mutually exclusive. Note that this isn't a new problem resulting from this project. The problem has always been there but has been ignored until now. To resolve this, we need to provide configure-time and possibly run-time controls for how this conversion is disambiguated. This resolution is performed as early as possible so that the middle/back-ends don't need to know anything about the ambiguity problem. --- To reply more directly to your email:> What can't be done to TargetMachine to avoid this serialization?TargetMachine already has the serialization (see TargetMachine::TargetTriple). We're not doing anything new here. We're simply replacing one object holding faulty information with a new object holding reliable information.> And a followup question: What can't be serialized at the function level in the IR to make certain things clear that aren't global? We already do this for a lot of command line options.The data I want to fix is global. I think the bit you may be getting hung up on here is that small portions of this global data can also be overridden at the function level. Those overrides aren't a problem and continue to operate in the same way as they do today.> And one more: What global options do we need to consider here?I'm not certain I understand this question. If you're talking command line options, it's things like –EL, -EB, -mips32, -mips32r[2356], -mips64, -mips64r[2356], -mabi=…. If you're talking about Triple -> TargetTuple mappings, there's quite a wide variety but the main ones for Mips are endian, architecture, default CPU, and default ABI.> The goal of the configuration level of the TargetMachine is that it controls things that don't change at the object level. > This is a fairly recently stated goal, but I think it makes sense for LLVM in general. TargetSubtargetInfo takes care of > everything that resides under this (as much as possible, some bits are still in transition, e.g. TargetOptions). This is part > of my suggestion to Daniel about the problems with MCSubtargetInfo and the assembler. Targets like Mips and ARM > were unfortunately designed to change things on the fly during assembly and need to collate or at least change defaults > as we're processing code. I definitely had to deal with a lot of the pain you're talking about when I was rewriting some > of the handling there during the TargetSubtargetInfo work.I generally agree with this. The key bit I need to draw attention to is that the 'defaults' don't change, but are instead overridden. These constant defaults are stored in TargetMachine and particularly TargetMachine::TargetTriple. These defaults are wrong for some toolchains since the information stored in TargetMachine::TargetTriple are wrong. It's the defaults I'm trying to fix rather than the overrides. I think I understand your proposed plan now and it's a few steps ahead of where we are and where we need to be. I agree that overridable state should be in TargetSubtargetInfo, however I can't initialize that state without the default values which come from the faulty information in TargetMachine::TargetTriple. This triple work is a pre-requisite to your plan and at first I don't need to override ABI's.> Right now I see TargetTuple as trying to take over all of the various arguments to TargetMachine and encapsulate them into a single thing. > I also don't see this is bad, but I also don't see it taking all of them right now and I'm not sure how it solves some of the existing problems > with data sharing that we've got which is where the push back you're both getting is coming from here. Ultimately library-wise I can agree > with some of the directions you're headed - I just don't see the unification and interactions right now.I think we'll end up with TargetTuple taking over many arguments to TargetMachine but that's not my goal at this stage. My goal is simply to fix the faulty information currently held in Triple and use the now-accurate information in TargetTuple to fix various blocking issues that prevent a proper Mips toolchain product based on Clang/LLVM. At the end of Phase 7, it become possible to fix a number of issues that are impossible to fix right now because the available data we can consult at the moment is incorrect. From: Eric Christopher [mailto:echristo at gmail.com] Sent: 16 September 2015 23:52 To: Renato Golin; Jim Grosbach Cc: Daniel Sanders; llvm-dev at lists.llvm.org Subject: Re: The Trouble with Triples Let's take a step back here. It appears that you and Daniel are trying to solve some problems. I think solving problems is good, I just want to make sure that we're solving them in a way that gets us a decent API at the end. I also want to make sure we're solving the right problems. TargetTuple appears to be related to the TargetParser as you bring up in this mail. They're two separate parts of similar problems - people trying to both serialize command line options and communication from the front end to the backend with respect to target information. This leads me to a question: What can't be done to TargetMachine to avoid this serialization? And a followup question: What can't be serialized at the function level in the IR to make certain things clear that aren't global? We already do this for a lot of command line options. And one more: What global options do we need to consider here? The goal of the configuration level of the TargetMachine is that it controls things that don't change at the object level. This is a fairly recently stated goal, but I think it makes sense for LLVM in general. TargetSubtargetInfo takes care of everything that resides under this (as much as possible, some bits are still in transition, e.g. TargetOptions). This is part of my suggestion to Daniel about the problems with MCSubtargetInfo and the assembler. Targets like Mips and ARM were unfortunately designed to change things on the fly during assembly and need to collate or at least change defaults as we're processing code. I definitely had to deal with a lot of the pain you're talking about when I was rewriting some of the handling there during the TargetSubtargetInfo work. Now a bit more on TargetParser + TargetTuple: TargetParser appears to be trying to solve the parsing in Triple in a nice way for ARM and also some of the "what kind of subtarget feature canonicalization can we do in llvm that makes sense to communicate to the front end". I like this particular idea and have often wanted a library of feature handling, but it seems to have stabilized at an ARM specific set of code with no defined interface. I can't even figure out how I'd use it in lib/Basic right now for any target other than ARM. This isn't a condemnation of TargetParser, but I think it's something that needs to be thought through a bit more. It's been hooked up well before I'd expected it to and right now if we moved it to the ARM backend from Support it'd make just as much sense as it does where it is now other than making clang depend on the ARM backend as well as the X86 backend :) Right now I see TargetTuple as trying to take over all of the various arguments to TargetMachine and encapsulate them into a single thing. I also don't see this is bad, but I also don't see it taking all of them right now and I'm not sure how it solves some of the existing problems with data sharing that we've got which is where the push back you're both getting is coming from here. Ultimately library-wise I can agree with some of the directions you're headed - I just don't see the unification and interactions right now. As a suggestion as a way forward here let's see if we can get my questions above answered and also show some of how the interactions between llvm's libraries are going to get fixed, moved to a better place, etc here. Thanks! -eric On Wed, Sep 16, 2015 at 3:02 PM Renato Golin <renato.golin at linaro.org<mailto:renato.golin at linaro.org>> wrote: On 16 September 2015 at 21:56, Jim Grosbach <grosbach at apple.com<mailto:grosbach at apple.com>> wrote:> Why do we care about GAS? We have an assembler.It's not that simple. There are a lot of old code out there, including the Linux kernel which we do care a lot, that only compiles with GAS. We're slowly moving the legacy code up to modern standards, and specifically some kernel folks are happy to move up not only the asm syntax, but the C standard and move away from GNU-specific behaviour. But we're not quite there yet, and might not be for a few more years. so, yes, we still care about GAS. But this is not just about GAS. As I said on my previous email, this is about clearing the bloat in target descriptions by both: removing the need for adding numerous CPU names, target features, architecture names (xscale, strongarm, etc), AND making sure all parties (front/middle/back-ends) speak the same language, produced from the same source. The TargetTuple is that common language, and the TargetParser created from the TableGen files is the common source. The Triple becomes a legacy constructor value for the Tuple. All other target information classes are already (or should be) generated from the TableGen files, so the ultimate source becomes the TableGen description, which I think it what you were aiming to on your comment. For simple architectures, like x86, you don't even need a TargetParser. You can easily construct the Tuple from a triple and use the Tuple as you've always used the triple. No harm done. But for the complex ones like ARM and MIPS, having a common interface generated from the same place the other interfaces are is important to avoid more bridges between front and middle and back end interpretations of the same target. Whatever legacy ARM or MIPS carry can be isolated in their own implementation, leaving the rest of the targets with a clean and simple interface. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150917/7916973a/attachment.html>