On 1 May 2017, at 11:53 pm, Tom Stellard via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:> First of all you may want to review the thread from a few years ago about putting a SPIR-V target into LLVM:> http://llvm.1065342.n5.nabble.com/RFC-Proposal-for-Adding-SPIRV-Target-td82552.htmlThanks I will take a look.> The fact that the SPIR-V target translates LLVM IR to SPIR-V directly and > does not use SelectionDAG/MachineInstrs or any of the standard lowering > mechanism is a strong case against having it in lib/Target.Can you even use tablegen as a not target for generating binary format descriptions and intrinsics (serious question)? I think it will require a custom tablegen backend anyway though. One of the primary reasons for being a target is that we can have intrinsics that map directly to core and OpenCL/Vulkan extension instructions, as opposed to the mangling hacks used at the moment, which hurt use by anyone who can’t mangle C++ and windows users because it doesn’t make ANY sense to mangle some stuff as Itanuinm and some stuff as MS. I am in the process of trying to make it “more traditional” where possible and it makes sense to do so, but I do not fully understand the backend pipeline and am just trying to express the intrinsics and binary format with tablegen at the moment. Regardless of the actual transformation pipeline used I think it should go in target for the advantages stated above, and the one that I missed was, it has its own target triples! Also IIRC the Mill CPUs will/(do?) only use one or two of the backend passes and I can’t imagine them not being considered targets.>There are > two solutions for this: have the code live outside of lib/Target or as an > out-of-tree project(maybe as part of SPIRV-Tools[1]) or rewrite it to use > the standard lowering mechanism of LLVM.The first loses the advantages of being a target, the second loses that and the advantages of more eyes of LLVM devs AND staying in sync with the rest of the codebase, not to mention releases! And I can’t see any advantages gained from either of those two possibilities.>In my opinion, doing LLVM IR->SPIR-V directly is a better option than >trying to convert it to a proper LLVM target. SPIR-V is too high level >to be able to gain any advantage from using LLVM's standard lowering >mechanism. > >You will lose a lot of the type information going through the >SelectionDAG/MachineInstr layers, which is a major disadvantage. Also, >since almost everything is legal in SPIR-V, you won't be getting the >same kind of advantages from it as other targets. > >SPIR-V and LLVM IR are actually fairly similar in terms of what level of the >compiler stack the were designed for, so I think doing a simple >LLVM IR -> SPIR-V conversion will be the easiest and give you the best >results in the the end.Perhaps, I do not understand the backend pipeline, but that is entirely orthogonal to targethood.>Also, if you are able to integrate it into SPIR-V Tools you will be able >to re-use the existing SPIR-V in memory representation and the >reader/writer APIs.The in memory representation is already there (inherited from SPIRV-LLVM) and will be refined once the tablegen work is complete, i.e. redundant code removed and integration with the tables.>I realize that having a separate library will make this more difficult to >integrate into clang, but there are other targets that use external tools >for linking/assembling so, you may be able to find a way to write a SPIR-V >driver for clang that called out to this external tool for the >LLVM IR -> SPIR-V phase.An external library looks to get the worst of all worlds. I have no interest in clang, as my work is for LDC. The external library approach would require me writing a driver for LDC and then someone else writing a driver for clang, duplicating both code AND effort/bug fixes across multiple projects. There is already enough to do w.r.t metadata for the producer without having to alter their compilation pipeline as well. I plan to reduce the dependance on metadata as much as possible, but still. Nic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170501/5e790780/attachment-0001.html>
On 05/01/2017 07:43 PM, Nicholas Wilson wrote:> >> On 1 May 2017, at 11:53 pm, Tom Stellard via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >> First of all you may want to review the thread from a few years ago about putting a SPIR-V target into LLVM: > >> http://llvm.1065342.n5.nabble.com/RFC-Proposal-for-Adding-SPIRV-Target-td82552.html > > Thanks I will take a look. > >> The fact that the SPIR-V target translates LLVM IR to SPIR-V directly and >> does not use SelectionDAG/MachineInstrs or any of the standard lowering >> mechanism is a strong case against having it in lib/Target. > > Can you even use tablegen as a not target for generating binary format descriptions and intrinsics (serious question)? I think it will require a custom tablegen backend anyway though. One of the primary reasons for being a target is that we can have intrinsics that map directly to core and OpenCL/Vulkan extension instructions, as opposed to the mangling hacks used at the moment, which hurt use by anyone who can’t mangle C++ and windows users because it doesn’t make ANY sense to mangle some stuff as Itanuinm and some stuff as MS. >You would probably need to write a new tablegen backend to generate instruction tables that would be used outside of LLVM.> I am in the process of trying to make it “more traditional” where possible and it makes sense to do so, but I do not fully understand the backend pipeline and am just trying to express the intrinsics and binary format with tablegen at the moment. Regardless of the actual transformation pipeline used I think it should go in target for the advantages stated above, and the one that I missed was, it has its own target triples! Also IIRC the Mill CPUs will/(do?) only use one or two of the backend passes and I can’t imagine them not being considered targets. >A "more traditional" backend is more than just using tablegen to generate the instruction definitions. It means implementing an instruction selector and using SelectionDAG and MachineInstrs. What I'm saying is that SPIR-V is not a good fit for the existing infrastructure, so I think it would be good to consider alternatives before going to far down that path.>>There are >> two solutions for this: have the code live outside of lib/Target or as an >> out-of-tree project(maybe as part of SPIRV-Tools[1]) or rewrite it to use >> the standard lowering mechanism of LLVM. > > The first loses the advantages of being a target, the second loses that and the advantages of more eyes of LLVM devs AND staying in sync with the rest of the codebase, not to mention releases! And I can’t see any advantages gained from either of those two possibilities. >In my opinion, doing LLVM IR->SPIR-V directly is a better option than >trying to convert it to a proper LLVM target. SPIR-V is too high level >to be able to gain any advantage from using LLVM's standard lowering >mechanism. > >You will lose a lot of the type information going through the >SelectionDAG/MachineInstr layers, which is a major disadvantage. Also, >since almost everything is legal in SPIR-V, you won't be getting the >same kind of advantages from it as other targets. > >SPIR-V and LLVM IR are actually fairly similar in terms of what level of the >compiler stack the were designed for, so I think doing a simple >LLVM IR -> SPIR-V conversion will be the easiest and give you the best >results in the the > end. > > Perhaps, I do not understand the backend pipeline, but that is entirely orthogonal to targethood. >Also, if you are able to integrate it into SPIR-V Tools you will be able >to re-use the existing SPIR-V in memory representation and the >reader/writer APIs. > > The in memory representation is already there (inherited from SPIRV-LLVM) and will be refined once the tablegen work is complete, i.e. redundant code removed and integration with the tables. > >>I realize that having a separate library will make this more difficult to >integrate into clang, but there are other targets that use external tools >for linking/assembling so, you may be able to find a way to write a SPIR-V >driver for clang that called out to this external tool for the >LLVM IR -> SPIR-V phase. > > An external library looks to get the worst of all worlds. I have no interest in clang, as my work is for LDC. The external library approach would require me writing a driver for LDC and then someone else writing a driver for clang, duplicating both code AND effort/bug fixes across multiple projects. There is already enough to do w.r.t metadata for the producer without having to alter their compilation pipeline as well. I plan to reduce the dependance on metadata as much as possible, but still. >I think we may be using different definitions of driver in this case. From the perspective of LDC, it doesn't seem like it would matter much if LLVM IR -> SPIR-V were in the LLVM tree or outside of it. In both cases you have a library that takes LLVM IR as input and produces SPIR-V as output. -Tom
> On 2 May 2017, at 8:48 am, Tom Stellard <tstellar at redhat.com> wrote: > You would probably need to write a new tablegen backend to generate > instruction tables that would be used outside of LLVM.I think I need to write one anyway because I need to generate lots of tables other than the instruction table (there are no registers and therefore no register table), at least one for each subsection of section 3 of the spec and then some. Wether or not it is inside or outside of LLVM is irrelevant. What is, is that I’m not sure that the same can be said for intrinsics, which will enable the removal of the mangling hacks.>> I am in the process of trying to make it “more traditional” where possible and it makes sense to do so, but I do not fully understand the backend pipeline and am just trying to express the intrinsics and binary format with tablegen at the moment. Regardless of the actual transformation pipeline used I think it should go in target for the advantages stated above, and the one that I missed was, it has its own target triples! Also IIRC the Mill CPUs will/(do?) only use one or two of the backend passes and I can’t imagine them not being considered targets. >> > > A "more traditional" backend is more than just using tablegen to generate > the instruction definitions. It means implementing an instruction selector > and using SelectionDAG and MachineInstrs. What I'm saying is that > SPIR-V is not a good fit for the existing infrastructure, so I think > it would be good to consider alternatives before going to far down that path.Indeed, tablegen is the first step. I intend to incorporate as much of the standard pipeline as makes sense (e.g. no register allocation because there are no registers). As I have said I know very little about the backend pipeline so I can’t comment a whole lot. Is SelectionDAG matching pieces of the IR and choosing instruction from them? Because I definitely want to do that, if it is a thing (the tablegen files are currently dag free and do not inherit from `Instruction` because I do not understand that process, I am merely transcribing the spec), any information is welcome.> I think we may be using different definitions of driver in this case. > From the perspective of LDC, it doesn't seem like it would matter > much if LLVM IR -> SPIR-V were in the LLVM tree or outside of it. > In both cases you have a library that takes LLVM IR as input and > produces SPIR-V as output. > > -TomMy point is that it is duplicated effort (i.e. one for LDC, one for clang and so on), and is redundant if it is a proper target. I am willing to put in the effort to allay concerns regarding the pipelining and legalisation, but I am steadfast in my opinion that it should be a target unless the benefits of not being a target can be shown to outweigh those of being a target.