I note that there was a talk recently at EuroLLVM SPIR-V and LLVM about and so I want to get this message out soon so as to avoid duplicated effort. I have an up to date backend for SPIR-V on an up to date fork (~2-3 weeks behind) of LLVM, transplanted and “modernised” from Khronos’ SPIRV-LLVM that I plan on integrating into LLVM trunk. While it is usable in it’s current form there are several issues. SPIRV-LLVM’s SPIRV support is not a real backend and does not live in /lib/Target/SPIRV. This has been solved, SPIRV is now a proper target(s). I have not copied the tests across yet, and they will need to be updated in light of the changes made/being made below. It only support’s the OpenCL “flavour” of SPIRV, but not the Vulkan. I plan on eventually supporting this but its priority is less than that of mainline integration. Intrinsics (for both Core and OpenCL extension instructions, Vulkan to follow) are done through C++ "Itanium with extensions” mangling. I plan to convert these to a table-gen format and cull the associated mangling code.. Likewise the instruction format is currently done through a home-brew table format, which I am in the process of converting to table-gen. The core instructions are almost complete, but I haven’t started on the OpenCL instructions. As noted in the talk the textual representation is different from the the reference implementation. I see no advantage of the current format over the reference implementation’s format, infact theirs is much easier to read, This is a low priority. The instruction table-gen tables contain a significant number of critical non-instruction tables (there are no registers) and therefore a proper table-gen backend needs to be written to accomodate for this. Required. There are some LLVM “infrastructure” things (TargetInfo, MCTargetDesc) that I have stubs in order to get LLVM to compile, I have no idea what I’m doing and those were mostly adapted from Sparc and RISCV. Due to the lack of a Dyld and the fact that SPIR-V is intermediate format, there is no point of a jit, Simple optimisations (CSE,DCE, some very simple inlining) would be nice to have, as would DebugInfo support. The code is available at my GitHub: SPIRV backend https://github.com/thewilsonator/llvm-target-spirv LLVM https://github.com/thewilsonator/llvm/tree/compute should you want to inspect/offer advice. I am very busy until about July, but I just want to put this out here to gather interest / feedback / contributors. Thanks, Nicholas Wilson -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170501/2c57102c/attachment.html>
Great job Nicholas! Actually one of the topics in the OpenCL BoF during EuroLLVM is to discuss about whether we should put SPIRV as one of the LLVM backends. Some of the guys in ARM and Codeplay participate that BoF. Although there is no concrete conclusion, I think your work would be a valuable materials for folks in both Khronos and LLVM to determine whether put SPIRV under lib/Target. Also, though I haven’t inspect your work in detail. I think you can take NVPTX and WebAssembly as reference instead of RISCV or SPARC, since the former two targets are both virtual targets, (unlimited amount of registers etc.), just like SPIRV. Best Regards, McClane> Nicholas Wilson via llvm-dev <llvm-dev at lists.llvm.org> 於 2017年5月1日 下午4:01 寫道： > > I note that there was a talk recently at EuroLLVM SPIR-V and LLVM about and so I want to get this message out soon so as to avoid duplicated effort. > > I have an up to date backend for SPIR-V on an up to date fork (~2-3 weeks behind) of LLVM, transplanted and “modernised” from Khronos’ SPIRV-LLVM that I plan on integrating into LLVM trunk. While it is usable in it’s current form there are several issues. > > SPIRV-LLVM’s SPIRV support is not a real backend and does not live in /lib/Target/SPIRV. This has been solved, SPIRV is now a proper target(s). I have not copied the tests across yet, and they will need to be updated in light of the changes made/being made below. > It only support’s the OpenCL “flavour” of SPIRV, but not the Vulkan. I plan on eventually supporting this but its priority is less than that of mainline integration. > Intrinsics (for both Core and OpenCL extension instructions, Vulkan to follow) are done through C++ "Itanium with extensions” mangling. I plan to convert these to a table-gen format and cull the associated mangling code.. > Likewise the instruction format is currently done through a home-brew table format, which I am in the process of converting to table-gen. The core instructions are almost complete, but I haven’t started on the OpenCL instructions. > As noted in the talk the textual representation is different from the the reference implementation. I see no advantage of the current format over the reference implementation’s format, infact theirs is much easier to read, This is a low priority. > The instruction table-gen tables contain a significant number of critical non-instruction tables (there are no registers) and therefore a proper table-gen backend needs to be written to accomodate for this. Required. > There are some LLVM “infrastructure” things (TargetInfo, MCTargetDesc) that I have stubs in order to get LLVM to compile, I have no idea what I’m doing and those were mostly adapted from Sparc and RISCV. Due to the lack of a Dyld and the fact that SPIR-V is intermediate format, there is no point of a jit, Simple optimisations (CSE,DCE, some very simple inlining) would be nice to have, as would DebugInfo support. > > The code is available at my GitHub: > SPIRV backend https://github.com/thewilsonator/llvm-target-spirv <https://github.com/thewilsonator/llvm-target-spirv> > LLVM https://github.com/thewilsonator/llvm/tree/compute <https://github.com/thewilsonator/llvm/tree/compute> > should you want to inspect/offer advice. > > I am very busy until about July, but I just want to put this out here to gather interest / feedback / contributors. > > Thanks, > Nicholas Wilson > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170501/d2f59982/attachment.html>
> On 1 May 2017, at 4:28 pm, Bekket McClane <bekket.mcclane at gmail.com> wrote: > > Great job Nicholas! >Thanks! I want to get this in so I don’t have to maintain a separate version for my work on LDC, the LLVM D Compiler.> Actually one of the topics in the OpenCL BoF during EuroLLVM is to discuss about whether we should put SPIRV as one of the LLVM backends. Some of the guys in ARM and Codeplay participate that BoF.Great, any help (especially by those who actually know what they doing when it come to LLVM backends, i.e. not me ;) ) is greatly appreciated. Co-ordination will be the key to getting this done in a reasonable timeframe.> Although there is no concrete conclusion, I think your work would be a valuable materials for folks in both Khronos and LLVM to determine whether put SPIRV under lib/Target.I think SPIRV should be a proper target (and therefore live in $ROOT/lib/Target) for a number of reasons. 1) Logically it is a target: it produces binary code and is the last stage in the compilation pipeline. 2) it makes it easier for consumers (LDC, clang) to set up the build system / CMake as they only need specify they wish to target SPIRV rather than specify they want to link to libLLVMSPIRV.a with some CMake hackery. This was part of the reason for making it a target, as I was unable to get LDC’s build system to pull in the required library from Khronos’ SPIRV-LLVM. 3) it will integrate much better into consumers existing consumers compilation pipelines, as for LDC, there were, until I made it a target, some rather ugly hacks to fit it in with the rest of the codebase that was setup to just pass the IR to a backend. 4) if its a target then it will stay up to date with LLVM and not stagnate fall many releases behind. There are probably some others I am missing.> Also, though I haven’t inspect your work in detail. I think you can take NVPTX and WebAssembly as reference instead of RISCV or SPARC, since the former two targets are both virtual targets, (unlimited amount of registers etc.), just like SPIRV.Haha, you overestimate my exposure to the LLVM codebase by a large margin. I used RISCV and SPARC because they are simple and I have very little idea what I’m doing, e.g. not knowing what files I need in order to make a backend (LLVMBuild.txt’s CMakeLists.txt,target info MCDesc etc.) no knowledge of the tablegen format (I’m still probably doin' it wrong), not because of architectural similarity. I have taken some Inspiration from NVPTX simply because my work on LDC involves GPUs in general, but thank for the tip anyway. Nic
On 05/01/2017 04:01 AM, Nicholas Wilson via llvm-dev wrote:> I note that there was a talk recently at EuroLLVM SPIR-V and LLVM about and so I want to get this message out soon so as to avoid duplicated effort. > > I have an up to date backend for SPIR-V on an up to date fork (~2-3 weeks behind) of LLVM, transplanted and “modernised” from Khronos’ SPIRV-LLVM that I plan on integrating into LLVM trunk. While it is usable in it’s current form there are several issues. >Thanks for taking the initiative to work on this. A well supported LLVM IR to SPIR-V transformation layer is something that could be very useful.> SPIRV-LLVM’s SPIRV support is not a real backend and does not live in /lib/Target/SPIRV. This has been solved, SPIRV is now a proper target(s). I have not copied the tests across yet, and they will need to be updated in light of the changes made/being made below. > It only support’s the OpenCL “flavour” of SPIRV, but not the Vulkan. I plan on eventually supporting this but its priority is less than that of mainline integration. > Intrinsics (for both Core and OpenCL extension instructions, Vulkan to follow) are done through C++ "Itanium with extensions” mangling. I plan to convert these to a table-gen format and cull the associated mangling code.. > Likewise the instruction format is currently done through a home-brew table format, which I am in the process of converting to table-gen. The core instructions are almost complete, but I haven’t started on the OpenCL instructions. > As noted in the talk the textual representation is different from the the reference implementation. I see no advantage of the current format over the reference implementation’s format, infact theirs is much easier to read, This is a low priority. > The instruction table-gen tables contain a significant number of critical non-instruction tables (there are no registers) and therefore a proper table-gen backend needs to be written to accomodate for this. Required. > There are some LLVM “infrastructure” things (TargetInfo, MCTargetDesc) that I have stubs in order to get LLVM to compile, I have no idea what I’m doing and those were mostly adapted from Sparc and RISCV. Due to the lack of a Dyld and the fact that SPIR-V is intermediate format, there is no point of a jit, Simple optimisations (CSE,DCE, some very simple inlining) would be nice to have, as would DebugInfo support. > > The code is available at my GitHub: > SPIRV backend https://github.com/thewilsonator/llvm-target-spirv > LLVM https://github.com/thewilsonator/llvm/tree/compute > should you want to inspect/offer advice. > > I am very busy until about July, but I just want to put this out here to gather interest / feedback / contributors. >First of all you may want to review the thread from a few years ago about putting a SPIR-V target into LLVM: http://llvm.1065342.n5.nabble.com/RFC-Proposal-for-Adding-SPIRV-Target-td82552.html The fact that the SPIR-V target translates LLVM IR to SPIR-V directly and does not use SelectionDAG/MachineInstrs or any of the standard lowering mechanism is a strong case against having it in lib/Target. There are two solutions for this: have the code live outside of lib/Target or as an out-of-tree project(maybe as part of SPIRV-Tools[1]) or rewrite it to use the standard lowering mechanism of LLVM. In my opinion, doing LLVM IR->SPIR-V directly is a better option than trying to convert it to a proper LLVM target. SPIR-V is too high level to be able to gain any advantage from using LLVM's standard lowering mechanism. You will lose a lot of the type information going through the SelectionDAG/MachineInstr layers, which is a major disadvantage. Also, since almost everything is legal in SPIR-V, you won't be getting the same kind of advantages from it as other targets. SPIR-V and LLVM IR are actually fairly similar in terms of what level of the compiler stack the were designed for, so I think doing a simple LLVM IR -> SPIR-V conversion will be the easiest and give you the best results in the the end. Also, if you are able to integrate it into SPIR-V Tools you will be able to re-use the existing SPIR-V in memory representation and the reader/writer APIs. I realize that having a separate library will make this more difficult to integrate into clang, but there are other targets that use external tools for linking/assembling so, you may be able to find a way to write a SPIR-V driver for clang that called out to this external tool for the LLVM IR -> SPIR-V phase. -Tom [1] https://github.com/KhronosGroup/SPIRV-Tools> Thanks, > Nicholas Wilson > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >
On 1 May 2017, at 11:53 pm, Tom Stellard via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:> First of all you may want to review the thread from a few years ago about putting a SPIR-V target into LLVM:> http://llvm.1065342.n5.nabble.com/RFC-Proposal-for-Adding-SPIRV-Target-td82552.htmlThanks I will take a look.> The fact that the SPIR-V target translates LLVM IR to SPIR-V directly and > does not use SelectionDAG/MachineInstrs or any of the standard lowering > mechanism is a strong case against having it in lib/Target.Can you even use tablegen as a not target for generating binary format descriptions and intrinsics (serious question)? I think it will require a custom tablegen backend anyway though. One of the primary reasons for being a target is that we can have intrinsics that map directly to core and OpenCL/Vulkan extension instructions, as opposed to the mangling hacks used at the moment, which hurt use by anyone who can’t mangle C++ and windows users because it doesn’t make ANY sense to mangle some stuff as Itanuinm and some stuff as MS. I am in the process of trying to make it “more traditional” where possible and it makes sense to do so, but I do not fully understand the backend pipeline and am just trying to express the intrinsics and binary format with tablegen at the moment. Regardless of the actual transformation pipeline used I think it should go in target for the advantages stated above, and the one that I missed was, it has its own target triples! Also IIRC the Mill CPUs will/(do?) only use one or two of the backend passes and I can’t imagine them not being considered targets.>There are > two solutions for this: have the code live outside of lib/Target or as an > out-of-tree project(maybe as part of SPIRV-Tools[1]) or rewrite it to use > the standard lowering mechanism of LLVM.The first loses the advantages of being a target, the second loses that and the advantages of more eyes of LLVM devs AND staying in sync with the rest of the codebase, not to mention releases! And I can’t see any advantages gained from either of those two possibilities.>In my opinion, doing LLVM IR->SPIR-V directly is a better option than >trying to convert it to a proper LLVM target. SPIR-V is too high level >to be able to gain any advantage from using LLVM's standard lowering >mechanism. > >You will lose a lot of the type information going through the >SelectionDAG/MachineInstr layers, which is a major disadvantage. Also, >since almost everything is legal in SPIR-V, you won't be getting the >same kind of advantages from it as other targets. > >SPIR-V and LLVM IR are actually fairly similar in terms of what level of the >compiler stack the were designed for, so I think doing a simple >LLVM IR -> SPIR-V conversion will be the easiest and give you the best >results in the the end.Perhaps, I do not understand the backend pipeline, but that is entirely orthogonal to targethood.>Also, if you are able to integrate it into SPIR-V Tools you will be able >to re-use the existing SPIR-V in memory representation and the >reader/writer APIs.The in memory representation is already there (inherited from SPIRV-LLVM) and will be refined once the tablegen work is complete, i.e. redundant code removed and integration with the tables.>I realize that having a separate library will make this more difficult to >integrate into clang, but there are other targets that use external tools >for linking/assembling so, you may be able to find a way to write a SPIR-V >driver for clang that called out to this external tool for the >LLVM IR -> SPIR-V phase.An external library looks to get the worst of all worlds. I have no interest in clang, as my work is for LDC. The external library approach would require me writing a driver for LDC and then someone else writing a driver for clang, duplicating both code AND effort/bug fixes across multiple projects. There is already enough to do w.r.t metadata for the producer without having to alter their compilation pipeline as well. I plan to reduce the dependance on metadata as much as possible, but still. Nic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170501/5e790780/attachment-0001.html>