llvm dev - Jan 2017 - Intro: DirectX Shader Compiler and LLVM

Greetings:
 
Microsoft team members have implemented a new DirectX Shader Compiler based on
LLVM and Clang. We would like to make sure that the LLVM community is kept fully
aware of it from now on:
 
The new compiler both includes and modifies the original clang to support HLSL
(a C/C++-flavored GPU programming language), and sets LLVM up to produce a
constrained form of LLVM IR. We call this output DXIL, where the specific
serialization conforms to a number of constraints that make it useful/usable by
GPU drivers when used with the Direct3D 12 runtime.
 
There are also a number of tweaks done to make it a suitable replacement for the
prior HLSL compiler which shipped as a dynamic library.
 
Originally we were hoping to contribute our changes back to clang and LLVM, but
as we implemented more and more of HLSL, it became clear that this is of
questionable value at this point in time. The code is still marked with various
conditional checks on whether LangOpts supports HLSL, a legacy of those early
days. At first it was an accumulation of small changes that made us start to
question whether this was a good idea, but after resolving all differences in
type promotions and conversion in various contexts, overload resolution and
handling of template-type constructs, it became clear this isn't a good idea
at the time - the changes are very intrusive and not a very good fit with the
current style of per-language handling.
 
We hope to evolve the language over time to a form where constructs are much
better aligned with C/C++, and hopefully we'll be able to revisit the idea
of contribution then.
 
Most of the other changes made were to align with other specific criteria:
 
- We're working on cleaning up many of the uses of global variables, to
allow for better use in multithreaded scenarios, which we see our customers use
in custom tooling.
+ For example, different passes can now be run with different options within
each, and the dynamic library provides enough information that both passes and
their options can be described to build a user interface to visualize and
potentially manage them.

- We've shimmed access to many operating system APIs, with an important
focus on file access, to improve some hosting / tooling scenarios. Still in
progress is proper use of memory allocators.

- We've worked to develop an API exported by the dynamic library that allows
component-sized access to various pieces of functionality (compilation,
optimization, validation, language services), which allowed most command-line
utilities to run by going through the dynamic library and avoid statically
linking in some of the larger core libraries.

- We've done work to reduce the disk space requirements of the library, via
a combination of "slicing off" libraries, preprocessor directives and
compile-time constants to dead-code large areas of code (a nifty trick with a
fixed/constant LangOpts took us a fair amount of the way on the clang side of
things).

- To improve cross-referencing documentation and code, we've written a
handful of scripts to generate both from documentation. The tradeoff with
TableGen was less regularity in the structures built, but higher expressiveness
in how we've built them ("intrinsics that are named with some regex
match get some attribute set on them" type of things). Not a particularly
impressive bit of code generation, but we've gotten a bunch of value of out
that, so we thought it was worth mentioning.
 
In closing, we stand on the shoulders of giants, and would like to thank the
LLVM and clang communities and express our hope that we may at some point
contribute back to the mainline whatever changes may be deemed of interest.
 
Marcelo & Chas.