llvm dev - Mar 2011 - [LLVMdev] [GSoC] "Microsoft Direct3D shader bytecode backend" proposal

Here's the other of my proposals for this year's Google Summer of Code.
(The first is on cfe-dev.) It was suggested to me by Dan Kegel (from the
Wine project, they really want this).

Title: Microsoft Direct3D shader bytecode backend

Abstract:

There is a distinct lack of open-source frameworks for compiling HLSL,
the shader language used by Direct3D, into bytecode that D3D can
understand. Currently, the only such framework is Ryan Gordon's
MojoShader, whose HLSL compiler component is still under heavy
development. By utilizing LLVM, it may be possible to generate
high-performance shader code from HLSL, just as Apple is known to do for
GLSL. The first step is a backend to generate D3D bytecode from LLVM IR.

Content:

1. Proposal

Currently, there are very few open-source frameworks for compiling
High-Level Shader Language (HLSL) into shader bytecode that can be
understood by Direct3D, a popular interface to 3D hardware in games.
This is because Microsoft provides its own interface in its D3DX (and
underlying D3DCompiler) DLLs. Most games therefore end up using D3DX to
compile their Direct3D shaders.

Microsoft seems to have paid no attention to optimization of the
resulting shader code, though. With LLVM, we can do better.

By using LLVM's optimizers, programs can potentially generate
high-performance shader code that works well on the platform on which
they are running. In addition, an open-source implementation would allow
the compiler to be embedded in many different applications--not the
least of which is Wine, which is in desperate need of a working shader
compiler.

The first step to this HLSL compiler is an LLVM backend for generating
D3D shader bytecode from LLVM IR. Therefore, I intend to implement such
a backend. Because implementing a full backend is a daunting task, I
intend to implement just enough to get simple examples working with
Wine's Direct3D implementation.

2. Interest

Over the past few years, I've become interested in graphics programming.
I'm also interested in learning how to design backends for LLVM.

3. Usefulness to LLVM

This should increase adoption of LLVM, particularly by games and other
3D graphical applications that use Direct3D.

4. Prior knowledge in compilers and LLVM in particular

I have been working with LLVM since the fall of 2009. I have a little
experience working with the x86 backend, but no experience writing my
own backend. I have, however, read the documents on building a new
backend, and I am very familiar with the TableGen syntax.

5. Academic, industry, etc. experience

I am a student at the Colorado School of Mines studying in Computer
Science. I am a Senior by year and credits, and I am set to graduate in
December of 2011. In addition, I have contributed to several open-source
projects, including Wine and LLVM.

6. Contact information

 - E-mail: cdavis at mymail.mines.edu
 - IRC nick: cdavis5x

Chip

On Mon, Mar 28, 2011 at 9:22 PM, Charles Davis <cdavis at
mymail.mines.edu>wrote:
> Here's the other of my proposals for this year's Google Summer of
Code.
> (The first is on cfe-dev.) It was suggested to me by Dan Kegel (from the
> Wine project, they really want this).
>
> Title: Microsoft Direct3D shader bytecode backend
>
> Abstract:
>
> There is a distinct lack of open-source frameworks for compiling HLSL,
> the shader language used by Direct3D, into bytecode that D3D can
> understand. Currently, the only such framework is Ryan Gordon's
> MojoShader, whose HLSL compiler component is still under heavy
> development. By utilizing LLVM, it may be possible to generate
> high-performance shader code from HLSL, just as Apple is known to do for
> GLSL. The first step is a backend to generate D3D bytecode from LLVM IR.
>
> Content:
>
> 1. Proposal
>
> Currently, there are very few open-source frameworks for compiling
> High-Level Shader Language (HLSL) into shader bytecode that can be
> understood by Direct3D, a popular interface to 3D hardware in games.
> This is because Microsoft provides its own interface in its D3DX (and
> underlying D3DCompiler) DLLs. Most games therefore end up using D3DX to
> compile their Direct3D shaders.
>
> Microsoft seems to have paid no attention to optimization of the
> resulting shader code, though. With LLVM, we can do better.
>
Do you have any sources for this?  In my experience, fxc is able to do some
clever tricks such as replacing if-then-else conditions with predicated
instructions and swizzles.

>
> By using LLVM's optimizers, programs can potentially generate
> high-performance shader code that works well on the platform on which
> they are running. In addition, an open-source implementation would allow
> the compiler to be embedded in many different applications--not the
> least of which is Wine, which is in desperate need of a working shader
> compiler.
>
I'm a bit confused how Wine would take advantage of a Direct3D bytecode
compiler.  Would they not want to re-compile Direct3D bytecode (most often
shipped with games in binary form instead of HLSL source) to something an
OpenGL implementation on *nix could handle?

>
> The first step to this HLSL compiler is an LLVM backend for generating
> D3D shader bytecode from LLVM IR. Therefore, I intend to implement such
> a backend. Because implementing a full backend is a daunting task, I
> intend to implement just enough to get simple examples working with
> Wine's Direct3D implementation.
>
Could you be a bit more specific on your goals?  A few quick questions that
come to mind are:

1. What shader models will you aim to support?
2. What types of shared will you aim to support?  e.g. vertex, pixel,
geometry, hull
3. How do you propose to handle vertex buffer semantics?  e.g. POSITION0,
TEXCOORD0, NORMAL, etc.

Perhaps a simple example would be nice, showing a very simple LLVM IR input
and the (proposed) bytecode output.

>
> 2. Interest
>
> Over the past few years, I've become interested in graphics
programming.
> I'm also interested in learning how to design backends for LLVM.
>
> 3. Usefulness to LLVM
>
> This should increase adoption of LLVM, particularly by games and other
> 3D graphical applications that use Direct3D.
>
> 4. Prior knowledge in compilers and LLVM in particular
>
> I have been working with LLVM since the fall of 2009. I have a little
> experience working with the x86 backend, but no experience writing my
> own backend. I have, however, read the documents on building a new
> backend, and I am very familiar with the TableGen syntax.
>
> 5. Academic, industry, etc. experience
>
> I am a student at the Colorado School of Mines studying in Computer
> Science. I am a Senior by year and credits, and I am set to graduate in
> December of 2011. In addition, I have contributed to several open-source
> projects, including Wine and LLVM.
>
> 6. Contact information
>
>  - E-mail: cdavis at mymail.mines.edu
>  - IRC nick: cdavis5x
>
> Chip
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 

Thanks,

Justin Holewinski
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20110329/0a54e425/attachment.html>

On 3/29/11 5:14 AM, Justin Holewinski wrote:
> On Mon, Mar 28, 2011 at 9:22 PM, Charles Davis <cdavis at
mymail.mines.edu
> <mailto:cdavis at mymail.mines.edu>> wrote:
> 
>     Here's the other of my proposals for this year's Google Summer
of Code.
>     (The first is on cfe-dev.) It was suggested to me by Dan Kegel (from
the
>     Wine project, they really want this).
> 
>     Title: Microsoft Direct3D shader bytecode backend
> 
>     Abstract:
> 
>     There is a distinct lack of open-source frameworks for compiling HLSL,
>     the shader language used by Direct3D, into bytecode that D3D can
>     understand. Currently, the only such framework is Ryan Gordon's
>     MojoShader, whose HLSL compiler component is still under heavy
>     development. By utilizing LLVM, it may be possible to generate
>     high-performance shader code from HLSL, just as Apple is known to do
for
>     GLSL. The first step is a backend to generate D3D bytecode from LLVM
IR.
> 
>     Content:
> 
>     1. Proposal
> 
>     Currently, there are very few open-source frameworks for compiling
>     High-Level Shader Language (HLSL) into shader bytecode that can be
>     understood by Direct3D, a popular interface to 3D hardware in games.
>     This is because Microsoft provides its own interface in its D3DX (and
>     underlying D3DCompiler) DLLs. Most games therefore end up using D3DX to
>     compile their Direct3D shaders.
> 
>     Microsoft seems to have paid no attention to optimization of the
>     resulting shader code, though. With LLVM, we can do better.
> 
> 
> Do you have any sources for this?  In my experience, fxc is able to do
> some clever tricks such as replacing if-then-else conditions with
> predicated instructions and swizzles.
I heard rumors that drivers tend not to like highly optimized bytecode
input. But that seems to count against optimization. I could take that
part out.
>  
> 
> 
>     By using LLVM's optimizers, programs can potentially generate
>     high-performance shader code that works well on the platform on which
>     they are running. In addition, an open-source implementation would
allow
>     the compiler to be embedded in many different applications--not the
>     least of which is Wine, which is in desperate need of a working shader
>     compiler.
> 
> 
> I'm a bit confused how Wine would take advantage of a Direct3D bytecode
> compiler.  Would they not want to re-compile Direct3D bytecode (most
> often shipped with games in binary form instead of HLSL source) to
> something an OpenGL implementation on *nix could handle?
They already do that. What they want right now (among other things) is a
D3DCompiler_*.dll implementation (see http://wiki.winehq.org/HLSLCompiler
).
>  
> 
> 
>     The first step to this HLSL compiler is an LLVM backend for generating
>     D3D shader bytecode from LLVM IR. Therefore, I intend to implement such
>     a backend. Because implementing a full backend is a daunting task, I
>     intend to implement just enough to get simple examples working with
>     Wine's Direct3D implementation.
> 
> 
> Could you be a bit more specific on your goals?  A few quick questions
> that come to mind are:
> 
> 1. What shader models will you aim to support?
SM1-3. SM4 was a huge break from SM3 (so I've gathered from reading Wine
source; all the opcodes seem to be different), so I'll do that
later.
> 2. What types of shared will you aim to support?  e.g. vertex, pixel,
> geometry, hull
Since I'm only doing up to SM3, vertex and pixel shaders
only.
> 3. How do you propose to handle vertex buffer semantics?  e.g.
> POSITION0, TEXCOORD0, NORMAL, etc.
I can think of several ways:

- Make the frontend declare special symbols, and handle these symbols
specially in the backend
- Decorate the struct members with metadata

I'm leaning towards the latter one.
> 
> Perhaps a simple example would be nice, showing a very simple LLVM IR
> input and the (proposed) bytecode output.
How about this (this is from
http://www.neatware.com/lbstudio/web/hlsl.html)?

struct a2v {
  float4 position : POSITION;
};

struct v2p {
  float4 position : POSITION;
};

void main(in a2v IN, out v2p OUT, uniform float4x4 ModelViewMatrix) {
  OUT.position = mul(IN.position, ModelViewMatrix);
}

This would generate something like this (assuming I took the metadata
route):

%struct.a2v = { <4 x float> !0 }
%struct.v2p = { <4 x float> !0 }

!0 = metadata !{ <something that indicates this is a position attribute> }

define void @main(%struct.a2v* %IN, %struct.v2p* %OUT, [4 x <4 x float>]
%ModelViewMatrix) {
  %OUT.Position = getelementptr %struct.v2p* %OUT, i32 0, i32 0
  %IN.Position = getelementptr %struct.a2v* %IN, i32 0, i32 0
  %0 = load <4 x float>* %IN.Position
  # Note the intrinsic for matrix multiplies
  %1 = call <4 x float> @llvm.d3d.mul.float4(<4 x float>%0, [4 x
<4 x
float>] %ModelViewMatrix)
  store <4 x float> %1, <4 x float>* %OUT.Position
}

and should generate assembly to the effect of:

vs_1_1
dcl_position v0
m4x4 oPos, v0, c0
mov oD0, c4

Thanks for your input.

Chip

On Tue, Mar 29, 2011 at 4:14 AM, Justin Holewinski
<justin.holewinski at gmail.com> wrote:
>> Here's the other of my proposals for this year's Google Summer
of Code.
>> (The first is on cfe-dev.) It was suggested to me by Dan Kegel (from
the
>> Wine project, they really want this).
>>
>> Title: Microsoft Direct3D shader bytecode backend
>...
> I'm a bit confused how Wine would take advantage of a Direct3D bytecode
> compiler.  Would they not want to re-compile Direct3D bytecode (most often
> shipped with games in binary form instead of HLSL source) to something an
> OpenGL implementation on *nix could handle?
Wine already does re-compile Direct3D bytecode to glsl (or arb
shaders), and it works.
The problem is, some games actually ship ascii hlsl and expect to
be able to compile it at startup time (or even later).
See e.g. http://bugs.winehq.org/buglist.cgi?quicksearch=D3DXCompileShader
and http://bugs.winehq.org/buglist.cgi?quicksearch=D3DCompile
- Dan