[LLVMdev] TargetRegisterInfo and "infinite" register files

Jakob Stoklund Olesen
Mon, 16 May 2011 17:00:10 -0000

On May 16, 2011, at 6:52 AM, Justin Holewinski wrote:

> Currently, the TableGen register info files for all of the back-ends define
concrete registers and divide them into logical register classes.  I would like
to get some input from the LLVM experts around here on how best to map this
model to an architecture that does *not* have a concrete, pre-defined register
file.  The architecture is PTX, which is more of an intermediate form than a
final assembly language.  The format is essentially three-address code, with
"virtual" registers instead of "physical" registers.  After
PTX code generation, the PTX assembly is compiled to a device binary with a
proprietary tool (ptxas) that does final register allocation (based on device
and user constraints).  However, exploiting register re-use at the LLVM/PTX
level has shown performance improvement over blindly using a new
"physical" register for each def and letting ptxas figure out all of
the register allocation details, so I would like to take advantage of the LLVM
register allocation infrastructure if at all possible.

What kind of optimizations can ptxas do? Can it hoist computations out of loops?
Can it split live ranges? Can it coalesce live ranges?

> Generally stated, I would like to solve the register allocation problem as
"allocate the minimum number of registers from an arbitrary set without
spill code" instead of the more traditional "allocate the minimum
number of registers from a fixed set."

It's a common misconception, but that is not what LLVM's register
allocators do. They try to minimize the amount of executed spill code given the
fixed set of registers.

I wouldn't recommend dynamically growing the register file. You are likely
to get super-linear compile time, and it is not clear that register allocation
would achieve anything compared to simply outputting virtual registers. Surely,
ptxas' register allocator can reuse a register for non-overlapping live
ranges. That is all you would get out of this.

> The current implementation defines an arbitrary set of registers that the
register allocator can use during code-gen.  This works, but is not scalable.
If the register allocator runs out of registers, spill code must be generated.
However, the "optimal" solution in this case would be to extend the
register file.  A few alternatives I have come up with are:
> 	• Bypass register allocation completely and just emit virtual registers,

This is worth a try. It is possible you want to run LLVM's 2-addr, phi-elim,
and coalescer passes first.

> 	• Remove register definitions from the TableGen files and create them at
run-time using the virtual register counts as an upper bound on the number of
registers needed, or

Don't do that.

> 	• Keep a small set of pre-defined physical registers, and craft spill code
that really just puts a new register definition in the final PTX and copies
to/from this register when spilling/restoring is needed

This could also work. Spill slots actually do what you want. The register
allocator tries to use as few as possible as long as performance doesn't
suffer. Later, StackSlotColoring will merge non-overlapping stack slot ranges to
save more space.

> I hesitate to use (1) or (3) as they rely too heavily on the final ptxas
tool to perform reasonable register allocation, which may not lead to optimal
code.  Option (2) seems promising, though I worry about the feasibility of the
approach.  Specifically, I am not yet sure if generating TargetRegisterInfo and
TargetRegisterClass instances on-the-fly will fit into the existing
architecture.
>
> Any thoughts from the experts out there?  Specifically, I am interested in
any non-trivial pros/cons for any of these approaches, or any new approaches I
have not considered.

Sorry to be backwards, but I think you should try (1) or (3).

Simply outputting virtual registers seems like a reasonable thing to to if ptx
is really an intermediate form. LLVM's instruction selector and phi-elim
tend to emit a lot of copies, so you probably want to run the coalescer before
emission. That will minimize the number of copies. This is also the fastest
thing you can do.

There are two reasons you may want to run the register allocator anyway:

- Coalescing is very aggressive. It creates long, interfering live ranges. If
ptxas doesn't have live range splitting, you may benefit from LLVM's.

- Passes like LICM and CSE will increase register pressure by hoisting redundant
computations. If ptxas cannot rematerialize these computations in high register
pressure situations, LLVM's register allocator can help you.

Note that if you always make sure there are 'enough' physical registers,
the register allocator will never split live ranges or rematerialize
computations. That's why (2) doesn't buy you anything over (1).

Use LLVM's register allocator like this:

- Provide a realistic number of physical registers. Make it similar to the
target architecture, but aim low.

- Map spill slots to PTX registers. That means 'spilling' is really a
noop, except you get live range splitting and remat. If you implement
TII::canFoldMemoryOperand() and TII::foldMemoryOperandImpl(), there will be no
inserted loads and stores.

The result should be code that is easy to register allocate for ptxas with some
live ranges that obviously should go in registers, and some that obviously
should spill. There will be a number of live ranges that can go either way,
depending on the actual number of registers targeted.

/jakob