llvm dev - Aug 2021 - GHC calling convention and the stack pointer register

tl;dr. GHC wants to change its calling convention to use the native
       stack pointer register (e.g. $rsp on x86-64) for its (segmented)
       stack. Is this going to be a problem for LLVM?


Hi all,

As you likely know, the Glasgow Haskell Compiler (GHC, [ghc]) has for many
years supported LLVM as a code generation backend. This is facilitated
via an LLVM calling convention (e.g. [ghc-cc]) which has been upstream since
LLVM 3.4 (IIRC). This calling convention matches the convention
used by GHC's internal code generator (the so-called NCG), allowing
object code produced by LLVM to be linked against that produced by the
NCG.

GHC's LLVM backend implements a simple lowering of C-- (GHC's C-like
imperative IR, built around GHC's STG abstract machine) to LLVM IR. C--
procedures are lowered to LLVM procedures, with the STG machine state
(namely register values) being passsed via arguments. The GHC calling
convention maps the STG machine's registers to physical registers of the
target machine.

One of these registers (Sp) points to the STG stack, which is a
segmented stack which must be traversable by GHC's garbage collector.
On x86-64 Sp is currently mapped to $rbp. However, recently we have been
considering [native-sp] modifying GHC's calling convention to rather use
the native stack pointer (e.g. $rsp) for Sp, allowing stack-sampling
performance measurement tools like `perf` to be used on Haskell
programs.

One of the open questions surrounding this change is how it will affect
the LLVM backend. It seems that LLVM makes rather strong assumptions
about the native stack register (e.g. freely using it for spilling).
Moreover, in looking through LLVM's other calling conventions I have yet
to find an example of a convention that uses the native stack pointer.

So I ask:

 * Can a TableGen-defined calling-convention in LLVM claim the native
   stack register for use by arguments?

 * Is there any way to affect LLVM's spilling behavior to spill to a
   stack pointed to be a register other than the native stack pointer
   register?

Thanks in advance for your help.

Cheers,

- Ben



[ghc]: https://www.haskell.org/ghc/
[ghc-cc]:
https://github.com/llvm/llvm-project/blob/8c8dbc10825cf099607af3da58b839e10c68320f/llvm/lib/Target/X86/X86CallingConv.td#L692
[native-sp]: https://well-typed.com/blog/2021/07/ghc-sp-profiling/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210729/f537c635/attachment.sig>

I think you hit the nail on the head: LLVM will try to use the native stack
to spill stuff. It's also going to generate prologues and epilogues to
update and restore it.

IIRC, GHC attempts to generate code that is almost "stackless": most
functions are leaf functions that simply tail call to the next function, so
there is nothing to spill and no stack frame. LLVM always assumes it has
permission to use the stack for something. Things may work, but I don't
think anyone can make any promises.

On Fri, Jul 30, 2021 at 9:17 AM Ben Gamari via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> tl;dr. GHC wants to change its calling convention to use the native
>        stack pointer register (e.g. $rsp on x86-64) for its (segmented)
>        stack. Is this going to be a problem for LLVM?
>
>
> Hi all,
>
> As you likely know, the Glasgow Haskell Compiler (GHC, [ghc]) has for many
> years supported LLVM as a code generation backend. This is facilitated
> via an LLVM calling convention (e.g. [ghc-cc]) which has been upstream
> since
> LLVM 3.4 (IIRC). This calling convention matches the convention
> used by GHC's internal code generator (the so-called NCG), allowing
> object code produced by LLVM to be linked against that produced by the
> NCG.
>
> GHC's LLVM backend implements a simple lowering of C-- (GHC's
C-like
> imperative IR, built around GHC's STG abstract machine) to LLVM IR. C--
> procedures are lowered to LLVM procedures, with the STG machine state
> (namely register values) being passsed via arguments. The GHC calling
> convention maps the STG machine's registers to physical registers of
the
> target machine.
>
> One of these registers (Sp) points to the STG stack, which is a
> segmented stack which must be traversable by GHC's garbage collector.
> On x86-64 Sp is currently mapped to $rbp. However, recently we have been
> considering [native-sp] modifying GHC's calling convention to rather
use
> the native stack pointer (e.g. $rsp) for Sp, allowing stack-sampling
> performance measurement tools like `perf` to be used on Haskell
> programs.
>
> One of the open questions surrounding this change is how it will affect
> the LLVM backend. It seems that LLVM makes rather strong assumptions
> about the native stack register (e.g. freely using it for spilling).
> Moreover, in looking through LLVM's other calling conventions I have
yet
> to find an example of a convention that uses the native stack pointer.
>
> So I ask:
>
>  * Can a TableGen-defined calling-convention in LLVM claim the native
>    stack register for use by arguments?
>
>  * Is there any way to affect LLVM's spilling behavior to spill to a
>    stack pointed to be a register other than the native stack pointer
>    register?
>
> Thanks in advance for your help.
>
> Cheers,
>
> - Ben
>
>
>
> [ghc]: https://www.haskell.org/ghc/
> [ghc-cc]:
>
https://github.com/llvm/llvm-project/blob/8c8dbc10825cf099607af3da58b839e10c68320f/llvm/lib/Target/X86/X86CallingConv.td#L692
> [native-sp
>
<https://github.com/llvm/llvm-project/blob/8c8dbc10825cf099607af3da58b839e10c68320f/llvm/lib/Target/X86/X86CallingConv.td#L692[native-sp>]:
> https://well-typed.com/blog/2021/07/ghc-sp-profiling/
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210804/46a23922/attachment.html>