llvm dev - Mar 2016 - RFC: Implementing the Swift calling convention in LLVM and Clang

Hi, all.

Swift uses a non-standard calling convention on its supported platforms. 
Implementing this calling convention requires support from LLVM and (to a lesser
degree) Clang.  If necessary, we’re willing to keep that support in “private”
branches of LLVM and Clang, but we feel it would be better to introduce it in
trunk, both to (1) minimize the differences between our branches and trunk and
(2) allow other language implementations to take advantage of that support.

We don’t expect this to be particularly controversial, at least in the abstract,
since LLVM already includes support for a number of variant, language-specific
calling conventions.  Some of Swift's variations are more invasive than
those existing conventions, however, so we want to make sure the community is on
board before we start landing patches or sending them out for review.

Here’s a brief technical summary of the convention:

In general, the calling convention lowers onto an existing C calling convention;
let’s call this the “intermediary convention”.  The intermediary convention is
not necessarily the target platform’s standard C convention; for example, we
intend to use a VFP convention on iOS ARM targets.  Aggregate arguments and
results are translated to sequences of scalar types (possibly just an indirect
argument/sret pointer) and, for the most part, passed and returned using the
intermediary convention’s rules for a function with that signature.  For
example, if struct A expands to the sequence [i32,float,i32], a function type
like (A,Int64) -> Bool) would be lowered basically like the C function type
bool(*)(int32_t, float, int32_t, int64_t).

There are four general points of deviation from the intermediary convention:

  - We sometimes want to return more values in registers than the convention
normally does, and we want to be able to use both integer and floating-point
registers.  For example, we want to return a value of struct A, above, purely in
registers.  For the most part, I don’t think this is a problem to layer on to an
existing IR convention: C frontends will generally use explicit sret arguments
when the convention requires them, and so the Swift lowering will produce result
types that don’t have legal interpretations as direct results under the C
convention.  But we can use a different IR convention if it’s necessary to
disambiguate Swift’s desired treatment from the target's normal attempts to
retroactively match the C convention.

  - We sometimes have both direct results and indirect results.  It would be
nice to take advantage of the sret convention even in the presence of direct
results on targets that do use a different (profitable) ABI treatment for it.  I
don’t know how well-supported this is in LLVM.

  - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.

  - We want a special “error” treatment for a certain argument/result.  A
pointer-sized value is passed in an integer register; a different value may be
present in that register after the call.  Much like the context treatment, the
caller may use the error treatment with a function that doesn’t expect it; this
should not trigger undefined behavior, and the existing value should be left in
place.  Like the context treatment, this suggests that the error value be passed
and returned in a register which is normally callee-save.

Here’s a brief summary of the expected code impact for this.

The Clang impact is relatively minor; it is focused on allowing the Swift
runtime to define functions that use the convention.  It adds a new calling
convention attribute, a few new parameter attributes constrained to that calling
convention, and some relatively un-invasive call lowering code in IR generation.

The LLVM impact is somewhat larger.

Three things in the convention require a possible change to IR:

  - Using sret together with a direct result may or may not “just work".  I
certainly don’t see a reason why it shouldn’t work in the middle-end. 
Obviously, some targets can’t support it, but we can avoid doing this on those
targets.

  - Opting in to the two argument treatments requires new parameter attributes. 
We discussed using separate calling conventions; unfortunately, error and
context arguments can appear either separately or together, so we’d really need
several new conventions for all the valid combinations.  Furthermore, calling a
context-free function with an ignored context argument could turn into a call to
a function using a mismatched calling convention, which LLVM IR generally treats
as undefined behavior.  Also, it wasn’t obvious that just a calling convention
would be sufficient for the error treatment; see the next bullet.

  - The “error” treatment requires some way to (1) pass and receive the value in
the caller and (2) receive and change the value in the callee.  The best way we
could think of to represent this was to pretend that the argument is actually
passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.

The convention also requires a few changes to the targets that support the
convention, to deal with the context and error treatments and to return more
values in registers.

Anyway, I would appreciate your thoughts.

John.

It's probably worth noting that the likely diffs involved can be
inferred from the "upstream-with-swift" branch at
"git at github.com:apple/swift-llvm.git", which is a fairly regularly
merged copy of trunk with the swift changes.

I'm steering well clear of the policy decision.

Tim.

On 2 March 2016 at 01:14, John McCall via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hi, all.
>   - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
Is this a back-end decision, or do you expect the front-end to tell
the back-end (via annotation) which parameters will be in regs? Unless
you also have back-end patches, I don't think the latter is going to
work well. For example, the ARM back-end has a huge section related to
passing structures in registers, which conforms to the ARM EABI, not
necessarily your Swift ABI.

Not to mention that this creates the versioning problem, where two
different LLVM releases can produce slightly different PCS register
usage (due to new features or bugs), and thus require re-compilation
of all libraries. This, however, is not a problem for your current
request, just a comment.

>   - We sometimes have both direct results and indirect results.  It would
be nice to take advantage of the sret convention even in the presence of direct
results on targets that do use a different (profitable) ABI treatment for it.  I
don’t know how well-supported this is in LLVM.
I'm not sure what you mean by direct or indirect results here. But if
this is a language feature, as long as the IR semantics is correct, I
don't see any problem.

>   - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
I think it's going to be harder to get all opts to behave in the way
you want them to. And may also require back-end changes to make sure
those registers are saved in the right frame, or reserved from
register allocation, or popped back after the call, etc.

> The Clang impact is relatively minor; it is focused on allowing the Swift
runtime to define functions that use the convention.  It adds a new calling
convention attribute, a few new parameter attributes constrained to that calling
convention, and some relatively un-invasive call lowering code in IR generation.
This sounds like a normal change to support language perks, no big
deal. But I'm not a Clang expert, nor I've seen the code.

>   - Using sret together with a direct result may or may not “just
work".  I certainly don’t see a reason why it shouldn’t work in the
middle-end.  Obviously, some targets can’t support it, but we can avoid doing
this on those targets.
All sret problems I've seen were back-end related (ABI conformance).
But I wasn't paying attention to the middle-end.

>   - Opting in to the two argument treatments requires new parameter
attributes.  We discussed using separate calling conventions; unfortunately,
error and context arguments can appear either separately or together, so we’d
really need several new conventions for all the valid combinations. 
Furthermore, calling a context-free function with an ignored context argument
could turn into a call to a function using a mismatched calling convention,
which LLVM IR generally treats as undefined behavior.  Also, it wasn’t obvious
that just a calling convention would be sufficient for the error treatment; see
the next bullet.
Why not treat context and error like C's default arguments? Or like
named arguments in Python?

Surely the front-end can easily re-order the arguments (according to
some ABI) and make sure every function that may be called with
context/error has it as the last arguments, and default them to null.
You can then later do an inter-procedural pass to clean it up for all
static functions that are never called with those arguments, etc.

>   - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
I think this logic is too high-level for the back-end to deal with.
This looks like a simple run of the mill pointer argument that can be
null (and is by default), but if it's not, the callee can change the
object pointed by but not the pointer itself, ie, "void foo(exception
* const Error = null)". I don't understand why you need this argument
to be of a special kind of SDNode.

cheers,
--renato

I've done the "environment passed in a callee-save register" thing
before,
just by using the C compiler's ability to reserve a register and map a
particular C global variable to it.

As you say, there is then no problem when you call code (let's call it
"library code") which doesn't expect it. The library code just
automatically saves and restores that register if it needs it.

However -- and probably you've thought of this -- there is a problem with
callbacks from library code that doesn't know about the environment
argument. The library might save the environment register, put something
else there, and then call back to your code that expects the environment to
be set up. Boom!

This callback might be a function that you passed explicitly as an
argument, a function pointed to by a global hook, or a virtual function of
an object you passed (derived from a base class that the library knows
about).

Any such callbacks need to either 1) not use the environment register, or
2) set up the environment register from somewhere else before using it or
calling other code that uses it, or 3) be a wrapper/thunk that sets up the
environment register before calling the real function.


On Wed, Mar 2, 2016 at 4:14 AM, John McCall via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi, all.
>
> Swift uses a non-standard calling convention on its supported platforms.
> Implementing this calling convention requires support from LLVM and (to a
> lesser degree) Clang.  If necessary, we’re willing to keep that support in
> “private” branches of LLVM and Clang, but we feel it would be better to
> introduce it in trunk, both to (1) minimize the differences between our
> branches and trunk and (2) allow other language implementations to take
> advantage of that support.
>
> We don’t expect this to be particularly controversial, at least in the
> abstract, since LLVM already includes support for a number of variant,
> language-specific calling conventions.  Some of Swift's variations are
more
> invasive than those existing conventions, however, so we want to make sure
> the community is on board before we start landing patches or sending them
> out for review.
>
> Here’s a brief technical summary of the convention:
>
> In general, the calling convention lowers onto an existing C calling
> convention; let’s call this the “intermediary convention”.  The
> intermediary convention is not necessarily the target platform’s standard C
> convention; for example, we intend to use a VFP convention on iOS ARM
> targets.  Aggregate arguments and results are translated to sequences of
> scalar types (possibly just an indirect argument/sret pointer) and, for the
> most part, passed and returned using the intermediary convention’s rules
> for a function with that signature.  For example, if struct A expands to
> the sequence [i32,float,i32], a function type like (A,Int64) -> Bool)
would
> be lowered basically like the C function type bool(*)(int32_t, float,
> int32_t, int64_t).
>
> There are four general points of deviation from the intermediary
> convention:
>
>   - We sometimes want to return more values in registers than the
> convention normally does, and we want to be able to use both integer and
> floating-point registers.  For example, we want to return a value of struct
> A, above, purely in registers.  For the most part, I don’t think this is a
> problem to layer on to an existing IR convention: C frontends will
> generally use explicit sret arguments when the convention requires them,
> and so the Swift lowering will produce result types that don’t have legal
> interpretations as direct results under the C convention.  But we can use a
> different IR convention if it’s necessary to disambiguate Swift’s desired
> treatment from the target's normal attempts to retroactively match the
C
> convention.
>
>   - We sometimes have both direct results and indirect results.  It would
> be nice to take advantage of the sret convention even in the presence of
> direct results on targets that do use a different (profitable) ABI
> treatment for it.  I don’t know how well-supported this is in LLVM.
>
>   - We want a special “context” treatment for a certain argument.  A
> pointer-sized value is passed in an integer register; the same value should
> be present in that register after the call.  In some cases, the caller may
> pass a context argument to a function that doesn’t expect one, and this
> should not trigger undefined behavior.  Both of these rules suggest that
> the context argument be passed in a register which is normally callee-save.
>
>   - We want a special “error” treatment for a certain argument/result.  A
> pointer-sized value is passed in an integer register; a different value may
> be present in that register after the call.  Much like the context
> treatment, the caller may use the error treatment with a function that
> doesn’t expect it; this should not trigger undefined behavior, and the
> existing value should be left in place.  Like the context treatment, this
> suggests that the error value be passed and returned in a register which is
> normally callee-save.
>
> Here’s a brief summary of the expected code impact for this.
>
> The Clang impact is relatively minor; it is focused on allowing the Swift
> runtime to define functions that use the convention.  It adds a new calling
> convention attribute, a few new parameter attributes constrained to that
> calling convention, and some relatively un-invasive call lowering code in
> IR generation.
>
> The LLVM impact is somewhat larger.
>
> Three things in the convention require a possible change to IR:
>
>   - Using sret together with a direct result may or may not “just
work".
> I certainly don’t see a reason why it shouldn’t work in the middle-end.
> Obviously, some targets can’t support it, but we can avoid doing this on
> those targets.
>
>   - Opting in to the two argument treatments requires new parameter
> attributes.  We discussed using separate calling conventions;
> unfortunately, error and context arguments can appear either separately or
> together, so we’d really need several new conventions for all the valid
> combinations.  Furthermore, calling a context-free function with an ignored
> context argument could turn into a call to a function using a mismatched
> calling convention, which LLVM IR generally treats as undefined behavior.
> Also, it wasn’t obvious that just a calling convention would be sufficient
> for the error treatment; see the next bullet.
>
>   - The “error” treatment requires some way to (1) pass and receive the
> value in the caller and (2) receive and change the value in the callee.
> The best way we could think of to represent this was to pretend that the
> argument is actually passed indirectly; the value is “passed” by storing to
> the pointer and “received” by loading from it.  To simplify backend
> lowering, we require the argument to be a special kind of swifterror alloca
> that can only be loaded, stored, and passed as a swifterror argument; in
> the callee, swifterror arguments have similar restrictions.  This ends up
> being fairly invasive in the backend, unfortunately.
>
> The convention also requires a few changes to the targets that support the
> convention, to deal with the context and error treatments and to return
> more values in registers.
>
> Anyway, I would appreciate your thoughts.
>
> John.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160302/6fc70dbc/attachment.html>

> On Mar 2, 2016, at 3:41 AM, Bruce Hoult <bruce at hoult.org> wrote:
> 
> I've done the "environment passed in a callee-save register"
thing before, just by using the C compiler's ability to reserve a register
and map a particular C global variable to it.
> 
> As you say, there is then no problem when you call code (let's call it
"library code") which doesn't expect it. The library code just
automatically saves and restores that register if it needs it.
> 
> However -- and probably you've thought of this -- there is a problem
with callbacks from library code that doesn't know about the environment
argument. The library might save the environment register, put something else
there, and then call back to your code that expects the environment to be set
up. Boom!
Yes, that’s a well-known problem with trying to reserve a register for a
ubiquitous environment.  That’s not what we’re doing here, though.  The error
result is more like a special argument / result to the function, in as much
it's only actually required that the value be in that register at the call
boundary.

Swift uses a non-zero-cost exceptions scheme for its primary error handling;
this result is used to indicate whether (and what) a function throws.  The basic
idea is that the callee sets the register to either null, meaning it didn’t
throw, or an error value, meaning it did; except actually it’s better for code
size if the caller sets the register to null on entry.

The considerations on the choice of register are as follows:

1. We want to be able to freely convert an opaque function value that’s known
not to throw to a function that can.  The idea here is that the caller
initializes the register to null.  If the callee is dynamically
potentially-throwing, it expects the register to be null on entry and may or may
not set it to be some other value.  If the callee is not dynamically
potentially-throwing, it leaves the register alone because it considers it to be
callee-save.  So there’s a hard requirement that whatever register we choose be
considered callee-save by the Swift convention.

2. Swift code frequently calls C code.  It’s good for performance and code size
if a function doesn’t have to save and restore the error-result register just
because it’s calling a C function.  So we really want it to be callee-save in
the C convention, too.  This also means that we don’t have to worry about the
dynamic linker messing us up.

3. We don’t want to penalize other Swift functions by claiming an
argument/result register that they would otherwise use.  Of course, they
wouldn’t normally use a callee-save register.

John.
> This callback might be a function that you passed explicitly as an
argument, a function pointed to by a global hook, or a virtual function of an
object you passed (derived from a base class that the library knows about).
> 
> Any such callbacks need to either 1) not use the environment register, or
2) set up the environment register from somewhere else before using it or
calling other code that uses it, or 3) be a wrapper/thunk that sets up the
environment register before calling the real function.
> 
> 
> On Wed, Mar 2, 2016 at 4:14 AM, John McCall via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi, all.
> 
> Swift uses a non-standard calling convention on its supported platforms. 
Implementing this calling convention requires support from LLVM and (to a lesser
degree) Clang.  If necessary, we’re willing to keep that support in “private”
branches of LLVM and Clang, but we feel it would be better to introduce it in
trunk, both to (1) minimize the differences between our branches and trunk and
(2) allow other language implementations to take advantage of that support.
> 
> We don’t expect this to be particularly controversial, at least in the
abstract, since LLVM already includes support for a number of variant,
language-specific calling conventions.  Some of Swift's variations are more
invasive than those existing conventions, however, so we want to make sure the
community is on board before we start landing patches or sending them out for
review.
> 
> Here’s a brief technical summary of the convention:
> 
> In general, the calling convention lowers onto an existing C calling
convention; let’s call this the “intermediary convention”.  The intermediary
convention is not necessarily the target platform’s standard C convention; for
example, we intend to use a VFP convention on iOS ARM targets.  Aggregate
arguments and results are translated to sequences of scalar types (possibly just
an indirect argument/sret pointer) and, for the most part, passed and returned
using the intermediary convention’s rules for a function with that signature. 
For example, if struct A expands to the sequence [i32,float,i32], a function
type like (A,Int64) -> Bool) would be lowered basically like the C function
type bool(*)(int32_t, float, int32_t, int64_t).
> 
> There are four general points of deviation from the intermediary
convention:
> 
>   - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
> 
>   - We sometimes have both direct results and indirect results.  It would
be nice to take advantage of the sret convention even in the presence of direct
results on targets that do use a different (profitable) ABI treatment for it.  I
don’t know how well-supported this is in LLVM.
> 
>   - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
> 
>   - We want a special “error” treatment for a certain argument/result.  A
pointer-sized value is passed in an integer register; a different value may be
present in that register after the call.  Much like the context treatment, the
caller may use the error treatment with a function that doesn’t expect it; this
should not trigger undefined behavior, and the existing value should be left in
place.  Like the context treatment, this suggests that the error value be passed
and returned in a register which is normally callee-save.
> 
> Here’s a brief summary of the expected code impact for this.
> 
> The Clang impact is relatively minor; it is focused on allowing the Swift
runtime to define functions that use the convention.  It adds a new calling
convention attribute, a few new parameter attributes constrained to that calling
convention, and some relatively un-invasive call lowering code in IR generation.
> 
> The LLVM impact is somewhat larger.
> 
> Three things in the convention require a possible change to IR:
> 
>   - Using sret together with a direct result may or may not “just
work".  I certainly don’t see a reason why it shouldn’t work in the
middle-end.  Obviously, some targets can’t support it, but we can avoid doing
this on those targets.
> 
>   - Opting in to the two argument treatments requires new parameter
attributes.  We discussed using separate calling conventions; unfortunately,
error and context arguments can appear either separately or together, so we’d
really need several new conventions for all the valid combinations. 
Furthermore, calling a context-free function with an ignored context argument
could turn into a call to a function using a mismatched calling convention,
which LLVM IR generally treats as undefined behavior.  Also, it wasn’t obvious
that just a calling convention would be sufficient for the error treatment; see
the next bullet.
> 
>   - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
> 
> The convention also requires a few changes to the targets that support the
convention, to deal with the context and error treatments and to return more
values in registers.
> 
> Anyway, I would appreciate your thoughts.
> 
> John.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160302/4371fbf6/attachment-0001.html>

> On Mar 2, 2016, at 1:33 AM, Renato Golin <renato.golin at linaro.org>
wrote:
> 
> On 2 March 2016 at 01:14, John McCall via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> Hi, all.
>>  - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
> 
> Is this a back-end decision, or do you expect the front-end to tell
> the back-end (via annotation) which parameters will be in regs? Unless
> you also have back-end patches, I don't think the latter is going to
> work well. For example, the ARM back-end has a huge section related to
> passing structures in registers, which conforms to the ARM EABI, not
> necessarily your Swift ABI.
> 
> Not to mention that this creates the versioning problem, where two
> different LLVM releases can produce slightly different PCS register
> usage (due to new features or bugs), and thus require re-compilation
> of all libraries. This, however, is not a problem for your current
> request, just a comment.
The frontend will not tell the backend explicitly which parameters will be
in registers; it will just pass a bunch of independent scalar values, and
the backend will assign them to registers or the stack as appropriate.

Our intent is to completely bypass all of the passing-structures-in-registers
code in the backend by simply not exposing the backend to any parameters
of aggregate type.  The frontend will turn a struct into (say) an i32, a float,
and an i8; if the first two get passed in registers and the last gets passed
on the stack, so be it.

The only difficulty with this plan is that, when we have multiple results, we
don’t have a choice but to return a struct type.  To the extent that backends
try to infer that the function actually needs to be sret, instead of just trying
to find a way to return all the components of the struct type in appropriate
registers, that will be sub-optimal for us.  If that’s a pervasive problem, then
we probably just need to introduce a swift calling convention in LLVM.
>>  - We sometimes have both direct results and indirect results.  It
would be nice to take advantage of the sret convention even in the presence of
direct results on targets that do use a different (profitable) ABI treatment for
it.  I don’t know how well-supported this is in LLVM.
> 
> I'm not sure what you mean by direct or indirect results here. But if
> this is a language feature, as long as the IR semantics is correct, I
> don't see any problem.
A direct result is something that’s returned in registers.  An indirect
result is something that’s returned by storing it in an implicit out-parameter.
I would like to be able to form calls like this:

  %temp = alloca %my_big_struct_type
  call i32 @my_swift_function(sret %my_big_struct_type* %temp)

This doesn’t normally happen today in LLVM IR because when C frontends
use an sret result, they set the direct IR result to void.

Like I said, I don’t think this is a serious problem, but I wanted to float the
idea
before assuming that.
>>  - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
> 
> I think it's going to be harder to get all opts to behave in the way
> you want them to. And may also require back-end changes to make sure
> those registers are saved in the right frame, or reserved from
> register allocation, or popped back after the call, etc.
I don’t expect the optimizer to be a problem, but I just realized that the main
reason is something I didn’t talk about in my first post.  See below.

That this will require some support from the backend is a given.
>> The Clang impact is relatively minor; it is focused on allowing the
Swift runtime to define functions that use the convention.  It adds a new
calling convention attribute, a few new parameter attributes constrained to that
calling convention, and some relatively un-invasive call lowering code in IR
generation.
> 
> This sounds like a normal change to support language perks, no big
> deal. But I'm not a Clang expert, nor I've seen the code.
> 
> 
>>  - Using sret together with a direct result may or may not “just
work".  I certainly don’t see a reason why it shouldn’t work in the
middle-end.  Obviously, some targets can’t support it, but we can avoid doing
this on those targets.
> 
> All sret problems I've seen were back-end related (ABI conformance).
> But I wasn't paying attention to the middle-end.
> 
> 
>>  - Opting in to the two argument treatments requires new parameter
attributes.  We discussed using separate calling conventions; unfortunately,
error and context arguments can appear either separately or together, so we’d
really need several new conventions for all the valid combinations. 
Furthermore, calling a context-free function with an ignored context argument
could turn into a call to a function using a mismatched calling convention,
which LLVM IR generally treats as undefined behavior.  Also, it wasn’t obvious
that just a calling convention would be sufficient for the error treatment; see
the next bullet.
> 
> Why not treat context and error like C's default arguments? Or like
> named arguments in Python?
> 
> Surely the front-end can easily re-order the arguments (according to
> some ABI) and make sure every function that may be called with
> context/error has it as the last arguments, and default them to null.
> You can then later do an inter-procedural pass to clean it up for all
> static functions that are never called with those arguments, etc.
Oh, sorry, I forgot to talk about that.  Yes, the frontend already rearranges
these arguments to the end, which means the optimizer’s default behavior
of silently dropping extra call arguments ends up doing the right thing.

I’m reluctant to say that the convention always requires these arguments.
If we have to do that, we can, but I’d rather not; it would involve generating
a lot of unnecessary IR and would probably create unnecessary
code-generation differences, and I don’t think it would be sufficient for
error results anyway.
>>  - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
> 
> I think this logic is too high-level for the back-end to deal with.
> This looks like a simple run of the mill pointer argument that can be
> null (and is by default), but if it's not, the callee can change the
> object pointed by but not the pointer itself, ie, "void foo(exception
> * const Error = null)". I don't understand why you need this
argument
> to be of a special kind of SDNode.
We don’t want checking or setting the error result to actually involve memory
access.

An alternative to the pseudo-indirect-result approach would be to model
the result as an explicit result.  That would really mess up the IR, though.
The ability to call a non-throwing function as a throwing function means
we’d have to provide this extra explicit result on every single function with
the Swift convention, because the optimizer is definitely not going to
gracefully handle result-type mismatches; so even a function as simple as
  func foo() -> Int32
would have to be lowered into IR as
  define { i32, i8* } @foo(i8*)

John.

On Tue, Mar 1, 2016 at 5:14 PM, John McCall via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> There are four general points of deviation from the intermediary
> convention:
>
>   - We sometimes want to return more values in registers than the
> convention normally does, and we want to be able to use both integer and
> floating-point registers.  For example, we want to return a value of struct
> A, above, purely in registers.  For the most part, I don’t think this is a
> problem to layer on to an existing IR convention: C frontends will
> generally use explicit sret arguments when the convention requires them,
> and so the Swift lowering will produce result types that don’t have legal
> interpretations as direct results under the C convention.  But we can use a
> different IR convention if it’s necessary to disambiguate Swift’s desired
> treatment from the target's normal attempts to retroactively match the
C
> convention.
>
You're suggesting that backends shouldn't try to turn returns of {i32,
float, i32} into sret automatically if the C ABI would require that struct
to be returned indirectly. I know there are many users of LLVM out there
that wish that LLVM would just follow the C ABI for them in "simple"
cases
like this, even though in general it's a lost cause. I think if you hide
this new behavior under your own swiftcc then we can keep those people
happy, ish.

  - We sometimes have both direct results and indirect results.  It
would
> be nice to take advantage of the sret convention even in the presence of
> direct results on targets that do use a different (profitable) ABI
> treatment for it.  I don’t know how well-supported this is in LLVM.
>
LLVM insists that sret functions be void because the C convention requires
the sret pointer to be returned in the normal return register. X86 Sys V
requires this, though LLVM does not leverage it, and was non-conforming for
most of its life. I don't see why Swift would need to use the 'sret'
attribute for indirect results, though, if it doesn't need to conform to
that part of the x86 convention. Am I missing something profitable about
reusing our sret support?

>   - We want a special “context” treatment for a certain argument.  A
> pointer-sized value is passed in an integer register; the same value should
> be present in that register after the call.  In some cases, the caller may
> pass a context argument to a function that doesn’t expect one, and this
> should not trigger undefined behavior.  Both of these rules suggest that
> the context argument be passed in a register which is normally callee-save.
>
As discussed later, these arguments would come last. I thought it was
already legal to call a C function with too many arguments without invoking
UB, so I think we have to keep this working in LLVM anyway.

>   - The “error” treatment requires some way to (1) pass and receive the
> value in the caller and (2) receive and change the value in the callee.
> The best way we could think of to represent this was to pretend that the
> argument is actually passed indirectly; the value is “passed” by storing to
> the pointer and “received” by loading from it.  To simplify backend
> lowering, we require the argument to be a special kind of swifterror alloca
> that can only be loaded, stored, and passed as a swifterror argument; in
> the callee, swifterror arguments have similar restrictions.  This ends up
> being fairly invasive in the backend, unfortunately.
>
This seems unfortunate. I guess you've already rejected returning an FCA. I
wonder if we should ever go back to the world of "only calls can produce
multiple values" as a special case, since that's what really happens at
the
MI level. I wonder if operand bundles or tokens could help solve this
problem.

---

In general, yes, I'm in favor of upstreaming this Swift CC support. The
main awkwardness here to me is the error value which is in memory in the
mid-level, but in registers in the backend. I feel like
intrinsic/token/bundle/glue stuff might actually be better.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160302/b1ed20eb/attachment.html>

> On Mar 2, 2016, at 12:00 PM, Reid Kleckner <rnk at google.com> wrote:
> On Tue, Mar 1, 2016 at 5:14 PM, John McCall via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> There are four general points of deviation from the intermediary
convention:
> 
>   - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
> 
> You're suggesting that backends shouldn't try to turn returns of
{i32, float, i32} into sret automatically if the C ABI would require that struct
to be returned indirectly. I know there are many users of LLVM out there that
wish that LLVM would just follow the C ABI for them in "simple" cases
like this, even though in general it's a lost cause. I think if you hide
this new behavior under your own swiftcc then we can keep those people happy,
ish.
Yes, that may be best.
>   - We sometimes have both direct results and indirect results.  It would
be nice to take advantage of the sret convention even in the presence of direct
results on targets that do use a different (profitable) ABI treatment for it.  I
don’t know how well-supported this is in LLVM.
> 
> LLVM insists that sret functions be void because the C convention requires
the sret pointer to be returned in the normal return register. X86 Sys V
requires this, though LLVM does not leverage it, and was non-conforming for most
of its life. I don't see why Swift would need to use the 'sret'
attribute for indirect results, though, if it doesn't need to conform to
that part of the x86 convention. Am I missing something profitable about reusing
our sret support?
For most platforms, it’s not profitable.  On some platforms, there’s a register
reserved for the sret argument; it would be nice to take advantage of that for
several reasons, including just being nicer to existing tools (debuggers, etc.)
in common cases.
>   - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
> 
> As discussed later, these arguments would come last. I thought it was
already legal to call a C function with too many arguments without invoking UB,
so I think we have to keep this working in LLVM anyway.
Formally, no, it’s UB to call (non-variadic, of course) C functions with extra
arguments.  But you’re right, it does generally work at runtime, and LLVM
doesn’t get in the way.
 
>   - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
> 
> This seems unfortunate. I guess you've already rejected returning an
FCA.
See one of my recent responses to Renato.  It’s really hard to make that work
cleanly with the ability to call functions that lack the result.
> I wonder if we should ever go back to the world of "only calls can
produce multiple values" as a special case, since that's what really
happens at the MI level. I wonder if operand bundles or tokens could help solve
this problem.
> 
> ---
> 
> In general, yes, I'm in favor of upstreaming this Swift CC support. The
main awkwardness here to me is the error value which is in memory in the
mid-level, but in registers in the backend. I feel like
intrinsic/token/bundle/glue stuff might actually be better.
I agree that the error-result stuff is the most intrinsically awkward part of
our current patch, and I would love to find alternatives.

John.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160302/460a953f/attachment.html>

On Tue, Mar 1, 2016 at 5:14 PM, John McCall via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
> Hi, all.
>
> Swift uses a non-standard calling convention on its supported platforms. 
Implementing this calling convention requires support from LLVM and (to a lesser
degree) Clang.  If necessary, we’re willing to keep that support in “private”
branches of LLVM and Clang, but we feel it would be better to introduce it in
trunk, both to (1) minimize the differences between our branches and trunk and
(2) allow other language implementations to take advantage of that support.
>
> We don’t expect this to be particularly controversial, at least in the
abstract, since LLVM already includes support for a number of variant,
language-specific calling conventions.  Some of Swift's variations are more
invasive than those existing conventions, however, so we want to make sure the
community is on board before we start landing patches or sending them out for
review.
>
> Here’s a brief technical summary of the convention:
>
> In general, the calling convention lowers onto an existing C calling
convention; let’s call this the “intermediary convention”.  The intermediary
convention is not necessarily the target platform’s standard C convention; for
example, we intend to use a VFP convention on iOS ARM targets.  Aggregate
arguments and results are translated to sequences of scalar types (possibly just
an indirect argument/sret pointer) and, for the most part, passed and returned
using the intermediary convention’s rules for a function with that signature. 
For example, if struct A expands to the sequence [i32,float,i32], a function
type like (A,Int64) -> Bool) would be lowered basically like the C function
type bool(*)(int32_t, float, int32_t, int64_t).
>
> There are four general points of deviation from the intermediary
convention:
>
>   - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
>
>   - We sometimes have both direct results and indirect results.  It would
be nice to take advantage of the sret convention even in the presence of direct
results on targets that do use a different (profitable) ABI treatment for it.  I
don’t know how well-supported this is in LLVM.
>
>   - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
>
>   - We want a special “error” treatment for a certain argument/result.  A
pointer-sized value is passed in an integer register; a different value may be
present in that register after the call.  Much like the context treatment, the
caller may use the error treatment with a function that doesn’t expect it; this
should not trigger undefined behavior, and the existing value should be left in
place.  Like the context treatment, this suggests that the error value be passed
and returned in a register which is normally callee-save.
>
> Here’s a brief summary of the expected code impact for this.
>
> The Clang impact is relatively minor; it is focused on allowing the Swift
runtime to define functions that use the convention.  It adds a new calling
convention attribute, a few new parameter attributes constrained to that calling
convention, and some relatively un-invasive call lowering code in IR generation.
>
> The LLVM impact is somewhat larger.
>
> Three things in the convention require a possible change to IR:
>
>   - Using sret together with a direct result may or may not “just
work".  I certainly don’t see a reason why it shouldn’t work in the
middle-end.  Obviously, some targets can’t support it, but we can avoid doing
this on those targets.
>
>   - Opting in to the two argument treatments requires new parameter
attributes.  We discussed using separate calling conventions; unfortunately,
error and context arguments can appear either separately or together, so we’d
really need several new conventions for all the valid combinations. 
Furthermore, calling a context-free function with an ignored context argument
could turn into a call to a function using a mismatched calling convention,
which LLVM IR generally treats as undefined behavior.  Also, it wasn’t obvious
that just a calling convention would be sufficient for the error treatment; see
the next bullet.
>
>   - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
>
> The convention also requires a few changes to the targets that support the
convention, to deal with the context and error treatments and to return more
values in registers.
>
> Anyway, I would appreciate your thoughts.
I consider it completely reasonable for Clang to support this calling
convention and the associated attributes, especially given the minor
impact you describe above.

> On Mar 2, 2016, at 12:22 PM, Richard Smith <richard at metafoo.co.uk>
wrote:
> On Tue, Mar 1, 2016 at 5:14 PM, John McCall via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
>> Hi, all.
>> 
>> Swift uses a non-standard calling convention on its supported
platforms.  Implementing this calling convention requires support from LLVM and
(to a lesser degree) Clang.  If necessary, we’re willing to keep that support in
“private” branches of LLVM and Clang, but we feel it would be better to
introduce it in trunk, both to (1) minimize the differences between our branches
and trunk and (2) allow other language implementations to take advantage of that
support.
>> 
>> We don’t expect this to be particularly controversial, at least in the
abstract, since LLVM already includes support for a number of variant,
language-specific calling conventions.  Some of Swift's variations are more
invasive than those existing conventions, however, so we want to make sure the
community is on board before we start landing patches or sending them out for
review.
>> 
>> Here’s a brief technical summary of the convention:
>> 
>> In general, the calling convention lowers onto an existing C calling
convention; let’s call this the “intermediary convention”.  The intermediary
convention is not necessarily the target platform’s standard C convention; for
example, we intend to use a VFP convention on iOS ARM targets.  Aggregate
arguments and results are translated to sequences of scalar types (possibly just
an indirect argument/sret pointer) and, for the most part, passed and returned
using the intermediary convention’s rules for a function with that signature. 
For example, if struct A expands to the sequence [i32,float,i32], a function
type like (A,Int64) -> Bool) would be lowered basically like the C function
type bool(*)(int32_t, float, int32_t, int64_t).
>> 
>> There are four general points of deviation from the intermediary
convention:
>> 
>>  - We sometimes want to return more values in registers than the
convention normally does, and we want to be able to use both integer and
floating-point registers.  For example, we want to return a value of struct A,
above, purely in registers.  For the most part, I don’t think this is a problem
to layer on to an existing IR convention: C frontends will generally use
explicit sret arguments when the convention requires them, and so the Swift
lowering will produce result types that don’t have legal interpretations as
direct results under the C convention.  But we can use a different IR convention
if it’s necessary to disambiguate Swift’s desired treatment from the
target's normal attempts to retroactively match the C convention.
>> 
>>  - We sometimes have both direct results and indirect results.  It
would be nice to take advantage of the sret convention even in the presence of
direct results on targets that do use a different (profitable) ABI treatment for
it.  I don’t know how well-supported this is in LLVM.
>> 
>>  - We want a special “context” treatment for a certain argument.  A
pointer-sized value is passed in an integer register; the same value should be
present in that register after the call.  In some cases, the caller may pass a
context argument to a function that doesn’t expect one, and this should not
trigger undefined behavior.  Both of these rules suggest that the context
argument be passed in a register which is normally callee-save.
>> 
>>  - We want a special “error” treatment for a certain argument/result. 
A pointer-sized value is passed in an integer register; a different value may be
present in that register after the call.  Much like the context treatment, the
caller may use the error treatment with a function that doesn’t expect it; this
should not trigger undefined behavior, and the existing value should be left in
place.  Like the context treatment, this suggests that the error value be passed
and returned in a register which is normally callee-save.
>> 
>> Here’s a brief summary of the expected code impact for this.
>> 
>> The Clang impact is relatively minor; it is focused on allowing the
Swift runtime to define functions that use the convention.  It adds a new
calling convention attribute, a few new parameter attributes constrained to that
calling convention, and some relatively un-invasive call lowering code in IR
generation.
>> 
>> The LLVM impact is somewhat larger.
>> 
>> Three things in the convention require a possible change to IR:
>> 
>>  - Using sret together with a direct result may or may not “just
work".  I certainly don’t see a reason why it shouldn’t work in the
middle-end.  Obviously, some targets can’t support it, but we can avoid doing
this on those targets.
>> 
>>  - Opting in to the two argument treatments requires new parameter
attributes.  We discussed using separate calling conventions; unfortunately,
error and context arguments can appear either separately or together, so we’d
really need several new conventions for all the valid combinations. 
Furthermore, calling a context-free function with an ignored context argument
could turn into a call to a function using a mismatched calling convention,
which LLVM IR generally treats as undefined behavior.  Also, it wasn’t obvious
that just a calling convention would be sufficient for the error treatment; see
the next bullet.
>> 
>>  - The “error” treatment requires some way to (1) pass and receive the
value in the caller and (2) receive and change the value in the callee.  The
best way we could think of to represent this was to pretend that the argument is
actually passed indirectly; the value is “passed” by storing to the pointer and
“received” by loading from it.  To simplify backend lowering, we require the
argument to be a special kind of swifterror alloca that can only be loaded,
stored, and passed as a swifterror argument; in the callee, swifterror arguments
have similar restrictions.  This ends up being fairly invasive in the backend,
unfortunately.
>> 
>> The convention also requires a few changes to the targets that support
the convention, to deal with the context and error treatments and to return more
values in registers.
>> 
>> Anyway, I would appreciate your thoughts.
> 
> I consider it completely reasonable for Clang to support this calling
> convention and the associated attributes, especially given the minor
> impact you describe above.
Great, thanks.

Since the response so far has been very positive on the idea, I think it’s
probably time to start sending out patches for review.  Manman will be leading
that on the LLVM side, since she did most of the work there.  On the Clang side,
I’ll land what I have and then progressively work on it in trunk.

John.