llvm dev - May 2020 - MTE -- discussion on Exception unwinding ABI

Hi everyone,

I believe the ABI for exception unwinding on a stack tagged with MTE
needs to be clarified -- hopefully we can start the discussion here?
(Please feel free to add people to the thread that you think would be
interested).

I'll outline some possible approaches that I think seem good below, I
know Evgenii and Peter have done a lot of investigation in this area for
HWASAN, so I'm hoping you can see any problems I've missed, or indeed
propose something better if there's another approach ;-)


---  Extra restriction on landing pads ---
No matter what the approach, I believe it would be helpful to require
stacks tagged for MTE to have landing pads which
1) Do not adjust SP to no longer cover local variables before calling
    _Unwind_Resume.
2) Never tail-call _Unwind_Resume.

With these restrictions (that Peter mentioned in the email I link below)
we can avoid the need to instrument landing pads and just rely on
instrumentation in the unwinder that untags stacks as it goes.
http://lists.llvm.org/pipermail/llvm-dev/2019-November/136807.html
I don't believe this should be a problem, since it seems that neither
LLVM nor GCC ever do this at the moment -- hence a new restriction
should not be too onerous.


---  Which part of the toolchain should untag the stack ---
In contrast to HWASAN I believe that the stack should be untagged in the
unwinder itself, rather than in a personality function.

For HWASAN the LLVM implementation of the untagging personality wrapper
introduces a new requirement that the frame record appears after any
locals. This is not required by AAPCS and GCC currently breaks that
restriction in most call-frames.
This new restriction was introduced so that the personality function can
find the range of the stack that needs to be untagged since there is no
API for the personality function to request this information from the
unwinder.

The unwinder already has access to information about the range of each
frame and its local variables.

As a minor additional point, the personality routine is designed as a
language-specific part of the unwinding mechanism, which clashes with
the language-independence of the MTE extension.


--- How should frames be marked as requiring untagging ---
I can see two options that make sense to me.
I prefer approach 1 but believe either is a good approach.

## Approach 1.
My preferred approach is to mark each frame that needs untagging with an
extra character (say 'T' for "Tagged") in the
"Augmentation String" of
the CFI information for each function (i.e. stored in the CIE).
This is similar to the way that AArch64 marks that a frame has used the
B-key for Pointer Authentication.

With this approach we
- Store the information in a language-independent place.
- Pass the information direct to the unwinder using the mechanism and
   code framework that is already in place for Pointer Authentication.

The downsides of this approach (in relation to option 2 below) are only
related to attempting to link a stack-tagged object file with a
non-MTE-aware toolchain.
- A non-MTE-aware linker will catch and warn about the extra character
   in the Augmentation String (with an error saying ~no .eh_frame_hdr
   table will be created~). Yet it will still generate a binary that
   appears to work without untagging the stack.
- While a non-MTE-aware unwinder will silently not untag the stack,
   leading to runtime MTE faults.

To mitigate this downside we could make a small test to check that the
users build environment is set up correctly -- then there's at least a
simple advertised procedure that the user can use to double check their
unwinder is MTE-aware.


## Approach 2.
An alternative is to introduce new values into the _Unwind_Reason_Code
enum so that a personality routine can inform the unwinder that a given
function has its stack tagged for MTE.

The untagging will still be done in the general unwinder, but those
functions which tag their stack use a different personality routine
(e.g. __gxx_personality_mte_v0) that returns a new value in this enum
indicating a combination of the value the old one returns plus a bit
indicating whether this function has a tagged stack.

With this approach, when attempting to link a stack-tagged object file
with a non-MTE-aware toolchain:
- The linker will complain about a missing personality function (and
   hence not create a binary).
(This benefit derives from the fact that we would be creating a *new*
personality function to annotate functions with).

However, this approach has the downsides that:
- Each language that wants to implememnt MTE stack tagging will need to
   write a new personality routine to return this extra enum.
- We're adding a new way for the compiler and unwinder to communicate
   (with this new enum value).


--- Summary ---
I'd appreciate any feedback on this, especially w.r.t. how much people
are worried about users trying to create a stack-tagged binary using old
unwinders/linkers.
If this does happen with approach 1., then a users program could fail at
runtime with an MTE exception in a strange place.

If that isn't much of a concern then implementing using the
"Augmentation String" seems like the approach which fits the use-case
best (being language-independent) and which introduces less complexity
to the ABI and code.


Regards,
MM

On Fri, May 1, 2020 at 7:25 AM Matthew Malcomson <matthew.malcomson at
arm.com>
wrote:
> Hi everyone,
>
> I believe the ABI for exception unwinding on a stack tagged with MTE
> needs to be clarified -- hopefully we can start the discussion here?
> (Please feel free to add people to the thread that you think would be
> interested).
>
> I'll outline some possible approaches that I think seem good below, I
> know Evgenii and Peter have done a lot of investigation in this area for
> HWASAN, so I'm hoping you can see any problems I've missed, or
indeed
> propose something better if there's another approach ;-)
>
>
> ---  Extra restriction on landing pads ---
> No matter what the approach, I believe it would be helpful to require
> stacks tagged for MTE to have landing pads which
> 1) Do not adjust SP to no longer cover local variables before calling
>     _Unwind_Resume.
> 2) Never tail-call _Unwind_Resume.
>
> With these restrictions (that Peter mentioned in the email I link below)
> we can avoid the need to instrument landing pads and just rely on
> instrumentation in the unwinder that untags stacks as it goes.
> http://lists.llvm.org/pipermail/llvm-dev/2019-November/136807.html
> I don't believe this should be a problem, since it seems that neither
> LLVM nor GCC ever do this at the moment -- hence a new restriction
> should not be too onerous.
>
>
> ---  Which part of the toolchain should untag the stack ---
> In contrast to HWASAN I believe that the stack should be untagged in the
> unwinder itself, rather than in a personality function.
>
> For HWASAN the LLVM implementation of the untagging personality wrapper
> introduces a new requirement that the frame record appears after any
> locals. This is not required by AAPCS and GCC currently breaks that
> restriction in most call-frames.
> This new restriction was introduced so that the personality function can
> find the range of the stack that needs to be untagged since there is no
> API for the personality function to request this information from the
> unwinder.
>
> The unwinder already has access to information about the range of each
> frame and its local variables.
>
> As a minor additional point, the personality routine is designed as a
> language-specific part of the unwinding mechanism, which clashes with
> the language-independence of the MTE extension.
>
>
> --- How should frames be marked as requiring untagging ---
> I can see two options that make sense to me.
> I prefer approach 1 but believe either is a good approach.
>
> ## Approach 1.
> My preferred approach is to mark each frame that needs untagging with an
> extra character (say 'T' for "Tagged") in the
"Augmentation String" of
> the CFI information for each function (i.e. stored in the CIE).
> This is similar to the way that AArch64 marks that a frame has used the
> B-key for Pointer Authentication.
>
> With this approach we
> - Store the information in a language-independent place.
> - Pass the information direct to the unwinder using the mechanism and
>    code framework that is already in place for Pointer Authentication.
>
> The downsides of this approach (in relation to option 2 below) are only
> related to attempting to link a stack-tagged object file with a
> non-MTE-aware toolchain.
> - A non-MTE-aware linker will catch and warn about the extra character
>    in the Augmentation String (with an error saying ~no .eh_frame_hdr
>    table will be created~). Yet it will still generate a binary that
>    appears to work without untagging the stack.
> - While a non-MTE-aware unwinder will silently not untag the stack,
>    leading to runtime MTE faults.
>
> To mitigate this downside we could make a small test to check that the
> users build environment is set up correctly -- then there's at least a
> simple advertised procedure that the user can use to double check their
> unwinder is MTE-aware.
>
>
> ## Approach 2.
> An alternative is to introduce new values into the _Unwind_Reason_Code
> enum so that a personality routine can inform the unwinder that a given
> function has its stack tagged for MTE.
>
> The untagging will still be done in the general unwinder, but those
> functions which tag their stack use a different personality routine
> (e.g. __gxx_personality_mte_v0) that returns a new value in this enum
> indicating a combination of the value the old one returns plus a bit
> indicating whether this function has a tagged stack.
>
> With this approach, when attempting to link a stack-tagged object file
> with a non-MTE-aware toolchain:
> - The linker will complain about a missing personality function (and
>    hence not create a binary).
> (This benefit derives from the fact that we would be creating a *new*
> personality function to annotate functions with).
>
> However, this approach has the downsides that:
> - Each language that wants to implememnt MTE stack tagging will need to
>    write a new personality routine to return this extra enum.
> - We're adding a new way for the compiler and unwinder to communicate
>    (with this new enum value).
>
>
> --- Summary ---
> I'd appreciate any feedback on this, especially w.r.t. how much people
> are worried about users trying to create a stack-tagged binary using old
> unwinders/linkers.
> If this does happen with approach 1., then a users program could fail at
> runtime with an MTE exception in a strange place.
>
> If that isn't much of a concern then implementing using the
> "Augmentation String" seems like the approach which fits the
use-case
> best (being language-independent) and which introduces less complexity
> to the ABI and code.

Approach 1 sounds perfect to me. Conveniently, both unwinders ignore
unrecognized characters in the augmentation string.

In our experience with ASan, errors caused by failing to unpoison/untag a
stack frame are cryptic, and pretty hard to debug. But they can be caused
by a number of other things, not just the unwinder - vfork, longjmp and
friends in libc, custom stack manipulation anywhere (ex. ART has longjmp
implemented in assembly). We could implement a "verified" mode to
catch
these cases - a compilation flag that checks that the entire frame is
SP-accessible at function entry.

We could use the letter "G" as that's what stands for
"tag" in the
instruction mnemonics (STG, IRG).

I had a thought about extending Dwarf with a way to specify a range of
offsets to be untagged within the frame (and default to the entire frame if
not specified). But it feels like the performance savings would not be
worth the extra complexity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200504/8ab8124d/attachment.html>

On 05/05/2020 01:32, Evgenii Stepanov wrote:
> 
> 
> 
> Approach 1 sounds perfect to me. Conveniently, both unwinders ignore 
> unrecognized characters in the augmentation string.
> 
> In our experience with ASan, errors caused by failing to unpoison/untag 
> a stack frame are cryptic, and pretty hard to debug. But they can be 
> caused by a number of other things, not just the unwinder - vfork, 
> longjmp and friends in libc, custom stack manipulation anywhere (ex. ART 
> has longjmp implemented in assembly). We could implement a
"verified"
> mode to catch these cases - a compilation flag that checks that the 
> entire frame is SP-accessible at function entry.
> 
> We could use the letter "G" as that's what stands for
"tag" in the
> instruction mnemonics (STG, IRG).
Awesome -- and the "G" letter seems like a good idea to me.

I'll wait until the end of the week to give others a chance to comment, 
but if no-one objects by then I think we can go ahead with Approach 1 
using the letter "G".
> 
> I had a thought about extending Dwarf with a way to specify a range of 
> offsets to be untagged within the frame (and default to the entire frame 
> if not specified). But it feels like the performance savings would not 
> be worth the extra complexity.
>