openssh unix dev - Sep 2018 - Concerns about enabling retpolines by default

We recently discovered that our OpenSSH distribution binaries contain
retpoline thunks.  It's due to this 

            OSSH_CHECK_CFLAG_COMPILE([-mfunction-return=thunk]) # gcc
            OSSH_CHECK_CFLAG_COMPILE([-mindirect-branch=thunk]) # gcc

This was quite surprising because at least the GNU/Linux userspace has
no provisions for retpolines.  You also fail to enable -fno-plt, so you
need a special linker that produces non-standard PLT stubs.  (And this
has to be repeated for all system libraries you call.)

In our case, it caused miscompilation because it triggered two different
GCC bugs (which are being fixed upstream):

  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87412>
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87414>

There have been other retpoline bugs in GCC which do not affect the
kernel (or affect only rarely used kernel features), but are potentially
visible in user space, so few distributions will backport those fixes to
their distribution compilers.  My expectation is that most of these
compilers will never get fixed because the goal was to get the kernel to
compile with retpoline support, and nothing more.

Maybe the expectation is that distributions fine-tune their build flags
according to their toolchain support level and simply configure with the
--without-hardening flag, but this still looks like an unnecessary trap
for those who build their OpenSSH binaries themselves.

Thanks,
Florian

Florian Weimer wrote:
> this still looks like an unnecessary trap for those who build their
> OpenSSH binaries themselves.
Do you intend to suggest something or only to create awareness? (Also good.)


//Peter

* Peter Stuge:
> Florian Weimer wrote:
>> this still looks like an unnecessary trap for those who build their
>> OpenSSH binaries themselves.
>
> Do you intend to suggest something or only to create awareness? (Also
> good.)
Oh, sorry for being unclear.  I think it would be best if you dropped
the configure.ac bits that try to enable retpolines.

Thanks,
Florian

On Wed, 26 Sep 2018 at 19:32, Florian Weimer <fweimer at redhat.com>
wrote:
> We recently discovered that our OpenSSH distribution binaries contain
> retpoline thunks.  It's due to this
>
>             OSSH_CHECK_CFLAG_COMPILE([-mfunction-return=thunk]) # gcc
>             OSSH_CHECK_CFLAG_COMPILE([-mindirect-branch=thunk]) # gcc
I was the one who added those.  It was shortly after the disclosure of
Spectre, and the concern was that ssh, sshd and particularly ssh-agent
hold secrets where the disclosure of those across trust boundaries
would be various levels of bad.

The documentation at the time was pretty sparse and it's not much
clearer now.  What should a userspace program do for Spectre?
> There have been other retpoline bugs in GCC which do not affect the
> kernel (or affect only rarely used kernel features), but are potentially
> visible in user space, so few distributions will backport those fixes to
> their distribution compilers.
Can we determine which versions are affected?  If there's one known to
work we can disable the check for versions prior to that.

-- 
Darren Tucker (dtucker at dtucker.net)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

* Darren Tucker:
> On Wed, 26 Sep 2018 at 19:32, Florian Weimer <fweimer at redhat.com>
wrote:
>> We recently discovered that our OpenSSH distribution binaries contain
>> retpoline thunks.  It's due to this
>>
>>             OSSH_CHECK_CFLAG_COMPILE([-mfunction-return=thunk]) # gcc
>>             OSSH_CHECK_CFLAG_COMPILE([-mindirect-branch=thunk]) # gcc
>
> I was the one who added those.  It was shortly after the disclosure of
> Spectre, and the concern was that ssh, sshd and particularly ssh-agent
> hold secrets where the disclosure of those across trust boundaries
> would be various levels of bad.
> 
> The documentation at the time was pretty sparse and it's not much
> clearer now.  What should a userspace program do for Spectre?
Our internal recommendation is: do nothing.  Userspace appears unfixable
without hardware support.

You can try processing data from different trust domains in different
processes, then the kernel mitigations should deliver some protection.
Kind of what ssh-agent does, I guess, or privilege separation.
>> There have been other retpoline bugs in GCC which do not affect the
>> kernel (or affect only rarely used kernel features), but are
potentially
>> visible in user space, so few distributions will backport those fixes
to
>> their distribution compilers.
>
> Can we determine which versions are affected?
I'm afraid not easily.  A lot of distribution compilers have seen some
backports for building the kernel, but the amount of fixes beyond the
initial backport is unclear.
> If there's one known to work we can disable the check for versions
> prior to that.
There is no released GCC version with the fix.

Thanks,
Florian