freebsd security - Sep 2021 - Important note for future FreeBSD base system OpenSSH update

On 9/12/2021 10:02, Markus Falb wrote:
>> On 09.09.2021, at 20:01, Ed Maste <emaste at freebsd.org> wrote:
>>
>> OpenSSH will disable the ssh-rsa signature scheme by default in the
>> next release.
>>
>> ...
>>
>> To check whether a server is using the weak ssh-rsa public key
>> algorithm, for host authentication, try to connect to it after
>> removing the ssh-rsa algorithm from ssh(1)'s allowed list:
>>
>>     ssh -oHostKeyAlgorithms=-ssh-rsa user at host
> FWIW, some of us may already have dealt with that.
> FIPS enabled RedHat Enterprise Linux (and probably other FIPS enabled
> systems) means effectively no ssh-rsa signature available in the sshd.
> I had that situation at the beginning of the year.
>
> As mentioned, ssh-rsa signature algorithm will stop working, but
> that does not automatically imply that every RSA key must be
> changed to something other. The signature algorithm is not a
> property that is inherent to the key.
>
> That said, existing RSA keys were working fine for me (my openssh
> client was rsa-sha2-256 and rsa-sha2-512 capable) but when I tested
> with some popular windows clients (filezilla, putty) it failed
> (apparently no rsa-sha2 algorithms available).
>
> I found it interesting that mentioned clients were ecdsa
> capable but did not support sha2 signatures with RSA keys.
> Maybe the situation changed in the meantime to the better.
>
> There are 3 scenarios:
>
> 1. both sides support rsa-sha2 signatures -> RSA keys still working
>
> 2. one side does not support sha2 signatures but does support other
> key types -> you can change key type
>
> 3. one side does not support sha2 and no other key type -> you loose
>
> A prominent candidate for 3. would be Cisco IOS
This has come up before with web browsers and is a serious PITA when 
there is no override available for those who need it on a targeted, 
specific basis.

I have in the field a BUNCH of "smart" rack power strips that have
this
problem; their management firmware does NOT support more-modern cipher 
sets and SSL requirements.? I get it, those older SSL versions are 
insecure and we know it.? But when the browser people all decided to 
kill the ability to connect to such servers with no override (that is, 
don't warn, DENY with no option to get around it) all of a sudden 
logging into those strips to change (for example) the name of a socket, 
the alarm limits and similar became literally impossible.? Contacting 
the manufacturer resulted in a middle finger back; "nope, we're not 
releasing new firmware for that."? I've seen the same thing with some 
older OOB management interfaces on server boards; they won't take an 
acceptably-long (by modern standards) HTTPS server key, and thus, same 
problem and same answer from the manufacturer.? These are 
perfectly-serviceable devices in their application and quite-expensive 
to replace when there's nothing wrong with them. On the server boards by 
now they've all been retired as people decided the better power budget 
and performance levels made changing them (and re-purchasing the RAM 
that went on them, which for larger servers is a non-trivial part of the 
total expense) a reasonable proposition.? This of course is not true for 
a smart power strip in the rack and makes both monitoring of energy and 
remote-hard-power-cycle available without a physical site visit or 
remote hands.

In the case of the power strips the "answer" was one of the
prepackaged,
self-contained old "portable" versions of FireFox which complains but 
the alert can be clicked through.? I recognize that exposing those 
devices to the Internet is unsafe but have never trusted that anyway; 
they're behind a gateway box with no port hole punch and if I'm
VPN'd in
then it's not possible for a random person to screw with it.

It would be sad indeed if the only answer here is "load up a partition 
with an older copy of FreeBSD on some device and use that."? Can we 
avoid that being the answer, as it became with the browser issues?

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4897 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-security/attachments/20210912/52f34bcf/attachment.bin>

I agree with Karl. To further the point:

"Secure by default" is a good idea, so removing ssh-rsa from the
default
list makes sense to alert people if its still in use.

Management ports for power strips, switches, UPSs, generators, thermostats,
radios, etc should already be isolated on a separate vlan or whatever. If
they're not, they're not secure anyway, regardless of the crypto
algorithms
because nearly all of their stacks have known security holes now anyway.
Taking away ssh-rsa is not necessary or helpful for improving anything's
security.

The only tactic for maintaining this older equipment is to keep an old
installation of FreeBSD in service past its support period on an
intermediate host. True, one can cherry-pick an old version of OpenSSH form
the old Ports tree, but the ports tree improves and drops support for
building old versions, as do the compilers. Over a 10-year time horizon,
building an old version will become unsustainable without an old
installation of FreeBSD.

Some of us will recall that this is already the answer for connecting to
hardware that only supports SSH1. I understand removing SSH1 more; I can
imagine it was more effort to keep up support for that being a different
protocol, etc, but ssh-rsa seems much simpler. However thinking longer term
as this situation evolves and OpenSSH continues adding new and removing old
support, the future of OpenSSH will also become unable to connect to these
intermediate hosts, requiring yet another intermediate host running a
frozen-in-time version of FreeBSD. And eventually the replacement hardware
for these becomes an issue.

All of this requires an insane amount of customization and time investment
for individual shops, when it could be solved by keeping ssh-rsa available
as part of the core project, and simply omit it from the default list.
Granted, I'm not one to speak credibly on the level of effort required for
this, nor is it necessary to convince me for FreeBSD to make a decision.
However as an educated observer, it appears to be easy enough to keep
support for ssh-rsa more or less forever, even to the point where its only
utility is to connect to hardware in museums.

-Leif



On Sun, Sep 12, 2021, 9:41 AM Karl Denninger <karl at denninger.net>
wrote:
> On 9/12/2021 10:02, Markus Falb wrote:
> >> On 09.09.2021, at 20:01, Ed Maste <emaste at freebsd.org>
wrote:
> >>
> >> OpenSSH will disable the ssh-rsa signature scheme by default in
the
> >> next release.
> >>
> >> ...
> >>
> >> To check whether a server is using the weak ssh-rsa public key
> >> algorithm, for host authentication, try to connect to it after
> >> removing the ssh-rsa algorithm from ssh(1)'s allowed list:
> >>
> >>     ssh -oHostKeyAlgorithms=-ssh-rsa user at host
> > FWIW, some of us may already have dealt with that.
> > FIPS enabled RedHat Enterprise Linux (and probably other FIPS enabled
> > systems) means effectively no ssh-rsa signature available in the sshd.
> > I had that situation at the beginning of the year.
> >
> > As mentioned, ssh-rsa signature algorithm will stop working, but
> > that does not automatically imply that every RSA key must be
> > changed to something other. The signature algorithm is not a
> > property that is inherent to the key.
> >
> > That said, existing RSA keys were working fine for me (my openssh
> > client was rsa-sha2-256 and rsa-sha2-512 capable) but when I tested
> > with some popular windows clients (filezilla, putty) it failed
> > (apparently no rsa-sha2 algorithms available).
> >
> > I found it interesting that mentioned clients were ecdsa
> > capable but did not support sha2 signatures with RSA keys.
> > Maybe the situation changed in the meantime to the better.
> >
> > There are 3 scenarios:
> >
> > 1. both sides support rsa-sha2 signatures -> RSA keys still working
> >
> > 2. one side does not support sha2 signatures but does support other
> > key types -> you can change key type
> >
> > 3. one side does not support sha2 and no other key type -> you
loose
> >
> > A prominent candidate for 3. would be Cisco IOS
>
> This has come up before with web browsers and is a serious PITA when
> there is no override available for those who need it on a targeted,
> specific basis.
>
> I have in the field a BUNCH of "smart" rack power strips that
have this
> problem; their management firmware does NOT support more-modern cipher
> sets and SSL requirements.  I get it, those older SSL versions are
> insecure and we know it.  But when the browser people all decided to
> kill the ability to connect to such servers with no override (that is,
> don't warn, DENY with no option to get around it) all of a sudden
> logging into those strips to change (for example) the name of a socket,
> the alarm limits and similar became literally impossible.  Contacting
> the manufacturer resulted in a middle finger back; "nope, we're
not
> releasing new firmware for that."  I've seen the same thing with
some
> older OOB management interfaces on server boards; they won't take an
> acceptably-long (by modern standards) HTTPS server key, and thus, same
> problem and same answer from the manufacturer.  These are
> perfectly-serviceable devices in their application and quite-expensive
> to replace when there's nothing wrong with them. On the server boards
by
> now they've all been retired as people decided the better power budget
> and performance levels made changing them (and re-purchasing the RAM
> that went on them, which for larger servers is a non-trivial part of the
> total expense) a reasonable proposition.  This of course is not true for
> a smart power strip in the rack and makes both monitoring of energy and
> remote-hard-power-cycle available without a physical site visit or
> remote hands.
>
> In the case of the power strips the "answer" was one of the
prepackaged,
> self-contained old "portable" versions of FireFox which complains
but
> the alert can be clicked through.  I recognize that exposing those
> devices to the Internet is unsafe but have never trusted that anyway;
> they're behind a gateway box with no port hole punch and if I'm
VPN'd in
> then it's not possible for a random person to screw with it.
>
> It would be sad indeed if the only answer here is "load up a partition
> with an older copy of FreeBSD on some device and use that."  Can we
> avoid that being the answer, as it became with the browser issues?
>
> --
> Karl Denninger
> karl at denninger.net <mailto:karl at denninger.net>
> /The Market Ticker/
> /[S/MIME encrypted email preferred]/
>

> On Sep 12, 2021, at 7:40 AM, Karl Denninger <karl at denninger.net>
wrote:
> 
> I have in the field a BUNCH of "smart" rack power strips that
have this problem; their management firmware does NOT support more-modern cipher
sets and SSL requirements.  I get it, those older SSL versions are insecure and
we know it.  But when the browser people all decided to kill the ability to
connect to such servers with no override (that is, don't warn, DENY with no
option to get around it) all of a sudden logging into those strips to change
(for example) the name of a socket, the alarm limits and similar became
literally impossible.  Contacting the manufacturer resulted in a middle finger
back; "nope, we're not releasing new firmware for that."  I've
seen the same thing with some older OOB management interfaces on server boards;
they won't take an acceptably-long (by modern standards) HTTPS server key,
and thus, same problem and same answer from the manufacturer.  These are
perfectly-serviceable devices in their application and quite-expensive to
replace when there's nothing wrong with them. On the server boards by now
they've all been retired as people decided the better power budget and
performance levels made changing them (and re-purchasing the RAM that went on
them, which for larger servers is a non-trivial part of the total expense) a
reasonable proposition.  This of course is not true for a smart power strip in
the rack and makes both monitoring of energy and remote-hard-power-cycle
available without a physical site visit or remote hands.
Blaming the browser and other client providers (OpenSSH, etc) for a problem that
is 100% because the devices are now abandoned by the manufacturer is the wrong
place to focus your anger. We have an enormous problem in the industry of crappy
embedded devices (like the OOB management plane) accruing technical security
debt while the manufacturers give "a middle finger back" as you say.
The supportability of the hardware needs to be baked into the purchasing
decision. Commitments from the manufacturers on supportability timeframes are
important to understand and budget into a hardware refresh cycle.
> In the case of the power strips the "answer" was one of the
prepackaged, self-contained old "portable" versions of FireFox which
complains but the alert can be clicked through.  I recognize that exposing those
devices to the Internet is unsafe but have never trusted that anyway;
they're behind a gateway box with no port hole punch and if I'm
VPN'd in then it's not possible for a random person to screw with it.
> 
> It would be sad indeed if the only answer here is "load up a partition
with an older copy of FreeBSD on some device and use that."  Can we avoid
that being the answer, as it became with the browser issues?
You are already accepting the risk of continuing to run devices with known bad
configurations. What's the problem with keeping that old FreeBSD host around
as well, it's just one more risk acceptance for issues that are pretty much
the same as what you are already accepting? Alternatively, compile and install
an older OpenSSH version on well-maintained host in a dedicated prefix which is
only used for that purpose.

We do need to remove the code entirely as putting it behind a compatibility or
some other "scary things are here" flag will guarantee that
manufacturers don't try to update their codebases to work with modern
protocols; they will just provide instructions on how to enable scary mode and
move on. In the interest of protecting everyone, we need to remove this code and
put it into the dustbin of history.

Best,
Gordon