Blumenthal, Uri - 0553 - MITLL
2017-Oct-18 17:27 UTC
Status of OpenSSL 1.1 support - Thoughts
OpenSSL developers believed that there was a need for a significant change. A
part of that change was a conscious choice to break (some of) the existing API.
They considered that pain unavoidable. So far I happen to agree with their
rationale and approach. Move from visible internal structures to accessor
functions is a good thing, regardless of what you may think of it. And the new
API *is* better, again like it or not.
I understand the frustration with lack of a ?migration library?, but how to you
see a ?shim? that allows code that relies on being able to directly access
members of structures, run unmodified (just recompiled)?
In my opinion, a reasonable thing for OpenSSH to do would be to port their code
to using accessor functions, and write a shim library to the ?old? way (exactly
as was proposed here before).
LibreSSL would have to do the same eventually. The pain of such migration can be
postponed, but not avoided indefinitely.
--
Regards,
Uri Blumenthal
On 10/18/17, 12:38, "openssh-unix-dev on behalf of Ingo Schwarze"
<openssh-unix-dev-bounces+uri=ll.mit.edu at mindrot.org on behalf of schwarze
at usta.de> wrote:
Hi Emmanuel,
Emmanuel Deloget wrote on Wed, Oct 18, 2017 at 05:45:40PM +0200:
> Important API change between major releases is something to be
> expected. Sometimes the changes are limited, sometimes thay aren't.
>
> The structure of the changes themselves is the reason why the openssl
> folks could not provide any migration path (the existance of this path
> would have negated the changes themselves).
True, but only in the 1.0 release series, while still attaining the
goal of making the transition manageable for application projects.
When you implement a major redesign, you can never expect to reap
the benefit in the old stable branch. The way OpenSSL did it doesn't
achieve that either.
> Ingo Schwarze wrote:
>> Oh, wait, LibreSSL has to do with it in one sense. It is available
>> as one possible way to *solve* the problem. Either temporarily or
>> for good, whichever you like.
> This is not a good idea IMHO. One want to be able to write
> security-related code easily, and the old OpenSSL API (and thus, the
> current LibreSSL interface) does not make that simple at all. As a
> result, we got many incorrect use of this very API in the past which
> led to security issues in many softwares (CVE-2009-0021). While some
> degree of API complexity is inherent to the role of the library,
I'm
> pretty sure you can agree that the simpler it is, the better.
>
> Given the fact that the knowledge of the structures of LibreSSL is
> needed to correclty use the library, I hope you'll understand my
> point: using LibreSSL as it is today instead of porting to OpenSSL is
> not a good idea :)
For the short term, it is of the same order of quality as using
the officially maintained OpenSSL-1.0 branch, or maybe even a bit
better because it may have fewer bugs. Of course, neither gets
the benefits of opaque structs, but both allow working around the
transition problem.
In the long term, LibreSSL will hopefully transition to opaque structs
as well, only less rushed and with a better transition plan, at which
point your argument will become moot.
> More precisely, there is only one migration path, and it's not
pretty.
> To allow applications to do the jump at their pace, you have to propose
> both interfaces and mark the old one as deprecated (and remove it in
> a subsequent release).
Precisely. That is how you do a large-scale change that is
incompatible both ways in a widely-used library:
1. Say you are at version 41.0.
2. Add the new interfaces in version 41.1 (minor bump).
3. Invite some application projects to switch to the new interfaces.
4. Help them with the switch and learn from the experience.
5. Quite probably, you will have to publish bug fix releases
41.1.1 and 41.1.2 to improve what you did in steps 2 to 4.
6. Invite *all* users to switch.
7. Once many of your users have switched, announce future deprecation.
8. Wait some time.
9. Announce end of life for a specific date.
10. Remove the old interfaces in version 42.0 (major bump).
OpenSSL jumped the shark and skipped steps 3 to 9, doing 1 and 10
in one single leap.
Of course, you can't get the full benefit until step 10, in particular
when the point is to improve encapsulation. But that's no excuse
for putting users in an impossible situation.
If, for whatever reason, you realize too late that you completely
forgot about steps 3 to 9, you can still mitigate part of the havoc
done by doing step 2 after step 10, enabling the forgotten migration
path. Better late than never.
> I'd volonteer to do this but given the gigantism of the task, any
help
> would be appreciated (not to mention that even if Joel agrees with the
> idea of having the 1.1 API in LibreSSL, that does not mean he would
> agree with my strategy).
You could offer your help for providing that migration path to the
OpenSSL project, if you feel qualified and willing to provide such
help. I think you should definitely use the expertise of the OpenSSL
developers to make sure that providing the migration path doesn't
introduce bugs and vulnerabilities, and avoid working as a lone
wolf if possible.
> But at the same time, some ditributions are phasing out OpenSSL 1.0.2
> and are integrating OpenSSL 1.1 while at the same time some supported
> LTS distributions still rely on older OpenSSL versions. So for a
> period of time you'll have to be compatible with both, whether you
> like it or not. The question is then: is it up to you to be compatible
> with both,
The OpenSSH project simply doesn't have the manpower to maintain
compatibility with both at the same time, and even if it had, doing
so would compromise the central project goal of security through
simplicity and diligence.
> or is it up to distributors to provide compatibilty?
That will be beyond the means of many distributors, in which case
they either need to use OpenSSL-1.0 or LibreSSL for OpenSSH.
Even for those distributors who can afford the required manpower,
i consider attempting to maintain a 3000+ line vendor patch risky,
and maintaining a non-official compat layer risky as well. Quite
respectable vendors have screwed up with three-line patches in the
past, how can anybody feel confident about a 3000-line vendor patch?
How sure can you feel that none of these 3000 lines causes non-obvious
problems? That also answers the question Jakub Jelen asked why i
would feel uncomfortable about using Fedora.
Yours,
Ingo
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev at mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5211 bytes
Desc: not available
URL:
<http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20171018/fc0a3ab7/attachment-0001.p7s>
Blumenthal, Uri - 0553 - MITLL wrote:> In my opinion, a reasonable thing for OpenSSH to do would be to > port their code to using accessor functions, and write a shim > library to the ?old? way (exactly as was proposed here before). > > LibreSSL would have to do the same eventually.It seems to be a bug that more than one project needs to do that. //Peter
Blumenthal, Uri - 0553 - MITLL
2017-Oct-18 19:38 UTC
Status of OpenSSL 1.1 support - Thoughts
Views, technologies, best practices, and threats evolve and change. So, regardless of what it is and how it came about, move to, e.g., opaque structures appears inevitable to me. And if LibreSSL hasn't done that yet, most likely it would have to in the long run. Regards, Uri Sent from my iPhone> On Oct 18, 2017, at 15:32, Peter Stuge <peter at stuge.se> wrote: > > Blumenthal, Uri - 0553 - MITLL wrote: >> In my opinion, a reasonable thing for OpenSSH to do would be to >> port their code to using accessor functions, and write a shim >> library to the ?old? way (exactly as was proposed here before). >> >> LibreSSL would have to do the same eventually. > > It seems to be a bug that more than one project needs to do that. > > > //Peter > _______________________________________________ > openssh-unix-dev mailing list > openssh-unix-dev at mindrot.org > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5801 bytes Desc: not available URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20171018/50093519/attachment.p7s>
On Wed, 18 Oct 2017, Blumenthal, Uri - 0553 - MITLL wrote:> OpenSSL developers believed that there was a need for a significant > change. A part of that change was a conscious choice to break (some > of) the existing API. They considered that pain unavoidable. So far I > happen to agree with their rationale and approach. Move from visible > internal structures to accessor functions is a good thing, regardless > of what you may think of it. And the new API *is* better, again like > it or not. > > I understand the frustration with lack of a ?migration library?, > but how to you see a ?shim? that allows code that relies on being > able to directly access members of structures, run unmodified (just > recompiled)?You've got this exactly backwards. We don't want a shim that allows OpenSSL-1.1 to present a OpenSSL-1.0 API. We want a shim that allows us to use the OpenSSL-1.1 API when using OpenSSL-1.0, so we don't have to maintain a forest of #ifdefs.
Blumenthal, Uri - 0553 - MITLL
2017-Oct-19 00:35 UTC
Status of OpenSSL 1.1 support - Thoughts
Well, in that case, didn't Fedora provide exactly what you say you want? Then the path forward for OpenSSH is sensible: port the code to the new API, and include Fedora-like library to interface with OpenSSL-1.0.x. So what's the problem? ;-) Regards, Uri Sent from my iPhone> On Oct 18, 2017, at 18:44, Damien Miller <djm at mindrot.org> wrote: > >> On Wed, 18 Oct 2017, Blumenthal, Uri - 0553 - MITLL wrote: >> >> OpenSSL developers believed that there was a need for a significant >> change. A part of that change was a conscious choice to break (some >> of) the existing API. They considered that pain unavoidable. So far I >> happen to agree with their rationale and approach. Move from visible >> internal structures to accessor functions is a good thing, regardless >> of what you may think of it. And the new API *is* better, again like >> it or not. >> >> I understand the frustration with lack of a ?migration library?, >> but how to you see a ?shim? that allows code that relies on being >> able to directly access members of structures, run unmodified (just >> recompiled)? > > You've got this exactly backwards. We don't want a shim that allows > OpenSSL-1.1 to present a OpenSSL-1.0 API. We want a shim that allows > us to use the OpenSSL-1.1 API when using OpenSSL-1.0, so we don't have > to maintain a forest of #ifdefs.-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5801 bytes Desc: not available URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20171019/d41da141/attachment-0001.p7s>
Hi, On Thu, Oct 19, 2017 at 09:43:41AM +1100, Damien Miller wrote:> You've got this exactly backwards. We don't want a shim that allows > OpenSSL-1.1 to present a OpenSSL-1.0 API. We want a shim that allows > us to use the OpenSSL-1.1 API when using OpenSSL-1.0, so we don't have > to maintain a forest of #ifdefs.For obvious reasons this shim cannot exist. If the structure member is not visible anymore (and might not actually look the way you think it does), you cannot provide structure definitons that magically give you access to the members again. Also, you do not need to maintain a forest of #ifdef - as soon as you switch the code to only use accessor functions, the only #ifdef you have is "one for the whole shim" or possibly "one per compat accessor function" - nicely encapsulated away from the code using the accessor. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany gert at greenie.muc.de fax: +49-89-35655025 gert at net.informatik.tu-muenchen.de