As far as I can see, here is a summary of the situation, and there's a point to this, but I only make it in step (4), needing the first three steps to set up a background to keep my own thoughts clear: 1) Fedora (via Jakub) shows it's possible to patch OpenSSH. 2) OpenVPN (via gert) shows it's possible to build a 'shim' of sorts that allows code to work with libreSSL and OpenSSL 1.1.0. 3) Using that phrase 'as far as I can see' again, it appears that OpenSSH doesn't really care that (1) and (2) are shown as possible. The changes required to implement these solutions, in the best view, can be seen as violating the 'simple/secure' precepts of OpenBSD - so they simply are not desired, independent of feasibility. 4) As a first result, with no judgement on anyone, just looking at the data - the root cause of this issue seems to be the split of LibreSSL from OpenSSL a while back and we are just dealing with the in-hindsight-obvious consequences of that split. With something as fundamental as the SSL/TLS stack forking between OpenBSD(LibreSSL dominant) and Linux(OpenSSL dominant), it is inevitable that applications written on one or the other will find it harder and harder over time to be compatible and usable in both OpenSSL and LibreSSL worlds. You think it's hard to build a compatibility layer NOW? What happens when OpenSSL 1.2 comes around, then LibreSSL version-next, etc... 2-3 years down the road, getting further and further apart, with not just accessor functions changing, but with semantics and 'overall interface design and philosophy' changing over time. In other words, I don't believe ANY package can, over a period of time, realistically support both OpenSSL and LibreSSL, given the fact neither seems to have a desire to maintain compatibility with the other (again, as far as I can see). OpenSSH's decision to not really want to support the changes OpenSSL made is just the canary in the coal mine here - others will get to that point, too. They just got there first. 5) As a final result, it seems to me that the OpenBSD and Linux worlds need to decide if they LIKE and TOLERATE the consequences of the long-term split between LibreSSL and OpenSSL - in particular, it being harder and harder to share packages between the OpenBSD and Linux worlds, if those packages need to interface with diverging SSL/TLS stacks. If they don't, they need to do something about it. This has to be dealt with by the LibreSSL and OpenSSL teams - looking at OpenSSH is looking at the wrong place. If those two SSL/TLS teams don't talk, it will just get harder for everyone. In the meantime, because I live in the Linux world and not the OpenBSD world, for good or ill, I have to face the fact that, as of today, my reliance on the OpenSSH package, which I love and trust, has an expiration date, and I need to investigate alternatives, all of which are less appealing to me by a significant margin. Oh well.
Hi, jpbion at jfwest.com wrote on Wed, Oct 18, 2017 at 05:53:21AM -0700:> 4) As a first result, with no judgement on anyone, just looking at the > data - the root cause of this issue seems to be the split of LibreSSL > from OpenSSLNo, you are totally misrepresenting the situation. The root cause is the split of OpenSSL-1.1 from OpenSSL-1.0, and that OpenSSL dumped the large and dangerous work of dealing with the large-scale API change on each and every application project instead of providing an official transition path that can be taken seriously. LibreSSL has almost nothing to do with the problem. Even if LibreSSL had never happened, the same problem would still exists. Oh, wait, LibreSSL has to do with it in one sense. It is available as one possible way to *solve* the problem. Either temporarily or for good, whichever you like.> OpenSSL and LibreSSL, given the fact neither seems to have a desire > to maintain compatibility with the other (again, as far as I can see).That is an unfounded allegation. Of course LibreSSL has a desire to eventually integrate the 1.1 API. Joel has said so long ago, in public, that in principle, opaque structs are a good concept [for example citation 1: Dec 30, 2016], and i have heard repeated discussions inside the LibreSSL project on how to get there. It is just a lot of work, it is made harder by the lack of a clear migration path, and it is of limited usefulness as long as application programs must still support the OpenSSL-1.0 API. That's what prevented it from getting done so far. Given that you got the facts wrong, your conclusions are misleading as well. All this was explained already, so your mail sounds almost trollish: It should already be well-known that the central design goal of LibreSSL is to be a compatible drop-in replacement for OpenSSL - at the time of the fork, that was OpenSSL-1.0. If, after the fork, OpenSSL breaks its own API and leaves users in the rain, blaming that on LibreSSL is quite dishonest. Even if the API break is so severe that it takes LibreSSL substantially more than a year to deal with it, even if LibreSSL hasn't yet solved the problem for its own purposes. The real problem is: How is OpenSSH supposed to support OpenSSL-1.0 and OpenSSL-1.1 at the same time, given that the API break is so severe that switching from one to the other requires a 3000+ line diff? Yours, Ingo [1] https://www.mail-archive.com/tech at openbsd.org/msg36437.html
(Re-sent as I used the wrong account before - edited slightly) I?m sorry you see my post as potentially trollish - it was not meant that way. So let me try to restate things. I have nothing against libreSSL, and I do think the OpenSSL api change was poorly executed. I am just speaking, as a solo semi retired hobbyist, who spends most of his server coding time on private weather station software, (I don?t do this stuff for work - this is all on my own time) I feel really left in a bind. OpenSSH is a fantastic tool, but I worry I will end up not being able to use it. I am not on all the mailing lists - this is why I repeatedly said ?as far as I can see? acknowledging my limited knowledge of the details. The end users of these tools have been dealing with the nightmare of the OpenSSL changes for about a year now. Nor do I see this as anyone?s ?fault? - except as I said OpenSSL created a pile of work for everyone. The real point of my note is: I don?t see OpenSSL and LibreSSL merging any time soon. Even if libreSSL puts in opaque structures, what if the two teams view ways to handle the issues of 2018, and beyond differently? Is API comparability, a goal at the start, going to remain a primary goal of both teams, especially if one of the two teams make choices seen as ill-founded by the other? I don?t know - so I ask - and without knowing, I worry that more API divergence will occur. Again, the OpenSSL changes were a huge gulp - and others may question the wisdom of staying in lock-step to those changes. As a result, i worry if it will be possible to maintain software, over time, that supports both variants of SSL stack. As such, I worry if a day comes that I can?t use openssh, because too many other things I depend upon CAN'T use libreSSL. Please tell me where this core concern of mine is misplaced - because I would surely like my concern to be ill-founded and mistaken. On 2017-10-18 07:15, Ingo Schwarze wrote:> Hi, > > jpbion at jfwest.com wrote on Wed, Oct 18, 2017 at 05:53:21AM -0700: > >> 4) As a first result, with no judgement on anyone, just looking at the >> data - the root cause of this issue seems to be the split of LibreSSL >> from OpenSSL > > No, you are totally misrepresenting the situation. The root cause > is the split of OpenSSL-1.1 from OpenSSL-1.0, and that OpenSSL > dumped the large and dangerous work of dealing with the large-scale > API change on each and every application project instead of providing > an official transition path that can be taken seriously. > > LibreSSL has almost nothing to do with the problem. Even if LibreSSL > had never happened, the same problem would still exists. > > Oh, wait, LibreSSL has to do with it in one sense. It is available > as one possible way to *solve* the problem. Either temporarily or > for good, whichever you like. > >> OpenSSL and LibreSSL, given the fact neither seems to have a desire >> to maintain compatibility with the other (again, as far as I can see). > > That is an unfounded allegation. Of course LibreSSL has a desire > to eventually integrate the 1.1 API. Joel has said so long ago, > in public, that in principle, opaque structs are a good concept > [for example citation 1: Dec 30, 2016], and i have heard repeated > discussions inside the LibreSSL project on how to get there. It > is just a lot of work, it is made harder by the lack of a clear > migration path, and it is of limited usefulness as long as application > programs must still support the OpenSSL-1.0 API. That's what > prevented it from getting done so far. > > Given that you got the facts wrong, your conclusions are misleading > as well. All this was explained already, so your mail sounds almost > trollish: It should already be well-known that the central design > goal of LibreSSL is to be a compatible drop-in replacement for > OpenSSL - at the time of the fork, that was OpenSSL-1.0. If, after > the fork, OpenSSL breaks its own API and leaves users in the rain, > blaming that on LibreSSL is quite dishonest. Even if the API break > is so severe that it takes LibreSSL substantially more than a year > to deal with it, even if LibreSSL hasn't yet solved the problem for > its own purposes. > > The real problem is: How is OpenSSH supposed to support OpenSSL-1.0 > and OpenSSL-1.1 at the same time, given that the API break is so > severe that switching from one to the other requires a 3000+ line > diff? > > Yours, > Ingo > > [1] https://www.mail-archive.com/tech at openbsd.org/msg36437.html
Hi Ingo, On Wed, Oct 18, 2017 at 4:15 PM, Ingo Schwarze <schwarze at usta.de> wrote:> Hi, > > jpbion at jfwest.com wrote on Wed, Oct 18, 2017 at 05:53:21AM -0700: > >> 4) As a first result, with no judgement on anyone, just looking at the >> data - the root cause of this issue seems to be the split of LibreSSL >> from OpenSSL > > No, you are totally misrepresenting the situation. The root cause > is the split of OpenSSL-1.1 from OpenSSL-1.0, and that OpenSSL > dumped the large and dangerous work of dealing with the large-scale > API change on each and every application project instead of providing > an official transition path that can be taken seriously.Important API change between major releases is something to be expected. Sometimes the changes are limited, sometimes thay aren't. The structure of the changes themselves is the reason why the openssl folks could not provide any migration path (the existance of this path would have negated the changes themselves). Anyway, this is done, and I'm not sure we have the power to go back in time to fix that (and I'm not even sure it's necessary to fix it). Anyway, I still agree with you: the problem is not the split, it's the API change.> LibreSSL has almost nothing to do with the problem. Even if LibreSSL > had never happened, the same problem would still exists. > > Oh, wait, LibreSSL has to do with it in one sense. It is available > as one possible way to *solve* the problem. Either temporarily or > for good, whichever you like.This is not a good idea IMHO. One want to be able to write security-related code easily, and the old OpenSSL API (and thus, the current LibreSSL interface) does not make that simple at all. As a result, we got many incorrect use of this very API in the past which led to security issues in many softwares (CVE-2009-0021). While some degree of API complexity is inherent to the role of the library, I'm pretty sure you can agree that the simpler it is, the better. Given the fact that the knowledge of the structures of LibreSSL is needed to correclty use the library, I hope you'll understand my point: using LibreSSL as it is today instead of porting to OpenSSL is not a good idea :) (Not to mention that more and more software implement compatibility with OpenSSL 1.1, so asking them to go the other way is not going to work well).>> OpenSSL and LibreSSL, given the fact neither seems to have a desire >> to maintain compatibility with the other (again, as far as I can see). > > That is an unfounded allegation. Of course LibreSSL has a desire > to eventually integrate the 1.1 API. Joel has said so long ago, > in public, that in principle, opaque structs are a good concept > [for example citation 1: Dec 30, 2016], and i have heard repeated > discussions inside the LibreSSL project on how to get there. It > is just a lot of work, it is made harder by the lack of a clear > migration path, and it is of limited usefulness as long as application > programs must still support the OpenSSL-1.0 API. That's what > prevented it from getting done so far.There is no easy migration path. More precisely, there is only one migration path, and it's not pretty. To allow applications to do the jump at their pace, you have to propose both interfaces and mark the old one as deprecated (and remove it in a subsequent release). I'd volonteer to do this but given the gigantism of the task, any help would be appreciated (not to mention that even if Joel agrees with the idea of having the 1.1 API in LibreSSL, that does not mean he would agree with my strategy).> Given that you got the facts wrong, your conclusions are misleading > as well. All this was explained already, so your mail sounds almost > trollish: It should already be well-known that the central design > goal of LibreSSL is to be a compatible drop-in replacement for > OpenSSL - at the time of the fork, that was OpenSSL-1.0. If, after > the fork, OpenSSL breaks its own API and leaves users in the rain, > blaming that on LibreSSL is quite dishonest. Even if the API break > is so severe that it takes LibreSSL substantially more than a year > to deal with it, even if LibreSSL hasn't yet solved the problem for > its own purposes.I agree with you on this one. The LibreSSL folks are not to be blamed for this situation.> The real problem is: How is OpenSSH supposed to support OpenSSL-1.0 > and OpenSSL-1.1 at the same time, given that the API break is so > severe that switching from one to the other requires a 3000+ line > diff?But at the same time, some ditributions are phasing out OpenSSL 1.0.2 and are integrating OpenSSL 1.1 while at the same time some supported LTS distributions still rely on older OpenSSL versions. So for a period of time you'll have to be compatible with both, whether you like it or not. The question is then: is it up to you to be compatible with both, or is it up to distributors to provide compatibilty? When more and more softwares propose their own shim it makes the later less and less understandable.> Yours, > IngoBR, -- Emmanuel Deloget