OpenSC has taken a different approach to OpenSSL-1.1. Rather then writing a shim for OpenSSL-1.1, the OpenSC code has been converted to the OpenSSL-1.1 API and a sc-ossl-compat.h" file consisting of defines and macros was written to support older versions of OpenSSL and Libressl. https://github.com/OpenSC/OpenSC/blob/master/src/libopensc/sc-ossl-compat.h The nice part of this approach is when using OpenSSL-1.1 sc-ossl-compat.h does not do anything. It sole purpose to provide calls to the older APIs that are not going to change and eventually the sc-ossl-compat.h could be removed. Only the OpenSSL routines used by OpenSC have added to sc-ossl-compat.h but others defines and macro could be added.There are a few utilities that use still use a few #ifdef's during initialization. On 6/23/2017 7:15 AM, The Doctor wrote:> On Fri, Jun 23, 2017 at 01:53:24PM +0200, Emmanuel Deloget wrote: >> Hello Ingo, >> >> On Fri, Jun 23, 2017 at 1:26 AM, Ingo Schwarze <schwarze at usta.de> wrote: >>> >>> Hi Emmanuel, >>> >>> Emmanuel Deloget wrote on Fri, Jun 23, 2017 at 12:26:47AM +0200: >>> >>>> * the openssl team has no real incentive to propose a shim ; >>> >>> If major application projects refuse to support their new release, >>> thus putting pressure on operating system distributions to not >>> completely switch to 1.1 either, that is not an incentive? >> >> Yes and no. The fact is that openssl is widely used by tons of projects. >> Even if a major project quits (and frankly, these days openssh is built >> around libressl, not openssl, so one car argue that openssh already leaved >> the boat) there is little incentive for the openssl to not continue in the >> same direction as other projects will follow anyway -- mostly because >> openssl is openssl, not because openssl is great (*). >> >> Why would the openssl team do that ? >> >> * it's time consuming, and their time is better spent on enhancing the >> security and features of the openssl libs >> * such a project is doomed. It would make the various downstream project >> move to the openssl 1.1 API, at which point the remaining distros will dump >> older versions, meaning that the shim will have a life duration of one, >> maybe two years. >> * yet during these two years, it might become a stable (as introduces by >> R.C. Martin : a stable software is one who as many clients and a limited >> set of dependencies), meaning that every little change in it might have >> tremendous effects on downstream projects. >> * its a rewrite of code they already wrote in the openssl library. >> * most of it is trivial >> * your specific downstream needs are different than another downstream's >> project need, so in order to satisfy everyone they'll have to build a >> complete library. Which is basically a rewrite of what they already did. >> >> So, to rephrase the question, why would they spend time to propose a >> difficult to maintain, not really needed, potentially large yet trivial, >> doomed library? >> >>>> Did I miss something? >>> >>> Maybe you are striving for the wrong goal. It is not a goal to >>> clobber something together and encourage OpenSSL to repeat such >>> recklessness in the future, and leave users out in the rain once >>> again. >> >> I fully understand that the API change was/is difficult. Yet, it's a >> proactive move (and not a bad one) to enhance downstream code and to limit >> the impact of further changes. You don't have to micromanage a quadrillion >> of structure members with confused semantics, meaning that there is less >> chance to misuse them. Furthermore you no longer have access to what are >> are really implementation details. >> >>> It is not a goal either to create a shim that is not >>> officially audited and thoroughly tested by the original authors >>> who know their original code best, to create a shim that creates >>> additional dangers for security. We are talking about security >>> software here, so this is not the place at all to lightly cobble >>> something together, in particular not in ways involving many lines >>> of additional code. >> >> >> And I fully get that as well. >> >> But since nobody will step in, one has to do the job. Luckily, many has >> jumped in (and I was ready to do that as well). Their shim code is, as >> expected, quite clear and it should be easy to spot anything suspicious or >> wrong (I suspect some BN_free() should be replaced with BN_clear_free() >> here and there, but for most part the BN_free() directly comes from the >> openssl code). >> >> The fact is that most added functions are setters and getters, so until you >> do things in the Widely Wrong Way, it should be ok :) I'm not talking about >> rewrite RSA_public_encrypt() :) >> >>> If a few important projects keep up resistance and refuse support >>> for 1.1 until OpenSSL rolls up their sleeves and fixes the mess >>> they have created, maybe they will eventually realize that they >>> started a job here, wandered off half-way, and failed to ever >>> properly finish it. >> >> Or another, less important project that did the jump will replace it >> because distros have limited patience w.r.t. softwares that do not want to >> evolve their API. Once the 1.0.2 support is out, distros will want to have >> 1.1 and softwares that do not want to move to that version will either be >> patched or replaced. Other implementations already exist, including >> dropbear, GNU lsh and probably others I don't know. Most of them are not as >> good as openssh and some of them are downright horrible to use (I'm looking >> at you, GNU lsh...) but the fact that some alternatives exists is what's >> important here. >> >> You also must take into account that some people want to have openssl 1.1 >> for wahetever reason (for example, TLS1.3 support in 1.1.1). For them, >> having to cope with multiple versions of the openssl library is not going >> to work well. Distro maintainers will also be rightly pissed off if only >> one important program does not want to take the jump. >> >>> So, such resistance has a chance to improve the situation for >>> everybody. And i can't think of many projects that are in as >>> widespread use as OpenSSH, and hence can be more valuable with >>> respect to such resistance. >> >> I'm not sure it's practical in the long term. Furthermore, one cannot >> really expect that the openssl team will deprecate the 1.1 API and move >> back to the Good Ol' Days of the 1.0.1 API. I'm not saying that resistance >> is futile but I'm not sure it will have the desired effect. >> > > Unfortunately, Openssl will not be heading that way. > > Opensl 1.0.X dies, that is it, Openssl 1.1.X upwards will be the standrard. > > Other packages like Exim, INN, Curl and Bind are adapating. > > Even Claamav will adapat, just a 2 line fix. > > In this case resistance will go nowhere. > > > >>> Just my personal 2 cents, >>> yours, >>> Ingo >> >> >> Best regards, >> >> -- Emmanuel Deloget >> >> (*) that does not mean openssl is not great. >> _______________________________________________ >> openssh-unix-dev mailing list >> openssh-unix-dev at mindrot.org >> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev >-- Douglas E. Engert <DEEngert at gmail.com>
I think that this is the better approach. The question I have is why the SSH logic should be dependent on the implementation details of ANY particular cryptographic library (be it openssl, libressl or whatever)? Proper software design would develop an abstraction layer with some measure of forward compatibility built in. On 6/23/2017 3:16 PM, Douglas E Engert wrote:> OpenSC has taken a different approach to OpenSSL-1.1. Rather then writing > a shim for OpenSSL-1.1, the OpenSC code has been converted to > the OpenSSL-1.1 API and a sc-ossl-compat.h" file consisting of defines and > macros was written to support older versions of OpenSSL and Libressl. > > https://github.com/OpenSC/OpenSC/blob/master/src/libopensc/sc-ossl-compat.h > > The nice part of this approach is when using OpenSSL-1.1 sc-ossl-compat.h > does not do anything. It sole purpose to provide calls to the older APIs > that are not going to change and eventually the sc-ossl-compat.h could be > removed. >
Hi George, On Sat, Jun 24, 2017 at 2:06 PM, George M. Garner Jr. <ggarner_online at gmgsystemsinc.com> wrote:> I think that this is the better approach. The question I have is why the > SSH logic should be dependent on the implementation details of ANY > particular cryptographic library (be it openssl, libressl or whatever)? > Proper software design would develop an abstraction layer with some measure > of forward compatibility built in.I'm all in favor of abstracting such a complex piece of code. Yet the sheer number of available crypto library out there makes it an XKCD "build another standard to aggregate all the existing standard, so now there is one more standard to deal with" kind of situation. Not to mention that these libraries may implement different philosophies so building some abstraction code above them could be quite complex. To be honest, I would not even know where to start :) BR, -- Emmanuel Deloget
Hello Douglas, On Fri, Jun 23, 2017 at 9:16 PM, Douglas E Engert <deengert at gmail.com> wrote:> OpenSC has taken a different approach to OpenSSL-1.1. Rather then writing > a shim for OpenSSL-1.1, the OpenSC code has been converted to > the OpenSSL-1.1 API and a sc-ossl-compat.h" file consisting of defines and > macros was written to support older versions of OpenSSL and Libressl. > >https://github.com/OpenSC/OpenSC/blob/master/src/libopensc/sc-ossl-compat.h> > The nice part of this approach is when using OpenSSL-1.1 sc-ossl-compat.h > does not do anything. It sole purpose to provide calls to the older APIs > that are not going to change and eventually the sc-ossl-compat.h could be > removed. > > Only the OpenSSL routines used by OpenSC have added to sc-ossl-compat.h > but others defines and macro could be added.There are a few utilities that > use still use a few #ifdef's during initialization.This might be because I'm kind of a failure when I try to speak English but this is what I assumed to be a shim :) Of course, I might be wrong again :) This is very similar to the approach taken by Kurt in his patch and to the work I did for OpenVPN ? [1]? . And I also think it's the way to go since it will allow to diss older versions of openssl and/or libressl with minimal change in the Code That Matters while not being terribly difficult to maintain. According to Kurt's patch, such a compat file for openssh would clock at roughly 500 lines of nearly trivial code (and I insist on this fact: code is quite trivial. The most complicated function would be this one : ?----8<-----? static int EVP_MD_CTX_reset(EVP_MD_CTX *ctx) { ? ? if (ctx == NULL) ? ? return 1; ? ? /* ? ? * Don't assume ctx->md_data was cleaned in EVP_Digest_Final, because ? ? * sometimes only copies of the context are ever finalised. ? ? */ ? ? if (ctx->digest && ctx->digest->cleanup ? ? && !EVP_MD_CTX_test_flags(ctx, EVP_MD_CTX_FLAG_CLEANED)) ? ? ctx->digest->cleanup(ctx); ? ? if (ctx->digest && ctx->digest->ctx_size && ctx->md_data ? ? && !EVP_MD_CTX_test_flags(ctx, EVP_MD_CTX_FLAG_REUSE)) { ? ? OPENSSL_clear_free(ctx->md_data, ctx->digest->ctx_size); ? ? } ? ? EVP_PKEY_CTX_free(ctx->pctx); #ifndef OPENSSL_NO_ENGINE ? ? ENGINE_finish(ctx->engine); #endif ? ? OPENSSL_cleanse(ctx, sizeof(*ctx)); ? ? return 1; } ?---->8----- Other functions are getter and setters such as : ?----8<-----? void DH_get0_key(const DH *dh, const BIGNUM **pub_key, const BIGNUM **priv_key) { if (pub_key != NULL) *pub_key = dh->pub_key; if (priv_key != NULL) *priv_key = dh->priv_key; } int DH_set0_key(DH *dh, BIGNUM *pub_key, BIGNUM *priv_key) { /* If the field pub_key in dh is NULL, the corresponding input * parameters MUST be non-NULL. The priv_key field may * be left NULL. */ if (dh->pub_key == NULL && pub_key == NULL) return 0; if (pub_key != NULL) { BN_free(dh->pub_key); dh->pub_key = pub_key; } if (priv_key != NULL) { BN_free(dh->priv_key); dh->priv_key = priv_key; } return 1; } ?int RSA_bits(const RSA *r) { return (BN_num_bits(r->n)); } ?---->8----- There are some simple security?-related functions such as OPENSSL_zalloc() or OPENSSL_clean_free() but these functions are quite simple too (and the reasoning behind them is, I assume, well known). BR, -- Emmanuel Deloget ?[1] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/openssl_compat.h ?
On 6/24/2017 11:35 AM, Emmanuel Deloget wrote:> Hello Douglas, > > On Fri, Jun 23, 2017 at 9:16 PM, Douglas E Engert <deengert at gmail.com <mailto:deengert at gmail.com>> wrote: > > OpenSC has taken a different approach to OpenSSL-1.1. Rather then writing > > a shim for OpenSSL-1.1, the OpenSC code has been converted to > > the OpenSSL-1.1 API and a sc-ossl-compat.h" file consisting of defines and > > macros was written to support older versions of OpenSSL and Libressl. > > > > https://github.com/OpenSC/OpenSC/blob/master/src/libopensc/sc-ossl-compat.h > > > > The nice part of this approach is when using OpenSSL-1.1 sc-ossl-compat.h > > does not do anything. It sole purpose to provide calls to the older APIs > > that are not going to change and eventually the sc-ossl-compat.h could be > > removed. > > > > Only the OpenSSL routines used by OpenSC have added to sc-ossl-compat.h > > but others defines and macro could be added.There are a few utilities that > > use still use a few #ifdef's during initialization. > > This might be because I'm kind of a failure when I try to speak English but this is what I assumed to be a shim :) Of course, I might be wrong again :)Yes it is a shim. https://en.wikipedia.org/wiki/Shim_(computing) "Shims can be used to support an old API in a newer environment, or a new API in an older environment." My reading of the discussion was to keep the older API to avoid changing the OpenSSH code so a shim would be to support an "old API in a newer environment(OpenSSL-1.1)". What we did was to convert OpenSC to the OpenSSL-1.1 API the write the shim for "new API in an older environment". Looking at your openssl_compat.h it looks like you have done something similar. With OpenSC we only added the routines and defines we needed in OpenSC.> > This is very similar to the approach taken by Kurt in his patch and to the work I did for OpenVPN > ? [1]? > . And I also think it's the way to go since it will allow to diss older versions of openssl and/or libressl with minimal change in the Code That Matters while not being terribly difficult to maintain. > According to Kurt's patch, such a compat file for openssh would clock at roughly 500 lines of nearly trivial code (and I insist on this fact: code is quite trivial. The most complicated function would > be this one : > > ?----8<-----? > static int EVP_MD_CTX_reset(EVP_MD_CTX *ctx) > { > ?? > if (ctx == NULL) > ?? > return 1; > ?? > /* > ?? > * Don't assume ctx->md_data was cleaned in EVP_Digest_Final, because > ?? > * sometimes only copies of the context are ever finalised. > ?? > */ > ?? > if (ctx->digest && ctx->digest->cleanup > ?? > && !EVP_MD_CTX_test_flags(ctx, EVP_MD_CTX_FLAG_CLEANED)) > ?? > ctx->digest->cleanup(ctx); > ?? > if (ctx->digest && ctx->digest->ctx_size && ctx->md_data > ?? > && !EVP_MD_CTX_test_flags(ctx, EVP_MD_CTX_FLAG_REUSE)) { > ?? > OPENSSL_clear_free(ctx->md_data, ctx->digest->ctx_size); > ?? > } > ?? > EVP_PKEY_CTX_free(ctx->pctx); > #ifndef OPENSSL_NO_ENGINE > ?? > ENGINE_finish(ctx->engine); > #endif > ?? > OPENSSL_cleanse(ctx, sizeof(*ctx)); > > ?? > return 1; > } > ?---->8----- > > > Other functions are getter and setters such as : > > ?----8<-----? > void DH_get0_key(const DH *dh, const BIGNUM **pub_key, const BIGNUM **priv_key) > { > if (pub_key != NULL) > *pub_key = dh->pub_key; > if (priv_key != NULL) > *priv_key = dh->priv_key; > } > > int DH_set0_key(DH *dh, BIGNUM *pub_key, BIGNUM *priv_key) > { > /* If the field pub_key in dh is NULL, the corresponding input > * parameters MUST be non-NULL. The priv_key field may > * be left NULL. > */ > if (dh->pub_key == NULL && pub_key == NULL) > return 0; > > if (pub_key != NULL) { > BN_free(dh->pub_key); > dh->pub_key = pub_key; > } > if (priv_key != NULL) { > BN_free(dh->priv_key); > dh->priv_key = priv_key; > } > > return 1; > } > > ?int RSA_bits(const RSA *r) > { > return (BN_num_bits(r->n)); > } > ?---->8----- > > > There are some simple security?-related functions such as OPENSSL_zalloc() or OPENSSL_clean_free() but these functions are quite simple too (and the reasoning behind them is, I assume, well known). > > > BR, > > -- Emmanuel Deloget > > ?[1] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/openssl_compat.h > ?-- Douglas E. Engert <DEEngert at gmail.com>
Having done this with libssh, this is far from trivial, even for the rather simple primitives required by SSH. Abstracting some concepts across very different libraries that deal with them in different ways (e.g. libcrypto vs libgcrypt) can introduce some nasty bugs. OpenSSH has always had KISS in mind so I wouldn't blame them to avoid supporting additional libraries or dropping OpenSSL 1.1 and sticking to LibreSSL altogether. On the subject of OpenSSL, Jakub Jelen provided us with an OpenSSL1.1-to-1.0 shim that works, but is not free of bugs. I would have definitively have appreciated that OpenSSL wrote that shim by themselves (they say it's trivial, of course it is not). The big problem currently is that any application that does nontrivial low-level cryptography cannot use a single API that will work with both of them, they're 100% incompatible. Aris On 24/06/17 14:06, George M. Garner Jr. wrote:> I think that this is the better approach. The question I have is why > the SSH logic should be dependent on the implementation details of ANY > particular cryptographic library (be it openssl, libressl or > whatever)? Proper software design would develop an abstraction layer > with some measure of forward compatibility built in. > > On 6/23/2017 3:16 PM, Douglas E Engert wrote: >> OpenSC has taken a different approach to OpenSSL-1.1. Rather then >> writing >> a shim for OpenSSL-1.1, the OpenSC code has been converted to >> the OpenSSL-1.1 API and a sc-ossl-compat.h" file consisting of >> defines and >> macros was written to support older versions of OpenSSL and Libressl. >> >> https://github.com/OpenSC/OpenSC/blob/master/src/libopensc/sc-ossl-compat.h >> >> >> The nice part of this approach is when using OpenSSL-1.1 >> sc-ossl-compat.h >> does not do anything. It sole purpose to provide calls to the older APIs >> that are not going to change and eventually the sc-ossl-compat.h >> could be >> removed. >> > _______________________________________________ > openssh-unix-dev mailing list > openssh-unix-dev at mindrot.org > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev >