openssh unix dev - Mar 2000 - Problem with 1.2.3pre4 and RSAref

The following code snippet will not compile support for RSAref on NetBSD even
if it exists on the system (which breaks OpenSSL):

        for WANTS_RSAREF in "" 1 ; do

                if test -z "$WANTS_RSAREF" ; then
                        LIBS="$saved_LIBS -lcrypto"
                else
                        LIBS="$saved_LIBS -lcrypto -lRSAglue -lrsaref"
                fi
....

It appears that the code referenced in configure just before line 1950 works
correctly even without the RSAglue and rsaref libraries, but "in
production"
work fails needing "_RSAPrivateDecrypt".

If someone wants to run a patch by me on this one, I'll be happy to
test it.

Thanks,
David


-- 
David W. Rankin, Jr.     Husband, Father, and UNIX Sysadmin. 
   Email: drankin at bohemians.lexington.ky.us   Address/Phone Number: Ask me.
"It's too bad she won't live! But then again, who does?"
             -- Gaff, _Blade Runner_

OK, I know it's not that tasteful to reply to your own post, but...

On Fri, Mar 17, 2000 at 04:25:53PM -0500, David Rankin
wrote:
> The following code snippet will not compile support for RSAref on NetBSD
even
> if it exists on the system (which breaks OpenSSL):
> 
>         for WANTS_RSAREF in "" 1 ; do
If the above line gets replaced with
	  for WANTS_RSAREF in 1 "" ; do
things seem to work. That said, I have to use rsaref, so I don't have
a system that can test the opposite assumptions.

Thanks
David

-- 
David W. Rankin, Jr.     Husband, Father, and UNIX Sysadmin. 
   Email: drankin at bohemians.lexington.ky.us   Address/Phone Number: Ask me.
"It's too bad she won't live! But then again, who does?"
             -- Gaff, _Blade Runner_

On Fri, 17 Mar 2000, David Rankin wrote:
> The following code snippet will not compile support for RSAref on
> NetBSD even if it exists on the system (which breaks OpenSSL):
[snip]
> It appears that the code referenced in configure just before line
> 1950 works correctly even without the RSAglue and rsaref libraries,
> but "in production" work fails needing
"_RSAPrivateDecrypt".
It looks like the test code is not complete enough.
> If someone wants to run a patch by me on this one, I'll be happy to
> test it.
Attached.

-d

--
| "Bombay is 250ms from New York in the new world order" - Alan Cox
| Damien Miller - http://www.mindrot.org/
| Email: djm at mindrot.org (home) -or- djm at ibs.com.au (work)


-------------- next part --------------
? configure
? config.h.in
? config.log
? config.cache
Index: configure.in
==================================================================RCS file:
/var/cvs/openssh/configure.in,v
retrieving revision 1.103
diff -u -r1.103 configure.in
--- configure.in	2000/03/17 12:26:46	1.103
+++ configure.in	2000/03/17 23:33:07
@@ -198,9 +198,14 @@
 				#include <openssl/bn.h>
 				#include <openssl/sha.h>
 				int main(void) 
-				{RSA *key; char seed[2048];memset(seed, 0, sizeof(seed));
-				RAND_seed(seed, sizeof(seed));key=RSA_generate_key(32,3,NULL,NULL);
-				return(key==NULL);}
+				{
+					RSA *key; char a[2048],b[2048];;
+					memset(a, 0, sizeof(a));memset(b, 0, sizeof(b));
+					RAND_seed(a, sizeof(a));
+					key=RSA_generate_key(32,3,NULL,NULL);
+					if (key==NULL) return(1);
+					return(-1==RSA_private_decrypt(RSA_size(key),a,b,key,RSA_NO_PADDING));
+				}
 			],
 			[
 				AC_DEFINE(HAVE_OPENSSL)
@@ -214,9 +219,14 @@
 				#include <ssl/bn.h>
 				#include <ssl/sha.h>
 				int main(void) 
-				{RSA *key; char seed[2048];memset(seed, 0, sizeof(seed));
-				RAND_seed(seed, sizeof(seed));key=RSA_generate_key(32,3,NULL,NULL);
-				return(key==NULL);}
+				{
+					RSA *key; char a[2048],b[2048];;
+					memset(a, 0, sizeof(a));memset(b, 0, sizeof(b));
+					RAND_seed(a, sizeof(a));
+					key=RSA_generate_key(32,3,NULL,NULL);
+					if (key==NULL) return(1);
+					return(-1==RSA_private_decrypt(RSA_size(key),a,b,key,RSA_NO_PADDING));
+				}
 			],
 			[
 				AC_DEFINE(HAVE_SSL)