openssh unix dev - Jan 2013 - Inconsisten declaration of ssh_aes_ctr_iv()

Hi,

The 20130110 snapshot fails to build against OpenSSL 0.9.8 and 1.0.0
with the following error:

gcc -g -O2 -Wall -Wpointer-arith -Wuninitialized -Wsign-compare
-Wformat-security -Wno-pointer-sign -fno-strict-aliasing -D_FORTIFY_SOURCE=2
-fno-builtin-memset -fstack-protector-all  -I. -I. 
-DSSHDIR=\"/tmp/foo/etc\"
-D_PATH_SSH_PROGRAM=\"/tmp/foo/bin/ssh\"
-D_PATH_SSH_ASKPASS_DEFAULT=\"/tmp/foo/libexec/ssh-askpass\"
-D_PATH_SFTP_SERVER=\"/tmp/foo/libexec/sftp-server\"
-D_PATH_SSH_KEY_SIGN=\"/tmp/foo/libexec/ssh-keysign\"
-D_PATH_SSH_PKCS11_HELPER=\"/tmp/foo/libexec/ssh-pkcs11-helper\"
-D_PATH_SSH_PIDDIR=\"/var/run\"
-D_PATH_PRIVSEP_CHROOT_DIR=\"/var/empty\"  -DHAVE_CONFIG_H -c
cipher-ctr.c
cipher-ctr.c:38: error: conflicting types for ?ssh_aes_ctr_iv?
openbsd-compat/openssl-compat.h:71: note: previous declaration of
?ssh_aes_ctr_iv? was here
cipher-ctr.c:117: error: conflicting types for ?ssh_aes_ctr_iv?
openbsd-compat/openssl-compat.h:71: note: previous declaration of
?ssh_aes_ctr_iv? was here
cipher-ctr.c: In function ?evp_aes_128_ctr?:
cipher-ctr.c:141: warning: assignment from incompatible pointer type
make: *** [cipher-ctr.o] Error 1

The following trivial patch addresses the compilation error, but does
not address an the incompatible pointer warning which is encountered at
line 141. (Note that there are similar warnings with cipher-bf1.c and
cipher-3des.c.)

Index: openbsd-compat/openssl-compat.h
==================================================================RCS file:
/cvs/openssh/openbsd-compat/openssl-compat.h,v
retrieving revision 1.21
diff -u -r1.21 openssl-compat.h
--- openbsd-compat/openssl-compat.h	9 Jan 2013 05:42:49 -0000	1.21
+++ openbsd-compat/openssl-compat.h	14 Jan 2013 18:39:27 -0000
@@ -68,7 +68,7 @@
 #define EVP_aes_192_ctr evp_aes_128_ctr
 #define EVP_aes_256_ctr evp_aes_128_ctr
 extern const EVP_CIPHER *evp_aes_128_ctr(void);
-extern void ssh_aes_ctr_iv(EVP_CIPHER_CTX *, int, u_char *, u_int);
+extern void ssh_aes_ctr_iv(EVP_CIPHER_CTX *, int, u_char *, size_t);
 #endif
 
 /* Avoid some #ifdef. Code that uses these is unreachable without GCM */

-- 
Iain Morgan

On Mon, 14 Jan 2013, Iain Morgan wrote:
> Hi,
> 
> The 20130110 snapshot fails to build against OpenSSL 0.9.8 and 1.0.0
> with the following error:
...

Try this:

Index: cipher-aes.c
==================================================================RCS file:
/var/cvs/openssh/cipher-aes.c,v
retrieving revision 1.9
diff -u -p -r1.9 cipher-aes.c
--- cipher-aes.c	7 Oct 2010 11:06:42 -0000	1.9
+++ cipher-aes.c	14 Jan 2013 21:14:23 -0000
@@ -46,9 +46,6 @@ struct ssh_rijndael_ctx
 	u_char		r_iv[RIJNDAEL_BLOCKSIZE];
 };
 
-const EVP_CIPHER * evp_rijndael(void);
-void ssh_rijndael_iv(EVP_CIPHER_CTX *, int, u_char *, u_int);
-
 static int
 ssh_rijndael_init(EVP_CIPHER_CTX *ctx, const u_char *key, const u_char *iv,
     int enc)
Index: cipher-ctr.c
==================================================================RCS file:
/var/cvs/openssh/cipher-ctr.c,v
retrieving revision 1.16
diff -u -p -r1.16 cipher-ctr.c
--- cipher-ctr.c	12 Dec 2012 21:18:56 -0000	1.16
+++ cipher-ctr.c	14 Jan 2013 21:13:03 -0000
@@ -34,9 +34,6 @@
 #include <openssl/aes.h>
 #endif
 
-const EVP_CIPHER *evp_aes_128_ctr(void);
-void ssh_aes_ctr_iv(EVP_CIPHER_CTX *, int, u_char *, size_t);
-
 struct ssh_aes_ctr_ctx
 {
 	AES_KEY		aes_ctx;
Index: openbsd-compat/openssl-compat.h
==================================================================RCS file:
/var/cvs/openssh/openbsd-compat/openssl-compat.h,v
retrieving revision 1.21
diff -u -p -r1.21 openssl-compat.h
--- openbsd-compat/openssl-compat.h	9 Jan 2013 05:42:49 -0000	1.21
+++ openbsd-compat/openssl-compat.h	14 Jan 2013 21:14:34 -0000
@@ -59,16 +59,16 @@
 # define EVP_aes_128_cbc evp_rijndael
 # define EVP_aes_192_cbc evp_rijndael
 # define EVP_aes_256_cbc evp_rijndael
-extern const EVP_CIPHER *evp_rijndael(void);
-extern void ssh_rijndael_iv(EVP_CIPHER_CTX *, int, u_char *, u_int);
+const EVP_CIPHER *evp_rijndael(void);
+void ssh_rijndael_iv(EVP_CIPHER_CTX *, int, u_char *, u_int);
 #endif
 
 #ifndef OPENSSL_HAVE_EVPCTR
 #define EVP_aes_128_ctr evp_aes_128_ctr
 #define EVP_aes_192_ctr evp_aes_128_ctr
 #define EVP_aes_256_ctr evp_aes_128_ctr
-extern const EVP_CIPHER *evp_aes_128_ctr(void);
-extern void ssh_aes_ctr_iv(EVP_CIPHER_CTX *, int, u_char *, u_int);
+const EVP_CIPHER *evp_aes_128_ctr(void);
+void ssh_aes_ctr_iv(EVP_CIPHER_CTX *, int, u_char *, size_t);
 #endif
 
 /* Avoid some #ifdef. Code that uses these is unreachable without GCM */

On Mon, Jan 14, 2013 at 15:15:43 -0600, Damien Miller
wrote:
> On Mon, 14 Jan 2013, Iain Morgan wrote:
> 
> > Hi,
> > 
> > The 20130110 snapshot fails to build against OpenSSL 0.9.8 and 1.0.0
> > with the following error:
> ...
> 
> Try this:
> 
Thanks. The snapshot builds with the supplied patch, but there is stall
a warning in cipher-ctr.c:

cipher-ctr.c: In function ?evp_aes_128_ctr?:
cipher-ctr.c:138: warning: assignment from incompatible pointer type

For completeness, the following warnings show up during the build.

cipher-acss.c:79: warning: assignment from incompatible pointer type
cipher-bf1.c:100: warning: assignment from incompatible pointer type
cipher-bf1.c:105: warning: assignment from incompatible pointer type
cipher-ctr.c:138: warning: assignment from incompatible pointer type
cipher-3des1.c:178: warning: assignment from incompatible pointer type

I ran the regression tests, but they fail when building against older
OpenSSL versions due to the lack of GCM support.

-- 
Iain Morgan

On Thu, Jan 17, 2013 at 17:19:35 -0600, Damien Miller
wrote:
> On Thu, 17 Jan 2013, Iain Morgan wrote:
> 
> > > Could you tell me the declaration of the function pointer
do_cipher in
> > > OpenSSL's evp.h on your system and the the
OPENSSL_VERSION_NUMBER from
> > > opensslv.h?
> > 
> > % grep -A1 do_cipher /usr/include/openssl/evp.h
> > 	int (*do_cipher)(EVP_CIPHER_CTX *ctx, unsigned char *out,
> > 			 const unsigned char *in, size_t inl);/* encrypt/decrypt data */
> > 
> > % grep OPENSSL_VERSION_NUMBER /usr/include/openssl/opensslv.h
> > #define OPENSSL_VERSION_NUMBER	0x10000003L
> 
> Is this a beta? This should help (on top of you existing diffs):
No, this is the version included with RHEL 6.3.

% /usr/bin/openssl version -a
OpenSSL 1.0.0-fips 29 Mar 2010
built on: Wed Aug 15 12:48:02 EDT 2012
platform: linux-x86_64
options:  bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) blowfish(idx) 
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT
-DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g
-pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DMD32_REG_T=int
-DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM
-DMD5_ASM -DAES_ASM -DWHIRLPOOL_ASM
OPENSSLDIR: "/etc/pki/tls"
engines:  aesni dynamic 

That cleared up the warnings.

-- 
Iain Morgan