dovecot - Feb 2012 - Dovecot 2.1 with custom OpenSSL fails to build

Hello Dovecot users!

There seems to be a new dependency in some modules (eg, lib-storage,
libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those
modules didn't require OpenSSL, but 2.1 does.

For the linking process the path to the OpenSSL library isn't
specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL
is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't
properly used as well.)

Example for lib-storage:

env SSL_CFLAGS="-I/usr/local/ssl/include"
SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto
-lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl
--with-ssldir=/usr/local/${DOVECOTVER}/etc/dovecot/certs

make

gcc -DHAVE_CONFIG_H -I. -I../..  -I../../src/lib -I../../src/lib-master
-I../../src/lib-settings -I../../src/lib-mail -I../../src/lib-storage
-DPKG_RUNDIR=\""/usr/local/Dovecot-2.1.1/var/run/dovecot"\" 
-std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations
-Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast
-fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include  -MT
master-connection.o -MD -MP -MF .deps/master-connection.Tpo -c -o
master-connection.o master-connection.c
mv -f .deps/master-connection.Tpo .deps/master-connection.Po
/bin/sh ../../libtool --tag=CC   --mode=link gcc  -std=gnu99 -g -O2 -Wall -W
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts
-Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2
-I/usr/local/ssl/include   -no-undefined -Wl,--as-needed  -o indexer-worker
indexer-worker.o indexer-worker-settings.o master-connection.o
../../src/lib-storage/libdovecot-storage.la  ../../src/lib-dovecot/libdovecot.la
-export-dynamic -ldl -lrt
libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes
-Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2
-Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2
-I/usr/local/ssl/include -Wl,--as-needed -o .libs/indexer-worker
indexer-worker.o indexer-worker-settings.o master-connection.o
-Wl,--export-dynamic  ../../src/lib-storage/.libs/libdovecot-storage.so
../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath
-Wl,/usr/local/Dovecot-2.1.1/lib/dovecot
/usr/bin/ld: warning: libcrypto.so.1.0.0, needed by
../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath
or -rpath-link)
/usr/bin/ld: warning: libssl.so.1.0.0, needed by
../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath
or -rpath-link)
../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to
`OBJ_txt2nid'
../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to
`SSL_set_ex_data'
[...]

# ldd src/lib*/.libs/*.so
src/lib-dovecot/.libs/libdovecot.so:
        linux-gate.so.1 =>  (0x00979000)
        libdl.so.2 => /lib/libdl.so.2 (0x00c31000)
        librt.so.1 => /lib/librt.so.1 (0x009ed000)
        libc.so.6 => /lib/libc.so.6 (0x0054d000)
        /lib/ld-linux.so.2 (0x00c12000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x008f4000)
src/lib-lda/.libs/libdovecot-lda.so:
        linux-gate.so.1 =>  (0x0096d000)
        libdovecot-storage.so.0 =>
/usr/local/src/dovecot-2.1.1/src/lib-storage/.libs/libdovecot-storage.so.0
(0x00ae4000)
        libdovecot.so.0 =>
/usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00110000)
        librt.so.1 => /lib/librt.so.1 (0x00189000)
        libc.so.6 => /lib/libc.so.6 (0x00192000)
        libcrypto.so.1.0.0 => not found
        libssl.so.1.0.0 => not found
        libdl.so.2 => /lib/libdl.so.2 (0x0031c000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00ebc000)
        /lib/ld-linux.so.2 (0x003f8000)
src/lib-sql/.libs/libdovecot-sql.so:
        linux-gate.so.1 =>  (0x00a95000)
        libdovecot.so.0 =>
/usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x002b6000)
        libdl.so.2 => /lib/libdl.so.2 (0x00ab8000)
        librt.so.1 => /lib/librt.so.1 (0x0072a000)
        libc.so.6 => /lib/libc.so.6 (0x008f7000)
        /lib/ld-linux.so.2 (0x00ebd000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00619000)
src/lib-ssl-iostream/.libs/libdovecot-ssl.so:
        linux-gate.so.1 =>  (0x00da6000)
        libcrypto.so.1.0.0 => not found
        libssl.so.1.0.0 => not found
        libdl.so.2 => /lib/libdl.so.2 (0x00df0000)
        librt.so.1 => /lib/librt.so.1 (0x00112000)
        libc.so.6 => /lib/libc.so.6 (0x00b5e000)
        /lib/ld-linux.so.2 (0x00e76000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x005a6000)
src/lib-storage/.libs/libdovecot-storage.so:
        linux-gate.so.1 =>  (0x006ed000)
        libcrypto.so.1.0.0 => not found
        libssl.so.1.0.0 => not found
        libdovecot.so.0 =>
/usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00c66000)
        libdl.so.2 => /lib/libdl.so.2 (0x00491000)
        librt.so.1 => /lib/librt.so.1 (0x00be2000)
        libc.so.6 => /lib/libc.so.6 (0x00110000)
        /lib/ld-linux.so.2 (0x0036b000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x0029a000)


Without SSL_CFLAGS and SSL_LIBS but just PKG_CONFIG_PATH,
it also won't work.

	Help appreciated ... Andreas

On Fri, 2012-02-24 at 02:22 +0100, Andreas M. Kirchwitz wrote:
> There seems to be a new dependency in some modules (eg, lib-storage,
> libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those
> modules didn't require OpenSSL, but 2.1 does.
> 
> For the linking process the path to the OpenSSL library isn't
> specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL
> is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't
> properly used as well.)
Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes
everything?

You can try a new nightly snapshot from http://dovecot.org/nightly/ once
it gets there in a few minutes.

Timo Sirainen <tss at iki.fi> wrote:

 >> There seems to be a new dependency in some modules (eg, lib-storage,
 >> libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those
 >> modules didn't require OpenSSL, but 2.1 does.
 >> 
 >> For the linking process the path to the OpenSSL library isn't
 >> specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL
 >> is in a non-standard path. (Haven't checked if SSL_CFLAGS
isn't
 >> properly used as well.)
 >
 > Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes
 > everything?
 
That's the way to go. Makes things better, but I've found three more
dependencies. This patch is against the daily snapshot 20120228.
(Sorry for the changes to Makefile.in which you won't need. But this way
I don't have to rebuild Makefile.in from Makefile.am when compiling. ;-)


diff -ur dovecot-20120228.original/src/auth/Makefile.am
dovecot-20120228/src/auth/Makefile.am
--- dovecot-20120228.original/src/auth/Makefile.am	2012-02-26 04:00:02.000000000
+0100
+++ dovecot-20120228/src/auth/Makefile.am	2012-02-29 01:38:41.000000000 +0100
@@ -167,7 +167,7 @@
 libauthdb_imap_la_LIBADD = \
 	../lib-imap-client/libimap_client.la \
 	../lib-ssl-iostream/libssl_iostream.la \
-	$(LIBDOVECOT)
+	$(LIBDOVECOT) $(SSL_LIBS)
 libauthdb_imap_la_CPPFLAGS = \
 	$(AM_CPPFLAGS) \
 	-I$(top_srcdir)/src/lib-imap \
diff -ur dovecot-20120228.original/src/auth/Makefile.in
dovecot-20120228/src/auth/Makefile.in
--- dovecot-20120228.original/src/auth/Makefile.in	2012-02-28 04:00:12.000000000
+0100
+++ dovecot-20120228/src/auth/Makefile.in	2012-02-29 01:38:44.000000000 +0100
@@ -499,7 +499,7 @@
 libauthdb_imap_la_LIBADD = \
 	../lib-imap-client/libimap_client.la \
 	../lib-ssl-iostream/libssl_iostream.la \
-	$(LIBDOVECOT)
+	$(LIBDOVECOT) $(SSL_LIBS)
 
 libauthdb_imap_la_CPPFLAGS = \
 	$(AM_CPPFLAGS) \
diff -ur dovecot-20120228.original/src/lib-storage/Makefile.am
dovecot-20120228/src/lib-storage/Makefile.am
--- dovecot-20120228.original/src/lib-storage/Makefile.am	2012-02-10
04:00:02.000000000 +0100
+++ dovecot-20120228/src/lib-storage/Makefile.am	2012-02-29 01:38:21.000000000
+0100
@@ -85,7 +85,7 @@
 
 pkglib_LTLIBRARIES = libdovecot-storage.la
 libdovecot_storage_la_SOURCES = 
-libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS)
+libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS)
 libdovecot_storage_la_DEPENDENCIES = $(shlibs)
 libdovecot_storage_la_LDFLAGS = -export-dynamic
 
diff -ur dovecot-20120228.original/src/lib-storage/Makefile.in
dovecot-20120228/src/lib-storage/Makefile.in
--- dovecot-20120228.original/src/lib-storage/Makefile.in	2012-02-28
04:00:15.000000000 +0100
+++ dovecot-20120228/src/lib-storage/Makefile.in	2012-02-29 01:38:21.000000000
+0100
@@ -397,7 +397,7 @@
 
 pkglib_LTLIBRARIES = libdovecot-storage.la
 libdovecot_storage_la_SOURCES = 
-libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS)
+libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS)
 libdovecot_storage_la_DEPENDENCIES = $(shlibs)
 libdovecot_storage_la_LDFLAGS = -export-dynamic
 test_programs = \
diff -ur dovecot-20120228.original/src/login-common/Makefile.am
dovecot-20120228/src/login-common/Makefile.am
--- dovecot-20120228.original/src/login-common/Makefile.am	2011-11-09
04:00:01.000000000 +0100
+++ dovecot-20120228/src/login-common/Makefile.am	2012-02-29 01:38:21.000000000
+0100
@@ -40,6 +40,6 @@
 
 pkglib_LTLIBRARIES = libdovecot-login.la
 libdovecot_login_la_SOURCES = 
-libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la
../lib-dovecot/libdovecot.la
+libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la
../lib-dovecot/libdovecot.la $(SSL_LIBS)
 libdovecot_login_la_DEPENDENCIES = liblogin.la
 libdovecot_login_la_LDFLAGS = -export-dynamic
diff -ur dovecot-20120228.original/src/login-common/Makefile.in
dovecot-20120228/src/login-common/Makefile.in
--- dovecot-20120228.original/src/login-common/Makefile.in	2012-02-28
04:00:18.000000000 +0100
+++ dovecot-20120228/src/login-common/Makefile.in	2012-02-29 01:38:21.000000000
+0100
@@ -320,7 +320,7 @@
 pkginc_lib_HEADERS = $(headers)
 pkglib_LTLIBRARIES = libdovecot-login.la
 libdovecot_login_la_SOURCES = 
-libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la
../lib-dovecot/libdovecot.la
+libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la
../lib-dovecot/libdovecot.la $(SSL_LIBS)
 libdovecot_login_la_DEPENDENCIES = liblogin.la
 libdovecot_login_la_LDFLAGS = -export-dynamic
 all: all-am

Timo Sirainen <tss at iki.fi> wrote:

 >> $ patch -p1 -s < ../dovecot-20120303-e540404debb7.patch
 >> $ env SSL_CFLAGS="-I/usr/local/ssl/include"
SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto
-lssl" ./configure --prefix=/usr/local/Dovecot-20120303 --with-ssl=openssl
--with-ssldir=/usr/local/Dovecot-20120303/etc/dovecot/certs && make
&& make install
 >
 > You would have needed to run autogen.sh again. It works with me now that
 > I tried in a test server with OpenSSL in non-standard dir.

Sorry, I didn't know that with "autogen.sh". Just grabbed Dovecot
2.1.2
(which is all properly set up - so I couldn't do anything wrong ;-) and
compiled it.

Compilation works. Great! The binaries find all their libraries.

But two libraries are not quite okay. They don't find their SSL libs:

	libdovecot-lda.so
	libdovecot-storage.so

Since libdovecot-lda.so doesn't contain the words libssl or libcrypto,
I guess that ldd just complains because it uses libdovecot-storage.so.
Thus, libdovecot-storage.so is the (only) one left with an incomplete
library search path.

Luckily, all binaries use some additional libraries which come with
a proper library path. So the whole things works, but it's more like
some kind of magic. It would be great if libdovecot-storage.so could
be fixed as well to make things finally perfect.

Thanks for all your effort. (I know this isn't top priority as most
people use precompiled stuff and never run into such kind of things.)

	Greetings, Andreas