Hi, we are using a round robin dns record for connections to our ldap system. This works fine for almost all cases. In particular, for dovecot does this mean, when an ldap server is stopped, dovecot instantly reconnects to another ldap server. But when the network connection to the active ldap server is broken, dovecot sticks to the failed ldap server. Is there any possibility to define a connection timeout? This is our current configuration for the ldap connection: # 2.2.devel (2d8f665): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (0de2a19) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.3 uris = ldaps://LDAP-SERVER dn = BINDDN dnpass = BINDPASS auth_bind = yes ldap_version = 3 base = BASEDN scope = subtree user_attrs= ATTRIBUTES user_filter = USERFILTER pass_filter = PASSFILTER iterate_filter = ITERATEFILTER Best regards, Gordon -- Technischer Leiter & stellv. Direktor Universit?tsrechenzentrum (URZ) E.-M.-Arndt-Universit?t Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86 1456 Fax. +49 3834 86 1401
On 29 Feb 2016, at 17:18, Gordon Grubert <gordon.grubert+lists at uni-greifswald.de> wrote:> > Hi, > > we are using a round robin dns record for connections to our ldap > system. This works fine for almost all cases. In particular, for > dovecot does this mean, when an ldap server is stopped, dovecot > instantly reconnects to another ldap server. > > But when the network connection to the active ldap server is broken, > dovecot sticks to the failed ldap server. Is there any possibility to > define a connection timeout?What should happen is that as long as new requests keep coming, Dovecot realizes after about 60 seconds that the LDAP server is hanging. It then reconnects and the reconnection should work. But... First of all, 60 seconds is likely a much too long timeout. But more importantly it looks like there's something weird now going on with OpenLDAP library. I added this somewhat recently and tested that it works: https://github.com/dovecot/core/commit/fb3178a1924dae52151d88c4d4ded879df43dd3f But now that I'm testing it, the timeout doesn't seem to be triggering. I don't know what happened to it that it suddenly doesn't work.. This also means that OpenLDAP seems to be internally stuck trying to connect to a server that isn't responding. Dovecot doesn't currently make the decisions on which LDAP server to connect to. It just passes through all the hosts to OpenLDAP library and lets it handle it. And it seems like OpenLDAP library can't right now do this failover. So maybe Dovecot should be responsible for that as well.. Anyway, for now you could set up haproxy to localhost and configure Dovecot LDAP to connect to haproxy and haproxy connect to the actual LDAP servers.
Hi, We have experienced the same or similar problem, and not just with dovecot but also with postfix. Thanks for your HAProxy suggestion! We have the feeling that when the ldap connection is actually DOWN (gone, terminated), OpenLDAP will reconnect to another server. But if the ldap server becomes 'stuck' (as in: returning no data anymore, but not actually terminating the connection) a failover does not happen. (we have had the second scenario, with samba4 AD ldap) MJ On 03/01/2016 10:51 PM, Timo Sirainen wrote:> But now that I'm testing it, the timeout doesn't seem to be > triggering. I don't know what happened to it that it suddenly doesn't > work.. This also means that OpenLDAP seems to be internally stuck > trying to connect to a server that isn't responding. Dovecot doesn't > currently make the decisions on which LDAP server to connect to. It > just passes through all the hosts to OpenLDAP library and lets it > handle it. And it seems like OpenLDAP library can't right now do this > failover. So maybe Dovecot should be responsible for that as well.. > > Anyway, for now you could set up haproxy to localhost and configure > Dovecot LDAP to connect to haproxy and haproxy connect to the actual > LDAP servers. >
Hi Timo, On 03/01/2016 10:51 PM, Timo Sirainen wrote:> On 29 Feb 2016, at 17:18, Gordon Grubert <gordon.grubert+lists at uni-greifswald.de> wrote: >> >> Hi, >> >> we are using a round robin dns record for connections to our ldap >> system. This works fine for almost all cases. In particular, for >> dovecot does this mean, when an ldap server is stopped, dovecot >> instantly reconnects to another ldap server. >> >> But when the network connection to the active ldap server is broken, >> dovecot sticks to the failed ldap server. Is there any possibility to >> define a connection timeout? > > What should happen is that as long as new requests keep coming, Dovecot realizes after about 60 seconds that the LDAP server is hanging. It then reconnects and the reconnection should work. But... First of all, 60 seconds is likely a much too long timeout. > > But more importantly it looks like there's something weird now going on with OpenLDAP library. I added this somewhat recently and tested that it works: > > https://github.com/dovecot/core/commit/fb3178a1924dae52151d88c4d4ded879df43dd3fthx a lot. I'll test this ASAP. IMHO, this will not really help, because the timeout is relevant when connecting to the LDAP server only and not for an active session, or?> But now that I'm testing it, the timeout doesn't seem to be triggering. I don't know what happened to it that it suddenly doesn't work.. This also means that OpenLDAP seems to be internally stuck trying to connect to a server that isn't responding. Dovecot doesn't currently make the decisions on which LDAP server to connect to. It just passes through all the hosts to OpenLDAP library and lets it handle it. And it seems like OpenLDAP library can't right now do this failover. So maybe Dovecot should be responsible for that as well..You're right, that there are some modifications in the OpenLDAP client. In 2014, the option BIND_POLICY in ldap.conf still existed. The current version does not support this option :-(> Anyway, for now you could set up haproxy to localhost and configure Dovecot LDAP to connect to haproxy and haproxy connect to the actual LDAP servers.I'll tke a look on it. Thx and best regards, Gordon -- Technischer Leiter & stellv. Direktor Universit?tsrechenzentrum (URZ) E.-M.-Arndt-Universit?t Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86 1456 Fax. +49 3834 86 1401
Hello Gordon, On 29.02.2016 16:18, Gordon Grubert wrote:> we are using a round robin dns record for connections to our ldap > system. This works fine for almost all cases. In particular, for > dovecot does this mean, when an ldap server is stopped, dovecot > instantly reconnects to another ldap server. > > But when the network connection to the active ldap server is broken, > dovecot sticks to the failed ldap server. Is there any possibility to > define a connection timeout?What about replicating the directory onto the dovecot host and send the ldap queries to localhost? Regards Christian -- No signature available. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5326 bytes Desc: S/MIME Cryptographic Signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20160303/36e556f6/attachment.p7s>
Hello Christian, On 03/03/2016 09:09 AM, Christian Schmidt wrote:> Hello Gordon, > > On 29.02.2016 16:18, Gordon Grubert wrote: >> we are using a round robin dns record for connections to our ldap >> system. This works fine for almost all cases. In particular, for >> dovecot does this mean, when an ldap server is stopped, dovecot >> instantly reconnects to another ldap server. >> >> But when the network connection to the active ldap server is broken, >> dovecot sticks to the failed ldap server. Is there any possibility to >> define a connection timeout? > > What about replicating the directory onto the dovecot host and send the > ldap queries to localhost?of course, this would be possible. E.g., for our DNS we are using exactly this solution. But this means, there is one additional daemon. Additionally, this leads to an mailserver interruption when updating the local ldap daemon. But you are right, the "connection loss" problem over the network can be prevented. Best regards, Gordon
Hi Timo, On 01.03.2016 22:51, Timo Sirainen wrote:> On 29 Feb 2016, at 17:18, Gordon Grubert <gordon.grubert+lists at uni-greifswald.de> wrote: >> >> Hi, >> >> we are using a round robin dns record for connections to our ldap >> system. This works fine for almost all cases. In particular, for >> dovecot does this mean, when an ldap server is stopped, dovecot >> instantly reconnects to another ldap server. >> >> But when the network connection to the active ldap server is broken, >> dovecot sticks to the failed ldap server. Is there any possibility to >> define a connection timeout? > > What should happen is that as long as new requests keep coming, Dovecot realizes after about 60 seconds that the LDAP server is hanging. It then reconnects and the reconnection should work. But... First of all, 60 seconds is likely a much too long timeout. > > But more importantly it looks like there's something weird now going on with OpenLDAP library. I added this somewhat recently and tested that it works: > > https://github.com/dovecot/core/commit/fb3178a1924dae52151d88c4d4ded879df43dd3f > > But now that I'm testing it, the timeout doesn't seem to be triggering. I don't know what happened to it that it suddenly doesn't work.. This also means that OpenLDAP seems to be internally stuck trying to connect to a server that isn't responding. Dovecot doesn't currently make the decisions on which LDAP server to connect to. It just passes through all the hosts to OpenLDAP library and lets it handle it. And it seems like OpenLDAP library can't right now do this failover. So maybe Dovecot should be responsible for that as well.. > > Anyway, for now you could set up haproxy to localhost and configure Dovecot LDAP to connect to haproxy and haproxy connect to the actual LDAP servers. >today I've upgraded to 2.2.21-1~auto+171 on debian 8 and made a lot of "interruption tests". Your fix not really solved the problem. But I found another interesting fact: The openldap client on debian 8 can handle hard communication interrupts correctly. I've added NETWORK_TIMEOUT 5 TIMEOUT 5 to ldap.conf because man 5 ldap.conf says: NETWORK_TIMEOUT <integer> Specifies the timeout (in seconds) after which the poll(2)/select(2) following a connect(2) returns in case of no activity. TIMEOUT <integer> Specifies a timeout (in seconds) after which calls to synchronous LDAP APIs will abort if no response is received. Also used for any ldap_result(3) calls where a NULL timeout parameter is supplied. We are using the ISC DHCP server with dynamic ldap connections. This daemon uses - like dovecot - the LDAP API of the openldap client for access to the ldap server. The DHCP opens a persistent ldap connection to handle all dhcp requests (same behavior like dovecot). Here, the timeouts for connection loss are working. Therefore, my question: Why does this not work for dovecot, too, when dovecot uses the same API? Dovecot does not get a response from the LDAP server and has to reconnect, only. IMAP server world domination requires a reconnect in case of connection timeouts ;-) Best regards, Gordon -- Technischer Leiter & stellv. Direktor Universit?tsrechenzentrum (URZ) E.-M.-Arndt-Universit?t Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86 1456 Fax. +49 3834 86 1401 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5333 bytes Desc: S/MIME Cryptographic Signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20160310/37247283/attachment.p7s>