dovecot - Jan 2018 - Dovecot auth SASL for exim and plain auth issue without initial response

Op 1/3/2018 om 8:31 AM schreef Daniel Kenzelmann:
> 3. Januar 2018 00:49, "Stephan Bosch" <stephan at
rename-it.nl> schrieb:
>
>> Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
>>
>>> Hi,
>>>
>>> I'm not entirely sure whether this issue is with exim or with
dovecot.
>>>
>>> First some background:
>>> I'm using exim with dovecot-auth which in turn is using LDAP
for
>>> authentication.
>>>
>>> When using AUTH PLAIN with the optional initial response argument,
>>> everything is fine.
>>>
>>> However when using AUTH PLAIN without the optional response
argument,
>>> instead of getting an empty challenge ("334 ") as per RFC
i am getting
>>> a "535 Incorrect authentication data".
>>>
>>> Example:
>>> Working:
>>> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
>>> EHLO test
>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>> 250-SIZE 52428800
>>> 250-8BITMIME
>>> 250-PIPELINING
>>> 250-AUTH PLAIN LOGIN
>>> 250-CHUNKING
>>> 250 HELP
>>> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=>>> 235 Authentication
succeeded
>>>
>>> NOT-WORKING:
>>> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
>>> EHLO test
>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>> 250-SIZE 52428800
>>> 250-8BITMIME
>>> 250-PIPELINING
>>> 250-AUTH PLAIN LOGIN
>>> 250-CHUNKING
>>> 250 HELP
>>> AUTH PLAIN
>>> 535 Incorrect authentication data
>>>
>>> Here the SASL mechanism should return an empty challenge as per RFC
>>> (i.e. "334 " in SMTP):
>> This is a an error produced by Exim. I find the Exim error handling in
>> Exim's implementation of the AUTH command rather peculiar. Still, I
>> managed to decipher at least part of it.
>>
>> That error is produced when FAIL status is returned from the driver:
>>
>> https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665
>>
>> This FAIL status can be returned by the driver itself, but -- in this
>> case more likely -- the Dovecot driver in Exim also returns FAIL status
>> when Dovecot auth service returns "FAIL":
>>
>> https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472
>>
>> So, this may very well be an issue triggered by Dovecot. What version
of
>> Dovecot is this? Some things were modified in initial response handling
>> recently (v2.3) and I may have messed up something.
>>
>> Does Dovecot log anything interesting with auth_verbose and auth_debug
>> enabled?
>>
>> Regards,
>>
>> Stephan.
>
> Hi,
>
> System is gentoo,
> dovecot version is 2.3.0
> exim version is 4.90
>
> Debug log does only show the following:
> auth: Debug: auth client connected (pid=0)
> auth: Debug: client in: AUTH   1       PLAIN   service=smtp    secured
rip=XX.XX.XX.XX       lip=XX.XX.XX.XX       nologin resp=<hidden>
> auth: plain(?,XX.XX.XX.XX): invalid input
> auth: Debug: client passdb out: FAIL   1
>
> I'm not 100% sure but i think it worked earlier, so this might be
connected to the 2.3 update. (if REALLY needed i can try to confirm by
downgrading dovecot)
Ok. I know what is going on already. This commit triggers the problem:

https://github.com/dovecot/core/commit/e4b72bd73bfffda7906faa248eab31f936cfc6fa

That fix was added to handle the EXTERNAL SASL mechanism properly when
used in ManageSieve, and somehow I didn't realize that the original
comment means that Exim would also send an empty resp field for an
absent initial response:

https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L403

This is now handled as an empty initial response instead (as it should
be), which -- in this case -- makes the PLAIN mechanism complain about
invalid data.

So, the fundamental blame lies with Exim for violating the protocol.
However, I don't think it is a good idea to break compatibility like
that, especially when we want to back-port this fix to Dovecot v2.2.

To solve this now, we can recognize an empty initial response for
service=smtp differently (EXTERNAL is not used there much I think) and
perhaps make that configurable with some setting.

Regards,

Stephan.

Op 1/3/2018 om 10:58 AM schreef Stephan Bosch:
> Op 1/3/2018 om 8:31 AM schreef Daniel Kenzelmann:
>> 3. Januar 2018 00:49, "Stephan Bosch" <stephan at
rename-it.nl> schrieb:
>>
>>> Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
>>>
>>>> Hi,
>>>>
>>>> I'm not entirely sure whether this issue is with exim or
with dovecot.
>>>>
>>>> First some background:
>>>> I'm using exim with dovecot-auth which in turn is using
LDAP for
>>>> authentication.
>>>>
>>>> When using AUTH PLAIN with the optional initial response
argument,
>>>> everything is fine.
>>>>
>>>> However when using AUTH PLAIN without the optional response
argument,
>>>> instead of getting an empty challenge ("334 ") as per
RFC i am getting
>>>> a "535 Incorrect authentication data".
>>>>
>>>> Example:
>>>> Working:
>>>> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
>>>> EHLO test
>>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>>> 250-SIZE 52428800
>>>> 250-8BITMIME
>>>> 250-PIPELINING
>>>> 250-AUTH PLAIN LOGIN
>>>> 250-CHUNKING
>>>> 250 HELP
>>>> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=>>>> 235
Authentication succeeded
>>>>
>>>> NOT-WORKING:
>>>> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
>>>> EHLO test
>>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>>> 250-SIZE 52428800
>>>> 250-8BITMIME
>>>> 250-PIPELINING
>>>> 250-AUTH PLAIN LOGIN
>>>> 250-CHUNKING
>>>> 250 HELP
>>>> AUTH PLAIN
>>>> 535 Incorrect authentication data
>>>>
>>>> Here the SASL mechanism should return an empty challenge as per
RFC
>>>> (i.e. "334 " in SMTP):
>>> This is a an error produced by Exim. I find the Exim error handling
in
>>> Exim's implementation of the AUTH command rather peculiar.
Still, I
>>> managed to decipher at least part of it.
>>>
>>> That error is produced when FAIL status is returned from the
driver:
>>>
>>> https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665
>>>
>>> This FAIL status can be returned by the driver itself, but -- in
this
>>> case more likely -- the Dovecot driver in Exim also returns FAIL
status
>>> when Dovecot auth service returns "FAIL":
>>>
>>>
https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472
>>>
>>> So, this may very well be an issue triggered by Dovecot. What
version of
>>> Dovecot is this? Some things were modified in initial response
handling
>>> recently (v2.3) and I may have messed up something.
>>>
>>> Does Dovecot log anything interesting with auth_verbose and
auth_debug
>>> enabled?
>>>
>>> Regards,
>>>
>>> Stephan.
>> Hi,
>>
>> System is gentoo,
>> dovecot version is 2.3.0
>> exim version is 4.90
>>
>> Debug log does only show the following:
>> auth: Debug: auth client connected (pid=0)
>> auth: Debug: client in: AUTH   1       PLAIN   service=smtp    secured
rip=XX.XX.XX.XX       lip=XX.XX.XX.XX       nologin resp=<hidden>
>> auth: plain(?,XX.XX.XX.XX): invalid input
>> auth: Debug: client passdb out: FAIL   1
>>
>> I'm not 100% sure but i think it worked earlier, so this might be
connected to the 2.3 update. (if REALLY needed i can try to confirm by
downgrading dovecot)
> Ok. I know what is going on already. This commit triggers the problem:
>
>
https://github.com/dovecot/core/commit/e4b72bd73bfffda7906faa248eab31f936cfc6fa
>
> That fix was added to handle the EXTERNAL SASL mechanism properly when
> used in ManageSieve, and somehow I didn't realize that the original
> comment means that Exim would also send an empty resp field for an
> absent initial response:
>
> https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L403
>
> This is now handled as an empty initial response instead (as it should
> be), which -- in this case -- makes the PLAIN mechanism complain about
> invalid data.
>
> So, the fundamental blame lies with Exim for violating the protocol.
> However, I don't think it is a good idea to break compatibility like
> that, especially when we want to back-port this fix to Dovecot v2.2.
>
> To solve this now, we can recognize an empty initial response for
> service=smtp differently (EXTERNAL is not used there much I think) and
> perhaps make that configurable with some setting.
Right, I can also just base behavior on the client protocol version.

Regards,

Stephan.

Op 1/3/2018 om 11:28 AM schreef Stephan Bosch:
> Op 1/3/2018 om 10:58 AM schreef Stephan Bosch:
>> Op 1/3/2018 om 8:31 AM schreef Daniel Kenzelmann:
>>> 3. Januar 2018 00:49, "Stephan Bosch" <stephan at
rename-it.nl> schrieb:
>>>
>>>> Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm not entirely sure whether this issue is with exim
or with dovecot.
>>>>>
>>>>> First some background:
>>>>> I'm using exim with dovecot-auth which in turn is using
LDAP for
>>>>> authentication.
>>>>>
>>>>> When using AUTH PLAIN with the optional initial response
argument,
>>>>> everything is fine.
>>>>>
>>>>> However when using AUTH PLAIN without the optional response
argument,
>>>>> instead of getting an empty challenge ("334 ") as
per RFC i am getting
>>>>> a "535 Incorrect authentication data".
>>>>>
>>>>> Example:
>>>>> Working:
>>>>> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
>>>>> EHLO test
>>>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>>>> 250-SIZE 52428800
>>>>> 250-8BITMIME
>>>>> 250-PIPELINING
>>>>> 250-AUTH PLAIN LOGIN
>>>>> 250-CHUNKING
>>>>> 250 HELP
>>>>> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=>>>>>
235 Authentication succeeded
>>>>>
>>>>> NOT-WORKING:
>>>>> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
>>>>> EHLO test
>>>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>>>> 250-SIZE 52428800
>>>>> 250-8BITMIME
>>>>> 250-PIPELINING
>>>>> 250-AUTH PLAIN LOGIN
>>>>> 250-CHUNKING
>>>>> 250 HELP
>>>>> AUTH PLAIN
>>>>> 535 Incorrect authentication data
>>>>>
>>>>> Here the SASL mechanism should return an empty challenge as
per RFC
>>>>> (i.e. "334 " in SMTP):
>>>> This is a an error produced by Exim. I find the Exim error
handling in
>>>> Exim's implementation of the AUTH command rather peculiar.
Still, I
>>>> managed to decipher at least part of it.
>>>>
>>>> That error is produced when FAIL status is returned from the
driver:
>>>>
>>>>
https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665
>>>>
>>>> This FAIL status can be returned by the driver itself, but --
in this
>>>> case more likely -- the Dovecot driver in Exim also returns
FAIL status
>>>> when Dovecot auth service returns "FAIL":
>>>>
>>>>
https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472
>>>>
>>>> So, this may very well be an issue triggered by Dovecot. What
version of
>>>> Dovecot is this? Some things were modified in initial response
handling
>>>> recently (v2.3) and I may have messed up something.
>>>>
>>>> Does Dovecot log anything interesting with auth_verbose and
auth_debug
>>>> enabled?
>>>>
>>>> Regards,
>>>>
>>>> Stephan.
>>> Hi,
>>>
>>> System is gentoo,
>>> dovecot version is 2.3.0
>>> exim version is 4.90
>>>
>>> Debug log does only show the following:
>>> auth: Debug: auth client connected (pid=0)
>>> auth: Debug: client in: AUTH   1       PLAIN   service=smtp   
secured rip=XX.XX.XX.XX       lip=XX.XX.XX.XX       nologin resp=<hidden>
>>> auth: plain(?,XX.XX.XX.XX): invalid input
>>> auth: Debug: client passdb out: FAIL   1
>>>
>>> I'm not 100% sure but i think it worked earlier, so this might
be connected to the 2.3 update. (if REALLY needed i can try to confirm by
downgrading dovecot)
>> Ok. I know what is going on already. This commit triggers the problem:
>>
>>
https://github.com/dovecot/core/commit/e4b72bd73bfffda7906faa248eab31f936cfc6fa
>>
>> That fix was added to handle the EXTERNAL SASL mechanism properly when
>> used in ManageSieve, and somehow I didn't realize that the original
>> comment means that Exim would also send an empty resp field for an
>> absent initial response:
>>
>> https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L403
>>
>> This is now handled as an empty initial response instead (as it should
>> be), which -- in this case -- makes the PLAIN mechanism complain about
>> invalid data.
>>
>> So, the fundamental blame lies with Exim for violating the protocol.
>> However, I don't think it is a good idea to break compatibility
like
>> that, especially when we want to back-port this fix to Dovecot v2.2.
>>
>> To solve this now, we can recognize an empty initial response for
>> service=smtp differently (EXTERNAL is not used there much I think) and
>> perhaps make that configurable with some setting.
> Right, I can also just base behavior on the client protocol version.
Fix pending (2 commits against master):

https://github.com/stephanbosch/dovecot-core/commits/fix-auth-exim

Regards,

Stephan.