dovecot - Jan 2018 - Dovecot auth SASL for exim and plain auth issue without initial response

Hi,

I'm not entirely sure whether this issue is with exim or with dovecot.

First some background:
I'm using exim with dovecot-auth which in turn is using LDAP for
authentication.

When using AUTH PLAIN with the optional initial response argument,
everything is fine.

However when using AUTH PLAIN without the optional response argument,
instead of getting an empty challenge ("334 ") as per RFC i am getting
a "535 Incorrect authentication data".

Example:
Working:
220 XXXX ESMTP 2018-01-02 22:32:33+0100
EHLO test
250-XXXX Hello XXXXX [x.x.x.x]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH PLAIN LOGIN
250-CHUNKING
250 HELP
AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=235 Authentication succeeded

NOT-WORKING:
220 XXXX ESMTP 2018-01-02 22:34:37+0100
EHLO test
250-XXXX Hello XXXXX [x.x.x.x]
250-SIZE 52428800
250-8BITMIME
250-PIPELINING
250-AUTH PLAIN LOGIN
250-CHUNKING
250 HELP
AUTH PLAIN
535 Incorrect authentication data


Here the SASL mechanism should return an empty challenge as per RFC
(i.e. "334 " in SMTP):

RFC 4954 - SMTP Service Extension for Authentication


4.  The AUTH Command

         [..]
         The optional initial response argument to the AUTH command is
         used to save a round-trip when using authentication mechanisms
         that support an initial client response. 
====>    If the initial
====>    response argument is omitted and the chosen mechanism requires
====>    an initial client response, the server MUST proceed as defined
====>    in Section 5.1 of [SASL].  In SMTP, a server challenge that
====>    contains no data is defined as a 334 reply with no text part.
====>    Note that there is still a space following the reply code, so
====>    the complete response line is "334 ".
         [..]



RFC 4422 - Simple Authentication and Security Layer (SASL)

5.  Mechanism Requirements

   SASL mechanism specifications MUST supply the following information:

   1) The name of the mechanism (see Section 3.1).  This name MUST be
      registered as discussed in Section 7.1.

   2) A definition of the server-challenges and client-responses of the
      authentication exchange, as well as the following:

     a) An indication of whether the mechanism is client-first,
        variable, or server-first. 

===>    If a SASL mechanism is defined as
===>    client-first and the client does not send an initial response
===>    in the authentication request, then the first server challenge
===>    MUST be empty
                      (the EXTERNAL mechanism is an example of this
        case).  If a SASL mechanism is defined as variable, then the
        specification needs to state how the server behaves when the
        initial client response in the authentication request is
        omitted (the DIGEST-MD5 mechanism [DIGEST-MD5] is an example of
        this case).  If a SASL mechanism is defined as server-first,
        then the client MUST NOT send an initial client response in the
        authentication request (the CRAM-MD5 mechanism [CRAM-MD5] is an
        example of this case).



Thanks,
Daniel

Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
> Hi,
>
> I'm not entirely sure whether this issue is with exim or with dovecot.
>
> First some background:
> I'm using exim with dovecot-auth which in turn is using LDAP for
> authentication.
>
> When using AUTH PLAIN with the optional initial response argument,
> everything is fine.
>
> However when using AUTH PLAIN without the optional response argument,
> instead of getting an empty challenge ("334 ") as per RFC i am
getting
> a "535 Incorrect authentication data".
>
> Example:
> Working:
> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
> EHLO test
> 250-XXXX Hello XXXXX [x.x.x.x]
> 250-SIZE 52428800
> 250-8BITMIME
> 250-PIPELINING
> 250-AUTH PLAIN LOGIN
> 250-CHUNKING
> 250 HELP
> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=> 235 Authentication succeeded
>
> NOT-WORKING:
> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
> EHLO test
> 250-XXXX Hello XXXXX [x.x.x.x]
> 250-SIZE 52428800
> 250-8BITMIME
> 250-PIPELINING
> 250-AUTH PLAIN LOGIN
> 250-CHUNKING
> 250 HELP
> AUTH PLAIN
> 535 Incorrect authentication data
>
>
> Here the SASL mechanism should return an empty challenge as per RFC
> (i.e. "334 " in SMTP):
This is a an error produced by Exim. I find the Exim error handling in
Exim's implementation of the AUTH command rather peculiar. Still, I
managed to decipher at least part of it.

That error is produced when FAIL status is returned from the driver:

https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665

This FAIL status can be returned by the driver itself, but -- in this
case more likely -- the Dovecot driver in Exim also returns FAIL status
when Dovecot auth service returns "FAIL":

https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472

So, this may very well be an issue triggered by Dovecot. What version of
Dovecot is this? Some things were modified in initial response handling
recently (v2.3) and I may have messed up something.

Does Dovecot log anything interesting with auth_verbose and auth_debug
enabled?

Regards,

Stephan.

3. Januar 2018 00:49, "Stephan Bosch" <stephan at rename-it.nl>
schrieb:
> Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
> 
>> Hi,
>> 
>> I'm not entirely sure whether this issue is with exim or with
dovecot.
>> 
>> First some background:
>> I'm using exim with dovecot-auth which in turn is using LDAP for
>> authentication.
>> 
>> When using AUTH PLAIN with the optional initial response argument,
>> everything is fine.
>> 
>> However when using AUTH PLAIN without the optional response argument,
>> instead of getting an empty challenge ("334 ") as per RFC i
am getting
>> a "535 Incorrect authentication data".
>> 
>> Example:
>> Working:
>> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
>> EHLO test
>> 250-XXXX Hello XXXXX [x.x.x.x]
>> 250-SIZE 52428800
>> 250-8BITMIME
>> 250-PIPELINING
>> 250-AUTH PLAIN LOGIN
>> 250-CHUNKING
>> 250 HELP
>> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=>> 235 Authentication
succeeded
>> 
>> NOT-WORKING:
>> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
>> EHLO test
>> 250-XXXX Hello XXXXX [x.x.x.x]
>> 250-SIZE 52428800
>> 250-8BITMIME
>> 250-PIPELINING
>> 250-AUTH PLAIN LOGIN
>> 250-CHUNKING
>> 250 HELP
>> AUTH PLAIN
>> 535 Incorrect authentication data
>> 
>> Here the SASL mechanism should return an empty challenge as per RFC
>> (i.e. "334 " in SMTP):
> 
> This is a an error produced by Exim. I find the Exim error handling in
> Exim's implementation of the AUTH command rather peculiar. Still, I
> managed to decipher at least part of it.
> 
> That error is produced when FAIL status is returned from the driver:
> 
> https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665
> 
> This FAIL status can be returned by the driver itself, but -- in this
> case more likely -- the Dovecot driver in Exim also returns FAIL status
> when Dovecot auth service returns "FAIL":
> 
> https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472
> 
> So, this may very well be an issue triggered by Dovecot. What version of
> Dovecot is this? Some things were modified in initial response handling
> recently (v2.3) and I may have messed up something.
> 
> Does Dovecot log anything interesting with auth_verbose and auth_debug
> enabled?
> 
> Regards,
> 
> Stephan.

Hi,

System is gentoo,
dovecot version is 2.3.0
exim version is 4.90

Debug log does only show the following:
auth: Debug: auth client connected (pid=0)
auth: Debug: client in: AUTH   1       PLAIN   service=smtp    secured
rip=XX.XX.XX.XX       lip=XX.XX.XX.XX       nologin resp=<hidden>
auth: plain(?,XX.XX.XX.XX): invalid input
auth: Debug: client passdb out: FAIL   1

I'm not 100% sure but i think it worked earlier, so this might be connected
to the 2.3 update. (if REALLY needed i can try to confirm by downgrading
dovecot)

Thanks,
Daniel

Op 1/3/2018 om 8:31 AM schreef Daniel Kenzelmann:
> 3. Januar 2018 00:49, "Stephan Bosch" <stephan at
rename-it.nl> schrieb:
>
>> Op 1/2/2018 om 10:48 PM schreef Daniel Kenzelmann:
>>
>>> Hi,
>>>
>>> I'm not entirely sure whether this issue is with exim or with
dovecot.
>>>
>>> First some background:
>>> I'm using exim with dovecot-auth which in turn is using LDAP
for
>>> authentication.
>>>
>>> When using AUTH PLAIN with the optional initial response argument,
>>> everything is fine.
>>>
>>> However when using AUTH PLAIN without the optional response
argument,
>>> instead of getting an empty challenge ("334 ") as per RFC
i am getting
>>> a "535 Incorrect authentication data".
>>>
>>> Example:
>>> Working:
>>> 220 XXXX ESMTP 2018-01-02 22:32:33+0100
>>> EHLO test
>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>> 250-SIZE 52428800
>>> 250-8BITMIME
>>> 250-PIPELINING
>>> 250-AUTH PLAIN LOGIN
>>> 250-CHUNKING
>>> 250 HELP
>>> AUTH PLAIN XXXXXXXXXXXXXXXXXXXXXXXX=>>> 235 Authentication
succeeded
>>>
>>> NOT-WORKING:
>>> 220 XXXX ESMTP 2018-01-02 22:34:37+0100
>>> EHLO test
>>> 250-XXXX Hello XXXXX [x.x.x.x]
>>> 250-SIZE 52428800
>>> 250-8BITMIME
>>> 250-PIPELINING
>>> 250-AUTH PLAIN LOGIN
>>> 250-CHUNKING
>>> 250 HELP
>>> AUTH PLAIN
>>> 535 Incorrect authentication data
>>>
>>> Here the SASL mechanism should return an empty challenge as per RFC
>>> (i.e. "334 " in SMTP):
>> This is a an error produced by Exim. I find the Exim error handling in
>> Exim's implementation of the AUTH command rather peculiar. Still, I
>> managed to decipher at least part of it.
>>
>> That error is produced when FAIL status is returned from the driver:
>>
>> https://github.com/Exim/exim/blob/master/src/src/smtp_in.c#L3665
>>
>> This FAIL status can be returned by the driver itself, but -- in this
>> case more likely -- the Dovecot driver in Exim also returns FAIL status
>> when Dovecot auth service returns "FAIL":
>>
>> https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L472
>>
>> So, this may very well be an issue triggered by Dovecot. What version
of
>> Dovecot is this? Some things were modified in initial response handling
>> recently (v2.3) and I may have messed up something.
>>
>> Does Dovecot log anything interesting with auth_verbose and auth_debug
>> enabled?
>>
>> Regards,
>>
>> Stephan.
>
> Hi,
>
> System is gentoo,
> dovecot version is 2.3.0
> exim version is 4.90
>
> Debug log does only show the following:
> auth: Debug: auth client connected (pid=0)
> auth: Debug: client in: AUTH   1       PLAIN   service=smtp    secured
rip=XX.XX.XX.XX       lip=XX.XX.XX.XX       nologin resp=<hidden>
> auth: plain(?,XX.XX.XX.XX): invalid input
> auth: Debug: client passdb out: FAIL   1
>
> I'm not 100% sure but i think it worked earlier, so this might be
connected to the 2.3 update. (if REALLY needed i can try to confirm by
downgrading dovecot)
Ok. I know what is going on already. This commit triggers the problem:

https://github.com/dovecot/core/commit/e4b72bd73bfffda7906faa248eab31f936cfc6fa

That fix was added to handle the EXTERNAL SASL mechanism properly when
used in ManageSieve, and somehow I didn't realize that the original
comment means that Exim would also send an empty resp field for an
absent initial response:

https://github.com/Exim/exim/blob/master/src/src/auths/dovecot.c#L403

This is now handled as an empty initial response instead (as it should
be), which -- in this case -- makes the PLAIN mechanism complain about
invalid data.

So, the fundamental blame lies with Exim for violating the protocol.
However, I don't think it is a good idea to break compatibility like
that, especially when we want to back-port this fix to Dovecot v2.2.

To solve this now, we can recognize an empty initial response for
service=smtp differently (EXTERNAL is not used there much I think) and
perhaps make that configurable with some setting.

Regards,

Stephan.