dovecot - Mar 2017 - Dovecot 2.2.27 proxy - enforcing per client IP connection limits

Sami Ketola writes:
>> Can anyone with Solr installed confirm/refute this:  does installing
>> Solr keep iOS clients from roofing the connection count?
>
> I doubt it, but since IMAP SEARCH goes all the way down to the backends
> mail_max_userip_connections can be used to limit the number of
> connections.
Understood -- that's the current situation I'm in now.  Our iOS users
would launch a search resulting in a connection burst, hit the connection
cap, log out all IMAP sessions out, then start the cycle again.  This
sometimes lasts for 10's of minutes.  I'm not sure what the users sees.

Sample logs entries:
 	Mar 19 01:21:30 server dovecot: imap-login: Login: user=<user> ...
 	[... 14 similar logins removed ...]
 	Mar 19 01:21:41 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:42 server dovecot: imap-login: Maximum number of connections from
user+IP exceeded (mail_max_userip_connections=16)
 	: user=<user> ...
 	Mar 19 01:21:42 server dovecot: imap(user): Logged out in=425 out=1107
 	[... 14 similar logouts removed ...]
 	Mar 19 01:21:42 server dovecot: imap(user): Logged out in=382 out=1107
 	Mar 19 01:21:42 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:42 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:43 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:44 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:44 server dovecot: imap(user): Logged out in=442 out=1173
 	Mar 19 01:21:44 server dovecot: imap(user): Logged out in=442 out=1155
 	Mar 19 01:21:44 server dovecot: imap(user): Logged out in=442 out=1166
 	Mar 19 01:21:44 server dovecot: imap(user): Logged out in=442 out=1174
 	Mar 19 01:21:44 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:47 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:47 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:48 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:48 server dovecot: imap-login: Login: user=<user> ...
 	Mar 19 01:21:49 server dovecot: imap-login: Login: user=<user> ...
 	{ ... and on and on for the next 10 minutes ... }

However, there is a pause between each login that might be long enough
to squeeze the search results in if given quickly enough.  From the I/O
stats, most of these searches have empty results.  It probably won't
prevent the connection cap problem, but it might minimize the length
and severity of these connection storms.

Of course, the real fix is for iOS mail-app developers to stop assuming
the IMAP server is owned exclusively by the user by configuring some
reasonable connection throttles.

Joseph Tam <jtam.home at gmail.com>

On 21/03/17 07:03, Joseph Tam wrote:
> Sami Ketola writes:
> 
>>> Can anyone with Solr installed confirm/refute this:  does
installing
>>> Solr keep iOS clients from roofing the connection count?
>>
>> I doubt it, but since IMAP SEARCH goes all the way down to the backends
>> mail_max_userip_connections can be used to limit the number of
>> connections.
> 
> Understood -- that's the current situation I'm in now.  Our iOS
users
> would launch a search resulting in a connection burst, hit the connection
> cap, log out all IMAP sessions out, then start the cycle again.  This
> sometimes lasts for 10's of minutes.  I'm not sure what the users
sees.
[...]
> Of course, the real fix is for iOS mail-app developers to stop assuming
> the IMAP server is owned exclusively by the user by configuring some
> reasonable connection throttles.
Thing is, one should never rely on the intentions or abilities of a 3rd 
party to fix their buggy code, especially when that 3rd party is Apple. 
Their IMAP implementation is shambolic at best and, by far and large, 
the clients using Apple mail clients are causing the most grief. Oh, did 
I mention that wonderful feature named iOS Profile which has so much 
potential if designed & implemented properly, but in A.D. 2017 it's 
still incomplete?
It's been more than obvious for years Apple can't be relied on for 
interoperability, the only way to improve the services offered to the 
clients is to look at the server side, whenever possible. And one of the 
options for limiting the IMAP client hammering is to enforce the limits 
on the proxies directly. Especially in an environment where the backend 
IMAP server isn't Dovecot and mail_max_userip_connections isn't an 
option. Even if the proxies don't exchange IMAP login information 
between them, being able to enforce the limit on the proxy can be a 
significant improvement to the current situation when the Courier-IMAP 
servers are open to IMAP abuse because they always see the proxy IP for 
the incoming connection.

Just my .02AUD

-- 
Adi Pircalabu