dovecot - Jul 2008 - login processes from attacks staying for hours

I'm in the process of rolling out new setups with dovecot on CentOS 5.2 
and I notice that dovecot doesn't handle the brute-force attacks too nice.
I reduced the limit a bit to some reasonable looking value:
login_max_processes_count = 32
to stop them earlier and the number of processes stops at that figure when 
an attack happens.
However, it stays at this count for hours although the attack is already 
over since long. For instance, my monitoring alerts me at the moment when 
the process count for pop3-login goes over 20 processes. This happened on 
three machines at 2 am with a brute-force attack from the same source that 
didn't last longer than a minute or so. However, the process count dropped 
only at 7am under 20 on two machines and on the third machine it was still 
over 20 when I was in the office at 9 am and finally killed them.
As these machines are all not in production yet, there weren't any other 
logins and the single brute-force ended within one minute according to the 
logs (obviously when pop3-logins hit the limit).
Shouldn't these processes go down to login_processes_count (3) within a 
few minutes? An strace shows that they are mostly doing gettimeofday 
lookups (=sleeping).
This is the default dovecot (1.07) coming with CentOS 5.2.
I've been running only one other instance of dovecot in production 
(0.99.11) on CentOS 4.6 so far and I don't know which behavior that 
displayed in the past as I just recognize that I accidentally ommitted it 
from monitoring. :-(

I had this mailing list searched for "brute-force" to see how others 
handle this and what dovecot provides to stop these attacks. I have found 
not many threads about this. There is one with a bit more information: 
"Delay on failed pw attempts" from January 1. Unfortunately, this 
functionality is only in a later version of dovecot and it's not clear if 
it was implemented or not or if it would be helpful. Was it implemented?

This thread also mentions fail2ban which may be one way to go, although I 
don't like this log parsing approach too much. Does anyone use iptables 
for rate-limiting per IP on the pop/imap ports to prevent brute-force 
attacks?



Kai

-- 
Kai Sch?tzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

On 7/23/2008, Kai Schaetzl (maillists at conactive.com)
wrote:
> I had this mailing list searched for "brute-force" to see how
others
> handle this and what dovecot provides to stop these attacks.
The best answer is to use a tool made for this kind of job, like fail2ban.

But as to why the processes remain for so long, I don't know...

-- 

Best regards,

Charles

Hi,

We you run 'netstat -tan' (or equivalent), what state are the packets 
in? If it is just a bunch of processes with no active connections then 
it should not be a big deal.

We've seen something on our SMTP servers that sounds similar (our IMAP 
servers haven't been hit yet). The problem is there is badly written 
spammer/hacker software that do not close connections correctly. We wind 
up with a number of useless connections, many of them in CLOSE_WAIT or 
FIN_WAIT* states.

TCP/IP kernel tuning is our solution to close connection states quicker. 
I don't know Centros. In Solaris the ndd parameters, which have stupid 
defaults, are tcp_time_wait_interval, tcp_fin_wait_2_flush_interval, 
tcp_ip_abort_interval and tcp_keepalive_interval. Redhat seems a bit 
better but the tcp_keepalive_time of 2 hours is a little high for my 
liking. I'ld pay attention to the connection state that seems to be the 
problem.

An strict iptables approach doesn't address the tcp teardown issue. Even 
with drops via iptables there still will be connections waiting to close.

Our servers are front-ended by load balancers and there is also a router 
at the border. This is where we block ip addresses, if we need to.

Hope this helps.

---Jack


Kai Schaetzl wrote:
> I'm in the process of rolling out new setups with dovecot on CentOS 5.2
> and I notice that dovecot doesn't handle the brute-force attacks too
nice.
> I reduced the limit a bit to some reasonable looking value:
> login_max_processes_count = 32
> to stop them earlier and the number of processes stops at that figure when 
> an attack happens.
> However, it stays at this count for hours although the attack is already 
> over since long. For instance, my monitoring alerts me at the moment when 
> the process count for pop3-login goes over 20 processes. This happened on 
> three machines at 2 am with a brute-force attack from the same source that 
> didn't last longer than a minute or so. However, the process count
dropped
> only at 7am under 20 on two machines and on the third machine it was still 
> over 20 when I was in the office at 9 am and finally killed them.
> As these machines are all not in production yet, there weren't any
other
> logins and the single brute-force ended within one minute according to the 
> logs (obviously when pop3-logins hit the limit).
> Shouldn't these processes go down to login_processes_count (3) within a
> few minutes? An strace shows that they are mostly doing gettimeofday 
> lookups (=sleeping).
> This is the default dovecot (1.07) coming with CentOS 5.2.
> I've been running only one other instance of dovecot in production 
> (0.99.11) on CentOS 4.6 so far and I don't know which behavior that 
> displayed in the past as I just recognize that I accidentally ommitted it 
> from monitoring. :-(
> 
> I had this mailing list searched for "brute-force" to see how
others
> handle this and what dovecot provides to stop these attacks. I have found 
> not many threads about this. There is one with a bit more information: 
> "Delay on failed pw attempts" from January 1. Unfortunately, this
> functionality is only in a later version of dovecot and it's not clear
if
> it was implemented or not or if it would be helpful. Was it implemented?
> 
> This thread also mentions fail2ban which may be one way to go, although I 
> don't like this log parsing approach too much. Does anyone use iptables
> for rate-limiting per IP on the pop/imap ports to prevent brute-force 
> attacks?
> 
> 
> 
> Kai
>

> and I notice that dovecot doesn't handle the brute-force attacks too
nice.
> I reduced the limit a bit to some reasonable looking value:
> login_max_processes_count = 32
> to stop them earlier and the number of processes stops at that figure when
> an attack happens.
Somewhat off original topic.  I cannot help but wander what the goal
of the brute force attack is.  I am guessing they want a working
username and password to relay junk email?

I have heard of users having there email address and password stolen
by a virus or spyware then used to authenticate and relay thousands of
pieces of junk email.  We enabled rate-limit on Exim which only allows
a given IP to send to X number of message recipients in X amount of
time.  We also added a plugin to Squirrel Mail to only allow so many
recipients per message and only so many messages per day.

Matt