dovecot - Aug 2013 - Logging passwords on auth failure/dealing with botnets

Hi,

Since upgrading our mail servers to Postfix/Dovecot, we've seen a rather
large increase in botnet brute force password attacks.  I guess our old servers
were too slow to suit their needs.

Now, when they hit upon a valid user, it's easy to see what passwords they
are trying (we've enabled auth_debug_passwords and set
auth_verbose_passwords = plain).  We can easily have log monitors pick up the
blatant password attempts (123, 1234, 12345, 3.1415926, etc) and firewall them.
Aug 22 00:21:47 host1 dovecot: auth-worker(1423): sql(aea,120.198.9.221):
Password mismatch (given password: 3.1415926)

However, when they are plugging away on an invalid user, all the logs show is
'unknown user'.  The only way to see what they are doing is to tcpdump
on the box.  Almost 99% of the attacks are through SMTP/SASL attempts, using
'auth login'. Postfix is configured to use Dovecot for the
authentication backend.  NOTE: parens contain the base64 decoded values, this is
not part of the data stream.
C: auth login
S: 334 VXNlcm5hbWU6  (334 Username:)
C: YWVi  (aeb)
S: 334 UGFzc3dvcmQ6  (334 Password:)
C: My4xNDE1OTI2  (3.1415926)
S: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6  (535 5.7.8 Error:
authentication failed: Password:)

Aug 22 11:00:31 host3 dovecot: auth-worker(19844): sql(aeb,120.198.9.221):
unknown user

How difficult would it be to add logging the given password, observing the
auth_debug_passwords and auth_verbose_passwords settings, for all authentication
failures?  When we see certain patterns, we want to aggressively block those
IPs, regardless of if it's a valid username or an unknown user.

Or another option, is there any good DNS based RBLs for botnet IPs, and is there
any way to tie that in to the dovecot auth system?  I've been looking for
botnet rbls, but what I've found so far doesn't seem to work very well. 
Most of the IPs that I've had to firewall don't exist in them.

--
Michael Smith

On Thu, Aug 22, 2013 at 04:16:51PM +0000, Michael Smith (DF)
wrote:
> Or another option, is there any good DNS based RBLs for botnet IPs, 
> and is there any way to tie that in to the dovecot auth system?  
> I've been looking for botnet rbls, but what I've found so far 
> doesn't seem to work very well.  Most of the IPs that I've had to 
> firewall don't exist in them.
I guess I would first have tried Spamhaus XBL, but I guess you 
checked that already.

The problem with using XBL, anyway, is that you might have legitimate 
logins from listed hosts. Example: a traveler using hotel wifi. We 
(TINW) really would need a new DNSBL type (or a special result) for 
this sort of abuse.

It's a nice idea, worth building upon, if someone can fund it (or 
find the time to develop it, which really amounts to the same thing.) 
Imagine also a Dovecot network of reporters, where brute force 
attempts worldwide are reported from Dovecots to the DNSBL, not 
merely a one-way tie in.

I'd also suggest listing SSH brute force attacks in the same DNSBL, 
possibly with a different result (127.0.0.$port, so IMAP attackers 
list as 127.0.0.143, SSH attackers as 127.0.0.22. Yes, we'd have to 
incorporate the third quad for ports > 255, but the general idea is 
for result codes to be both machine and human readable as much as 
possible.)
-- 
  http://rob0.nodns4.us/ -- system administration and consulting
  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:

"Michael Smith (DF)" writes:
> Or another option, is there any good DNS based RBLs for botnet IPs, and
> is there any way to tie that in to the dovecot auth system? I've been
> looking for botnet rbls, but what I've found so far doesn't seem to
> work very well.  Most of the IPs that I've had to firewall don't
exist
> in them.
/dev/rob0 writes:
> The problem with using XBL, anyway, is that you might have legitimate
> logins from listed hosts. Example: a traveler using hotel wifi. We
> (TINW) really would need a new DNSBL type (or a special result) for
> this sort of abuse.
>
> It's a nice idea, worth building upon, if someone can fund it (or
> find the time to develop it, which really amounts to the same thing.)
> Imagine also a Dovecot network of reporters, where brute force
> attempts worldwide are reported from Dovecots to the DNSBL, not
> merely a one-way tie in.
>
> I'd also suggest listing SSH brute force attacks in the same DNSBL,
> possibly with a different result (127.0.0.$port, so IMAP attackers
> list as 127.0.0.143, SSH attackers as 127.0.0.22. Yes, we'd have to
> incorporate the third quad for ports > 255, but the general idea is
> for result codes to be both machine and human readable as much as
> possible.)
I use bl.blocklist.de as a DNSRBL for ssh BFD, but I think it also
detects BFD for other protocols:

 	http://www.blocklist.de/en/index.html

The nice thing about this RBL is that you can also contribute by
configuring your Fail2Ban/DenyHost to forward logs to the maintainers,
to widen the detection network.  I get about a 60% hit on ssh BFD attacks.

I also found

 	http://openbl.org

but they distribute it as a downloadable file rather than as a DNSRBL.
Maybe I can introduce the latter to the former.

Joseph Tam <jtam.home at gmail.com>

Michael Smith writes:
> We're already running fail2ban, but it doesn't seem that effective
> against botnets, when they only do one attempt per IP.
Yeah, distributed BFDs are tough to block unless you can characterize
the clients well.
> That leaves us back to getting dovecot to log the tried password for
> unknown users.
Another tactic might be to hook in a authentication script:

 	http://wiki2.dovecot.org/AuthDatabase/CheckPassword

You can run this as an external plugin and won't have to muck into the
dovecot innards.  From here, you can log attempts, keep track of  bad
IPs, or take action if you spot a username/password combination that
merits instant blacklisting.

Joseph Tam <jtam.home at gmail.com>