-------- Original-Nachricht --------
Betreff: STARTTLS bug - background story
Datum: Mon, 7 Mar 2011 15:08:09 -0500 (EST)
Von: Wietse Venema <wietse at porcupine.org>
An: Postfix users <postfix-users at postfix.org>
CERT/CC announces a flaw today in multiple STARTTLS implementations.
This problem was silently fixed in Postfix 2.8 and 2.9. Updates
for Postfix 2.[4-7] are made available via the usual channels.
Wietse
Plaintext injection in multiple implementations of STARTTLS
==========================================================
This is a writeup about a flaw that I found recently, and that
existed in multiple implementations of SMTP (Simple Mail Transfer
Protocol) over TLS (Transport Layer Security) including my Postfix
open source mailserver. I give an overview of the problem and its
impact, technical background, how to find out if a server is affected,
fixes, and draw lessons about where we can expect similar problems
now or in the future. A time line is at the end.
On-line information is/will be available at:
http://www.kb.cert.org/vuls/id/555316
http://www.postfix.org/CVE-2011-0411.html
Problem overview and impact
==========================
The TLS protocol encrypts communication and protects it against
modification by other parties. This protection exists only if a)
software is free of flaws, and b) clients verify the server's TLS
certificate, so that there can be no "man in the middle" (servers
usually don't verify client certificates).
The problem discussed in this writeup is caused by a software flaw.
The flaw allows an attacker to inject client commands into an SMTP
session during the unprotected plaintext SMTP protocol phase (more
on that below), such that the server will execute those commands
during the SMTP-over-TLS protocol phase when all communication is
supposed to be protected.
The injected commands could be used to steal the victim's email or
SASL (Simple Authentication and Security Layer) username and password.
This is not as big a problem as it may appear to be. The reason
is that many SMTP client applications don't verify server TLS
certificates. These SMTP clients are always vulnerable to command
injection and other attacks. Their TLS sessions are only encrypted
but not protected.
A similar plaintext injection flaw may exist in the way SMTP clients
handle SMTP-over-TLS server responses, but its impact is less
interesting than the server-side flaw.
SMTP is not the only protocol with a mid-session switch from plaintext
to TLS. Other examples are POP3, IMAP, NNTP and FTP. Implementations
of these protocols may be affected by the same flaw as discussed here.
Technical background: SMTP over TLS
==================================
For a precise description of SMTP over TLS, see RFC 3207, on-line
at http://www.ietf.org/rfc/rfc3207.txt.
SMTP over TLS uses the same TLS protocol that is also used to encrypt
traffic between web clients and web servers. But, there is a subtle
difference in the way TLS is used, and that makes this flaw possible.
SMTP sessions over TLS begin with an SMTP protocol handshake in
plaintext. Plaintext means no encryption (thus no privacy), and no
protection against modification (no integrity). The plaintext
handshake is needed because SMTP has always worked this way. Simply
skipping this plaintext phase would seriously break internet email.
During the plaintext handshake phase, the SMTP server announces
whether it is willing to use TLS. If both SMTP client and server
support TLS, the client sends a "STARTTLS" request to turn on TLS.
Once TLS is turned on, all further traffic is encrypted and protected
from modification. The client and server repeat the entire SMTP
protocol handshake, and the client starts sending mail.
Demonstration
============
The problem is easy to demonstrate with a one-line change to the
OpenSSL s_client command source code (I would prefer scripting, but
having to install Perl CPAN modules and all their dependencies is
more work than downloading a .tar.gz file from openssl.org, adding
eight characters to one line, and doing "./config; make").
(The OpenSSL s_client command can make a connection to servers that
support straight TLS, SMTP over TLS, or a handful other protocols
over TLS. The demonstration here focuses on SMTP over TLS only.)
The demonstration with SMTP over TLS involves a one-line change in
the OpenSSL s_client source code (with OpenSSL 1.0.0, at line 1129
of file apps/s_client.c).
Old: BIO_printf(sbio,"STARTTLS\r\n");
New: BIO_printf(sbio,"STARTTLS\r\nRSET\r\n");
With this change, the s_client command sends the plaintext STARTTLS
command ("let's turn on TLS") immediately followed by an RSET
command
(a relatively harmless protocol "reset"). Both commands are sent
as plaintext in the same TCP/IP packet, and arrive together at the
server. The "\r\n" are the carriage-return and newline characters;
these are necessary to terminate an SMTP command.
When an SMTP server has the plaintext injection flaw, it reads the
STARTTLS command first, switches to SMTP-over-TLS mode, and only
then the server reads the RSET command. Note, the RSET command was
transmitted during the plaintext SMTP phase when there is no
protection, but the server reads the command as if it was received
over the TLS-protected channel.
Thus, when the SMTP server has the flaw, the s_client command output
will show two "250" SMTP server responses instead of one. The first
"250" response is normal, and is present even when the server is
not flawed. The second "250" response is for the RSET command, and
indicates that the SMTP server has the plaintext injection flaw.
$ apps/openssl s_client -quiet -starttls smtp -connect server:port
[some server TLS certificate details omitted]
250 some text here <=== Normal response, also with "good" server.
250 more text here <=== RSET response, only with flawed server.
How would an attacker exploit this? It would play man-in-the-middle
on the connection between SMTP client and server, perhaps using ARP
spoofing at a public WIFI access point. Instead of adding a harmless
RSET command, it could steal email or authentication credentials.
Anatomy of the flaw
==================
The flaw is made possible by two ingredients: I already discussed
the switch mid-session from plaintext SMTP to SMTP over TLS. This
allows an attacker to piggy-back commands onto the SMTP client's
plaintext STARTTLS ("let's turn on TLS") request, such that the
server may read those commands after the switch to TLS is completed,
as if the commands arrived through the TLS-encrypted session.
The second ingredient is a layered software architecture through
which those piggy-backed commands bubble up from the network to the
application. In the case of SMTP, we have the following major
layers before and after the switch to TLS:
Before switch to TLS After switch to TLS
SMTP protocol engine SMTP protocol engine
|| ||
TCP/IP protocol engine TLS protocol engine
|| ||
Internet TCP/IP protocol engine
||
Internet
Each layer hides details of what is happening inside. Layering makes
it easy to switch from plaintext to TLS, with minimal changes to
existing code.
To insert the TLS layer between the SMTP engine and the O/S TCP/IP
engine, simply adjust the plumbing between the layers, and make all
information flow through the TLS layer.
It's all about the plumbing
==========================
Whether a program may have the plaintext injection flaw depends on
how it adjusts the plumbing as it inserts the TLS protocol layer
in-between the SMTP protocol layer and the O/S TCP/IP protocol
layer. I illustrate this with examples from three open source MTAs:
Postfix, Sendmail and Exim.
The diagram below zooms in on the plumbing between the SMTP and TLS
layers, and shows how different MTAs handle the switch from plaintext
to TLS. It is best viewed with a fixed-width font (for example,
from the Courier family).
Postfix MTA Sendmail MTA Exim MTA
before/after before/after before/after
switch to TLS switch to TLS switch to TLS
SMTP SMTP SMTP SMTP SMTP SMTP <= SMTP layer
|| || || || || ||
stream stream stream stream' || ||
buffers buffers buffers buffers' rw r'w' <=
stream layer
rw r'w' rw r'w' || ||
|| || || || || ||
|| TLS || TLS || TLS <= TLS layer
|| || || || || ||
O/S O/S O/S O/S O/S O/S <= TCP/IP layer
As shown in the diagram, both Postfix and Sendmail use an application-
level stream abstraction, where each stream has properties such as
read/write buffers, read/write functions (indicated with rw), and
other properties that are omitted for brevity.
When Postfix switches to SMTP over TLS, it replaces the plaintext
read/write functions (rw) with the TLS read/write functions (r'w').
Postfix does not modify any of the other stream properties including
the read/write buffers. A patch for qmail that introduces TLS
support uses the same approach. This approach of replacing only
the stream read/write functions, but not the buffers or other stream
properties, can introduce the plaintext injection flaw.
When Sendmail switches to SMTP over TLS, it replaces the entire
stream, along with its read/write buffers and read/write functions.
Exim. on the other hand, does not have a stream abstraction like
Postfix, Sendmail or qmail. Instead of replacing streams or stream
properties, Exim replaces plaintext read/write functions with TLS
read/write functions. Because of their program structure, Sendmail
and Exim didn't suffer from the plaintext injection flaw.
Switching world views
====================
When the switch from plaintext to TLS mode is made, all layers above
the TLS layer need to purge all information that they have received
through the plaintext session. This "world view switch" needs to
be implemented consistently. Otherwise, information that was sent
as unprotected plaintext may slip through the cracks.
In the case of Postfix and other affected MTAs, this "world view
switch" was incomplete. Postfix it did not account for information
that lingered in stream buffers at the boundary between two layers.
This allowed an attacker to piggy-back commands onto the plaintext
"let's turn on TLS" request, such that the commands could be read
after the switch to TLS was completed, as if they had arrived through
the TLS-encrypted session.
Fixing the problem
=================
There are two solutions to address the flaw, and both solutions can
be used together.
- Report an error when unexpected plaintext is received after the
STARTTLS command. As documented in RFC 3207, STARTTLS must be
the last command in a pipelined group. If plaintext commands are
received after STARTTLS, then that is a protocol violation.
This measure can also be implemented outside the MTA, for example
in a protocol-aware firewall.
- If a program uses the same input buffer before and after the
switch to TLS, it should discard the contents of the input buffer,
just like it discards SMTP protocol information that it received
during the plaintext protocol phase.
Conclusion
=========
This plaintext injection problem is likely to recur when some
development moves the plaintext-to-ciphertext switch outside the
application: for example, into the kernel, into the local hardware,
into a proxy, or into other infrastructure. This encourages
applications to use the same application-level streams and buffers
and read/write functions before and after the switch to ciphertext.
When this migration happens, plaintext injection becomes once more
a possibility.
Postfix did not reject plaintext commands that were piggy-backed
onto the plaintext "let's turn on TLS" request. This reflects what
once was the primary mission of Postfix: to deliver mail, not to
force other systems to implement all the Internet RFCs correctly.
Nowadays, strict protocol compliance is becoming a requirement for
senders to get their email delivered. As this episode shows, stricter
protocol enforcement by receivers can bring security benefits,
besides blocking spambots.
Time line
========
Jan 5 2011: While finishing Postfix for its annual release, I found
this flaw in the server and client implementations of SMTP over
TLS. It had been sitting there for six years ever since TLS support
was adopted into Postfix. Not wanting to delay the release schedule,
I silently fixed the problem and sent an email to co-developer
Victor Duchovni.
Jan 6-10 2011: As we investigated the scope of the problem, Victor
discovered quickly that many other SMTP over TLS implementations
were also affected. Among those affected were email service providers,
email anti-spam/virus service providers, anti-spam/virus appliances,
as well as other mail server implementations. It was clear that
this problem's resolution was going to involve many organizations.
Jan 11 2011: Contact CERT/CC to help coordinate with the problem's
resolution.
Mar 7 2011: Public announcement.
