Keith Moore <moore(_at_)cs(_dot_)utk(_dot_)edu> writes:
That's the only way that makes sense. If the SMTP client expected the
server's cert to match the domain in the email address (the one used to
query for MX records), it would not be feasible to use the same SMTP
server to accept mail for multiple MX domains.
The problem with taking this approach is that you've now reduced the
security of your TLS session to the level of security of your DNS lookup.
If an attacker can insert their own MX record for the domain, the TLS
public key authentication will succeed.
I'm very suspicious of any place where TLS is done expecting any
certificate other than the one matching the name that the user typed into
their configuration or program, since it opens an avenue for attack.
You have correctly identified the problem, though, namely that there's no
way prior to the TLS negotiation for the client to tell the server which
certificate it should present. HTTP has this same problem (thus making it
currectly impossible to use TLS on a server farm without IP-based virtual
hosts).
Thankfully, the TLS working group has released a fix for this that
actually solves the problem in the correct way. See section 3.1 of RFC
3546. Unfortunately, the server_name extension is probably still not
widely implemented.
--
Russ Allbery (rra(_at_)stanford(_dot_)edu)
<http://www.eyrie.org/~eagle/>