When a client MTA tries to perform TLS authentication on a server MTA,
and the authentication fails, should the client MTA move on to the next
MX host, or give up?
I have an OpenSSL-based SMTP MTA client that attempts to authenticate
the server whenever offered the STARTTLS EHLO keyword. It gets the
server's certificate, verifies that the certificate is valid, follows
and verifies the trust chain, and then verifies that the hostname on the
cert matches the hostname to which the client connected (normally the MX
host). If any part of the authentication fails, the client drops the
connection and then moves on to the next MX host, repeating until it has
tried them all.
I'm struggling with trying to decide if this is correct behavior or not;
for some reason it seems intuitively wrong, although I can make a good
argument for it being right. I've got one case where the receiver has a
phalanx of MTAs all sharing the same certificate; the client happily
runs through all of them, failing them all. I've got another case,
though, where the receiver has to MX hosts configured as primary and
backup; the higher-precedence primary has an expired cert, but the
secondary has a valid one. So the mail is delivered successfully.
Any RTFMs or pointers to other lists are welcome.
<csg>