If it was that simple to match up received e-mail to sent e-mail,
everyone would be doing it already as a whitelisting mechanism.
Unfortunately, there are things like mailing lists, aliases, people who
use multiple e-mail service providers, and so on.
This is a readily addressable issue. I previously stated that an email service
provider could maintain a list of outgoing emails sent by each user. Incoming
challenges could then be filtered out if they did not correspond to the
Asrg mailing list