The problem with a "test suite" is that it would allow a software creator
to test that his software READS openpgp compliant data, but not that it
CREATES compliant data. The latter testing is more difficult because as
Jon Callas pointed out there is a lot of randomness in what is created
and so you can't just compare against a "known good" output.
I am inclined to think that traditional interoperability testing is a
better way to approach this problem. At least with openpgp we don't all
have to get together, it can all be done online.
We would define a specific set of message/key types to create, and
everyone who wanted to participate would create messages/keys of those
types (or whatever subset they support), then everyone else would try
to read them.
See the S/MIME page at
http://www.rsasecurity.com/standards/smime/interop_center.html for an
example. They did it differently by comparing with a reference
implementation, but since we have only a few implementations I think
it is feasible to have N by N comparisons.
Hal Finney