It is true that Phil Zimmermann has been the chief advocate of making
PGP data structures as concise as possible. (I don't know whether Jon
actually goes around the office singing songs about it or not.) Phil is
out of the country now so I will try to relay some of his thinking.
First, the difficulty is really not as large as people are making out.
The code to read the packet lengths in the various cases is less than a
page, and it's very simple. Read a byte to see if it is the old or new
packet formats. In the old case, read 1, 2, or 4 bytes to get the length;
in the new case, read 1 byte and if it's 0xc0 or over, read a second.
The payoff is that you have slightly reduced the size of all the PGP
exchanged data. By itself it is not so much, but as a philosophy applied
throughout the design, these savings add up. The alternative philosophy
of making everything 32 or 64 bits can lead to bloated data structures.
It's easier on the coder, but every user pays the cost in terms of
larger data.
Phil contrasts PGP's philosophy of a clean interface with SMIME. Look at
a PGP cleartext signature:
-----BEGIN PGP SIGNATURE-----
Version: PGP for Business Security 5.5.2
iQA/AwUBNHijtqy7FkvPc+xMEQJ9EQCg98ARQWaclA7enOcIroIRiaGJeQsAn1om
OZVLRqiOTCRBMJSZ6QX2pbUZ
=Juy1
-----END PGP SIGNATURE-----
Phil has proposed simplifications to shorten this to:
-----BEGIN PGP SIGNATURE-----
iQA/AwUBNHijtqy7FkvPc+xMEQJ9EQCg98ARQWaclA7enOcIroIRiaGJeQsAn1om
OZVLRqiOTCRBMJSZ6QX2pbUZ=Juy1
-----END PGP SIGNATURE-----
(He'd like to see the -----END line go away, too, since it is redundant.)
You can hardly imagine a more concise signature block than this.
Compare that with SMIME. If you've seen an SMIME signed message
in any public forum, you've probably also seen complaints about it.
The signature block is about 50+ lines long. Now, this is largely a
design error in the client software, in that it is including the whole
certificate chain in the signature. But it is probably going to be a
factor in slowing acceptance of SMIME. Nobody likes a technology which
brings them tons of flames every time they send a clearsigned message.
Phil has worked from the beginning to make PGP be as usable and attractive
to end users as possible, taking into consideration "real world" issues.
Where PEM envisioned a complex hierarchical certification structure which
didn't exist except on paper, PGP created the web of trust which could
work locally and be useful from the beginning. In the case of SMIME,
probably the designers of the client software envisioned a world where
everyone was using MIME and no one would care how big and bloated the
signatures were, since they wouldn't see them. But that's not the
real world.
PGP's philosophy has been one of being willing to pay a cost in
implementation complexity in order to reduce objectionable aspects of the
software to end users. Part of that has been making the data structures
as compact as possible.
These efforts to use concise data structures are costly to implementors,
but beneficial to users. You pay the cost once per implementation -
a few pages of code for this and similar features. But the benefit is
felt every time a message is sent. It doesn't take much to swing the
cost-benefit balance in favor of making things nicer for users.
Let me make one technical point, in case all this philosophizing is
unpersuasive. Part of the purpose of the new packet length headers is
to allow "one pass" processing. You can put out a packet length header
which says "will be followed by 2K of data, then another packet length
header". This allows you to start emitting data even when you don't
know the total length of the data in advance, and also allows lengths
greater than 2^32 bits.
This is one of several technical changes which are designed to allow
one-pass processing. Yesterday I explained how the new ascii armor header
line and the one-pass signature packets also supported this capability.
It's an important feature which allows software to get by without the use
of temporary files.
Any changes to another packet length header format should provide some
way of allowing one-pass processing. A simple 32 bit length will not
be adequate.
Hal Finney
hal(_at_)rain(_dot_)org
hal(_at_)pgp(_dot_)com