I like the basic proposal. I think that deprecation is better than banning, and
consequently we ought to be doing it with SHOULDs and SHOULD NOTs rather than
MUSTs, as that’s banning rather than deprecating.
However, I want to add that implementations today can deprecate it on their own
and no changes in the standard need not be there. If an implementation creates
a key with compression preferences set to no compression then any other
implementation is bound by the standard not to compress. I advocate this as
well. As OpenPGP sits today, any implementer can unilaterally not only
deprecate, but eliminate compression. In fact, I’ll go so far as to advocate
that all implementers, even GnuPG ought to just start making keys by default
that eschew compression. More on this below.
My rationale for getting rid of compression is different. I think that the
security-based reasons in this thread are largely unconvincing and often just
not quite correct. I don’t want to discuss that in this thread, because I agree
with the destination — let’s move away from compression, and am afraid that if
we debate the reasons for it we distract from the result we mutually want. We
should deprecate compression, but not because of security, but because of
simplicity, and in short a case of “that was then, this is now."
OpenPGP has many things in it that are needlessly complex, and compression is
one of them. (Other needless complexities include the way that packet sizes are
computed, and others.) Much of this comes from its heritage as a utility, and a
utility that was designed for DOS and BBSes. The considerations of networking
and communications in 1991 are not at all what we should have for 2021.
One of these differences is that zip-style compression is just not as useful
now. Obviously, the wins for compression are on things that are the largest.
Most of the large things we transfer today are already compressed, and thus
compressing them again is mostly a waste of cpu cycles. There are some file
formats that have zip-style compression built in. For example, GIF and PNG
graphics are explicitly compressed. Other formats like JPEG are a different
sort of compression but have the same characteristic that it is reasonably
mathematically pseudo-random and not readily compressed further. Ditto for
audio formats like MP3 and AAC, while FLAC and ALAC are like GIF and PNG in
that they’re losslessly compressed.
Today, the large things we send are media, and therefore there’s no need to
compress them. The things we send that are easily compressible tend to be
smaller.
In the cases where large files are sent that can be compressed, the best
solution is for that system to compress the file itself. Remember, OpenPGP’s
history comes from DOS programs on small computers and they didn’t even have a
way to pipe a tar or zip command to an encryption program. That’s not the case
today.
Historically, compression has been the largest share of the time it takes to
process a file with an OpenPGP implementation, and thus we are doing something
by default that burns CPU for little gain. We should stop doing it by default.
Again, the best way to do this is to create keys that have a compression
preference of none. We can do this now.
Over a decade ago, the PGP program had a feature of the “PGP Zip” file. This
was literally nothing more than a tar file that was run through gzip, and then
encrypted with OpenPGP without compression. PGP itself had some nice viewers
that let someone manipulate the container just as other directory compression
systems did, but any unix system could handle the file by just decrypting with
gnupg and piping that to tar. Later on, as I remember (correct me if I’m wrong,
Derek, or someone), we shifted to bz2. The same strategy can be improved as
compression tools improve.
The compression inside of OpenPGP is also hard to implement correctly. The
default compression, the “DEFLATE” option is a modified implementation of
ZIP-style programs from the era of the late 1980s. As those programs coalesced
into default (not precisely standard, merely ubiquitous) implementations, they
went in one direction and that era’s PGP stayed with the variant. When we did
RFC 2440, we added in ZLIB (RFC 1950) encryption so that an implementation
wouldn’t have to hack the compression software. This is another reason to move
away from compression inside OpenPGP. RFC 4880 added in an option for BZ2 (it
was the new hotness at the time), and these days the cool kids are using 7-Zip
as it's even better than bzip. The best way to keep up with advances in
compression is piping from a compressor to some OpenPGP implementation, rather
than continuing to chase advancements.
The underlying reality is even worse. Go look at section 5.6 of 2440, and there
are interoperability hints for working with PGP2 because it had further
limitations in it for internal table sizes. This is another reason to get rid
of it for simplicity. It flat isn’t needed.
Simply not having compression aids implementation, as well. A number of years
ago, I was putting together a Javascript implementation and there were no good
compression libraries at the time, so we just created keys that said “no
compression, please” and ran with it. Every implementation can handle decoding
an OpenPGP object that is not compressed, so there are fewer bits of backwards
compatibility to transition away from it.
Lastly, I’ll note that everything I’ve said here could be applied to banning
encryption in 4880-bis, as well. I am not opposed to a ban, but I think
deprecation is better, particularly since every implementation can just stop
doing compression by default and everything will work just fine. Repeating
myself, I even advocate this. Everyone who makes some OpenPGP program can stop
today, and likely should. I’ll even promote that to SHOULD.
(The biggest reason against an outright ban is that there are a lot of systems
that use OpenPGP in the midst of some internal process, like moving around
large, sensitive files. There are thus likely a whole lot of shell scripts
somewhere that someone’s got to find and change, and I wouldn’t be surprised if
something breaks if there’s a sudden shift. If we simply start creating keys
with preference of no compression, it gets maximum quick uptake, and can even
be improved with some options in gnupg.conf or equivalent.)
So to sum up, I completely support deprecating compression, because it improves
and simplifies the standard. Today, compression by default is of limited
benefit, and it simplifies implementation and understanding of OpenPGP to phase
it out. I am an enthusiastic supporter of implementations just making the
changes now, without a document change. I’m also in favor of deprecating. I’m
not opposed to a ban, but I think it’s unneeded.
Jon
_______________________________________________
openpgp mailing list
openpgp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/openpgp