Re: [openpgp] Deprecating compression support

I agree with deprecation at a reasonable pace. SHOULD looks right.

I would primarily like to correct what PGP Zip is. It is a TAR file with
standard OpenPGP compression. The compression layer was not provided by
gzip (that would not have worked in Windows environment). PGP Zip doesn't
need a license  / is free. Therefore, we should expect that OpenPGP
compression is likely used by users of commercial PGP on Windows. In
business world, many users have PGP installed because their employer wants
every machine to use PGP disk, and so they get PGP Zip available to them.

On Mon, Mar 18, 2019, 1:46 PM Jon Callas <joncallas=
40icloud(_dot_)com(_at_)dmarc(_dot_)ietf(_dot_)org> wrote:

> I like the basic proposal. I think that deprecation is better than
> banning, and consequently we ought to be doing it with SHOULDs and SHOULD
> NOTs rather than MUSTs, as that’s banning rather than deprecating..
>
> However, I want to add that implementations today can deprecate it on
> their own and no changes in the standard need not be there. If an
> implementation creates a key with compression preferences set to no
> compression then any other implementation is bound by the standard not to
> compress. I advocate this as well. As OpenPGP sits today, any implementer
> can unilaterally not only deprecate, but eliminate compression. In fact,
> I’ll go so far as to advocate that all implementers, even GnuPG ought to
> just start making keys by default that eschew compression. More on this
> below.
>
> My rationale for getting rid of compression is different. I think that the
> security-based reasons in this thread are largely unconvincing and often
> just not quite correct. I don’t want to discuss that in this thread,
> because I agree with the destination — let’s move away from compression,
> and am afraid that if we debate the reasons for it we distract from the
> result we mutually want. We should deprecate compression, but not because
> of security, but because of simplicity, and in short a case of “that was
> then, this is now."
>
> OpenPGP has many things in it that are needlessly complex, and compression
> is one of them. (Other needless complexities include the way that packet
> sizes are computed, and others.) Much of this comes from its heritage as a
> utility, and a utility that was designed for DOS and BBSes. The
> considerations of networking and communications in 1991 are not at all what
> we should have for 2021.
>
> One of these differences is that zip-style compression is just not as
> useful now. Obviously, the wins for compression are on things that are the
> largest. Most of the large things we transfer today are already compressed,
> and thus compressing them again is mostly a waste of cpu cycles. There are
> some file formats that have zip-style compression built in. For example,
> GIF and PNG graphics are explicitly compressed. Other formats like JPEG are
> a different sort of compression but have the same characteristic that it is
> reasonably mathematically pseudo-random and not readily compressed further.
> Ditto for audio formats like MP3 and AAC, while FLAC and ALAC are like GIF
> and PNG in that they’re losslessly compressed.
>
> Today, the large things we send are media, and therefore there’s no need
> to compress them. The things we send that are easily compressible tend to
> be smaller.
>
> In the cases where large files are sent that can be compressed, the best
> solution is for that system to compress the file itself. Remember,
> OpenPGP’s history comes from DOS programs on small computers and they
> didn’t even have a way to pipe a tar or zip command to an encryption
> program. That’s not the case today.
>
> Historically, compression has been the largest share of the time it takes
> to process a file with an OpenPGP implementation, and thus we are doing
> something by default that burns CPU for little gain. We should stop doing
> it by default. Again, the best way to do this is to create keys that have a
> compression preference of none. We can do this now.
>
> Over a decade ago, the PGP program had a feature of the “PGP Zip” file.
> This was literally nothing more than a tar file that was run through gzip,
> and then encrypted with OpenPGP without compression. PGP itself had some
> nice viewers that let someone manipulate the container just as other
> directory compression systems did, but any unix system could handle the
> file by just decrypting with gnupg and piping that to tar. Later on, as I
> remember (correct me if I’m wrong, Derek, or someone), we shifted to bz2.
> The same strategy can be improved as compression tools improve.
>
> The compression inside of OpenPGP is also hard to implement correctly. The
> default compression, the “DEFLATE” option is a modified implementation of
> ZIP-style programs from the era of the late 1980s. As those programs
> coalesced into default (not precisely standard, merely ubiquitous)
> implementations, they went in one direction and that era’s PGP stayed with
> the variant. When we did RFC 2440, we added in ZLIB (RFC 1950) encryption
> so that an implementation wouldn’t have to hack the compression software.
> This is another reason to move away from compression inside OpenPGP. RFC
> 4880 added in an option for BZ2 (it was the new hotness at the time), and
> these days the cool kids are using 7-Zip as it's even better than bzip. The
> best way to keep up with advances in compression is piping from a
> compressor to some OpenPGP implementation, rather than continuing to chase
> advancements.
>
> The underlying reality is even worse. Go look at section 5.6 of 2440, and
> there are interoperability hints for working with PGP2 because it had
> further limitations in it for internal table sizes. This is another reason
> to get rid of it for simplicity. It flat isn’t needed.
>
> Simply not having compression aids implementation, as well. A number of
> years ago, I was putting together a Javascript implementation and there
> were no good compression libraries at the time, so we just created keys
> that said “no compression, please” and ran with it. Every implementation
> can handle decoding an OpenPGP object that is not compressed, so there are
> fewer bits of backwards compatibility to transition away from it.
>
> Lastly, I’ll note that everything I’ve said here could be applied to
> banning encryption in 4880-bis, as well. I am not opposed to a ban, but I
> think deprecation is better, particularly since every implementation can
> just stop doing compression by default and everything will work just fine..
> Repeating myself, I even advocate this. Everyone who makes some OpenPGP
> program can stop today, and likely should. I’ll even promote that to
> SHOULD.
>
> (The biggest reason against an outright ban is that there are a lot of
> systems that use OpenPGP in the midst of some internal process, like moving
> around large, sensitive files. There are thus likely a whole lot of shell
> scripts somewhere that someone’s got to find and change, and I wouldn’t be
> surprised if something breaks if there’s a sudden shift. If we simply start
> creating keys with preference of no compression, it gets maximum quick
> uptake, and can even be improved with some options in gnupg.conf or
> equivalent.)
>
> So to sum up, I completely support deprecating compression, because it
> improves and simplifies the standard. Today, compression by default is of
> limited benefit, and it simplifies implementation and understanding of
> OpenPGP to phase it out. I am an enthusiastic supporter of implementations
> just making the changes now, without a document change. I’m also in favor
> of deprecating. I’m not opposed to a ban, but I think it’s unneeded.
>
>         Jon
>
>
> _______________________________________________
> openpgp mailing list
> openpgp(_at_)ietf(_dot_)org
> https://www.ietf.org/mailman/listinfo/openpgp
_______________________________________________
openpgp mailing list
openpgp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/openpgp