ietf-openpgp
[Top] [All Lists]

Re: [openpgp] AEAD Chunk Size

2019-03-29 15:19:56
As an implementor (iPGMail on iOS), I strongly support Jon's suggestion
(16K "MUST", 256K "SHOULD", everything else "MAY").  I dont want to deal
with lots of optional sizes that just introduce complexity with little
practical benefit beyond some outliers that want to consume huge files in
one big bite.  Mobile devices are somewhat memory limited (though that is
every growing), but 64K or 256K is certainly not a problem.

-Wyllys Ingersoll


On Fri, Mar 29, 2019 at 3:59 PM Jon Callas <joncallas=
40icloud(_dot_)com(_at_)dmarc(_dot_)ietf(_dot_)org> wrote:



On Mar 29, 2019, at 2:07 AM, Neal H. Walfield 
<neal(_at_)walfield(_dot_)org> wrote:

On Fri, 29 Mar 2019 03:19:39 +0100,
Jon Callas wrote:
Like some interim replies, particularly Bart Butler, I thought we had a
rough consensus and that it was approximately:

* MUST support 16KB chunks.
* SHOULD support 256K chunks, as these are common (Protonmail).
* MAY support larger up to the present very large size.
* MAY reject or error out on chunks larger than 16KB, but repeating
ourselves, SHOULD support 256K.

I still think this is a bad idea.

Noted. It was a position that I threw out because I thought it encompassed
a lot of things that people said and also allows you, the implementer to
say, “screw it, we’re doing 256K only and to heck with all of you” and it
would meet the standard. You could even implement 64K, thus generating a
brinksmanship debate with you and Protonmail and I don’t have to listen to
it.

I think that my proposal above is flawed because there’s this squishy
space between 64K and 256K, and really hoped someone would either say,
“Fine. I’ll give up on 256K” or “Fine. 256K is fine.” And then we’d modify
the proposal and probably be done. It was, however a position I thought we
might reason from.

I could also get behind a hard limit of 2^30 on the grounds that that’s
what we had for partial body lengths, but I understand the comment that
there are things like multi-terabyte S3 buckets and out and out forbidding
those to be single-chunk is a bit churlish, but only a bit.

This is the bit that I don't understand.  Clearly you see some
advantage to having large chunk sizes.

Well, actually what I was doing there was offering a number lower than
2^whatever for the max max max oh-dear-god-why-do-you-want-to-do-this size
at a (to me) very, very generous gigabyte to the other side on the max size
that you can then safely ignore.



But, let me try to use your framing:

I propose that we remove the chunk size parameter from the AEAD header
and fix it to a small value (e.g., 64 KB or 256 KB), because 1.) no
one has shown that a large chunk size is useful, 2.) large chunk sizes
encourage implementations to release unauthenticated plaintext, which
is a serious security concern [1], 3.) if an implementation releases
unauthenticated plaintext for large chunks, then an attacker can
always convince it to release unauthenticated plaintext for the first
chunk in a message [2], 4.) making the chunk size variable increases
complexity, which is a security concern.

This isn’t actionable.

There’s a lot of rationale here, but not an actual proposal.

My proposal (left at the top of this missive for easy referral) was a
concrete proposal. The above paragraph has a lot of justification in it and
is interesting polemic but it is not an actual proposal.

If you had said:

* Chunk size is 256K. Always. If you have less than 256K, pad it with
zeroes.

Then that would be a proposal.

As I understand your position, you want to allow implementations a
broad degree of flexibility in choosing the chunk size.  Can you
please explain why this is useful?  I've reread your messages, but
beyond what appears to me to be some hand wavy performance argument,
and an apparent misunderstand (chunk size != message size) [3],
neither of which are convincing, I can't find any other arguments.
I'm sorry if you did and I didn't understand.

You don’t understand my position.

My position is that there are a bunch of people (like you, at least I
think you) who want to build a general-purpose implementation that you hope
will be used all over the place. You have a legitimate need for small
chunks.

There are other people who observe that there’s also a legitimate need for
huge chunks because they want to do storage encryption on very large things
and — well, they have considerations that I’m not paying a lot of attention
to.

My proposal above lets them do their thing and lets you ignore them.
That’s my position. Making as many people happy as possible.


I'm a bit confused.  In my original mail, I included a PR:


https://mailarchive.ietf.org/arch/msg/openpgp/XH098WlJe8lkOypIaB1IXTde2sY

Was that not actionable enough?  Should I link to the PR more often?

It would help, but even better would be to send what the proposal is. I
really did read it and it’s a patch file. Patch files are hard for humans
to read. It’s really nice to have a redline, but it’s also nice to know
what the full thing is without having to mentally emulate git.


 I've spent some time thinking about use cases for different chunk
 sizes, and I can't come up with any modulo some, IMHO, insignificant
 performance tweaks.  Can you please give some examples of use cases
 that would profit from different chunk sizes?

I’m not the one proposing the large chunk sizes. And as I understand the
people who are doing it, performance isn’t their issue, data integrity is.

There’s a dilemma in here.

One is that there’s a belief that


 What should / would you recommend an implementation do if it
 encounters a chunk that it can't buffer?  I see two choices: report
 an error, or release unauthenticated plaintext.

Report an error. I’ve said that many times.

Regarding our implementation: it doesn't actually have tight resource
constraints.  Our primary goal is to write a secure implementation.

Okay, so you’re arguing resources when you don’t really mean it. Gotcha. I
thought that was the case that the things you were saying didn’t jibe
together.

From that goal, we derived the constraint that we must always work in
a bounded amount of space.  The current version of AEAD doesn't allow
this without potentially rejecting messages that other implementations
can process by being insecure (releasing unauthenticated plaintext),
which we don't want to do.

That constraint is a long-standing meta-requirement in OpenPGP, so cool.

However, I detect that there is an impossible-to-solve problem here, and
I’ll outline it in another note later.


But, my concerns are not only about my implementation.  I'm concerned
about the ecosystem.  And, the current proposal encourages insecure
implementations as demonstrated by GnuPG and RNP processing
unauthenticated plaintext.  I think the standard should not
proactively make it harder to write a secure implementation.  And that
is what I see the AEAD chunk size doing.

I think I understand where you’re coming from, but this is a standards
organization. Standards are concerned with interoperability between
implementors. I learned that from Jeff Schiller ages ago. Think of a
standard as being like a dictionary and grammar. When you send me a
message, I use that grammar/dictionary to determine what it means.
Similtharly, if I want to encode a message that I want you to understand,
then I encode it according to that grammar as well.

My proposal above says to an encoder that the safest thing to do is to use
a chunk size of 64K. Everyone can decode that, and you’re fine. Many, many,
many of them are going to be okay with 256K, and odds are that anyone of
any import will do fine with it. (That’s what the SHOULD means). It also
says there are people out in Lala-land who might be playing with chunks
that are out to 2^whatever, and that’s kinda cool, but ignore them.

That’s why I made it. It gives a sandbox for people who want to go out on
the edge that the mainstream can ignore.



Thanks,

Neal

P.S. I hope it is clear that I'm trying to engage in a constructive
manner.  I sincerely haven't understand your position, and I really
want to.

I think that’s because I’m trying to support a community of people who
have reasonable, differing needs, and you want to mandate your
implementation ideas on the universe. I apologize if that’s harsh, but it’s
what I perceive. If I’m mistaken, let me know.

        Jon


_______________________________________________
openpgp mailing list
openpgp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/openpgp

_______________________________________________
openpgp mailing list
openpgp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/openpgp
<Prev in Thread] Current Thread [Next in Thread>