In the working group meetings, I believe it was decided that binary pieces
can be specified in the PKCS-7 envelope as "Content-Transfer-Encoding:
binary", which removes the size overhead for the first BASE64 encoding.
...
Am I wrong about this?
Unfortunately, yes.
The choice of transfer encoding, between bin64 and binary will
depend upon the available transfer paths between originator and recipients.
Most paths cannot support binary. Most paths therefore require bin64.
Overhead remains.
Dave is absolutely right, but not for the reason he gives. Dave, the reason
you've given here doesn't apply since the outer encoding on the PCKS#7
part turns the whole thing into base64, which in turn eliminates any
transport issues.
The real reason that Dave is right is that support for binary MIME is far from
universal. A few agents can handle it (enough to demonstrate interoperability
of the concept at least) but most cannot, at least not in an email context.
(HTTP is another matter -- binary is the default there and support is nearly
universal.) Think for a moment about what it means to have multiple binary
parts enclosed by a multipart structure and you will see why this is so
tricky. MIME goes to a lot of trouble to make sure binary structures can
be done correctly, consistently, and interoperably, but this doesn't
mean that existing agents handle such things correctly.
Furthermore, there is no requirement that agents support binary MIME -- its
optional as far as email MIME conformance goes. So this presents significant
problems when the resulting binary object is removed from its PKCS#7 and
presented to a MIME agent for display.
I must confess that I missed this when I read over the S/MIME specification. We
discussed this whole issue at considerable length when we prepared the security
multiparts specification. We eventually decided that in the case of a signed
object use of 7bit encodings had to be a requirement, but we left restrictions
on the form of the enclosed within multipart/encrypted up to the security
service to specify. The logic here was that some security services may be
defined that only apply to specific domains where binary MIME is known to be
(or not to be) possible. Unlike signed objects, an encryped object is
completely useless once it leaves the domain of the security service that
produced it. Signed objects are potentially useful anywhere, hence the
need to restrict them so they will work over any transport.
Note also that this problem exists in S/MIME regardless of whether you're
signing or encrypting. In the case of signing the overhead really starts to add
up -- 33% for the base64 encoding of binary content, 100% on top of that for
the alternative part, and 33% on 1/2 of the total so far for the PKCS#7 piece.
Security multiparts have similar problems in the case of encryption, but not
when signing. But this whole area is a known problem, and work is underway to
define lighter encodings with far less overhead that can be used in these
situations. These encodings will not cause any problems with agents that don't
support binary MIME, but of course the agents will have to have suppport for
the new encoding added... But a new textual encoding is *much* easier to add
than support for binary is.
In general the overhead of such an "inner" encoding can be reduced to 4-5%
maximum and to even less if a preprocessing pass is used. I expect to see
proposals in this area out as Internet Drafts in the next couple of months.
There are still some issues regarding tradeoffs between preprocessing and
overhead reduction that need to be resolved.
Ned