Hal,
I appreciate your clarification.
I can see that the real challenge will be to deal with the 3 levels of
PBLs. When I detect that the EncryptedDataPacket is using PBLs, can I
safely assume that the enclosed compressed data and literal data packets
have been put into PBLs also? It is possible for a sender to prepare the
literal and compressed data packets as a whole without using PBLs and
then to send the encrypted data out in PBLs, even though it doesn't make
sense and defeats the purpose of employing PBLs. I'm not sure if the
specification prevents them from doing this sort of PBL encoding at the
outer layer only.
Regards
Hasnain
-----Original Message-----
From: Hal Finney [mailto:hal(_at_)finney(_dot_)org]
Sent: Monday, December 29, 2003 6:36 AM
To: hal(_at_)finney(_dot_)org; Hasnain Mujtaba; ietf-openpgp(_at_)imc(_dot_)org
Subject: RE: Compression and partial length chunks
Hasnain -
If I must have hold the whole source data in memory before I can
compress and then encrypt it, then I would always know before-hand the
lengths of the resulting packets (literal, compressed, and encrypted).
So when exactly does the need arise to use PBLs? Even when I am
streaming data, I cannot avoid the basic constraint that the entire
source data must be in memory before it can be processed. It is not
the
case, as you pointed out, that I am feeding chunks of the source data
to
the compression and encryption routines and then sending them out as
PBLs.
I'm afraid I misled you in my earlier comments. Nothing in the OpenPGP
algorithms or formats requires you to hold the entire document in
memory.
The compression and encryption algorithms can operate on chunks at a
time,
as long as they are in order: first chunk, second chunk, third chunk,
etc.
You asked earlier whether you could process the chunk "independently of
the chunks already received" and my answer was no. What I meant was
that you have to have processed those other chunks already, in order,
and retained state information from those chunks. The widely used zlib
decompression library provides this functionality, for example, as do
most decryption libraries which provide the CFB chaining mode used by
PGP.
I am new to the interesting world of PGP. I think it would be useful,
if
at all possible, to be able to deal with the chunks independently so
as
to avoid physical memory limitations.
I hope my clarification above will show that it is possible to do this,
but that state must be retained from chunk to chunk in the decompression
and decryption modules.
Hal Finney