On Fri, 4 Apr 2003, Martin Stecher wrote:
a) Thus, somebody needs to tell OPES processor
and callout server what is an "application message"
definition that they should both use during OCP
communications. Should OCP support auto-negotiation or
rely on out-of-band (e.g., manual) configuration?
How about this:
We discussed that we probably end with multiple application
protocol bindings of the OCP. One application protocol binding
could be a generic binding which is not really related to any
Internet protocol but handles data as if it was a file on disk.
Application messages could have a very simple and generic format
here. Many callout services may want to support this protocol
binding and an OPES processor may have pre-processing code to
translate Internet protocol data to that simple format.
Other protocol bindings will handle native HTTP and SMTP messages.
Every application protocol binding defines a fixed application
message format.
I think both Reinaldo Penno and myself want a scheme very similar to
what you described above. There will be a set of supported application
protocols and message encodings. Known message "formats" will be
documented. OPES agents will support some subset of them. It is
probably a good idea to document a scheme where OCP sees only message
payloads and is unaware of the actual application protocol (still, one
would need to agree on things like payload encoding).
Negotiations between OPES processors and callout services is then
limited to exchange supported application protocol bindings.
This is the only uncertain area. You are, essentially, voting for
supporting auto-negotiations (over a finite set of known
protocols/encodings, of course). If nobody objects to this automation,
OCP will implement auto-negotiation. My understanding is that Reinaldo
is already working on specific messages to handle that.
e) Do we need to group processing of a single application
message together using OCP transaction concept? Should
OCP transaction mean something else? Do we need OCP
transactions at all?
Yes, I think so.
In most cases not more than one application message will be
transferred from CLIENT to SERVER and back within a transaction.
But we have two examples where multiple messages make sense:
1. Allowing the SERVER to return different SMTP messages for
one email to multiple recipients it received from the CLIENT.
2. Pre-processing of a message at CLIENT which results in
multiple application messages, e.g. MIME sections of an email
to be transferred as multiple application messages
The first case requires to have the messages being wrapped into
a single transaction.
The second case could benefit from a transaction wrapping
because the SERVER may want to know which MIME sections belong
together.
The current draft does not support the second case in a situation where
both cases happen at the same time (e.g., multiple responses to a
pre-processed message part). This combination can be supported within one
OCP transaction only if server responses specify which pre-processed
application message [part] they are adapting. That is, an
app-message-start message from a callout server will have to include a
second am-id parameter, carrying the am-id of the corresponding
application message part from the processor. While possible, this
defeats the purpose of having an OCP transaction -- since all server
responses have to carry two am-ids, the transaction wrapping is useless
for anything other than aborting the whole thing with one message.
I think the second case must be supported by generating multiple
(possibly interleaved) OCP transactions, one for each pre-processed part.
The overhead, if any, will be minimal, and the only missing
functionality would be an ability to abort all parts with one message. We
can add 'xaction-ends' to support the latter if needed.
I agree with Alex that the transaction overhead could (and should)
be limited to a minimum so that no additional transport layer packet
is needed. For the normal case the xaction-start app-message-start
couple could almost look like a single message I think.
But let's keep the xaction-start message two allow support of above
cases.
Yes, there is not much difference between parsing two simple messages
with 1 attribute and one complex message with 2 attributes.
- The sentence "A SERVER MUST NOT send this message" should also
be added to the connection-service message and maybe also to
connection-priority
Ack.
- I am not sure whether the ability to change the service with
connection-service and within xaction-start is really needed.
It increases the implementation requirements for a callout
server but is it really needed?
Setting a service will usually also mean to negotiate further
parameters.
Couldn't we just put this into connection-start?
Using different services will then mean to start different
connections. Isn't this just ok?
Yeah, that will work. The rationale was that you can use the
connection to specify common/default services and then overwrite that
on a transaction basis. You make a good point regarding associated
negotiations though. Will add this question to the to-do list. The
answer will probably depend on what kind of negotiations we would end
up supporting.
- I forget about the motivation of the "ack" flag and data-ack
messsage. I thought we wanted to have this done by the
transport layer binding. Can you point me to the thread in
which this has been discussed?
Ack/Data-ack sequence is useful if one side wants to track (or debug)
progress in receiving/parsing OCP messages on the other side. Perhaps
OPES processor can optimize its buffer management by knowing how fast
the callout server is reading the data. Do you think that enough
motivation to support the ack/data-ack pair?
- Isn't there a problem with data-pause?
If the SERVER sends data-pause to the CLIENT, the CLIENT
MUST stop sending but before the CLIENT receives and
processes this message it may already have sent additional
data-have messages. The SERVER will then receive unexpected
data any MAY abort.
Indeed. I've added the "MAY abort" part at the last minute and
missed the bug. Will fix.
Second problem: A CLIENT may not be able to pause as long
as the server likes it to pause. If it is not able to keep
a copy of the complete message and if it is not be able to
pause the origin server for the same time, it may easily run
into an internal buffer overflow problem.
I do not think that can happen because the CLIENT can always stop
reading incoming data if it has no place to put it. Right?
There is a deadlock problem, however (see the "copying commitment and
deadlock" thread on this list). That problem is on the to-do list.
How about defining that data-pause MUST NOT be send by the
SERVER but adding a new message data-pause-request which is
only to be sent by the SERVER. Upon receiving this message
the CLIENT MAY pause by sending data-pause to the SERVER.
The above logic is what data-pause from the SERVER was supposed to mean.
We can fix the current definition or use data-pause-request instead.
Will fix.
Thanks a lot for the comments,
Alex.