Hello Alex,
I read your comments...answers inline...
-----Original Message-----
From: Alex Rousskov [mailto:rousskov(_at_)measurement-factory(_dot_)com]
Sent: Wednesday, April 02, 2003 3:52 PM
To: OPES Group
Subject: Re: Comments on ocp-00
On Wed, 2 Apr 2003, Reinaldo Penno wrote:
1)
" application message: A sequence of octets that OPES processor
designates for callout service processing. Usually, an
application message is the basic unit of application protocol
communication, as defined by that application protocol (e.g.,
HTTP/1.1 message). Before OCP is applied, OPES processor may
pre-process raw application messages to extract and manipulate
well-known parts (e.g., HTTP message headers or SMTP
message body)
in order to subject just those parts to callout
services. For the
purpose of OCP, the result of such preprocessing is
an application
message. Naturally, OPES processor and callout services it is
configured to use must agree on what application messages are
acceptable. Establishing such an agreement is beyond
OCP scope."
Everything after "Before OCP..." has nothing to do with the
definition
of application message anymore. This has to do with the workings of
the protocol.
Agreed. The second part is an explanation/motivation for the
not-so-obvious definition. Do you think it is worth keeping,
by moving it elsewhere? Or should we completely get rid of that prose?
Although I suggest to rip out the last part of this definition I
should say that I disagree (if I understood correctly) with your
statement "Establishing such an agreement is beyond OCP scope".
Capability negotiation can help you establish what application
messages are acceptable on each connection or callout server as a
whole.
I agree that it can help. I hope we can keep things simple (no
negotiation) in this context without loosing too much
functionality or interoperability. I suspect that callout
implementation abilities will be well known to those who add
a callout server/service to an OPES intermediary. These
administrators would be able to configure the system appropriately.
Humm...I'm really not sure about that. I guess the sysadmin can learn it,
but it would be much easier and debug friendly (if something goes wrong) to
have capability negotiations.
Not to mention that as OPES services become more "rich" a lot of new
capabilities will appear.
IMO, there are two cases where auto-negotiation is important:
- dynamic config changes: a new service is automatically
added to the list of services available to an OPES
processor;
- dynamic ability changes: the same service suddenly can no
longer handle complete HTTP messages and demands just
HTTP message bodies in certain encoding.
Both cases seem unusual and not worth supporting. Do you see
other use cases for auto-negotiation? Or do you think the
above two cases are worth supporting in OCP?
See above. I think that relying alone in CLI configuration is not the way to
go. IMHO if you look at routing protocols, PPP, and text protocols like SIP
and HTTP there is some form of capability negotiation. SOAP also has some
form of capability negotiation.
Suggested version.
" application message: A sequence of octets that OPES processor
designates for callout service processing. Usually, an
application message is the basic unit of application protocol
communication, as defined by that application protocol (e.g.,
HTTP/1.1 message).
Agreed. Please let me know whether you think we should still
keep the explanation/motivation part elsewhere in the specs.
Yes, we should...in the protocol working section.
2)
"A callout server may send this data back to
the OPES processor, with or without modifications."
Suggestion
"A callout server may send or not this data back to the OPES
processor. When the callout server sends data back to the OPES
processor, it can be modified or not."
I think "may" implies "may not", but I will change to be explicit.
3)
"The primary purpose of OCP communications is modification of
application message payloads..."
Do not want to seem picky, but the primary purpose of OCP
is to ship
packets back and forth between Callout Server and OPES Processor.
Whatever is done with message has nothing to do with OCP
anymore. It
is a function totally contained within the callout server.
I guess you can say
"The primary purpose of the OCP is to send data to the
callout server
where it will be examined and possibly changed"
OK.
4)
" OCP transaction is a logical sequence of OCP messages
processing a
single original application message. The result of the
processing may
be zero or more application messages, adapted from the
original. A
typical transaction consists of two message flows: a
flow from the
CLIENT to the SERVER (sending original application message) and a
flow from the SERVER to the CLIENT (sending adapted application
messages). The number of application messages produced
by the SERVER
and whether the SERVER actually modifies original
application message
depends on the requested callout service. The CLIENT or
the SERVER
can terminate the transaction by sending a corresponding
message to
the other side."
Hummm...I guess I'm not confortable with the idea of mixing the OCP
semantics per se and whatever it carries..The problem with this
approach is that we will start asking things like "what is the
original application message?". "Is the the whole HTTP 1.1 message
before TCP fragmentation or after?" , "And if is protocol
X..and Y",
etc, etc.
This is my (lean) suggestion using your definitions:
" An OCP transaction is a logical exchange of OCP messages.
A OCP transaction starts with a xaction-start message, followed
by zero or more data-xxx messages, zero or more appl-xxx and ends
with a xaction-end message. "
Here I disagree. The whole purpose of having transactions is
to group together processing of a single application message
(which is a defined term). That grouping is not strictly
necessary but it brings structure and is actually required by
the OCP requirements draft (see the "3.3 Callout
Transactions" section).
I guess this is more of a performance/device intelligence discussion. My
first point is that having to send xaction-start/stop for every single
application message is quite an overhead. Actually this might be the biggest
overhead I've ever seen. If a HTTP message fits into a TCP packet we will
have to process 2 extra packets for every single "real" (for the lack of a
better word) packet.
The other point I would like to make is that having to determine start/end
of application messages means that the OPES processor will need to
understand all these applications, which is not necessarily desirable. Let's
suppose I have a callout server that deals with content filtering. The OPES
processor gets TCP packets with HTTP in it (fragmented or not) and ships
them to the Callout server, which will reconstruct the HTTP message if
needed and send it back to the OPES processor. The OPES processor then
decapsulates the IP/TCP/HTTP from OCP and sends it on its way.
The same example would apply for RTP where I just want to send packets to
the callout server for content adaption. I would not like to have to
implement a RTP/MPEG-1 decoder in my OPES processor to know where
application messages end/start.
Finally, let's suppose the first packet of a video session was lost and I
can recognize that. Since there is no retransmissions, what should I do
since I can only send whole application messages to the Callout server? I
will never see that first packet again. In the case of HTTP I would need to
wait for a retransmission before I can send anything to the callout server.
And who can guarantee me that I will ever receive everything?
If we adopt your wording, there is no need to have a notion
of OCP transaction at all, right? Every app-message message
from the callout server would simply have to have an extra ID
pointing to the original application message it modifies (so
that the OPES processor can put things back together and
forward the message as appropriate).
See above.
I agree that "original" qualifier should be removed, changed,
or defined.
As for "And if is protocol X..and Y", it is a valid question
that we will need to address, but it does not seem to be
related to the definition of an OCP transaction?
5)
"xid: OCP transaction identifier. Uniquely identifies an OCP
transaction originated by a given CLIENT. Since each
transaction
deals with a single application message, "xid" also identifies
that application message."
As you can guess, IMHO the part ""xid" also identifies that
application message." is troublesome.
My suggestion
"xid: OCP transaction identifier. Uniquely identifies an OCP
transaction originated by a given CLIENT."
OK.
6)
"am-id: Application message identifier. Uniquely identifies an
application message within an OCP transaction."
I guess it is one more reason to make the modification I mention in
number
5) above.
I am not sure I follow. Am-id exists so that a callout server
can modify one [original] application message into more than
one [adapted] application messages. Since there could be more
than one adapted message, we need am-id to be able to
interleave their data and to abort them.
I understand your point. We can still have am-id if the OPES processor wants
to do something with the message after it receives it from the Callout
server. Again, see above.
7)
" am-source: Information about the source of an
application message.
For example, HTTP client or origin server IP address and TCP
connection ID.
am-destination: Information about the destination of an
application
message. For example, HTTP client or origin server address."
I'm not sure I follow this...TCP connection ID? Information
about the
source of an application message?
Callout server actions may depend on meta-information that is
not available in the message itself. For example, a callout
service may insert user-specific pieces of HTML but user
authentication may happen outside of the current message
scope and, hence, user ID would need to be relayed by OPES
processor to the callout server (e.g., via the destination parameter).
However, I consider current "am-source" and "am-destination"
approach inadequate because meta-information can include more
than source and destination descriptions. I will propose and
argue for a more general replacement shortly. Let's ignore
this broken part for now.
8)
"size: Application data size in octets. The value either applies to
the data attached to an OCP message or suggests data
size to be
sent in response to an OCP message."
Is this the original value (which I guess before reaseembly
it cannot
be known)? The value of the OCP payload?
If it is the size of the data attached to an OCP message,
then it is OCP payload (e.g., "here are the next 100 bytes").
If it is a suggestion (e.g., "give me 100 more bytes"), then
it is not OCP payload. See data-have and data-need messages.
Note that "data" is a defined term.
The "size" attribute has little to do with application
message size. It is about the size of an application message
chunk being sent or chunks being requested. The value of
"size" would be equal to the transfer-length of the
application message only if the entire application message is
transmitted via one data-have OCP message.
Okay, I guess this very answer that you gave me could go into the draft.
9)
"modified: A boolean parameter indicating that the attached
application data has been modified by he SERVER.
Only SERVER may
send this flag. This parameter can be used with any
OCP message
that has am-id parameter."
Suggestion
"modified: A boolean parameter indicating that the attached
application data has been modified by the SERVER.
Only the SERVER may
send this flag. This parameter can be used with any
OCP message
that has am-id parameter."
I guess the caveat is that if you but the flag in one OCP message
containing am-id, you have to put in all of them for the same am-id.
Actually, not. The granularity may be one level lower. The
SERVER tells the CLIENT that a particular message data
_chunk_ got modified. This implies that the entire message
got modified, but provides more information and does not
require repeating the same flag in subsequent data-have messages.
Of course, if the flag is used with, say, app-message-start
message, then it applies to the entire application message
without [yet] saying which chunk got modified.
I'm still working on the frequency that you can haver more than one
application message into one transaction.
10)
"copied: A flag indicating that a copy of the attached application
data is being kept at the CLIENT. Only the CLIENT may
send this
flag. This parameter can be used with any OCP message that may
carry application message data. (XXX: it is yet unclear when
CLIENT commitment to preserve the data may end.)"
Same caveat as above.
Same answer as above. Here, the chunk-level granularity is
even more important. See the long e-mail thread about copying
commitment. Copying a data chunk is relatively easy and
painless. Commitment to copy the entire application message
is a very different animal.
Perhaps the "data" definition must be modified to stress the
level of granularity here?
Yep.
12)
"error: A flag indicating abnormal conditions at the sender that
cannot be expressed via result parameter. "
Such as? I guess maybe we could maintain the HTTP/SIP
approach where
we have status code and reason phrases for everything.
We could. Some argue that abnormal termination must be
signaled more explicitly. See messages from Oskar Batuner
related to the xaction-abort issue.
"It is RECOMMENDED that
the recipient deletes all state associated with the
corresponding
OCP message. For example, if the message is
'app-message-end' and
the application message contained user credentials, such
credentials should be deleted."
My suggestion
""It is RECOMMENDED that
the recipient deletes all state associated with the
corresponding
OCP message."
Do you mean deleting the example? Examples are not normative;
do you think the presence of an example hurts in this case?
No. I guess examples should go on sections of their own.
13)
"feature: A OCP feature identifier with optional feature parameters.
Used to declare support and negotiate use of OCP
optional features
(e.g., copying of data chunks at the CLIENT)."
I guess this is another one we need to iron out...
Yes, definitely.
14)
"If a malformed message is received, the recipient MUST terminate
processing of the corresponding OCP connection using
'connection-end'
message with an error flag"
Why not send a status code and reason phrase instead of this error
flag?
Just say "400 Bad Request"...or "400 Malformed Message"
As Oskar argued, the semantics of "400 Malformed Message" may
be different from "400 Malformed Message, and the error on
this side was so bad that you probably want to delete all
state associated with this transaction even if that state
looks valid to you". We could have a special code for the
latter, but we would need it for every error response; that
is why Oskar proposed special *-abort messages and I
suggested a special "error" flag.
I'm not going to argue too much about this. I will just say that other
text/binary based protocols have been working in a while without the need
for a error flag. This is an infinite cycle. Say, how about it is a really
nasty error, worst than "so bad"? Should we have a flag for that?
If it is bad (that is worst than 400), but not worst than "so bad"? You know
where I'm getting at?
We just need to have a different message (if needed) and document the
recommended behaviour. For example..."if you receive a messge with a error
code of 40X you should delete all state associated with this transacation"
"Raw" HTTP ignores this problem because it does not document
handling of message-independent state (e.g., authentication
information).
As I said, will not argue too much about this. IMHO I do not feel this is
really a good design.
15)
"This is the only OCP message that may carry application data. There
MUST NOT be any gaps in data supplied by data-have and data-as-is
messages (i.e., the offset of the next data message must
be equal to
the offset+size of the previous data message) (XXX: we
do not need
offset then; should we keep it as a validation mechanism?) (XXX:
document what to do when this MUST is violated). Zero size is
permitted and is useful for communicating predictions
without sending
data."
Well, although we must not have gaps, we might loose
packets...we need
to be able to recover and not throw all the OCP messages
received so
far away...
I am under impression that section 3.1 of
draft-ietf-opes-protocol-reqs-03.txt prohibits OCP packet
loss. Are you talking about application protocol packet loss?
If so, do you really want to inform the SERVER of lost data
on OCP level? Or should we assume that if application
protocol is lossy, the callout server would wither know how
to detect/handle the losses or will not care? If possible, I
would like data size and offset fields to represent received
application message octets, nothing else (because it keeps
things simple and application-agnostic).
I guess this goes back to examples I gave previously.
[ ],
Reinaldo
Thanks a lot for all your comments!
Alex.