At 05:30 25/02/03, Alex Rousskov wrote:
On Wed, 26 Feb 2003, jfcm wrote:
> No. The question was about parallel command pipe or not. My response
> was to try to see if we can have a consistent response. Obviously
> triggering a command should probably go the same way as aborting it.
If I understand the above, that is what the current predraft proposal
does. Triggering transaction processing on the other end uses the same
mechanism as aborting that processing (an OPES message with the
corresponding transaction ID xid).
Yes. And this is what I underlined in asking that command and
abort be on the same pipe.
> The idea of a piped protocol being accepted, you must zap the pipe
> if you want to abort. This means a priority somewhere in the
> processus. Either in the pipe with a zapper swallowing the data. Or
> a prioritized command pipe ordering to clear the four buffers.
Priority processing is not required to terminate a transaction.
Priority processing is an optimization -- it allows to terminate
transaction on the other side _faster_. How much faster? Depends on
how exactly one organizes priority queues and on how long those queues
are at the abortion time.
If you recall, the predraft protocol allows for out-of-order (i.e.,
prioritized) commands but does not document how the transport
connections are handled (yet). If a OPES processor opens more
connection to the callout server than there are concurrent application
messages, then a <xaction-end> message can be sent using "spare"
[control] connections, possibly triggering a faster abort.
This is the issue. You can either think of a spare connection
or of a command connection. When you establish a relation
between servers you may decide that conenction 0 will be
the command pipe. I do prefer that approach as this is a
good pipe to signal the sever is still up and running. Also, in
my prefered wording, I would say that connection 0 would
go to the service administrator.
I do not know what you mean by "zapper swallowing the data" in the
pipe. Can you clarify, please? Since we are talking about TCP-like
connections here, it may not be possible to "swallow" anything that
was sent. The only reliable option is to ignore messages with unknown
(e.g., already aborted) xid and/or terminate the corresponding
transport connection. Perhaps that's what you meant by swallowing?
Yes and more.
Even under TCP/IP you transport data in blocks/messages
(IMHO due to TCP/IP there could be interest in multiplexed
datagrams to go faster and get easier acks). This means that
the non yet used data are in buffers and will be then processed.
When I say swallowing I mean that the buffers are cleaned (and
that the current processes are signaled to die). I cannot signal
processes but I can clean the I/O buffers (and on the path buffers
as the processes can be multi/machines).
Please recall that some processes can be quite CPU
consuming. Just ignoring the next protocol message
may result in a lot of unecessary finishing processing.
> Then there is the message telling the server to abort and and ack to
> the requester.
An ACK would be a waste of resources unless you want to support OPES
transaction resumption after the event of transport-layer connectivity
loss. By "resumption", I mean resuming at the place of last
synchronization, transparently to the consumer and producer.
This is a good idea (this puts a process on hold or to recover
from a bad situation). No, I meant that the interrupted service
acks that he died. Either when dying or through a service admin
obituary message.
The "waste of resources" comes from the trade-off. The whole
internet vision is based upon the TCP/IP trade-off you quote.
The acks do exist but they are implicit and come through
layer violations (the response of the application shows that
it received the data).
If you have a TCP/IP pipe you can use it as tansport for an
higher level block transport protocol and design a extremely
fast, reliable and secure value added virtual circuit switch
solution. Please remember that we are in very well
defined closed network. Please review my corrected draft.
I kept dispatcher/services/ends blocks as equal
because the command pipe issue has not been decided.
But, let assume that the ONES system is built only
of dispatchers and ends (and may be the ends also
with a dispatcher). This means that the "dispatcher"
is acting as a real time agent (like in X.400/e-mail but
real time).
The architecture for a virtual server would then be:
----> dispatcher -----> service admin
|----> service A occurence 1
|----> service A occurence 2
|----> service B occurence 1
|----> service B occurence 2
|----> service B occurence 3
| etc.
And that would be very near from the architecture I
introduced in my first mail about my prototype.
The inter-dispatcher protocol could be one of the
services and support soap, etc. under straight
TCP/IP or your OCP under a TCP/IP+ multiplexed
datagram solution with a behaviour and a response
time much closer of a message passing OS.
This could provide a very broad set of consistent
solutions?
I doubt the benefits of transaction resumption outweigh increased
protocol complexity. It seems to me that for application protocols we
know about (HTTP and SMTP), an application transaction abort or an
automated retry would be more appropriate solutions. Do you think OPES
transactions should be resumable if the transport layer fails?
Let be clear. I discuss here what I name ONES of which
I think OPES is a fist level. I certainly do not think that
HTTP simple systems to replace a string in a page to
correct an URL or to change a .jpg into a .gif needs that.
But I think that an OPES may come to a point they have
to behave as a ONES because of the load or of the speed.
My site does not need much, but Amazone yes (we have
not discussed services running in parallel and the data
to keep from page to page to support a User cession?).
Then SOAP is useless. In talking about ONES, we know
which environement we might want to stay consistent with.
Also I do not see here a clear boarder defined yet.
IAB says HTTP. You add SMTP as I did. IAB says one
way, yet there seem to be some real support for both
ways. IAB does not say redirect and we are divided on
this.
Also, we may find that a more global vision may simlify
and make more robust the OPES. Or extend the field
of their immediate applictations.
Example. I wish to have a new vision of the DNS. I
send calls/mails using domain names and a 0.0.0.0
IP address that an OPES will replace with the proper
current IP of the DN savnig a lot of time: no more
first erroneous resolution and further redirect. And
a lot of flexibility in addressing.
> The real issue is that at minimum there are four flows involved
> user/dispatcher/service/dispatcher/organizer in my own words instead
> of one. Following all the possiblities calls for an Excell
> spreadshit.
In such cases, a higher level simplification or abstraction often
works better than enumerating endless possibilities in a table (IMO).
True. See above. But to make sure about it you must be
very basic first and see if the proposed abstraction resits
to the list of possible cases/conflicts. We fully agree.
The proposed OCP protocol is such an abstraction (at this point in
time) because it does not really care what the consumer, producer,
source, or destinations are.
True. This is why you should not embarass it with client/server
issues if it is bidi. And let consider seriously an inter dispatcher
network only. The disptacher as a smart plug everywhere,
including on the users side (what is a firewall?) where the minimum
access to a service is only a single (yes/no) rule (what is a
power switcher?).
The number of flows is not important as
long as all flows are mostly independent and there is a well-known
"forking" or "initial scheduling" point (the OPES processor in our
case).
True. See above. But I prefer the word "dispatcher" from the dratf.
Because it is more generic and services are processed outside
of the dispatcher.
The proposed protocol handles at least two flows per
application transaction and should scale well with the number of
flows.
Yes. Except that these two flows can be encapsulated
into a value added transport protocol for speed and
flow management. That way you may have cheap
and even virtual acks or service messages.
Please remember that we are real time. We cannot make
users wait because another user has big data to process.
We must permit to optimise dispatcher and services CPU
time. This certainly means priority assignement in transmitting
data under the protocol. Maximum flexibility can only come
from a service admin talking with the remote dispatcher. The
dispatcher must then act both as a link and as transport
dispatcher.
jfc