On 6/17/02, Timur Shemsedinov wrote:
SC> Remote Direct Data Placement (rddp)
SC>-----------------------------------
SC> 1) A (transportindependent) protocol core to support
SC> direct data placement from the NIC into specified
SC> memory, usually application buffers.
SC> 2) A (transportindependent) protocol core layered on
SC> top of the direct data placement protocol that
SC> specifies control of RDMA.
SC> 3) A mapping of the direct data placement protocol
SC> and the RDMA control protocol to SCTP.
I have some questions.
Whether it is supposed what server module will be inside
the application (or in separate service)? This question
has arisen because processes in the majority of
operational systems have different address spaces.
Direct Data Placement protocols typically support anonymous
target buffers (essentially the same semantics as SCTP) as
well as tagged buffers.
Each placement specifies a tag and an address to be
interpreted in the context of that tag. How tags are
obtained is a local interface issue. How they are
distributed is a ULP issue.
However, the typical local interface is that the application
registers a portion of its own address space, and thereby
obtains tags that it can distribute to its peers.
You can find examples of such interfaces with InfiniBand
(http://www.infinibandta.org), the Virtual Interface
Archtiecture (http://www.vidf.org) and the Direct Access
Transport API (http://www.datcollaborative.org).
InfiniBand is an excellent example of a specific solution.
The DAT specifications present a technology neutral
definition of Direct Data Placement capable transports.
So, application should use mechanisms of memory sharing
and cross-application interaction or RDMA module should be
included into each application.
Typically, the local interface is indeed a user mode
library. However, it is local interface, so it can be
implemented many ways and not affect interoperability.
The reason it will tend to be a user mode library is to
avoid unnecessary user/kernel mode transitions. Typically
kernel interactions will be done when memory is registered
and endpoints are created. The goal is to leave the kernel
out of the loop on a per data transfer basis to the greatest
extent possible. Current local bus architectures,
particularly how interrupts work, limit the degree to which
that can be done. However things can still be minimized, and
most features of the local interfaces are intended to keep
user/kernel interactions as minimal as possible without
creating security holes.
And how to understand "transport independent" in this case?
The meaning of a sucessfully delivered DDP or RDMA message
is not dependent upon the underlying IP transport protocol
that was used to deliver it.
The goal is to define protocols that run over ubiquitous
transports, not to define a new transport. Applications
should not be restricted to SCTP, but free to choose other
IP transports that are suitable to their needs. One example
would be use of DCP for applications (such as streaming
media) that prefer unreliable delivery because real-time
delivery requirements severely limit the time window for
retries.
This is similar to many protocols above layer 4 that run
over multiple transports, including RPC and SNMP.
Caitlin Bestler
Asomi Network Technologies