ISCSI Extensions for RDMA


The iSCSI Extensions for RDMA is a computer network protocol that extends the Internet Small Computer System Interface protocol to use Remote Direct Memory Access. RDMA is provided by either the Transmission Control Protocol with RDMA services that uses existing Ethernet setup and therefore no need of huge hardware investment, RoCE that does not need the TCP layer and therefore provides lower latency, or InfiniBand.
It permits data to be transferred directly into and out of SCSI computer memory buffers without intermediate data copies and without much of the CPU intervention.

History

An RDMA consortium was announced on May 31, 2002, with a goal of product implementations by 2003.
The consortium released their proposal in July, 2003.
The protocol specifications were published as drafts in September 2004 in the Internet Engineering Task Force and issued as RFCs in October 2007.
The OpenIB Alliance was renamed in 2007 to be the OpenFabrics Alliance, and then released an open source software package.

Description

The motivation for iSER is to use RDMA to avoid unnecessary data copying on the target and initiator.
The Datamover Architecture defines an abstract model in which the movement of
data between iSCSI end nodes is logically separated from the rest of the iSCSI protocol; iSER
is one Datamover protocol. The interface between the iSCSI and a Datamover protocol, iSER
in this case, is called Datamover Interface.
The main difference between the standard iSCSI and iSCSI over iSER is the execution of
SCSI read/write commands. With iSER the target drives all data transfer by issuing RDMA write/read operations, respectively.
When the iSCSI layer issues an iSCSI command PDU, it calls the Send_Control primitive,
which is part of the DI. The Send_Control primitive sends the STag with the PDU. The iSER
layer in the target side notifies the target that the PDU was received with the Control_Notify
primitive. The target calls the Put_Data or Get_Data primitives to perform an RDMA write/read operation respectively. Then, the target calls the Send_Control primitive to send a response to the initiator. An example is
shown in the figures.
All iSCSI control-type PDUs contain an iSER header, which allows the
initiator to advertise the STags that were generated during buffer registration. The target
will use the STags later for RDMA read/write operations.