In computing, Array of Structures , Structure of Arrays and Array of Structures of Arrays refer to contrasting ways to arrange a sequence of records in memory, with regard to interleaving, and are of interest in SIMD and SIMT programming.
Structure of Arrays
Structure of arrays is a layout separating elements of a record into one parallel array per field. The motivation is easier manipulation with packed SIMD instructions in most instruction set architectures, since a single SIMD register can load homogeneous data, possibly transferred by a wide internal datapath. If only a specific part of the record is needed, only those parts need to be iterated over, allowing more data to fit onto a single cache line. The downside is requiring more cache ways when traversing data, and inefficient indexed addressing. For example, to store N points in 3D space using a Structure of Arrays: struct pointlist3D ; struct pointlist3D points; float get_point_x
Array of Structures
Array of structures is the opposite layout, in which data for different fields is interleaved. This is often more intuitive, and supported directly by most programming languages. For example, to store N points in 3D space using an Array of Structures: struct point3D ; struct point3D points; float get_point_x
Array of Structures of Arrays
Array of Structures of Arrays is a hybrid approach between the previous layouts, in which data for different fields is interleaved using tiles or blocks with size equal to the SIMD vector size. This is often less intuitive, but can achieve the memory throughput of the SoA approach, while being more friendly to the cache locality and load port architectures of modern processors. For example, to store N points in 3D space using an Array of Structures of Arrays with a SIMD register width of 8: struct point3Dx8 ; struct point3Dx8 points; float get_point_x
Alternatives
It is possible to split some subset of a structure into a parallel array, and this can actually improve locality of reference if different pieces of fields are used at different times in the program. Some SIMD architectures provide strided load/store instructions to load homogeneous data from the SoA format. Yet another option used in some Cell libraries is to de-interleave data from the AoS format when loading sources into registers, and interleave when writing out results. Some vector maths libraries align floating point 4D vectors with the SIMD register to leverage the associated data path and instructions, while still providing programmer convenience, although this does not scale to SIMD units wider than four lanes.
4D vectors
AoS vs. SoA presents a choice when considering 3D or 4D vector data on machines with four-lane SIMD hardware. SIMD ISAs are usually designed for homogeneous data, however some provide a dot product instruction and additional permutes, making the AoS case easier to handle. Although most GPU hardware has moved away from 4D instructions to scalar SIMT pipelines, modern compute kernels using SoA can still give better performance due to memory coalescing.
