NVLink


NVLink is a wire-based communications protocol serial multi-lane near-range communication link developed by Nvidia.
Unlike PCI Express, a device can consist of multiple NVLinks, and devices use mesh networking to communicate instead of a central Hub. The protocol was first announced in March 2014 and uses a proprietary High-Speed Signaling interconnect.

Principle

NVLink is a wire-based communications protocol for near-range semiconductor communications developed by Nvidia that can be used for data and control code transfers in processor systems between CPUs and GPUs and solely between GPUs. NVLink specifies a point-to-point connection with data rates of 20 and 25 Gbit/s per differential pair. Eight differential pairs form a "sub-link" and two "sub-links", one for each direction, form a "link". The total data rate for a sub link is 25 Gbit/s and the total data rate for a link is 50 Gbit/s. Each V100 GPU supports up to six links. Thus, each GPU is capable of supporting up to 300 Gbit/s in total bi-directional bandwidth. NVLink products introduced to date focus on the high-performance application space. Announced May 14, 2020, NVLink 3.0 increases the data rate per differential pair from 25 Gbit/s to 50 Gbit/s while halving the number of pairs per NVLink from 8 to 4. With 12 links for an Ampere-based A100 GPU the brings the total bandwidth to 600 Gbit/sec.

Performance

The following table shows a basic metrics comparison based upon standard specifications:
InterconnectTransfer
Rate		Line CodeEff. Payload Rate
per Lane
per Direction		Max total
Lane Length
PCIe 1.x2.5 GT/s8b/10b~0.25 GB/s20" = ~51 cm
PCIe 2.x5 GT/s8b/10b~0.5 GB/s20" = ~51 cm
PCIe 3.x8 GT/s128b/130b~1 GB/s20" = ~51 cm
PCIe 4.016 GT/s128b/130b~2 GB/s8−12" = ~20−30 cm
PCIe 5.032 GT/s128b/130b~4 GB/s
NVLink 1.020 Gbit/s~2.5 GB/s
NVLink 2.025 Gbit/s~3.125 GB/s
NVLink 3.050 Gbit/s~6.25 GB/s

The following table shows a comparison of relevant bus parameters for real world semiconductors that all offer NVLink as one of their options:
SemiconductorBoard/Bus
delivery variant		InterconnectTransmission
Technology
Rate 		Lanes per
Sub-Link
Sub-Link Data Rate
Sub-Link
or Unit
Count		Total Data Rate
Total
Lanes
Total
Data Rate
Nvidia GP100P100 SXM,
P100 PCI-E		PCIe 3.08 GT/s16 + 16 Ⓑ128 Gbit/s = 16 GByte/s116 + 16 GByte/s32 Ⓒ32 GByte/s
Nvidia GV100V100 SXM2,
V100 PCI-E		PCIe 3.08 GT/s16 + 16 Ⓑ128 Gbit/s = 16 GByte/s116 + 16 GByte/s32 Ⓒ32 GByte/s
Nvidia TU104GeForce RTX 2080,
Quadro RTX 5000		PCIe 3.08 GT/s16 + 16 Ⓑ128 Gbit/s = 16 GByte/s116 + 16 GByte/s32 Ⓒ32 GByte/s
Nvidia TU102GeForce RTX 2080 Ti,
Quadro RTX 6000/8000		PCIe 3.08 GT/s16 + 16 Ⓑ128 Gbit/s = 16 GByte/s116 + 16 GByte/s32 Ⓒ32 GByte/s
Nvidia XavierPCIe 4.0 Ⓓ
2 units: x8
1 unit: x4
3 units: x1		16 GT/s
8 + 8 Ⓑ
4 + 4 Ⓑ
1 + 1
128 Gbit/s = 16 GByte/s
64 Gbit/s = 8 GByte/s
16 Gbit/s = 2 GByte/s		Ⓓ
2
1
3		Ⓓ
32 + 32 GByte/s
8 + 8 GByte/s
6 + 6 GByte/s		40 Ⓑ80 GByte/s
IBM Power9PCIe 4.016 GT/s16 + 16 Ⓑ256 Gbit/s = 32 GByte/s396 + 96 GByte/s96192 GByte/s
Nvidia GA100Ampere A100PCIe 4.016 GT/s16 + 16 Ⓑ256 Gbit/s = 32 GByte/s132 + 32 GByte/s32 Ⓒ64 GByte/s
Nvidia GP100P100 SXM,
NVLink 1.020 GT/s8 + 8 Ⓐ160 Gbit/s = 20 GByte/s480 + 80 GByte/s64160 GByte/s
Nvidia XavierNVLink 1.020 GT/s8 + 8 Ⓐ160 Gbit/s = 20 GByte/s
IBM Power8+NVLink 1.020 GT/s8 + 8 Ⓐ160 Gbit/s = 20 GByte/s480 + 80 GByte/s64160 GByte/s
Nvidia GV100V100 SXM2
NVLink 2.025 GT/s8 + 8 Ⓐ200 Gbit/s = 25 GByte/s6150 + 150 GByte/s96300 GByte/s
IBM Power9NVLink 2.0
25 GT/s8 + 8 Ⓐ200 Gbit/s = 25 GByte/s6150 + 150 GByte/s96300 GByte/s
NVSwitch
NVLink 2.025 GT/s8 + 8 Ⓐ200 Gbit/s = 25 GByte/s2 * 8 + 2
= 18		450 + 450 GByte/s288900 GByte/s
Nvidia TU104GeForce RTX 2080,
Quadro RTX 5000		NVLink 2.025 GT/s8 + 8 Ⓐ200 Gbit/s = 25 GByte/s125 + 25 GByte/s1650 GByte/s
Nvidia TU102GeForce RTX 2080 Ti,
Quadro RTX 6000/8000		NVLink 2.025 GT/s8 + 8 Ⓐ200 Gbit/s = 25 GByte/s250 + 50 GByte/s32100 GByte/s
Nvidia GA100Ampere A100NVLink 3.050 GT/s8 + 8 Ⓐ400 Gbit/s = 50 GByte/s6300 + 300 GByte/s96600 GByte/s

Note: Data Rate columns were rounded by being approximated by transmission rate, see real world performance paragraph
Real world performance could be determined by applying different encapsulation taxes as well usage rate. Those come from various sources:
Those physical limitations usually reduce the data rate to between 90 and 95% of the transfer rate.
NVLink benchmarks show an achievable transfer rate of about 35.3 Gbit/s for a 40 Gbit/s NVLink connection towards a P100 GPU in a system that is driven by a set of IBM Power8 CPUs.

Usage with Plug-In Boards

For the various versions of plug-in boards that are exposing extra connectors for joining them into a NVLink group a similar amount of slightly varying, relatively compact, PCB based interconnection plugs does exist. Typically only boards of same type will mate together due to their physical and logical design. For some setups two identical plugs need to be applied for achieving the full data rate. As of now the typical plug is U-shaped with a fine grid edge connector on each of the end strokes of the shape facing away from the viewer. The wide of the plugs determines how far away the plug-in cards need to be seated to the main board of the hosting computer system - a distance of for the placement of the card is commonly determined by the matching plug. The interconnect is often referred as SLI from 2004 for its structural design and appearance even if the modern NVLink based design is of a quite different technical nature with different features in its basic levels compared to the former design. Reported real world devices are:
By means of the NVML-API offers for the Tesla, Quadro and Grid line of products NVidia a set of functions for programmatically controlling some aspects of NVLink interconnects on Windows and Linux systems, such as component evaluation and versions along with status/error querying and performance monitoring. Further with the provision of the NCCL library developers in the public space shall be enabled for realizing e.g. powerful implementations for artificial intelligence and similar computation hungry topics atop NVLink. The page "3D Settings" => "Configure SLI, Surround, PhysX" from the NVidia Control panel and the CUDA sample application "simpleP2P" are supposed using such APIs as mentioned upfront for realizing their services in respect to their NVLink features. At least on the Linux platform the command line application with a certain sub-command "nvidia-smi nvlink" provides a similar set of advanced information and control.

History

On 5 April 2016, Nvidia announced that NVLink would be implemented in the Pascal-microarchitecture-based GP100 GPU, as used in, for example, Nvidia Tesla P100 products. With the introduction of the DGX-1 high performance computer base it was possible to have up to eight P100 modules in a single rack system connected to up to two host CPUs. The carrier board allows for a dedicated board for routing the NVLink connections – each P100 requires 800 pins, 400 for PCIe + power, and another 400 for the NVLinks, adding up to nearly 1600 board traces for NVLinks alone. Each CPU has direct connection to 4 units of P100 via PCIe and each P100 has one NVLink each to the 3 other P100s in the same CPU group plus one more NVLink to one P100 in the other CPU group. Each NVLink offers a bidirectional 20 GB/sec up 20 GB/sec down, with 4 links per GP100 GPU, for an aggregate bandwidth of 80 GB/sec up and another 80 GB/sec down. NVLink supports routing so that in the DGX-1 design for every P100 a total of 4 of the other 7 P100s are directly reachable and the remaining 3 are reachable with only one hop. According to depictions in Nvidia's blog based publications from 2014 NVLink allows bundling of individual links for increased point to point performance so that for example a design with two P100s and all links established between the two units would allow the full NVLink bandwidth of 80 GB/s between them.
At GTC2017, Nvidia presented its Volta generation of GPUs and indicated the integration of a revised version 2.0 of NVLink that would allow total i/o data rates of 300 GB/s for a single chip for this design, and further announced the option for pre-orders with a delivery promise for Q3/2017 of the DGX-1 and DGX-Station high performance computers that will be equipped with GPU modules of type V100 and have NVLink 2.0 realized in either a networked of or a fully interconnected fashion of one group of four V100 modules.
In 2017-2018, IBM and Nvidia delivered two supercomputers for the US Department of Energy named "Summit" and "Sierra", which combine IBM's POWER9 family of CPUs and Nvidia's Volta architecture, using NVLink 2.0 for the CPU-GPU and GPU-GPU interconnects and InfiniBand EDR for the system interconnects.