Chip interconnect means the chips and circuits that join one processor to the next, and to the network switches, so that many processors can work as one machine. These links must be fast, because thousands of GPUs that train one AI model must exchange results all the time. Whenever the links are slow, the GPUs sit idle, waiting.
Two measures matter, bandwidth and latency. Bandwidth is how much data a link carries each second. Latency is the delay before the first data arrives.
At the edge of each processor sits a circuit called a SerDes, short for serialiser-deserialiser. Inside a chip, data moves on many parallel wires at a modest speed. The SerDes takes these signals and sends them as one very fast signal on a single pair of wires. At the other end, another SerDes turns it back into parallel signals. Today's fastest SerDes lanes carry about 100 to 200 gigabits per second each, and a GPU uses many lanes side by side. Inside a server, GPUs are joined directly to each other by links of this kind, along copper wires a few centimetres long. Between servers, the links run through switches built on networking chips, often over fibre. Connectivity standards such as PCIe and Ethernet define the signals, so that chips from different makers can work together.
Here is a typical case. A research team in Hyderabad trains a language model on 1,024 GPUs in a data centre in India. The model has 7 billion weights. In each training step, every GPU works on its own batch of examples and works out how each weight should change. Then all 1,024 GPUs must add up their changes and share the total, so that every GPU holds the same model. This exchange is called all-reduce. Each GPU's changes take about 14 GB, at 2 bytes for each weight. If each GPU can send data at 400 gigabits per second, which is 50 GB per second, the exchange takes about half a second. If the links were half as fast, it would take about a second. Over a training run of 1,00,000 steps, that extra half-second in each step adds about 15 hours, during which all 1,024 GPUs wait.
So the people who build AI cluster networking try to make every link as fast as possible. The chips that do this come from a few companies, some of which also design custom AI chips for cloud companies, because a custom chip needs the same links. Between servers, the links reach data centre switches and routers.
The links are also reaching a limit. Fast electrical signals weaken quickly along copper, so each extra gain in speed shortens the distance a copper link can reach and raises the power it needs. The next step is to carry the signals as light, on fibre, from right beside the chip. This is called co-packaged optics. Much of the design of these links is done in chip design centres in Bengaluru and Hyderabad.