AI clusters and rack-scale systems are the machines built for training and running AI. An AI cluster is a large group of GPU servers that are wired, cooled and networked so that they work on one job together, such as training an AI model. A rack-scale system is one rack of such servers, sold and delivered as a single finished unit. Both are built around AI accelerators, the GPUs and other chips that train and run AI models.
Why can a large model not be trained on one server? Training a large model means doing an enormous amount of arithmetic on a very large amount of data. One server with 8 GPUs might take years. So the work is divided among hundreds or thousands of GPUs. After each small step, every GPU must share its results with all the others before the next step can begin. If one link between them is slow, every GPU waits. So a cluster is designed as a whole: the servers, the network between them, the storage and the cooling.
Take, for example, an AI company in Chennai that is training a speech recognition model for Indian languages. It uses 512 GPUs in 64 servers, with 8 GPUs in each server. AI cluster networking gives each GPU its own connection of 400 gigabits per second. A parallel file system supplies thousands of hours of recorded speech to all the GPUs at once. Training runs day and night for three weeks. Every 30 minutes, the cluster saves a copy of the model's progress, called a checkpoint. When one GPU fails on the ninth day, the job restarts from the last checkpoint. It loses at most 30 minutes of work, instead of nine days.
AI racks use far more power than ordinary racks. A rack-scale system with 72 GPUs can draw more than 100 kW, so it needs direct-to-chip liquid cooling from the data centre's power and cooling equipment. Such racks arrive from the factory already wired, cooled and networked, ready to switch on. Some companies also build clusters to order from separate parts, and manage them after they are installed.
Not every company that trains models owns a cluster. Many rent GPUs by the hour in a GPU cloud. In India, the government's IndiaAI Mission pays part of the cost of GPUs that start-ups and researchers rent from Indian providers. Governments that want AI models trained and run inside their own country buy clusters for a sovereign AI cloud.