GPU cloud is the renting of servers with GPUs, by the hour, to train and run AI models. It is also called GPU as a service. A GPU, or graphics processing unit, is a chip with thousands of small cores that do the same calculation at the same time. It was built to draw graphics in games. Today it does most of the maths behind AI, as the page on AI accelerators explains.
Why rent a GPU and not buy one? A single data-centre GPU of the latest kind costs about ₹25 lakh or more. It also needs a server, power and cooling around it. A small team uses it only part of the time. Between experiments, a GPU that the team owns sits idle. A rented one is simply switched off, and the bill stops.
There are two kinds of AI work, explained on the page on training and inference. Training teaches a model by showing it large amounts of data, and can keep GPUs busy for days. Inference is running the finished model to answer each request, such as turning one voice note into text.
Imagine a six-person AI start-up in Hyderabad that builds speech recognition for Telugu. Speech recognition is software that turns spoken words into text. The team has 2,000 hours of recorded Telugu speech. It wants to fine-tune an open model on these recordings. Fine-tuning means taking a model that someone else has already trained, and training it further on your own data.
On a developer cloud, an engineer chooses a server with one high-end GPU, at about ₹250 an hour. The server comes with the GPU drivers and AI libraries already installed. She copies the recordings to the server's disk and starts the training. The run takes 40 hours, so it costs about ₹10,000. When it ends, she saves the new model to storage and deletes the server. For the app's daily use, she rents a smaller GPU at about ₹60 an hour, which turns users' voice notes into text.
Why have GPUs been scarce? AI companies around the world want the same chips, and the chip makers cannot make enough of them. So whoever owns a room full of GPUs can sell GPU time. New providers called neoclouds rent out only GPU servers. The software they use to share GPUs among customers, and to measure each customer's use for the bill, is described on the page on GPU-as-a-service platforms. Governments that want this computing inside their own country build a sovereign AI cloud. The very largest training runs use AI clusters, thousands of GPUs wired to work as one machine.
Rented GPUs serve other work too. A visual-effects studio may rent extra GPUs to turn its 3D scenes into finished frames in the weeks before a film's release, through its VFX pipeline.