A distributed computing framework is open-source software that splits one large computing job, such as training or serving an AI model, across hundreds or thousands of machines. Programmers can write the job as if it ran on one computer, and the framework spreads it out. The best-known framework of this kind for AI is Ray. The company started by Ray's makers also sells it as a managed platform, which means it runs and maintains Ray on the customer's behalf.
Why is this needed? A large model does not fit on one GPU, and training it on one GPU would take years. So the work is cut into pieces. Each GPU trains on a different slice of the data, and the GPUs regularly combine what they have learned. The framework decides which machine runs which piece. It moves data between machines and keeps track of the results. If a machine fails, it starts that piece again on another machine. Without a framework, every team would have to write this coordination code itself.
In Ray, a programmer marks an ordinary Python function as a task. Ray then runs thousands of copies of that task at once, on whichever machines are free. The same framework covers the steps before and after training. It can clean and prepare the data, train the model, test many settings of the model, and later serve the trained model to users.
Take, for example, the research team of an AI start-up in Pune that is adapting an open model to answer questions in Marathi. It has rented 64 GPU servers from a cloud. The team writes its training code once. Ray starts the code on all 64 servers and gives each server a different part of 2 crore Marathi sentences. On the third day, one server's GPU fails. Ray notices within seconds and moves that server's work to a spare server. Training continues from the last checkpoint, a copy of the model saved every hour. The team loses one hour of work instead of three days.
The training data usually comes from a data lakehouse. The files are read through a parallel file system, so that thousands of GPUs can read at once. The GPUs sit in an AI cluster and are programmed through the CUDA and GPU software stack. When training ends, the model goes to a model deployment platform, which serves it.
Ray is free to download. The company behind it earns by running it as a service, which is the open source business model. How is Ray different from Kubernetes? Kubernetes starts and stops containers on machines. Ray runs inside those containers, and splits one program's work among them.