Generative AI applications are apps built on top of a large language model or another AI model, which write text, answer questions or take actions for their users. A generative AI model creates new content, such as a reply or a summary, instead of only sorting or scoring what it is given. A model on its own is just a large file of numbers. A lot of other software has to be added before that file becomes a product that people can use.
Imagine a start-up in Bengaluru that wants to sell a Hindi customer-support assistant to banks. A bank customer in Varanasi will type, 'Mera debit card block ho gaya, kya karun?' and expect a correct answer in Hindi within seconds. To get there, the start-up has to choose or adapt a model and run it for thousands of users at once. It has to connect the model to each bank's documents and systems. And it has to prove that the answers are right and safe. It buys most of these pieces from other companies, and they fall into five layers.
The first layer is the AI models themselves. A large language model (LLM) reads and writes text. Speech and voice AI turns speech into text and text into speech, and a robotics foundation model controls machines. The largest models are foundation models, which other companies build their own products on.
The second layer is model training and inference. Training teaches the model from data, on a cluster of thousands of GPU servers. Inference, also called serving, means running the trained model to answer each user's request. The page on training vs inference explains why the two jobs need different machines. A model deployment platform keeps the start-up's model answering reliably.
The third layer is AI agents and RAG, the layer through which an app uses a model. An LLM gateway sends each request to the right model provider. Retrieval-augmented generation (RAG) finds the right passages in the bank's own documents, such as its card rules, and hands them to the model with each question. An AI agent framework gives the model tools, so that it can, for example, block a card in the bank's system.
The fourth layer is AI evaluation and data labelling. Data labelling companies write the examples a model learns from. LLM evaluation tools test whether its answers are correct, before release and after.
The fifth layer is AI security. An assistant that can read account details and act in a bank's systems can also be tricked. For instance, a user may type instructions that try to make it ignore its rules, an attack called prompt injection. LLM guardrails check what goes into the model and what comes out. AI agent identity and security gives an agent its own login and only the permissions it needs, the rule called least privilege.
Most of the models the world uses are made by a few large companies outside India. In India, the companies that build the other four layers post more engineering jobs than the model makers do. India is also training its own models on Indian languages, for reasons the page on sovereign AI explains.