A model is a file. A lot has to happen before that file becomes a product:
- the cluster of machines it is trained on
- the servers it answers requests from
- the data it learns from
- the tests that say whether it is any good
- the gateway that decides which model each request goes to
- the memory that gives it context
All of that is the business of this Market Segment. In India, it has more job postings than Market Segment 9.1, Foundation model companies.
A model is a file; the cluster it is trained on and the servers it answers from, the gateway and the memory, and the data it learns from and the tests that judge it are what turn it into a product, and they are the three sub-segments here.
Picture a Bengaluru start-up building a Hindi customer-support assistant for banks, on top of a language model. It has to train or adapt the model, and run it for thousands of users. It has to connect the model to each bank's systems and documents, and prove that its answers are right and safe. It buys each of those pieces from the companies in this Market Segment.
Every company named here has posted software engineering jobs in India. Famous companies that don't actively hire software engineers in India are left out.
This Market Segment follows the model from the lab to real use, in three sub-segments:
Training and serving: the platforms that run the cluster the model is trained on, and the servers it answers from. Answering requests is called serving.
The gateway, the agent and the retrieval: the layer through which an app uses a model, and gives it tools and memory.
The data and the judge: the labelled examples a model learns from, and the evaluation that decides whether it can be trusted.
It ends with a link to the chips built for models.
Training and serving
Distributed compute frameworks, MLOps and LLMOps platforms, automated machine learning, AI compute clouds, edge AI, quantum-and-AI platforms
Back to the start-up from the start of this Market Segment. Its model has to be trained across many machines, usually on GPUs, the graphics chips that AI runs on. Then it has to be served to thousands of users at once, and watched once it is live. The software for deploying and running models in production is called MLOps, or LLMOps for language models. The companies in this sub-segment run the model.
Anyscalehas engineers in Bengaluru, and the second most job postings in this Market Segment. It is the company behind Ray, the open-source framework on which many of the largest models are trained and served across thousands of machines. It sells Ray as a managed platform.
TrueFoundryfounded in Bengaluru, has engineers there. It sells the platform on which a company deploys, scales and monitors its models, and now its agents, on its own cloud. It is the largest Indian-founded company in this sub-segment.
H2O.aiwith engineers in Bengaluru, is the older name. It sells automated machine learning, which builds a model from a table of data without a data scientist. It has rebuilt that product around language models.
RapidCanvaswith engineers in Mumbai, sells the same automation for business users, with AI agents on top.
SandLogican Indian company in Bengaluru, builds models and the software to run them at the edge: on a device, rather than in a data centre.
Four companies stand for the computing power itself:
G42the Abu Dhabi AI group, builds sovereign AI clouds and models: ones built for, and controlled by, a single country. It has engineers in Bengaluru.
QpiAIin Bengaluru, and Qubrid AI, in Kolkata, both sell platforms that put quantum computing and AI on the same cloud. Quantum computers use quantum physics to solve certain problems. These two are the one Indian corner of this Market Segment where the hardware is not a GPU.
One company covered mainly in another Market Segment belongs here too:
hosted-aisells the control software that turns a room full of GPUs into a cloud that a provider can sell. (More on it in Market Segment 5.1, Cloud infrastructure and web hosting providers, in Industry Vertical 5, Cloud providers and OS makers.)
The gateway, the agent and the retrieval
LLM gateways, agent frameworks, retrieval-augmented generation, domain model APIs
Back to the start-up from the start of this Market Segment. Its assistant has to reach the model, look up a customer's account, and read the bank's policy documents. The companies in this sub-segment sell the layer between the app and the model:
The gatewayone API in front of many model providers, which sends each request to the right model.
The agent frameworksoftware for building agents, AI models given tools, memory and a goal.
Retrievallooking up the right passages in a company's documents, and handing them to the model with each question. This is called retrieval-augmented generation, or RAG.
Between the app and the model sit an agent framework that gives the model tools, memory and a goal, retrieval that hands it the right passages from the bank's documents, and a gateway that routes each request to a model provider or a domain model.
Concentrate AIa Bengaluru start-up, sells the gateway. It routes each request, switches to another provider when one is down, and tracks what the company spends.
Lyzr AIwith engineers in Bengaluru, sells a framework for building agents. It is for large companies that want their agents to run inside their own systems.
PipesHuba three-person Bengaluru company, sells the open-source version of the same. It includes the retrieval pipeline and the connectors through which an agent reads a company's documents.
Nugenwith engineers in Mumbai, sells domain-specific models through an API: models trained for one field, such as finance or law. It is a reminder that the model an app calls is more and more often a specialist.
One company covered mainly in another Market Segment belongs here too:
Wand AIis here for the infrastructure its agents run on. (More on it in Market Segment 10.5, Enterprise AI agent platforms, in Industry Vertical 10, ERP and business automation.)
The agent as the unit of software.
The companies in this sub-segment share one belief: the typical app of the next few years will be an agent. An agent needs a gateway to choose the model, a framework to give it tools, and a retrieval layer to give it memory. That is the plumbing this sub-segment sells. The memory is usually kept in a vector database, which stores text as lists of numbers so a model can find passages by meaning. It is this category's best-known new product, but no vector-database company is covered in this Market Segment.
You're reading as a guest. Sign in free to follow links for five minutes, once an hour.
Sign in
The data and the judge
Data labelling and annotation, model evaluation and observability, AI testing, interpretability and alignment
Back to the start-up from the start of this Market Segment. Its model has to learn what a good answer to a bank customer looks like. Then someone has to check that its answers are right and safe. The companies in this sub-segment decide what a model learns from, and whether it can be trusted.
The data. A model learns from labelled examples: data that people have marked up, such as a question paired with the correct answer. Making these is called data labelling, or annotation.
Surge AIwith job postings in Delhi, Mumbai and Chennai, is the labelling company whose expert annotators produce the examples and the human feedback the largest models are trained on.
DATAmundiin Bengaluru, is the Indian annotation company doing the same for speech and language data.
Large Indian data-services firms also do labelling as a service, at scale. (More on them in the services-world collection.)
The judge. The companies that check models are mostly Indian-founded. Their work is called evaluation: measuring whether an answer was right, safe and on topic. Watching a model in production, and tracing why an answer went wrong, is called observability.
Galileowith engineers in Bengaluru, and Fiddler AI, also in Bengaluru, sell evaluation and observability for models in production. (A security team uses the same tools. More in Market Segment 4.13, AI security.)
Maxim AIsells evaluation and simulation of agents, before and after they ship.
RagaAIsells testing for models and the retrieval pipelines behind them.
Lexsi Labshas the most job postings in this Market Segment, in Mumbai, from a team of just 19 people. It is the only company here with more job postings than staff. It is the research arm of the Indian company Arya.ai. It builds tools for interpretability, alignment and compression. Interpretability means seeing inside a model. Alignment means making it behave as intended. Compression means making it smaller, so it runs cheaper.
A model learns from labelled examples, such as a question paired with its correct answer; evaluation then judges it from outside by asking questions and scoring the answers, while interpretability, the frontier, reads what it has learned from the inside.
The model examined from the inside.
Evaluation from outside, asking questions and scoring the answers, is the established business in this sub-segment. Lexsi Labs stands for the frontier: interpretability, reading what a model has learned from its insides. It is the research problem the whole field is now hiring for.
The chips built for models
Connector to Market Segment 35.3, AI chip start-ups
The frameworks in the first sub-segment exist to keep AI chips busy. Most models train on NVIDIA's GPUs. (More on NVIDIA in Market Segment 35.1, Processor and GPU makers, in Industry Vertical 35, Semiconductors and chip design.) Newer companies design chips built only for training and running models: Cerebras, SambaNova, d-Matrix, Tenstorrent and Graphcore. Several of them have large design centres in Bengaluru. (More on them in Market Segment 35.3, AI chip start-ups, in Industry Vertical 35, Semiconductors and chip design.)
So the start-up from the start of this Market Segment needs plumbing at every step. It needs a cluster to train on and servers to answer from. It needs a gateway, an agent framework and memory to connect to each bank. And it needs labelled data to learn from, and a judge to prove it can be trusted.