Every answer from an AI chatbot is worked out on chips in a data centre. Today, most of those chips are NVIDIA's graphics processors, or GPUs. They are expensive, and for years there have not been enough of them. A wave of start-ups is betting that chips designed for AI from the start can do the job better or more cheaply.
Two jobs for an AI chip
Trainingteaches a model, once, using a huge amount of data. It needs raw computing power, and thousands of chips linked together.
Inferenceis running the trained model to answer each request. It happens billions of times a day. Here, what matters most is the speed of each answer and the energy it uses.
Two jobs for an AI chip: training teaches a model once, with huge amounts of data and thousands of linked chips, while inference runs the trained model for every request, billions of times a day, where the speed and energy of each answer matter most.
New designs. The start-ups are trying several ideas:
The whole wafer as one chipChips are made many at a time on a thin disc of silicon, called a wafer, which is then cut into separate chips. One company keeps the whole wafer as one very large chip, so that data never has to leave it.
Computing inside the memoryMuch of the energy a chip uses goes on moving data between the memory and the processor. Doing the arithmetic inside the memory saves most of that energy.
Tiny, low-power chipsthat run small AI models inside earphones and sensors, with no need for the cloud.
Open designsSome build their AI chips on RISC-V, the open instruction set. (More on RISC-V in Market Segment 35.2, Semiconductor IP companies.)
The software problem. A new chip is useless if programmers cannot run their models on it. Most AI software is written for NVIDIA's tools. (More on NVIDIA in Market Segment 35.1, Processor and GPU makers.) So each start-up must build its own compiler and software, to take models written for GPUs and run them on its chip. This is a large part of why these companies hire software engineers.
The start-ups' four bets: the whole wafer as one chip (Cerebras Systems), computing inside the memory (d-Matrix and EnCharge AI), tiny low-power chips (Syntiant Corp.) and open designs on RISC-V (Tenstorrent). Each needs its own compiler and software to run models written for NVIDIA's GPU tools on its chip.
How they earn. Some sell chips, or whole systems built around them. Some run their chips in their own cloud and sell access to AI models, charging for each answer. Some license their designs to others.
Several of these start-ups do much of their design work in Bengaluru.
Every company named here has posted software engineering jobs in India. Famous companies that don't actively hire software engineers in India are left out.