A custom AI ASIC is an AI chip designed for one company's own work, usually the work of one of the largest cloud companies. ASIC stands for application-specific integrated circuit: a chip built for one particular job, instead of a general-purpose chip sold to everyone. The cloud company decides what the chip must compute. A specialist chip company designs the rest of the chip, has it made and delivers it.
Why would a cloud company want its own chip? The largest cloud companies, most of them based in the United States, run AI models for hundreds of millions of users: search, video recommendations, advertising and chatbots. They buy GPUs in very large numbers. A GPU is built for every kind of AI work, so it carries circuits that one company's models may never use. A chip built only for its own models can leave those circuits out. Across hundreds of thousands of chips, the savings in price and electricity add up to very large sums. A chip of its own also makes the company less dependent on a single GPU supplier.
Why not design the whole chip alone? A cloud company knows its own models very well. But a modern AI chip also needs parts that are very hard to design. It needs the high-speed links that join it to other chips, called chip interconnect. It needs a connection to stacked high-bandwidth memory (HBM). It needs a package, the case that holds the chip and its memory and joins them to a circuit board. And someone must manage the work with the foundry, the factory that makes the chip. A chip company that already makes networking chips has all these skills. So the work is split between the two.
Take, for example, a large cloud company in the United States that wants a chip to run the model that ranks videos for its users. Its own engineers design the compute engine, the part of the chip that does the model's arithmetic. The chip partner adds the links, the memory connection and the package, and checks that the whole design works. The partner sends the design to a foundry in Taiwan. A few months later, the first chips come back, and both teams test them. The cloud company then installs the chips in racks in its own data centres, as part of an AI cluster. From the first plan to chips in service usually takes two to three years. The cloud company pays the partner for the design work and for each chip delivered.
For the chip partner, this is a design win. If the first chip works well, the partner usually designs the next generation too. Much of this design and testing work is done in the partners' centres in Hyderabad and Bengaluru.
How does a custom chip differ from an AI accelerator? An accelerator is sold, or rented out, to many buyers. A custom chip has one owner, and it is not sold to anyone else. An Indian company can use one only by renting it as a cloud service from its owner. Like accelerators, custom chips are used for both training and inference, which the page on training vs inference explains.