An AI PC is a laptop or desktop whose processor chip includes an engine built to run AI models on the machine itself, instead of sending the work to a data centre. That engine is called a neural processing unit, or NPU. The same chip also holds the usual processor cores, the CPU, and a graphics engine, the GPU.
Why add a separate engine? Running an AI model is mostly a huge number of multiplications and additions. The CPU can do them, but slowly. The GPU can do them quickly, but it uses a lot of battery. An NPU is built to do only this kind of arithmetic, many sums at once, at low power. Its speed is measured in TOPS, or trillions of operations per second. Many current laptop NPUs are rated at about 40 to 50 TOPS.
The laptop only runs models; it does not train them. Training a model takes thousands of GPUs in a data centre for weeks. Running a trained model to answer a request is called inference, and the difference is explained on training vs inference. To fit on a laptop, a model must be small. A typical on-laptop language model has a few billion parameters, the numbers the model learnt in training. Each number is stored in only 4 bits, so the whole model takes about 2 GB of memory.
There are three reasons to run a model on the laptop. First, privacy: the data never leaves the machine. Second, speed: there is no wait for a server, and it works without the internet. Third, cost and battery: there is no cloud bill for each request, and the NPU uses less power than the GPU.
Say a lawyer at a law firm in Mumbai records a 40-minute meeting with a client on her laptop. The firm's rules say that client recordings must not leave the firm's own machines. The transcription app runs a speech recognition model on the NPU, with the Wi-Fi switched off. About 5 minutes later, the full transcript is ready. She then asks for a one-page summary, and a small language model on the laptop writes it. Next, she asks the assistant to compare the client's contract with 200 earlier contracts stored on the firm's servers. That job is too large for the laptop's model. So the software offers to send it to a larger model in the cloud, and asks her to confirm first.
This choice, deciding what runs on the laptop and what runs in the cloud, is written by the PC makers and the software companies. Much of that work is done in India. Running models on devices instead of in data centres is called edge AI. Phones have done the same for years, with an AI engine on the phone's system-on-chip. PC makers hope that AI PCs will make people and companies replace their laptops sooner than the usual three to five years.