Edge AI means AI models, and the software that runs them, placed on a device such as a phone, a camera, a vehicle or a machine, instead of in a distant data centre. The device answers by itself, often in a few milliseconds, and it keeps working when there is no network. The word edge refers to the edge of the network, where the data is created, as the page on edge computing explains.
Why not send everything to the cloud? There are three reasons. The first is speed: a round trip to a data centre over a mobile network can take half a second or more, which is too slow for some decisions. The second is the network itself, which may be weak or absent. The third is privacy and cost. Sending video from thousands of cameras all day is expensive, and some data should not leave the device at all.
The difficulty is size. A model trained in a data centre may need gigabytes of memory and a large GPU. A camera has a small chip and little memory. So edge AI companies shrink the model. One method, called quantisation, stores each of the model's numbers with fewer bits, for example 4 bits instead of 16. This makes the model about four times smaller. Another method removes the parts of the model that matter least. The company then compiles the model for the device's chip, which may be a phone processor, a TinyML chip or a small AI accelerator. The page on AI accelerators covers these chips.
For instance, a truck carrying cement from Chittorgarh to Ahmedabad has a camera facing the driver. On some stretches of the highway, the mobile signal drops for minutes at a time. A small model on the camera's own chip checks the driver's eyes 15 times a second. When the eyes stay closed for more than two seconds, the camera sounds an alarm in the cab at once. It stores the clip and uploads it later, when the signal returns. This is how an AI dash cam works.
The same approach runs inside machine vision cameras on factory lines, video analytics on security cameras and AI PCs. New versions of a model are prepared centrally and sent to the devices through a model deployment platform. Compression tools from AI interpretability and alignment companies can also make a model small enough for a device.
The trade-off is accuracy. A small model makes more mistakes than a large one. So many products split the work. The device makes the quick decision, and the cloud checks the harder cases later.