Shelf image recognition is software that reads a photo of a shop shelf and checks each product on it: whether it is in stock, placed as agreed with the shop, and priced correctly. The photo is taken by a company's sales representative on a phone, or by a camera fixed in the store. The software uses computer vision, software that recognises objects in images. Checking shelves in this way is called retail execution, or a shelf audit.
The buyers are mostly consumer-goods companies. In India they are called FMCG companies, for fast-moving consumer goods, such as biscuits, soap and shampoo. Retailers use the software too. A product that is not on the shelf does not sell. So these companies send sales representatives to lakhs of shops every week.
Consider a sales representative of a noodles brand who visits a kirana store in Indore, a small neighbourhood shop. She photographs the shelf, which holds noodles from four brands. Within seconds, the app on her phone shows the result. Her brand has 6 packs facing the shopper, against 10 agreed with the shopkeeper. Its new masala flavour is missing. And one pack is marked ₹15, while the MRP printed on it is ₹14. MRP, the maximum retail price, is the highest price a shop may charge. The representative corrects the price label and takes an order for the missing flavour. Her field sales software sends the order to the brand's distributor management system (DMS). The local distributor uses that system to deliver the stock.
To do this, the model must be trained on photos of every pack the brand sells, and of its rivals' packs, at many angles and in many kinds of light. Packs change often, for example for a festival offer. So the model is retrained regularly. The agreed layout that each photo is compared with is called a planogram.
The data has a second use. When a brand collects thousands of photos a day, it can measure its share of the shelf in each city, and against each rival. Companies that sell retail measurement data, which shows what sells where, use data of a similar kind.
Photos are not the only way to find a gap on the shelf. Out-of-stock detection software finds the same problem from a store's sales data. In large supermarkets, cameras fixed on the shelves, or robots that move around the store, can check every shelf every hour. But in India, millions of small shops sell most consumer goods. A photo on the representative's phone is the method that can reach all of them.