Tech, ExplainedE-commerce, travel and logistics
Product

Robotic picking

The last warehouse job still done by hand.

BEFORE ITAFTER ITRobotic pickingMowitoCOVERSVision modelsGrasp choiceSuction grippersUnseen itemsGoods-to-person robotsbrings binsto the armRobotics foundation model↗ GENERATIVE AI APPLICATIONSsupplies thecontrol modelEach item is placed in itsorder's tote, ready for packingCONCEPTS IT USESComputer visionBEFORE ITAFTER ITRobotic pickingMowitoCOVERSVision modelsGrasp choiceSuctiongrippersUnseen itemsGoods-to-personrobotsbrings binsto the armRoboticsfoundation model↗ GENERATIVEAI APPLICATIONSsupplies thecontrol modelEach item isplaced in itsorder's tote,ready for packingCONCEPTS IT USESComputer visionBEFORE ITGoods-to-person robots↓ brings bins to the armRobotics foundation model↗ GENERATIVE AI APPLICATIONS↓ supplies the control modelRobotic pickingMowitoVision modelsGrasp choiceSuction grippersUnseen itemsAFTER IT↓Each item is placed in its order's tote,ready for packingCONCEPTS IT USESComputer vision
A goods-to-person robot brings a mixed bin to a robotic arm in Coimbatore, the camera finds a face-wash tube among other items, and the arm lifts it by suction into the order's tote.
opens a page inside this one a page in the same scene a page in another scene a concept page a label only, with no page

Robotic picking is the use of a robotic arm, guided by vision models, to pick single items out of a bin, including items it has never seen before. It is also called piece picking. It is the last step in most warehouses that is still done mainly by hand.

Robots already move shelves, bins and pallets. But choosing one item from a crowded bin is hard for a machine. A large warehouse may stock hundreds of thousands of different products. They come in boxes, soft pouches, bottles and plastic bags. In a bin, they lie on top of each other at every angle. Shiny wrapping reflects light, and clear plastic is hard to see. A person handles all of this without thinking. A robot programmed separately for each product cannot, because there are too many products, and new ones arrive every week.

So the arm uses computer vision, software that recognises objects in images. A camera above the bin takes a colour image and a depth image, which records how far away each point is. A model finds the separate items in the images, and chooses the point where the gripper can best hold one of them. The gripper is often a suction cup, which holds a flat surface, or soft fingers for a bag. The arm lifts the item, the camera checks that it has lifted only one, and the arm places it in the tote for the order. The model has learnt from millions of images of many kinds of objects. That is why it can find a grip on an item it has never seen. Newer robotics foundation models are trained to control many kinds of robot with one model.

For instance, consider a beauty products seller's warehouse in Coimbatore. A goods-to-person robot brings a bin of mixed items to a station where an arm works instead of a person. The order needs one tube of face wash. The camera sees the tube, two lipstick boxes and a pouch of sheet masks, all overlapping. The model picks out the tube, finds a flat area on its side and places the suction cup there. The arm lifts the tube, the camera confirms that it holds one item, and the arm drops it into the order's tote. A few seconds later, the next bin arrives. If the model is not sure of a grip, the station sends the bin to a person instead.

The arm must decide quickly, at the station itself. So the model runs on a computer beside the arm, not in a distant data centre. This is called edge AI. Robotic picking is still slower than an experienced person for many items, and it fails more often on soft or see-through packaging. So most warehouses use it beside people, for the items it handles well, and keep people for the rest.

India, with its quick-commerce dark stores and its robotics start-ups, is both building and testing these arms. Robotic picking differs from an ordinary industrial robot, which repeats the same movement on the same part thousands of times. A picking arm must work out a new movement for every item.

Tech, Explained is a part of ingrid.fyi. Every page explains one piece of technology in plain words.