Tech, ExplainedIndustrial tech and mobility
Part

AI robotic picking

Picking up an object it has never seen.

INSIDE: INDUSTRIAL ROBOTS AND COBOTSHOW IT WORKS · PICKING A STEEL BRACKET FROM A MIXED BINA bin holds 40steel bracketsof severalshapes,jumbled togetherTwo camerasphotograph thebin fromslightlydifferent anglesSoftware buildsa 3D model ofthe bracketon top, whichit has neverseen beforeIt finds twopoints wherethe grippercan hold thebracket safelyThe arm moves tothe point on apath that avoidsthe bin walls,and closesthe gripperA camera checksthat the bracketis held, and thearm puts it onthe conveyorALSO TALKS TOMachine visionRobotics foundation model↗ GENERATIVE AI APPLICATIONSINSIDE: INDUSTRIAL ROBOTS AND COBOTSHOW IT WORKS · PICKING A STEEL BRACKET FROM AMIXED BINA bin holds 40 steelbrackets of severalshapes, jumbled togetherTwo cameras photographthe bin from slightlydifferent anglesSoftware builds a 3Dmodel of the bracket ontop, which it has neverseen beforeIt finds two points wherethe gripper can hold thebracket safelyThe arm moves to thepoint on a path thatavoids the bin walls, andcloses the gripperA camera checks thatthe bracket is held,and the arm putsit on the conveyorALSO TALKS TOMachine visionRobotics foundation model↗ GENERATIVE AI APPLICATIONSINSIDE: INDUSTRIAL ROBOTS AND COBOTS · PICKINGA STEEL BRACKET FROM A MIXED BINA bin holds 40 steel brackets of severalshapes, jumbled together↓Two cameras photograph the bin fromslightly different angles↓Software builds a 3D model of the bracketon top, which it has never seen before↓It finds two points where the gripper canhold the bracket safely↓The arm moves to the pointon a path that avoids the binwalls, and closes the gripper↓A camera checks that the bracket is held,and the arm puts it on the conveyorALSO TALKS TOMachine visionRobotics foundation model↗ GENERATIVE AI APPLICATIONS
AI robotic picking uses cameras to see a bin of objects, builds a 3D shape of each, chooses a grip and moves the arm to pick one.
opens a page inside this one a page in the same scene a page in another scene a concept page a label only, with no page

AI robotic picking is the use of cameras and AI so that a robot can pick up an object it has never seen before. Normally a robot is programmed point by point by a specialist, who tells it exactly where the part will be and how to grip it. That works only if the part is always in the same place.

With AI robotic picking, the robot looks at the object and works out the grip itself. The robot's cameras take pictures of the bin. Software builds a 3D shape of each object, finds the points where the gripper can hold it, and plans a path for the arm that does not hit the bin. After the pick, the cameras check that the object is held properly.

Picking from a jumbled bin is far harder than picking from a conveyor belt. On a belt, the parts lie flat and spaced apart, and a simple camera can find them. In a bin, parts lie on top of each other, hide each other and point in every direction. A shiny metal part also reflects the light and confuses the camera. So the software must decide which part is on top, and whether the arm can reach it without knocking the others.

For example, a bicycle-parts maker in Ludhiana changes its product every few days. A conventional robot would need a specialist to reprogram the grip and the path for every new part. A robot that picks by sight needs only a few pictures or the part's drawing, and it can handle a part that the factory has never made before.

This part depends on machine vision, which is the same camera technology used for inspection, and on a robotics foundation model. The robot arm is described under industrial robots and cobots. The idea belongs to computer vision in general.

The hard part is speed and certainty. A picking robot must pick at a speed close to a person's, and it must not drop a part, because a drop stops the line. Because of these two needs, picking from a jumbled bin is still one of the hardest factory jobs to automate.

Tech, Explained is a part of ingrid.fyi. Every page explains one piece of technology in plain words.