Visual search lets you search a shopping app with a photo instead of words. It is also called search by image or photo search. You photograph a chair, a kurta or a lamp, and the app shows products that look like it. Some apps also let you place a product, such as a sofa, in a live camera view of your own room before you buy it.
Words are often the hardest part of a search. A buyer may not know that a chair is called a "cane accent chair" or that a print is called "ikat". A photo needs no words.
Here is a typical case. A buyer in Kochi sees a cane chair at a friend's home and photographs it. The shopping app uses computer vision, software that recognises objects in photos, to find the chair in the picture and cut out the background. A model then turns the chair into a vector. A vector is a long list of numbers that describes the chair's shape, colour and pattern. Two chairs that look alike get vectors whose numbers are close together.
The marketplace has already turned every product photo in its catalogue into a vector in the same way. Say it has 2 lakh chair photos. The app compares the buyer's vector with all of them and finds the 20 closest, in well under a second. It shows those 20 chairs, priced from ₹4,500 to ₹11,000. The buyer can then filter them by price or colour, as in an ordinary search. The ranking and the filters are run by e-commerce search and recommendations software, which treats the photo as one more kind of query.
The buyer picks a chair for ₹6,200 and taps "View in your room". The phone's camera measures the floor and the walls. The app then draws the chair at its real size in the live picture, so the buyer can see whether it fits beside the window. This is called augmented reality (AR): a digital object drawn into a live camera view.
Visual search matters most for goods that a buyer cannot touch before buying, such as furniture, home decor and fashion. Home buyers who want a whole room designed, not one chair, use a home interior design platform instead.
How is visual search different from text search? Text search matches the words that a buyer types with the words in each listing. Visual search matches what the products look like, even when the listing uses different words. Many online marketplaces offer both, side by side.