Recommendation systems

Recommendation systems are software that decides what each person sees first: which film, post, video, product or profile. A recommendation system is also called a recommendation engine, or a recommender. It makes its choice from what that person did before, and from what other people with similar habits did. Almost every app that has more items than a person could ever scroll through uses one. Streaming services, short-video apps, shopping apps and matrimony sites all do.

Here is how it works in general. First, the system collects signals. A signal is any action that shows interest: a play, a like, a share, how long the person watched, or a video they skipped after two seconds. Second, from a catalogue of millions of items, the system picks a few hundred candidates that the person might like. It finds them in two main ways. It looks for items similar to ones the person liked before, which is called content-based filtering. It also looks at what was liked by other people who liked the same things, which is called collaborative filtering. Many systems now turn each item and each person into a list of numbers, called a vector embedding, so that finding similar items becomes a quick calculation. Third, a ranking model scores each candidate by how likely this person is to watch it, click it or reply to it. The highest-scoring items are shown first. The person's next action becomes a new signal, and the cycle repeats.

A recommendation system is only as good as the goal it is given. If it is scored only on how long people keep watching, it may keep showing more extreme or repetitive clips. So companies add other rules too: show some variety, include new items, keep out harmful posts, and give new creators a chance. A new user with no history is the hardest case. This is called the cold start problem. The system then starts with what is popular in that user's region and language.

Take, for example, a streaming service. Its content recommendation engine arranges the home screen differently for each viewer, so one viewer sees cricket highlights first and another sees Korean dramas.

Consider a short-video app. Within a few minutes, it learns from what a new user watches to the end, and what they skip, which clips to show next.

Say a family is looking for a match on a matrimonial website. The site ranks profiles by the family's stated preferences, and by which profiles similar families contacted.

How are recommendation systems different from search? In search, the person types what they want, and the system finds it. A recommendation system offers items before the person asks for anything. Shopping sites use both. The way a shopping site changes its pages for each shopper is described in e-commerce personalisation.