Trust and safety investigation is software in which an AI agent reads a report of abuse or a fake account, gathers the evidence and drafts a decision for a human reviewer. An AI agent is an AI model that can take several steps on its own, such as searching records and opening files, to finish a task. Before such tools, a reviewer collected all the evidence by hand.
Large apps receive thousands of reports a day. Users report posts that harass someone, accounts that pretend to be someone else, and messages that ask for money. Each report needs evidence before anyone can decide on it fairly. Collecting that evidence takes most of a reviewer's time. The decision itself often takes a minute.
Here is an example of one report. A user of a vernacular social media app reports an account that pretends to be a well-known doctor in Vijayawada. The account posts in Telugu and sells a "cure" for diabetes, paid by UPI. The agent opens the report and collects the facts. The account was created four days ago. Its profile photo also appears on the real doctor's clinic website. Nine other accounts were created from the same phone and post the same text. Three other users have already reported the same account. The agent writes a two-paragraph draft. It recommends removing all ten accounts for impersonation, and keeping the evidence in case the police ask for it. It lists each fact with a link to where it was found.
A reviewer reads the draft, opens two of the links and agrees. She removes the accounts. The whole case takes her four minutes instead of forty. This is human in the loop: the model drafts, and a person decides. The person stays responsible, because a wrong removal harms a real user.
The agent can also call other tools. It can send a suspect photo or voice note to deepfake detection. It can search public websites and news, which is called open-source intelligence (OSINT). And when the fake account copies a company's name, it can pass the case to digital risk protection, which gets fake sites and accounts taken down.
In India, the timing matters. Under the IT Rules 2021, a platform's grievance officer must acknowledge each complaint within 24 hours and resolve it within 15 days. Faster investigation helps the platform meet these limits. Reports also arrive from a dating app or a matrimonial website, where fake profiles ask people for money.