Automated red teaming is the use of an AI agent that plans and runs an attack on a company's own systems by itself, to find the weak points before a real attacker does. A red team is a group that attacks its own side to test the defence. An AI agent is software built on an AI model. It is given a goal, a set of tools and permissions, and it decides its own next steps.
Human red teams are expensive, because they need many days of skilled people's time. So most companies test only once or twice a year. An automated red team can run every week. The newest products have the agent do the scouting and the first attempts, and keep human testers for what the agent cannot find. Attackers already use AI models in the same way, as AI in cybersecurity explains.
The agent works in a loop. It looks at what it can reach, chooses a technique, runs it, reads the result and decides what to try next. It can write and run small programs as it goes, in the same way as AI coding agents. It is given a fixed scope: which systems it may attack, which actions it may never take, and when it must stop and ask a person. These AI agent guardrails matter more here than anywhere else, because this agent attacks on purpose.
Automated red teams now also attack AI systems themselves. Consider a health insurer in Indore with a customer chatbot built on a language model. The chatbot answers questions about each customer's own policy. The insurer gives the red-team agent one goal: make the chatbot reveal another customer's policy. In one night, the agent sends 2,000 messages. Some ask politely, some pretend to come from an employee, and some hide instructions inside a pasted document. This kind of attack is called prompt injection. Thirty-one of the messages succeed. In each of them, the chatbot reveals another customer's name and policy number.
The agent's report lists the 31 messages and what they had in common. The insurer adds a rule to its LLM guardrails, the layer that checks what goes into the model and what comes out. It also changes the chatbot so that it can fetch only the logged-in customer's policy. The agent runs again, and none of the 2,000 messages succeed. The insurer then adds the 2,000 messages to its LLM evaluation tests, which run before every new version of the chatbot.
When the agent attacks the network rather than a chatbot, the security team watches the SIEM at the same time. If the SIEM raises no alert during the attack, the team has found a second gap, this time in detection.