We are not ready for what AI has done to red teaming
CSO Online ·
Something shifted in offensive security over the last eighteen months that most organizations haven’t fully absorbed yet. It isn’t that attackers got smarter. It’s that the economics of attack changed entirely — and those of us who run red teams have had a front-row seat to the transformation. The conversation about AI in cybersecurity tends to fixate on defense: better anomaly detection, faster triage, and AI-assisted SOC analysts. That framing misses the more urgent story. The attack side of this ledger has already been rewritten. A small, well-equipped offensive team using frontier AI models today can outperform what was possible with a much larger traditional operation just two years ago, by a margin that should unsettle every security leader reading this. Through our work on the front lines of offensive security, the consensus is clear: if you are not augmenting your operations with AI, you are already at a disadvantage. Before addressing the operational shift, we must clarify a fundamental distinction that is often conflated in industry discussions: AI red teaming: This involves probing Large Language Models (LLMs) for vulnerabilities, testing agentic pipelines, and identifying prompt-injection vectors or unintended behaviors. Red teaming with AI: Using AI as an operator’s tool to execute traditional offensive operations. It is this second category, Red Teaming with AI, where the most immediate, drastic, and dangerous transformation is occurring. The staggering scale of productivity AI-augmented operations transitioned from a nice-to-have luxury to a non-negotiable requirement. The democratization of frontier models introduces asymmetry between well-equipped adversaries and traditional defensive teams relying on manual triage. The productivity leaps we are seeing in the field are staggering. When AI is effectively weaponized by offensive teams, some of the initial findings speak for themselves: Metric Traditional red teaming AI-augmented red teaming Annual team output 60 to 70 findings (3-4 person team) 90+ findings generated in a single after-hours exercise Single app engagement 7 to 10 findings 100+ true positive findings Large-scale scanning Takes appropriate sequencing and pipelines, or times out entirely Enables you to build capabilities that can scan millions of IPs in 2-3 hours (for roughly $20k-$30k) Reconnaissance of external applications 150-200 valid findings/year 100+ findings a month (Data above aggregated by DayBlink Consulting over past 10 years of engagements and validated by Robert Pimentel as representative of his Red team experience.) AI across the engagement lifecycle We are no longer just using AI to write phishing emails. Organizations and threat actors alike are adopting a structured approach to integrate AI throughout every phase of the kill chain: Pre-operation (setup and reconnaissance): The weeks-long setup phase has been accelerated into mere moments. Gathering Open-Source Intelligence (OSINT), provisioning infrastructure, building organizational charts, and configuring command-and-control (C2) servers can now be delegated to agentic AI. This significantly reduces the time and manual effort needed to get an operation off the ground, though complex infrastructure decisions still require experienced operators. Pre-op usually starts off exciting (new op coming) but often ends in a grind. We love the acceleration AI has provided here. Mid-operation (enumeration and attack): during active engagements, AI acts as a real-time, on-demand expert. Operators use AI interactively to generate custom audit scripts, rapidly adapt strategies, and query unfamiliar technology stacks on the fly. By utilizing an LLMs-as-judges workflow, where one model’s output is vetted by another for correctness and risk before execution, attackers can ensure highly confident, automated actions. Enumeration and attack is where all the excitement and adrenaline happens. We are using AI to help here, but it’s not replacing us humans (yet). Post-operation (reporting and deconfliction): reporting has historically been the most tedious burden of any engagement. Today, nine to ten months of raw C2 logs can be piped into a frontier model to generate a comprehensive deconfliction report in under 20 minutes. These tools can map every executed command to the MITRE ATT&CK framework , cross-reference threat intelligence, and translate highly technical data into clear, actionable narratives for board-level executives. The output still requires manual review. AI gets you to a first draft in minutes instead of days, but an operator needs to verify findings, validate ATT&CK mappings, and ensure accuracy before delivery. Ticketing and follow-up are the bane of most red teams. AI has become an awesome sidekick to our teams to build our reports and do lots of the tedious work. The talent divide: judgment as the ultimate differentiator With AI automating basic enumeration, scanning, and scaffolding, the core red-teaming skill set — critical thinking, architectural understanding, adversarial judgment — is as valuable as it’s ever been. AI doesn’t replace that. But operators who can leverage AI alongside those skills will outperform those who can’t. The differentiator isn’t AI versus no AI; it’s whether you have the foundational knowledge to direct it effectively and validate what it gives you. This is fundamentally shifting the required profile of a successful cybersecurity professional. As a result, the industry is quickly diverging into two distinct categories of practitioners: The “unicorns” (high performers): These are the practitioners who bring critical thinking and deep adversarial judgment to AI. They understand exactly where trust boundaries fail and where architectures are weak. They use AI to execute on these insights efficiently, offloading the busywork to focus on complex exploitation. The “slop cannons”: Operators who use AI as a mindless copy-paste shortcut. They rely entirely on unverified outputs without understanding the underlying mechanics, producing noisy, easily detectable work that wastes time and triggers alarms. Consequently, hiring practices must change. The primary screening criterion for new talent can no longer be a stack of legacy certifications; it must be their hands-on curiosity, their fundamental understanding of network architecture, and their experience building tools with frontier AI models. Certifications still matter, but hiring must also evaluate whether candidates can think critically about systems and leverage AI tools effectively — not just operate them blindly Strategic imperatives for 2027 For security leaders looking to survive this paradigm shift, the path forward requires immediate action. Based on our observations, here are the critical priorities for the next 18 months: Formalize AI budgets: Access to frontier models is expensive, but it is a necessary operational cost. Teams without the budget to experiment and build will fall behind adversaries. Implement continuous, programmatic testing: As enterprises deploy thousands of their own agentic integrations, manual testing is no longer viable. Continuous, automated adversarial testing must become the standard operating procedure. Prepare for hybrid threat vectors: As digital perimeters harden through automated defense, attacks will increasingly blend hyper-automated digital exploits with highly sophisticated physical and social engineering vectors. Hire for curiosity: Build teams anchored by practitioners who are actively experimenting with AI and possess the high-level judgment needed to guide it safely and effectively. Organizations that fail to adopt an AI-augmented mindset risk being overwhelmed by adversaries operating faster, smarter, and at an unprecedented scale. Success in the modern cyber landscape relies on one core principle: seamlessly combining elite human judgment with intelligent automation.