Future defenses

The field is evolving. Defences must too.

As agentic AI scales into more roles and capabilities, organisations must anticipate new risks. The joint guidance highlights three directions security practitioners and researchers should invest in now to keep pace.

Source: Careful adoption of agentic AI services, co-authored by ASD's ACSC, CISA, NSA, Canadian Cyber Centre, NCSC-NZ, NCSC-UK (pp. 24–25).

1. Expand threat intelligence through collaboration

Existing frameworks (OWASP 2025 Top 10 for LLMs and GenAI, MITRE ATLAS™) focus on LLM vulnerabilities; vendor reports emphasise platform misuse. Attack vectors unique to agentic AI are not yet fully captured. Closing that gap is a community problem.

Strengthen stakeholder collaboration

Coordinate with major AI developers and government cyber organisations to compile and maintain threat information specific to agentic AI.

Adopt collaborative playbooks

Reference CISA's AI Cybersecurity Collaboration Playbook for shared incident-response and information-sharing patterns.

Track adversaries over time

Implement alerting, data collection, and tracking methods for malicious actors and techniques targeting agentic systems.

Harmonise threat taxonomies

Build shared threat taxonomies across industries to improve threat modelling and tighten mitigation design.

2. Develop robust, agent-specific evaluations

Existing evaluation methods are sensitive to minor semantic changes, vary by scenario, and only partially capture real-world deployment conditions. That makes reliable validation of agent security and architecture nearly impossible. Better evaluations are the prerequisite for everything else.

Generate realistic benchmarks

Create benchmark datasets that cover new domains and represent realistic deployment contexts — not just model-level tasks.

Validate emerging practices

Use evaluation results to validate emerging security practices and identify failure points in agents.

Cover deployment edges

Include edge cases beyond typical training conditions: unfamiliar tools, hostile third parties, partial-failure environments.

Share findings publicly

Share evaluation findings to strengthen security assessments across the field.

3. Use system-theoretic approaches

Agentic AI is an ecosystem of LLMs, humans, guardrails, datasets, tools, and hardware where security risks emerge from interactions between components, not isolated flaws. Component-level analysis is insufficient. System-theoretic methods are designed for exactly this class of problem.

STPA — System-Theoretic Process Analysis

Hazard analysis treating safety as a control problem; identifies unsafe control actions across an entire system architecture.

STPA-Sec — security extension

Applies STPA to security; identifies security issues, assesses mission risk, informs mitigations across notional or operational systems.

CAST — Causal Analysis using System Theory

Post-incident: investigate security incidents and identify root causes at the system level rather than blaming individual components.

Why this matters

Cascades and emergent behaviours are exactly the failures STAMP-family methods were designed to find before they occur. The MIT STAMP materials and Systems thinking for safety and security are the canonical references; we use them in plan reviews when the agent topology gets non-trivial.

Subscribe to The 101s Brief

Notified when allied agencies and standards bodies publish new agentic-AI guidance. Plain-English impact for your stack.