An agent is governed through what it can access, invoke and change, and only partly through what it produces. Tool calls, data reach, spending authority, work handed to another agent: most AI governance frameworks in circulation were written before any of them existed.
What changes when the system acts
Approval stops being a single decision. A use case approved in March describes an agent that has since gained a new tool, a wider data scope and a second agent downstream of it.
Human oversight has to be designed in. Someone has to be able to see what the agent did, assess whether the action was appropriate, and intervene where required. A policy sentence will not produce that capability.
Accountability spreads. When an agent calls another agent which calls a vendor’s model, the question of who answers for the outcome has to be settled before anything goes to production.
Agent lifecycle governance
Intake → classification → permissions → testing → approval → deployment → monitoring → incident response → retirement
Each stage produces something the next stage needs. Where a stage is missing, the chain breaks and nobody notices until an auditor or an incident makes them look.
What has to be controlled at each stage
Agents do not need a separate management system. They need the controls ISO/IEC 42001 already requires, implemented for systems that act.
| Stage | What has to be controlled | ISO/IEC 42001 |
|---|---|---|
| Intake | The agent registered as a system in its own right, with its tools, data access and autonomy written down | A.4 resources for AI systems, including data, tooling and human resources |
| Classification | Impact on individuals and society, and which role your organisation holds for this system | A.5 assessing impacts; clause 6.1.4 AI system impact assessment |
| Permissions | Agent identity, tool permissions and data reach: explicit, attributable, revocable. What it may decide alone, what needs a human, what is prohibited outright | A.6.2 AI system requirements and specification |
| Testing | Criteria met before release, including the threat model below | A.6.2 verification and validation |
| Approval | A named approver and a decision record, not a signature on a use case from March | A.6.2 design and development documentation |
| Deployment | A documented plan, with requirements met before anything goes live | A.6.2 AI system deployment |
| Monitoring | What is measured on the live agent, what threshold triggers a look, and who looks | A.6.2 operation and monitoring |
| Incidents | Escalation wired into the incident process you already run, and how incidents reach affected parties | A.8 information for interested parties, communication of incidents |
| Retirement | How the agent is decommissioned, what happens to its access, what record survives it | A.6.2 operation and monitoring |
| Throughout | Event logs that make the agent’s actions reconstructable | A.6.2 recording of event logs |
| Throughout | Which agents may call which, what they may pass, and who answers for the model underneath | A.10 third-party and customer relationships |
An agent acting under its builder’s credentials cannot be governed separately from that person. That is usually the first thing to fix.
Threat model
Prompt injection, excessive agency and tool misuse, identity and privilege abuse, memory and context poisoning, insecure communication between agents, and cascading failure across a chain of them. The reference points are the OWASP Top 10 for Large Language Model Applications and the OWASP Top 10 for Agentic Applications.
AI agents under the EU AI Act
The AI Act does not treat agents as a separate legal category. Agentic architectures can nevertheless affect how a system is characterised, how it is used and which obligations apply, and those determinations have to be made for the specific system and use case.
Where an agent is built on a general purpose AI model, the obligations of the model provider and those of the organisation deploying the agent are distinct, and worth separating early.
Where to start
Pick one agent you have in production. Write down every tool it can call and every data source it can reach. If that list takes longer than an hour to produce, or if anyone disagrees with it, that gap is your first project.