AI governance is moving out of the policy document and into the operating layer.
That shift matters because AI agents no longer only generate text. They can call tools, access company data, update records, trigger transactions, and pass work to other agents. Once an agent can act, the key question is not simply, “Which model are we using?” It is, “What controls the complete chain of action?”
Several recent signals point in the same direction. Dataiku is positioning agents as a distinct unit for discovery, monitoring, and risk management. F5 argues that governance must cover what agents do at runtime, including permissions, tool use, approvals, and chains of authority. Thoughtworks has launched a governed runtime and fleet-level control plane. Anthropic’s trustworthy-agent framework emphasises human control, security, transparency, and privacy across the whole agent system.
The shared operator implication is clear: the next layer of enterprise AI is the agent control plane.
Large organisations may buy a platform for this. SMEs still need the discipline, even if they implement it with simpler systems. The controls do not have to be expensive. They do have to be explicit.
Why model governance is no longer enough
Model governance asks useful questions: Which model is approved? Where is data processed? How is privacy protected? How is output evaluated?
Agent governance adds another set of questions:
- Which agents are running, and who owns each one?
- What business outcome is each agent allowed to pursue?
- Which tools, records, and credentials can it access?
- Which actions can it take without approval?
- What happens when context is missing, instructions conflict, or cost rises?
- Can we reconstruct the request, evidence, action, approval, and result?
- Can we pause or stop the agent immediately?
A chatbot can give a poor answer. An agent can create a poor business outcome. The governance boundary therefore has to move from the model to the workflow.
The seven parts of an SME agent control plane
1. Agent inventory
Start with a register of every agent in use, including agents created inside existing SaaS platforms. Record the owner, purpose, systems accessed, tools allowed, data classification, risk level, current status, review date, and shutdown contact.
This sounds basic, but it solves a growing problem: useful agents can be created quickly inside CRM, productivity, automation, finance, and support tools. Without a register, the company loses visibility before it realises there is a fleet to manage.
For an SME, a controlled table may be enough at first. The important point is that every digital coworker has a named human owner and a stated job.
2. Purpose and outcome boundaries
Do not define an agent by its software label. Define it by the business outcome it is allowed to support.
“Finance agent” is too broad. “Prepare an accounts-receivable follow-up list from approved invoices, then draft messages for human review” is governable. It identifies the source, task, output, and approval boundary.
Each agent needs:
- a permitted outcome;
- approved sources and systems;
- prohibited actions;
- acceptance criteria;
- stop and escalation rules.
Better models reduce the value of rigid step-by-step scripts. They do not reduce the value of clear outcomes and boundaries.
3. Agent identity and scoped permissions
Agents should not inherit unrestricted human accounts. Give each agent the least privilege needed for the current task. Start read-only where possible. Separate preparing from approving, drafting from sending, and recommending from committing.
Useful controls include:
- task-specific identities;
- record-level access;
- approved tool and connector registries;
- time-limited credentials;
- transaction and spending caps;
- permission checks at each handoff.
This is especially important when agents use connectors or MCP servers. Connectivity is not the same as trust. Every tool needs an owner, a permission scope, an update process, and a clear reason to exist in the workflow.
4. Human approval gates
“Human in the loop” should not mean a person watches every action. That destroys the capacity advantage. It should mean that human judgment appears at the moments where consequences rise.
Require approval before:
- sending customer-facing communication;
- moving money or changing financial records;
- deploying to production;
- granting access or exposing sensitive data;
- making a contractual, legal, or governance commitment;
- taking an action that cannot be easily reversed.
The approval screen must give the reviewer enough context to decide: proposed action, evidence, applicable rule, expected result, uncertainty, and rollback option. A bare “approve” button is not a control. It is a transfer of ambiguity.
5. Runtime telemetry
Traditional monitoring asks whether a server is available. Agent monitoring asks whether the agent still does the job it was built to do.
Track:
- completed and accepted outcomes;
- correction and reopen rates;
- exceptions and escalations;
- tool calls and permission denials;
- approval frequency and wait time;
- latency and cost per accepted outcome;
- changes in the types of requests the agent receives;
- quality drift after model, prompt, data, or tool updates.
An agent can remain technically “up” while its business performance quietly degrades. Telemetry turns that hidden drift into an operating signal.
6. Evidence and audit trail
An audit trail does not need the model’s private chain of thought. It needs the business chain of evidence and authority.
Capture the request, relevant context, source records, policy applied, tools called, action proposed, human approval, final result, exception, and rollback. This allows an operator to reconstruct what happened without reading thousands of raw messages.
Good auditability also improves day-to-day management. When an output is rejected, the team can see whether the problem came from poor context, weak instructions, an unreliable tool, an outdated rule, or a bad approval design.
7. Cost and stop controls
Every agent workflow consumes resources: model tokens, software calls, staff review time, and the cost of corrections. Set budgets and stopping conditions before autonomy expands.
Measure cost per accepted business outcome, not token volume alone. A more expensive run may still be better if it produces a usable result with less review. A cheap agent that creates rework is not efficient.
Every production agent should also have a tested pause and shutdown path. If an agent starts calling the wrong tool, processing the wrong records, or generating excessive cost, the operator should not need a technical investigation before stopping it.
A practical 30-day path for SMEs
Week 1: Discover
List every agent and agent-like automation already in use. Include features embedded in software your teams already license. Assign an owner and remove or pause anything with no clear purpose.
Week 2: Bound
Choose one valuable workflow. Define the outcome, approved inputs, tools, permissions, prohibited actions, approval points, and stop rules. Start with read-only access and reversible tasks.
Week 3: Instrument
Add logs for requests, tool actions, approvals, exceptions, results, and cost. Define two quality measures and one business-outcome measure. Test missing data, conflicting instructions, suspicious content, unavailable tools, and excessive cost.
Week 4: Operate
Run the workflow with a small volume. Review rejected outputs and exceptions. Fix the workflow before adding more autonomy. Only expand permissions when the evidence shows that the agent is reliable inside its current boundary.
Governance should accelerate useful autonomy
The wrong way to govern agents is to bury every experiment under a large approval process. The right way is to make the operating boundaries visible, testable, and proportional to risk.
Low-risk research can run with light controls. Customer, financial, production, and sensitive-data actions need stronger identity, approval, evidence, and rollback. The control plane should make that difference explicit.
This is the practical meaning of “orchestrate, don’t operate.” Humans should not perform every task or watch every keystroke. They should define outcomes, permissions, checkpoints, and acceptance tests—then use telemetry and evidence to supervise the digital workforce.
For SMEs, the advantage will not come from having the most agents. It will come from knowing which agents exist, what they are allowed to do, how well they perform, and when a human must take control.
