From assistant to worker
For a year or two, enterprise AI meant an assistant: a person asked, the model answered, the person decided. The risk was contained because nothing happened until a human acted.
That phase is ending. The agents now arriving inside organisations do not just answer. They update a customer record in the CRM, reconcile a batch of transactions against the ledger, raise a ticket in ServiceNow and close it, chase a supplier for a missing invoice. Analysts expect a large share of enterprise applications to ship with task-specific agents built in within a year, and most organisations we talk to already have one or two running somewhere — usually in finance operations, usually further along than the governance around them.
That is the change that matters. A system that enforces a policy creates one kind of risk. An agent that takes actions across several systems, continuously, creates another — and the old control model was not built for it.
Why “a human in every loop” does not scale
The instinctive response is to keep a person approving every step. It is the right instinct early on: it builds confidence and gives the team a way to check outputs before the agent has earned any trust.
But if every action needs a human review, the process has not changed. The organisation has added AI to the same bottleneck. Analysis arrives faster; nothing moves faster, because a person still has to push each step through.
The goal is not to remove people. It is to reposition them: setting policy, defining boundaries, watching performance, handling exceptions, and making the judgment calls where the stakes are high. They should not be approving routine, low-risk actions because nobody designed a control model that could tell the difference.
What a job description looks like
Treat the agent as you would a new member of staff with system access, and the questions become familiar ones — the ones HR, security, and the data team already know how to answer.
- Role and scope. What work is it for, and what is explicitly out of scope? “Resolves supplier invoice queries under $10,000” is a job. “Helps with finance” is not.
- Systems it may act in. Name them: read access to the data warehouse, write access to the ticketing queue, no access to the payment run. Integration is where this gets real — an agent’s permissions are only as precise as the APIs and roles the underlying systems expose.
- Data it may use. Which tables, which documents, which customer records. This is where the catalog and the warehouse’s access model do the work: an agent grounded in governed data inherits the lineage, classification, and row-level controls the business already trusts. An agent grounded in an extract someone emailed inherits nothing.
- Decisions it may take alone. Low-risk, predictable, reversible actions can run autonomously. Monitored workflows get periodic review. Sensitive or irreversible actions escalate immediately, every time.
- When it must stop. Escalation rules written down in advance: confidence below a threshold, a value above a limit, a customer flagged as high-risk, an action the policy does not cover.
None of this is new thinking. It is the access-control and segregation-of-duties discipline enterprises already apply to people, extended to a worker that never sleeps and never goes off-script unless you let it.
Trust has to be designed in, not reviewed in
The early model of trust was review: someone checked the answer. That still has a place, but it cannot be the only mechanism once the agent is acting rather than advising. Trust has to be built into the process before the action happens.
That means permissions that are context-aware rather than blanket. Controls proportionate to the risk of the specific action. And an audit trail that records not just what the agent did, but what information it used and why it chose that action — captured in the system of record where the action landed, not in a chat transcript nobody will ever search.
This last point is where the information-systems view earns its keep. If an agent writes to the ERP, the ERP’s change history is the audit trail. If it decides using warehouse data, the warehouse’s lineage tells you which version of the truth it saw. Organisations that already run a well-governed data platform are further ahead here than they realise: the evidence infrastructure exists. It just needs the agent wired into it rather than around it.
The next question is not how many, but how managed
The next phase of enterprise AI will be judged less by how many agents an organisation deploys and more by how well it manages them. That means an inventory of agents, maintained the way the inventory of users, systems, and access rights is maintained. Clear ownership for each one. Telemetry on what they do. Governance that is revisited as the agents improve, the workflows change, and new risks appear.
Agents will only get more capable. The work is making sure they do not outrun the business — being clear about what each one may do on its own, where a person steps in, and who is accountable when the work is done. Write the job description before the agent starts. It is far easier than writing it after the first incident.
