Your AI Center of Excellence needs to govern what agents can do, not just what teams can build.
This week in my diary, I’m revisiting the four-part AI CoE series I wrote last year. The question isn’t whether I’d throw it away. It’s what I’d make more explicit before giving an agent permission to change something in the business.
The original series was already about the agentic era. Its foundation was Platform + People + Patterns. I still believe in that foundation. This is a short update, not a replacement.
If you missed the originals, here are all four:
The overall model: Winning with AI CoEs in the Agentic Era
Platform: The Foundation of an AI CoE
Patterns: Your Real Competitive Advantage in AI
Think of that last pillar as the bridge to processes: reusable ways of building, releasing, operating, and retiring AI systems.
The moment the operating model changes
Imagine Maya, a customer-operations lead at a retailer. Her AI assistant explains the refund policy. Now she wants it to issue refunds, which means giving it permission to move company money.
The agent can recommend a refund, but the refund service must check whether the action is allowed and whether human approval is required. The agent cannot grant itself permission.
That is what I would add to the CoE’s responsibilities: define what agents can do, enforce those limits, and name the people accountable for their actions.
1. People: name the owner of the action
In the people edition, I proposed a simple division of responsibility: the CoE provides shared tools, rules, and support, while business teams remain responsible for how they use AI and the results it produces.
For every agent, I’d make three responsibilities clear:
The business owner decides what the agent is allowed to do and when it must ask for help.
The technical owner manages its access, tests its behavior, monitors it, and handles technical failures.
The human decision-maker approves or rejects actions that require human judgment.
These do not have to be three different people. For Maya’s refund agent, Maya could own the refund rules, while an engineering lead maintains the controls. Everyone should know who can stop the agent if something goes wrong.
Before deciding how much authority to give an agent, I’d ask three questions from Anthropic’s architect course.
For Maya, drafting a recommendation can stay with the agent. Current order facts and refund rules belong in existing systems. High-stakes exceptions need an authorized person before execution.
Anthropic also discusses calibrated confidence: confidence scores that have been checked against actual results to see whether they are reliable. These scores can help prioritize human review, but the potential harm and difficulty of undoing an action matter more. High confidence alone is not permission to act.
When an action needs human approval, the system should show what the agent wants to do, the relevant facts, and why approval is needed. For Maya’s refund agent, the reviewer should check the order details, refund amount, and applicable policy rather than simply trust the agent’s explanation.
Not every routine response needs this level of review. Teams can spot-check lower-risk work where policy allows, but actions requiring approval must always wait for that approval.
2. Platforms: enforce limits outside the model
The platform edition already included gateways, budgets, approvals, and observability. I’d now be more precise about where enforcement happens.
Below is an AI landing Zone Architecture from Microsoft. I receommend checking out Agent Landing Zone.
The platform team owns the shared infrastructure and guardrails agents run on: identity, gateways, budgets, approval mechanisms, and monitoring. The application team wires these controls into Maya’s refund workflow, while the refund service enforces business permissions before money moves.
A model gateway manages requests to the model, but the company’s refund API, the backend that actually processes refunds, must independently check whether the agent is authorized to issue that refund.
For Maya’s refund agent, I’d require:
Scoped access: Give the agent access only to the orders and refund functions/tools it needs, not a shared administrator account.
Pre-action authorization: The refund service must check who is requesting the refund, which order it concerns, the amount, and the current rules. If permission is missing, it must not proceed.
Bounded Approval: Human approval applies to a specific order and amount. If either changes, the agent must request approval again.
Bounded execution: Limit how long the agent can run, how much it can spend, and how often it can retry. If a request times out, check whether the refund happened before trying again. Use the service’s duplicate-payment protection, keeping the same retry key within its supported time limit. If the outcome is still unclear, stop and let a person investigate.
Traceability and recovery:: Record what was requested, why it was allowed or denied, who approved it, and what happened. Keep only necessary sensitive information and restrict access to it. Make sure the team can disable the agent and take over unresolved cases. Stopping the agent does not reverse money already refunded.
This is consistent with OWASP’s excessive-agency guidance: minimize functionality and permissions, require approval for high-impact actions, and enforce authorization downstream rather than trusting the LLM to decide.
3. Patterns and processes
In the patterns edition, I argued that reusable patterns should include guardrails and lifecycle discipline. I’d extend every golden path with an operating contract: allowed actions, owners, approval rules, evaluation cases, stop conditions, and a recovery procedure.
I’d first agree with the business team on what the agent must do, what it must never do, and how we’ll test both. We should define these requirements before writing production code, rather than change our definition of success to match whatever the prototype can already do.
I’d also correct my earlier claim that automated evaluations mean “no manual checks needed.” Tests help show whether an agent behaves as expected, but passing them does not give it permission to issue a refund, remove required human approval, or prove that the system meets regulatory requirements.
For Maya, I’d test unauthorized orders, malicious instructions in customer messages, changed amounts after approval, and ambiguous payment timeouts. Customer text is evidence to assess, not authority to override policy. (Prompt Injection Attacks)
For each guardrail, I’d record the control, owner, and evidence that it works. Suppose Maya’s policy requires human approval for refunds above $100. The refund service must block those payments until approval is recorded. An engineering lead owns that check, and the evidence includes a passing test showing that an unapproved refund is rejected.
I’d also decide who responds when something goes wrong. If the agent repeatedly attempts refunds outside its permissions, a named owner should investigate, suspend its access if necessary, and verify that the problem is resolved. Scheduled reviews should check that permissions and approval rules still match the current policy. A dashboard can flag a problem, but someone must own the response.
For Maya’s team, I’d expand the agent’s authority in stages:
Read-only assistance: The agent explains the refund policy but cannot send money.
Approval-required actions: The agent prepares a refund, but a person must approve it before payment.
Limited automatic actions: If testing and observed results justify it, the agent handles routine refunds within explicitly permitted limits. Required approvals still apply.
If a fixed workflow can handle the task reliably, I’d use that instead of an autonomous agent.
Changes to the model, tools, permissions, or policy should trigger relevant tests again. For example, a shorter return window requires new eligibility tests. I’d measure correctly resolved cases, unauthorized-action attempts, staff corrections, and cost per resolved case.
Adding more agents is not progress if staff spend more time fixing their mistakes.
Final thoughts
I’d keep the people, platform, and patterns framework, but sharpen what the AI CoE provides: clear ownership rules, shared platform controls, and reusable patterns for testing, approval, and recovery.
The CoE’s mandate should be to set common rules and ensure they are enforced across business units, especially for high-stakes or hard-to-reverse agent actions. Each business unit should name who is accountable for those actions and when human approval is required.
Business teams would still own their workflows and outcomes. The CoE’s job is to make it easier for every team to deploy agents with clear permissions and accountability, without rebuilding those safeguards from scratch.
♻️ If this was useful, share it with someone building with AI.
✉️ Subscribe at newsletter.karuparti.com so you never miss an edition.
P.S. Want more? 👋
1/ My visual guide to agentic AI → Gumroad
2/ Daily deep dives on agentic AI architecture → LinkedIn
References
Anurag Karuparti, Winning with AI CoEs in the Agentic Era, October 17, 2025. Original three-pillar framework.
Anurag Karuparti, Platform: The Foundation of an AI CoE, October 24, 2025. Shared platform, controls, and federated ownership; the automated-evaluation wording revisited above.
Anurag Karuparti, People: The Force Multiplier of the AI CoE, October 31, 2025. Responsibilities and operating model.
Anurag Karuparti, Patterns: Your Real Competitive Advantage in AI, November 7, 2025. Reuse, embedded controls, versioning, and retirement.
OWASP, LLM06:2025 Excessive Agency. Least privilege, human approval, and downstream authorization.
Anthropic, Building effective agents. Simplicity, environmental feedback, testing, human checkpoints, and stopping conditions. The refund operating contract is my proposed application, not a quoted standard.
Anthropic, Claude Certified Architect Professional course. Module 1: delegation through reversibility, stakes, and accountability. Module 2: evaluations as acceptance criteria. Module 3: risk-based review, calibrated confidence, and controls with owners and evidence. Module 4: scheduled reviews and operational response. Paraphrased from the course material; lesson access may require registration.
Anu Karuparti Creator, Diary of an AI Architect How enterprises actually ship AI to production
Read by 3,000+ FDEs, AI Architects, and Engineering Leaders from Microsoft, Google, IBM, PwC and others.
Connect with me on LinkedIn.
Want to partner? Email me at anurag.karuparti@gmail.com.