This week, I watched an incredible All-In interview with Satya Nadella. I highly recommend making time for it. What stayed with me was his starting point for safe AI: it should serve humanity and remain under human control.
He also emphasized stronger engineering practices and independent testing, making safety something we build into systems, not just talk about.
Separately, I’ve been digging into Jev, TypeSafe AI’s new model, now in early access.
It’s built for bounded semantic decisions: choosing a category, scoring against a rubric, or assessing whether a statement holds, rather than generating paragraphs.
I can see the developer appeal: typed outputs, uncertainty signals, questions evaluated in parallel, and decision logic you compose in code.
That doesn’t make every judgment correct, but it gives us an interesting way to build around uncertainty.
I’m researching where it could genuinely help in production. More on Jev soon. Stay tuned.
This week in my diary, though, I’m exploring a related question: are we paying for useful decisions, or just more tokens?
Let’s get into 10 ways to move from token maxing to value maxing and build smarter AI workflows
Token-maxxing asks: How do I throw the most intelligent model, the most context, and the most tokens at the problem?
Value-maxxing asks: How do I use the right model, context, tools, and compute to produce a correct, useful, and trustworthy outcome at the lowest total cost?
That distinction matters.
A frontier model is wasteful if a smaller model produces the same outcome.
A cheaper model is not cheaper if it causes three retries.
More context is not better if most of it is irrelevant.
More reasoning is wasted compute if the task is straightforward.
An autonomous agent is not productive if it creates more human cleanup.
The real optimization target is not tokens used or tokens saved.
It is total cost per successful task.
If you learn better visually, check out my Visual AI Library on Gumroad. I use these cheat sheets myself before customer meetings to refresh key concepts and explain them more clearly. There is a lot of noise in AI. This library gives you the distilled version, so you can quickly review important concepts before interviews, exams, presentations, or customer conversations.
1. Spray and pray → Intent first
Do not default every workload to the strongest model.
Match the model to the task. Use a mid-tier model option that passes your quality checks, and reserve stronger models for work that genuinely needs them.
This is the first step from model-centric optimization toward workload-centric optimization.
Architecture takeaway: Define acceptable errors and rollback conditions before comparing models. A good average can hide failures in one ticket category, so include edge cases rather than grading only easy requests.
Example: I’d start with Claude Sonnet on labeled billing, delivery, and technical tickets, then test Haiku on the same set. I’d switch only if Haiku meets the quality bar in each category at lower total cost. Unclear cases go to a person, not whichever model sounds most confident.
2. No tools → Scoped tools
Do not spend tokens asking a model to reason about information it does not have.
When a task needs live information, more reasoning cannot replace access to it. A Model Context Protocol, MCP, server can expose the specific tools the job needs, with permissions enforced outside the prompt.
This moves optimization from better prompting toward better system design.
Architecture takeaway: Split responsibilities. The model interprets the request. The system of record supplies current facts. Application code enforces access. Do not ask the model to perform all three roles.
Example: For “Where is order #1042?”, I’d expose a read-only order-status tool. The backend verifies that the signed-in customer owns the order before returning its carrier, tracking link, and latest status. No refund or payroll tools. A lookup timeout means “I couldn’t check,” not “your order doesn’t exist.”
3. Connector sprawl → One governed gateway
As your agent gains capabilities, integration architecture starts mattering as much as model architecture.
For a growing integration estate, a shared gateway can centralize authentication, policy checks, observability, and audit records.
MCP alone does not provide that governance, and adding a gateway does not automatically save tokens. The value comes from reducing duplication, inconsistent controls, and operational complexity.
Architecture takeaway: Centralize the integration, not everyone’s privileges. Shared infrastructure also concentrates failures, so name an operational owner and decide what stops safely when the gateway is unavailable.
Example: I’d put customer-record lookup and shipment lookup behind one gateway. A support rep can read assigned customer records. A sales rep cannot inherit that access just by using the same endpoint. Check identity and permissions per action, retain scoped downstream credentials, and run redundant gateway instances, not one shared admin login.
4. Dumping everything → Scoped, not starved
More context is not always better context.
Every token should earn its place, but aggressive trimming can make a system worse if you remove information that changes the answer.
The goal is contextual precision.
Architecture takeaway: Cut irrelevant history, not facts that could change the decision. Preserve exact identifiers, amounts, applicable policy versions, and where the evidence came from when shortening context.
Example: For a refund on order #1042, I’d send the purchase date, $79 paid, the damage report, and the applicable 30-day refund policy with its effective date and damaged-item exception. Leave out unrelated chats and card details.
5. Agentic by default → Match the pattern
Autonomy has a cost.
Every additional agent step can introduce tokens, latency, tool calls, retries, and new failure modes.
Use a deterministic workflow when the steps are known. Use an agentic loop when the next step genuinely depends on what the system discovers.
Architecture takeaway: Define “done” and “stuck” in observable terms before allowing a loop. A call limit bounds the work. A useful handoff preserves findings, attempted actions, and what remains unresolved.
Example: For invoices, I’d extract supplier, amount, and purchase-order number, validate them in code, then route exceptions to accounts payable. For a suspected duplicate charge, an agent could compare invoices and payment records until the discrepancy is verified or progress stalls. I’d also cap it at five tool calls, then hand off unresolved findings to a person.
6. Fix it later → Read, plan, act
Value-maxxing also means reducing the cost of mistakes.
For consequential changes, review a short action plan before execution. Plan approval is not a substitute for checking the result.
The more expensive or irreversible the action, the stronger the checkpoint should be.
Architecture takeaway: Place checkpoints where mistakes are costly or hard to reverse. Give the reviewer the supporting evidence and proposed changes, not just an “Approve” button and a confident summary.
Example: I’d show a preview for 20 CRM contacts: record ID, old value, proposed value. A person approves only that change set. The application rechecks permissions and record versions before writing, verifies saved values afterward, and requests fresh approval if the records or scope changed.
7. Tool bloat → Load on demand
Tools consume context too.
If every tool definition enters every prompt, your system pays for capabilities that may never be used.
Expose a small task-specific set, or use tool search to load definitions when discovered, before calling them.
Architecture takeaway: Context management and permission management are different jobs. Discovering a tool makes its instructions available. It does not authorize execution. Measure discovery overhead before adding another step.
Example: I’d give a meeting assistant calendar lookup and availability tools first. When someone asks, “Turn that action item into a ticket,” discover and load the ticket-creation definition, then check permissions and confirm the write before calling it. If those are the only three tools, I’d skip search and load them up front.
8. No visibility → Full transparency
You cannot value-max a system you cannot measure.
Make available permissions and actual tool calls visible. Track model usage, latency, retries, failures, human escalations, and successful outcomes without turning your logs into a copy of sensitive customer data.
The unit of optimization should move from cost per token to cost per completed outcome.
Architecture takeaway: Monitoring needs a response, not just a dashboard. For each important signal, define the trigger, the person responsible, and the corrective action, including quality declines that do not produce errors.
Example: Suppose a trace shows six repeated order lookups after a timeout. I’d return a typed timeout error marked retryable, allow one retry, then hand off if it fails again. Log tokens, latency, retries, and outcomes with redacted arguments and restricted access. Compare cost per resolved ticket and quality, not hidden chain-of-thought.
9. Wasteful outputs → Right output
Generation itself can create downstream work.
Do not ask the model for a five-paragraph explanation when another system needs three fields.
Specify what the next person or system actually needs. A usable answer beats a long answer that needs another prompt to summarize, extract, or reformat it.
Architecture takeaway: Separate format checks from meaning checks. Code can reject missing fields or invalid dates. Checking whether a decision was actually agreed requires comparison with the source, sometimes by a person.
Example: I’d ask for one row per meeting decision: decision, owner, due date. If nobody committed to a deadline, write “not agreed” rather than invent Friday. For software, require those fields in a validated JSON schema, allow null for missing values, and check each row against the transcript. Valid JSON is not proof of truth.
10. Compute as a crutch → Targeted reasoning
The most reasoning is not always the best reasoning.
Spend additional compute where it materially improves the outcome. Where your model supports configurable effort, test lower settings for straightforward tasks instead of defaulting to maximum.
This is where token-maxxing and value-maxxing converge.
You reduce compute when it adds little value, and spend more when the additional reasoning prevents expensive mistakes.
Architecture takeaway: Diagnose the gap before buying more reasoning. Missing facts need retrieval. Unclear instructions need clarification. Extra computation earns its place only when evaluation shows it improves the task’s outcome.
Example: On a Sonnet version that supports effort controls, I’d test low effort for “Turn this verified shipment record into a two-sentence update.” I’d test high effort for “Compare these two migration plans for downtime and rollback risk.” Keep each setting only if representative tests meet the quality bar.
Final thoughts…
Value-maxxing means optimizing the entire workflow, not one model call.
Measure total cost per successful task, including model usage, tools, retries, latency, human review, and rework.
Pick one recurring workflow, establish a baseline, change one variable, and measure the impact.
The goal is simple: spend the right amount of compute and effort to produce a result you can trust.
♻️ If this was useful, share it with someone building with AI.
✉️ Subscribe at newsletter.karuparti.com so you never miss an edition.
Anu Karuparti Creator, Diary of an AI Architect How enterprises actually ship AI to production
Read by 3,000+ FDEs, AI Architects, and Engineering Leaders from Microsoft, Google, IBM, PwC and others.
Connect with me on LinkedIn.
Want to partner? Email me at anurag.karuparti@gmail.com.
P.S. Want more? 👋
1/ My visual guide to agentic AI → Gumroad
2/ Daily deep dives on agentic AI architecture → LinkedIn
3/ Real-time takes on breaking AI news → X
4/ Casual hot takes and community → Threads
5/ Visual frameworks and carousels → Instagram
6/ 60-second production lessons → TikTok
7/ The full newsletter, free → newsletter.karuparti.com