How Coinbase cut its AI spend by 50% with 5 smart strategies
Tokenomics is fast becoming a core skill for any organization that wants to justify its AI spend and stay ahead of competitors. Coinbase just showed us what that looks like in practice.
Coinbase cut its AI bill in half while token usage went up. Costs went down, consumption went up, and that combination is not luck. It is engineering.
On June 28, Brian Armstrong posted the exact playbook his team used. Five moves. Most teams are running none of them, which is why most teams watch their AI spend climb while their output barely moves.
I am an applied AI architect at Microsoft, and I have put production-grade agents inside Fortune 500 environments. So when I say Coinbase's playbook is smart, I mean it: it is exactly the kind of move the recent progress in open-weight models now makes possible.
The reflex answer most teams reach for is the wrong one. They renegotiate the contract, throttle usage, or wait for prices to drop. Coinbase did something different. They left the models alone and re-engineered the system around them. The bill fell by half and usage went up, because lower cost per task meant teams could afford to point AI at more tasks
This week’s edition breaks down those five moves, why each one works, and how they compound into a system instead of a list of tips.
Prefer to watch first? Here is the 80-second version:
The team bleeding tokens
Picture the head of engineering at a Series B fintech. Call her Anjali. Her org adopted an AI coding assistant and a handful of internal agents last year, and the productivity gains were real enough that adoption spread fast.
So did the bill. It tripled in a single quarter. The CFO started asking pointed questions, and the easy political move was on the table: cap everyone’s usage, or pull the most expensive tool entirely.
That move would have been a mistake, and Anjali knew it. The usage was creating value. The problem was not that her engineers were using AI too much. The problem was that every token was being spent at the worst possible price.
When her platform team finally looked under the hood, the pattern was almost comical. Routine code edits were being sent to the most expensive frontier model by default. The cache hit rate was close to zero, so the system re-paid for the same context on every single call. Agent sessions stayed open for hours, dragging a bloated context window into every request. And no engineer could see their own spend, so no one had any reason to care.
Anjali’s team did not have a model problem. They had a systems problem. And a systems problem has a systems fix.
The framework: spend on models versus engineer around them
There are two ways to respond to a high AI bill.
The reflex: Bigger bill -> pay more / throttle / wait for prices to drop -> bigger bill
The fix: Bigger bill -> treat the model as one component you control -> smaller bill, more usage
The reflex treats the model as a fixed cost you are at the mercy of. The fix treats the model as one part inside a system you design, where the default, routing, caching, context, and visibility are all knobs you own.
That reframe is the whole game. Once you stop seeing the model as the system and start seeing it as a component in the system, the cost stops being something that happens to you and becomes something you engineer.
The graph below, spend vs. token usage at Coinbase, proves it. So how did they pull it off?
The five moves are the knobs:
Change the default model: stop paying frontier prices for routine work
Build a routing layer: let the system pick the model, not the human
Fix your cache: the single biggest cost driver almost nobody looks at
Keep context lean: wasted tokens are a silent tax on every call
Make spend visible: align incentives instead of capping behavior
Let me go through each one.
Move 1: Change the default model
The first move is the one that sounds too simple to matter and matters more than anything else.
Coinbase switched its engineers to open-weight models like GLM 5.2 and Kimi 2.7 by default. The frontier model did not go away. It just stopped being the lazy default for every keystroke.
“Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task.” - Brian Armstrong
Here is why this works. The gap between open-weight models and the closed frontier has narrowed dramatically over the last two years. For a huge share of real engineering work, code completion, refactors, test generation, routine reasoning, the open-weight model is already good enough. You were paying a premium for capability you were not using.
The default is a pricing decision disguised as a quality decision. Nobody chose to send every routine prompt to the most expensive model. It was just the value that shipped in the box, and inertia did the rest. Flip the default, and the small fraction of work that genuinely needs the frontier can still reach for it on demand.
For Anjali’s team, this one change alone could move the needle more than any contract renegotiation could have, because it attacked the volume where the volume actually was.
Move 2: Build a routing layer
Changing the default gets you most of the way. A routing layer gets you the rest, and it does it without asking a single human to think about pricing.
The idea is to route every prompt by task difficulty and price, automatically. Hard, ambiguous, plan-this-whole-feature work goes to the frontier. Routine, well-scoped execution goes to the cheaper open-weight model. The system makes the call, in real time, on every request.
This is a known pattern with real research behind it. Model routing, formalized in work like RouteLLM, learns to predict whether a cheaper model will be good enough for a given prompt and only escalates when it will not. The RouteLLM authors reported cutting cost by more than two times in some cases without dropping response quality. Think of it as a load balancer, except instead of balancing for traffic, it balances for the cheapest model that can still do the job.
The architectural caution is the classifier. A router is only as good as its difficulty estimate, so you need a fallback path: if the cheap model fails or the confidence is low, escalate automatically and log it. The goal is not to be cheap on every prompt. The goal is to be exactly as expensive as each prompt requires, and no more.
The human picking the model is the bottleneck and the leak. Take the human out of the pricing decision and the system optimizes itself thousands of times a day.
Move 3: Fix your cache
This is the move that almost nobody looks at, and at Coinbase it was the single biggest cost driver.
Their cache hit rate went from 5% to 60%. Sit with that number, because it is enormous. On Anthropic’s published pricing, a cached input token is billed at one-tenth the price of a fresh one. OpenAI and Google pass through similar discounts. Going from 5% to 60% cache hits means the majority of your input tokens suddenly cost a fraction of what they did, on the exact same workload.
Prompt caching is a real, documented feature across Anthropic, OpenAI, and Google. The mechanism rewards a specific structure: put your stable content first, the system prompt, the tool definitions, the long reference docs, and put the volatile, request-specific content last. When the stable prefix does not change between calls, the provider serves it from cache instead of reprocessing it.
Most teams never structure prompts this way, so their cache hit rate sits near that 5% floor by accident. They are paying full freight to re-send the same system prompt and the same documentation on every call, thousands of times a day, and the line item never gets questioned because nobody is watching that number.
This is the highest-ROI move on the entire list. It requires no model change, no new vendor, and no behavior change from your engineers. It is pure architecture, and it is sitting unclaimed in almost every AI system I review.
Move 4: Keep context lean
If the cache is the token you should not re-pay for, context bloat is the token you should never have sent in the first place.
The discipline here is unglamorous and powerful. Start fresh sessions when you switch tasks instead of dragging an hour of irrelevant history into a new problem. Narrow the file scope so the agent sees the three files it needs, not the whole repository. Disconnect the tools you are not using, because every connected tool spends tokens describing itself in the context window.
Every token in the context window is paid for on every single call. A bloated window is not a one-time cost, it is a tax that compounds across every request in that session. The session that carries forty thousand tokens of stale context pays for those forty thousand tokens again and again until someone closes it.
This connects directly to my earlier edition on context engineering, where I argued that managing the context window is the core craft of building reliable agents. Here is the financial corollary: the same lean context that makes an agent more accurate also makes it dramatically cheaper. Precision and economy are the same discipline. You do not have to trade one for the other.
Wasted tokens are a silent tax precisely because they are invisible. Nobody sees the bloated window. They just see the bill, and they blame the model.
Move 5: Make spend visible
The last move is the one that holds the other four together, and it is not a technical move at all. It is an incentive move.
Coinbase did not impose hard caps. Instead, every engineer can see their own usage, and the message attached to it is explicit: more spend is fine, as long as it means more impact. Spend is not a sin to be punished. It is a resource to be deployed well, and visibility is what makes deploying it well possible.
Hard caps are the intuitive answer and the wrong one. A cap punishes the engineer doing high-leverage work that genuinely needs more tokens, while doing nothing to teach the engineer who is wasting them. Caps optimize for staying under a number, not for getting value per token.
Visibility flips the incentive. The same way you would not cap an engineer’s cloud spend but you would absolutely make it visible and tie it to outcomes, AI spend should be a number every engineer can see and feel ownership of. This is FinOps applied to tokens. When people can see what they spend and understand that spend is judged against impact, they self-correct without anyone writing a policy.
For Anjali’s team, this was the cultural unlock. The moment her engineers could see their own token spend on a dashboard, the wasteful patterns started disappearing on their own, because for the first time the cost was attached to a name instead of hidden in an aggregate invoice.
How the five moves compound
The reason this works as a system and not a checklist is that the moves hand off to each other. The default change shrinks the volume at the top. The router catches what the default misses. The cache discounts what survives. Lean context shrinks what gets cached and sent. Visibility keeps the whole thing honest.
Notice that no single move is the whole answer. A cheaper default with a five percent cache rate still leaks. A perfect cache behind a bloated context still overpays. Visibility with no routing just shows people a big number they cannot do anything about. The savings compound only when the moves run together, each one cutting cost at a different layer of the same request.
That is why teams who try just one of these are disappointed. They switch the default model, see a modest dip, and conclude the hype was overblown. They were missing the other four layers.
How I think about applying this
When I review an AI system’s cost now, I do not start by asking which model they are on or what they are paying per token. I ask one question first: at how many points in the request can you choose to spend less without spending worse?
If the answer is “the model picks itself, the cache is whatever it is, the context is whatever accumulated, and nobody sees their spend,” then I know exactly why the bill is what it is. There are five control points sitting unused, and no model negotiation will recover what those five points are leaking.
For Anjali’s team, the fix was not a cheaper contract. It was the control points they had never touched: a default flip, a router, a restructured prompt for cache reuse, a leaner context discipline, and a spend dashboard. The bill came down by more than half within two months, and adoption actually went up, because once each task got cheaper, AI became affordable for work that was previously too expensive to automate. This is the same lesson I kept hitting in my edition on LLM cost optimization: the cheapest token is the one you engineered your way out of sending.
The teams that win the next two years on cost are not the ones who found the cheapest model. Everyone has access to the same models. The winners are the ones who engineered the system around the models, because the model is one line item and the system is the other ten.
The bottom line
Coinbase cut its AI bill in half while usage went up, and they did it without a single model breakthrough. They changed the default model, built a routing layer, fixed their cache, kept context lean, and made spend visible. Five moves, all architecture and discipline, none of them a new model.
Before you negotiate your next AI contract or throttle your team, audit those five control points. The savings are almost never in finding a cheaper model. They are in engineering the system around the model you already have.
Thank you for reading my newsletter!
Anu Karuparti
Creator, Diary of an AI Architect
How enterprises actually ship AI to production
Read by 3,000+ FDEs, AI Architects, and Engineering Leaders from Microsoft, Google, IBM, PwC and others.
Connect with me on LinkedIn.
Want to partner? Email me at anurag.karuparti@gmail.com.
Disclaimer: The Coinbase figures in this article are drawn from a public post by Brian Armstrong and are attributed as such. The story of Anjali and her team is hypothetical, inspired by patterns observed across similar real-world experiences. It is used to convey key concepts more effectively and does not represent any specific individual or organization.
P.S. Want more? 👋
1/ My visual guide to agentic AI → Gumroad
2/ Daily deep dives on agentic AI architecture → LinkedIn
3/ Real-time takes on breaking AI news → X
4/ Casual hot takes and community → Threads
5/ Visual frameworks and carousels → Instagram
6/ 60-second production lessons → TikTok
7/ The full newsletter, free → newsletter.karuparti.com