Welcome back to Diary of an AI Architect. Today, we are stepping outside our usual solo deep dives to bring you a guest essay by Rohan Kodialam, co-founder and CEO of Sphinx AI, a company building a knowledge and reliability layer for enterprise AI. Before Sphinx, Rohan was an AI research leader at Citadel.
This week in my diary, I’m returning to a question behind my earlier posts on context engineering and decision quality and the three-layer context stack: what happens to that investment when you change models? Rohan brings a founder’s perspective on business context to an uncomfortable argument: access to another model is not the same as the ability to move your business knowledge to it. Read this with your own model-switching plan in mind. Below are his words.
Enterprise AI teams like to talk about model optionality. The idea is straightforward: don’t become dependent on OpenAI, Anthropic, Google, or any single provider. Build your stack so you can use whichever model offers the best combination of intelligence, cost, latency, security, and reliability. The logic is increasingly difficult to argue with. Menlo Ventures estimates that Anthropic captured 40% of enterprise LLM spending in 2025, up from 24% a year earlier, while OpenAI accounted for 27% and Google reached 21%. In a market moving this quickly, betting your enterprise architecture on one permanent winner makes little sense.
But there is a problem with how we think about model optionality. Changing which API processes a request is relatively easy. Moving everything your AI system has learned about your business is much harder. An enterprise AI system needs to know far more than what exists in a database or document repository.
It needs to understand that finance and sales may define the same metric differently, that one system should take precedence when two sources disagree, that a particular field has an unusual historical meaning, or that a workflow contains exceptions understood by employees but poorly documented anywhere else.
If that knowledge becomes embedded in model specific prompts, fine tuning, memory systems, or application logic, the enterprise may technically be able to switch models while remaining practically locked into the system built around the old one.
That matters because the future of enterprise AI will almost certainly be multi model. IDC argues that enterprises should design around portfolios of models rather than a single stack, continuously choosing among models based on performance, cost, latency, and the use case.
Gartner has similarly warned that relying on a single AI model is a common enterprise pitfall and recommends composite approaches that combine multiple models and techniques. One model may be best for complex reasoning, another for coding, another for high volume repetitive tasks, and a smaller or open weight model may make more sense when privacy, cost, or control is paramount.
The architecture therefore needs a clean separation between what is replaceable and what is durable. Models are replaceable. A company’s institutional knowledge is not. Definitions, relationships, business rules, permissions, historical decisions, and the knowledge required to interpret internal data should belong to the enterprise rather than to whichever model happens to be reasoning over them.
Think of the model as an intelligence engine. You should be able to replace that engine without rebuilding everything the system knows about the organization around it. When a new model appears, the question should be, “How well does this model perform using our existing business context?” rather than, “How do we teach another model how our company works?”
There is an economic argument for this architecture as well. Enterprise generative AI spending reached an estimated $37 billion in 2025, more than triple the $11.5 billion spent in 2024. As that number grows, companies will increasingly care about routing workloads intelligently. Using the most capable frontier model for every request may make little sense if a smaller model can handle repetitive tasks at a fraction of the cost.
Optionality also creates negotiating leverage and resilience. Providers can change prices, experience outages, alter policies, or fall behind competitors. The ability to move meaningful workloads elsewhere is valuable only if switching does not require reconstructing the institutional knowledge those workloads depend on.
There is also a broader constraint that enterprise architecture discussions often overlook: model choice is not purely an individual company decision. At global scale, choice is constrained by where compute capacity exists and which models can actually serve enormous volumes of production demand. An open model may look compelling on benchmarks, but the entire market cannot necessarily migrate to it tomorrow. The combination of rapidly changing model quality, economics, and available compute makes predicting the dominant provider several years from now nearly impossible.
That is why enterprises should architect for a model market they cannot predict. The goal is not simply to avoid committing to one API. It is to ensure the most valuable part of the AI stack, the accumulated understanding of how your business actually works, remains yours regardless of which model comes next. True model optionality means being able to change the intelligence engine without forcing the new engine to relearn your business.
Connect with Rohan on LinkedIn.
From my diary: test the portability, not just the API
Editor’s closing note, Anurag Karuparti
My earlier context-engineering editions asked what an agent needs to know and how we govern the data that supplies it. Rohan adds a useful ownership test: can that context survive the next model choice?
Here is the exercise I would take into the next architecture review. Pick one workflow and identify:
Meaning: Where do we record what metrics mean, which sources to trust, and any exceptions?
Control: Can we move that knowledge, track changes, and keep the same access rules?
Evidence: Can another model pass the same tests, including handling conflicting sources and blocking unauthorized access?
Portable context does not guarantee identical behavior. Test the replacement model’s tool calls, outputs, and permission boundaries before moving a production workload. Keep the knowledge reusable; earn confidence in each new engine.
♻️ If this was useful, share it with someone building with AI.
✉️ Subscribe at newsletter.karuparti.com so you never miss an edition.
Anu Karuparti Creator, Diary of an AI Architect How enterprises actually ship AI to production
Read by 3,000+ FDEs, AI Architects, and Engineering Leaders from Microsoft, Google, IBM, PwC and others.
Connect with me on LinkedIn.
Want to partner? Email me at anurag.karuparti@gmail.com.
P.S. Want more? 👋
1/ My visual guide to agentic AI → Gumroad
2/ Daily deep dives on agentic AI architecture → LinkedIn
3/ Visual frameworks and carousels → Instagram
4/ 60-second production lessons → TikTok
References
Menlo Ventures, 2025: The State of Generative AI in the Enterprise, December 9, 2025. Source for the guest’s spending and provider-share estimates. These are Menlo’s 2025 estimates, not current September 2026 market shares. The study surveyed 495 U.S. enterprise AI decision-makers; share estimates approximate spending from production API usage. The $37 billion estimate excludes chips, inference/model serving, and AI features embedded in existing software. Its $11.5 billion 2024 comparator is restated on the same inference-excluding basis.
Tim Law and Zhenshan Zhong, IDC, Beyond LLMs: Why AI Strategy Now Requires Multi-Model, Multimodal, and Multi-Agent Architectures, April 13, 2026. Supports model portfolios, use-case matching, and cost/performance/latency evaluation.
Ben Yan, Pieter den Hamer, Deepak Seth, and Joe Antelmi, Gartner, Go Beyond a Single Model and Use Composite AI to Enhance Enterprise AI Adoption, July 2, 2025. The public abstract supports the guest’s attribution; the full research requires access. Composite AI includes different techniques, not only switching LLM providers.
Sphinx, Sphinx launches with $9.5M to redefine how AI works with data, September 9, 2025, and the company’s current product description. Sources for the editor’s short bio: co-founder/CEO, prior Citadel AI research role, and company focus. Product positioning is the company’s description, not an independent performance endorsement.
Anurag Karuparti, How context engineering can affect your organization’s decision quality, March 6, 2026. Earlier discussion of instructions, memory, retrieval, tools, and structured outputs as components of enterprise context.
Anurag Karuparti, How to build a strong enterprise AI moat with context engineering and data estate, April 24, 2026. The three-layer context stack: unified data foundation, retrieval architecture, and governance/documentation. Public preview available; full edition is subscriber-restricted.
