The next key AI skill will not be writing better prompts. It will be graph engineering.
Graph engineering determines when agents work sequentially or concurrently, where evaluators and humans enter the loop, and how routing and handoffs lead to a final business outcome.
This week in my diary, I am breaking that system into five control layers: prompt, context, harness, loop, and graph.
I have helped ship more than 50 production generative AI systems. The teams that struggle are rarely blocked by the wording of one prompt. They are struggling with what reaches the model, what the model can touch, how the agent recovers, and how the work moves across a larger system.
The graph layer in this framework was sharpened by Greg Isenberg’s recent explanation of graph engineering. His video is a useful companion if you want the outermost layer in plain English.
Here is the mental model I now use with enterprise teams: prompts steer, context informs, harnesses constrain, loops persist, and graphs coordinate.
My YouTube short explains the key differences between these five layers in 90 seconds.
Priya’s agent passed the demo and failed the system
Imagine Priya, a staff AI architect at a regional bank. Her team builds a coding agent to fix low-risk defects in an internal payments service.
The demo looks excellent. The prompt tells the agent to inspect the issue, avoid unrelated changes, run the tests, and return a pull request summary. On a clean sample repository, it finds the bug and produces a reasonable patch.
Then the pilot reaches a real repository. The agent misses an architecture decision buried in the docs, selects a shell command with a broader scope than intended, retries the same failing test without changing its hypothesis, and sends the pull request to the wrong review path.
The team asks, “How should we improve the prompt?” Priya asks a better question: which control layer failed? The instruction was only one part of the system, and it was not the part that caused most of the damage.
The framework: one model call versus a controlled system
Think of the five concerns as control layers around the model:
MODEL CALL = prompt + context
AGENT = model call + harness + loop
SYSTEM = agents + deterministic steps + humans connected by a graph
EVALS = evidence that every layer works
Each layer answers a different architecture question.
Prompt and context operate closest to the model. Harness and loop turn the model into an agent. Graph engineering turns agents, functions, and human decisions into a system.
Layer 1: prompt engineering
Prompt engineering controls the instruction: role, goal, constraints, examples, output contract, and success criteria.
Priya’s coding agent might receive this:
Analyze the reported defect and propose the smallest safe fix.
Do not change unrelated behavior.
Return the root cause, files changed, test evidence, and residual risk.
Stop and ask for approval if the fix changes an external contract.
The unit being optimized is one model interaction. A stronger prompt reduces ambiguity and makes the expected output inspectable, but it cannot supply a missing design document, restrict a dangerous tool, or decide which reviewer receives the result.
Prompts still matter. In an agent system, the prompt is the steering wheel, not the car.
Layer 2: context engineering
Context engineering controls what reaches the context window: retrieved documents, conversation history, tool results, code files, policies, summaries, and the information deliberately excluded.
Ask the same model to summarize the risks in an 80-page contract.
Setup A dumps the document into the window with no retrieval plan.
Setup B retrieves the liability, indemnification, termination, and data-use clauses, includes the company’s risk policy, preserves linked definitions, and allows the agent to pull the full text when a cross-reference matters.
The prompt did not change. The evidence available to answer it did.
Setup B is designed to produce a more useful risk review, provided the retrieval layer is tested for omissions.
This connects to my earlier edition, “Context engineering is about to become your organization’s biggest moat.” A brilliant instruction with a poor working set still produces a poor decision.
For Priya, the missing architecture decision is a context failure. Rewording the prompt may hide the symptom for one test case, but fixing retrieval and context assembly repairs the class of failure.
Layer 3: harness engineering
I use harness engineering as shorthand for the runtime envelope around the model: tools, file access, shell access, MCP connections, sandboxing, permissions, logging, approval boundaries, timeouts, and error surfaces.
The model decides, “I need to run the tests.” The harness decides whether test execution is available, which commands are allowed, which directory is visible, how long the process can run, what output returns, and what gets recorded.
MCP can standardize how an AI application connects to external tools, resources, and prompts. It does not decide that an agent deserves production write access. Identity, authorization, least privilege, and approval policy still belong in the host and its surrounding control plane.
This is where capability meets control. In regulated environments, this is often the layer the security team cares about first.
For Priya, the broad shell command is a harness failure. The fix is not “please be careful” in the prompt. The fix is a narrower tool, an allowlist, a sandbox, and an approval gate for commands with a larger blast radius.
Learn how to implement Agent Harness with Microsoft Agent Framework here.
Layer 4: loop engineering
Loop engineering controls the cycle: act, observe, evaluate, adjust, and repeat. It owns retry policy, validators, completion criteria, budgets, stop conditions, and escalation.
The distinction from the harness matters:
Harness: Can the agent execute the test, in which sandbox, with what permission and timeout?
Loop: Does a failed test trigger another attempt, what must change before retrying, how many attempts are allowed, and what counts as done?
You can have a good harness and a terrible loop. An agent can be perfectly sandboxed, fully logged, and still repeat the same failed fix until it exhausts its budget.
This is the layer I explored in my earlier edition on loop engineering. Priya’s repeated test failure needs a new hypothesis requirement, a retry cap, and an escalation rule, not broader filesystem access.
Layer 5: graph engineering
Graph engineering controls the topology of the whole workflow. Nodes may be agents, deterministic functions, data transformations, evaluators, or human gates. Edges define sequencing, routing, parallel branches, recovery paths, and where loops live.
Loop asks, “How should this agent keep working?” Graph asks, “Which component works next, and how does the system coordinate?”
LangGraph describes itself as a low-level orchestration runtime for long-running, stateful workflows. Its useful design idea is not simply drawing boxes. It is mixing deterministic steps with model-driven steps while preserving state, durable execution, and human interrupts.
Learn how to build agent graphs with Microsoft Agent Framework here.
Greg Isenberg’s graph engineering framing makes this layer easy to see: stop asking one chat to plan, research, write, and approve its own work. Split responsibilities, draw the real dependencies, and keep the human where consequences become expensive.
The evidence also argues against graph maximalism. Anthropic reported that its multi-agent research system outperformed a single-agent setup by 90.2% on an internal breadth-first research evaluation, but its multi-agent runs used about 15 times as many tokens as chat interactions. That result is specific to Anthropic’s research workload, not a universal multiplier, and it captures the trade-off perfectly: graphs earn their complexity only when the task value and parallelism justify the bill.
For Priya, the wrong review path is a graph failure. The system needs an explicit route from code changes to tests, security review, contract review when needed, and human approval before merge.
All five layers at once
The cleanest way to understand the model is one coding workflow using every layer simultaneously.
Prompt: “Fix this defect without changing unrelated behavior.”
Context: Issue, repository files, architecture decisions, test failures, ownership metadata, and relevant git history.
Harness: Scoped terminal, filesystem tools, git operations, sandbox, permissions, secrets isolation, and traces.
Loop: Inspect, form a hypothesis, modify, test, diagnose, retest, and stop or escalate against explicit criteria.
Graph: Triage, planner, coding agent, deterministic tests, security reviewer, human approval, and merge.
These layers do not replace one another. A production agent uses all five at the same time, and the weakest control often sets the reliability ceiling for the whole system.
Three honest caveats
I want you to use this framework, but I also want you to use it accurately.
The terms are not equally mature. Prompt engineering and context engineering are established industry language. Agent harness is a real category, while loop engineering and graph engineering are newer labels for concerns practitioners also call agent loops, workflows, and orchestration.
The boundaries leak. Memory can belong to context, harness, or runtime state. Verification can sit inside a tool boundary, a retry loop, or a separate graph node. These are five concerns, not five perfectly isolated software components.
The layers are not a build order. I often sketch the graph first (workflow), define the loops and control boundaries next, then tune prompts last. Think concentric controls around the model, not a waterfall pipeline.
The value of the framework is diagnostic clarity. It gives the team a better answer than “the AI was weird.”
The concern that spans all five: evals
There is a sixth concern that cuts across every layer: evals.
How do you know the prompt follows instructions? Did context retrieval include the decisive evidence? Did the harness allow the right tool and deny the unsafe one? Did the loop stop for the right reason? Did the graph route the risky case to a human?
Measure each layer with evidence that matches its job:
OpenAI’s evaluation guidance starts with the same discipline: define the desired behavior, run representative test inputs against explicit criteria, analyze the results, and iterate. Evals are not a sixth ring outside the architecture. They are the measurement system running through all five.
The bottom line
If your team is still treating prompt engineering as the whole job, you are staffing for one fifth of the architecture. Prompts steer, context informs, harnesses constrain, loops persist, and graphs coordinate.
Build all five. Measure all five. Then ask your team which layer is weakest before you rewrite another prompt.
♻️ If this was useful, share it with someone building with AI.
✉️ Subscribe at newsletter.karuparti.com so you never miss an edition.
Thank you for reading my newsletter and supporting my work. I truly appreciate it.
Anu Karuparti Creator, Diary of an AI Architect How enterprises actually ship AI to production
Read by 3,000+ FDEs, AI Architects, and Engineering Leaders from Microsoft, Google, IBM, PwC and others.
Connect with me on LinkedIn.
Want to partner? Email me at anurag.karuparti@gmail.com.
P.S. Want more? 👋
1/ My visual guide to agentic AI → Gumroad
2/ Daily deep dives on agentic AI architecture → LinkedIn
3/ Real-time takes on breaking AI news → X
4/ Casual hot takes and community → Threads
5/ Visual frameworks and carousels → Instagram
6/ 60-second production lessons → TikTok
7/ The full newsletter, free → newsletter.karuparti.com
References
Greg Isenberg. “Why Graph Engineering will 10x your Claude/Codex.” YouTube. Source for the graph-engineering framing and the shift from one overloaded chat to designed handoffs.
Anthropic. “Building effective agents.” Definitions of workflows and agents, the augmented LLM, agent loops, parallelization, evaluator-optimizer, human checkpoints, and the recommendation to add complexity only when it improves outcomes. https://www.anthropic.com/engineering/building-effective-agents
Anthropic. “How we built our multi-agent research system.” Source for the 90.2% internal research-eval result, approximately 15x token-use figure, orchestration pattern, evaluation lessons, and production reliability trade-offs. https://www.anthropic.com/engineering/multi-agent-research-system
LangChain. “Runtimes, frameworks, and harnesses.” Current distinctions among agent frameworks, runtimes, and agent harnesses. https://docs.langchain.com/oss/python/concepts/products
LangChain. “LangGraph overview.” Low-level orchestration, deterministic and agentic steps, persistence, durable execution, and human-in-the-loop controls. https://docs.langchain.com/oss/python/langgraph/overview
Model Context Protocol. “What is MCP?” MCP as an open standard for connecting AI applications to external data, tools, and workflows. https://modelcontextprotocol.io/docs/getting-started/intro
Model Context Protocol. “Architecture overview.” Host-client-server responsibilities, tools and resources, transports, and authorization boundaries. https://modelcontextprotocol.io/docs/2026-07-28/learn/architecture
OpenAI. “Working with evals.” Defining desired behavior, representative test data, explicit grading criteria, and iterative analysis. https://developers.openai.com/api/docs/guides/evals
Disclaimer: The stories and scenarios in this article are hypothetical, inspired by patterns observed across similar real-world experiences. They are used to convey key concepts more effectively and do not represent any specific individual or organization.