Insight / Blog

AI Agent Series · 2/6

What AI Agents need before they can execute real work

Once you stop thinking of an AI Agent as an answer generator, a more practical question appears.

What has to exist outside the model before the agent can act safely in a real work environment?

Episode 1 of this series focused on the first design question: who decides the next action? An agent differs from a fixed workflow because the language model can make part of that decision as the work unfolds.

But deciding what to do next is not the same as executing it reliably.

Consider a customer asking an AI Agent:

“Can you cancel last week’s order and tell me whether I qualify for a refund?”

A polished answer is not enough.

The agent has to find the order, check the refund policy, inspect the shipping status, confirm the user’s identity and permissions, and decide whether approval is required. If a system call fails halfway through, it also needs to know what completed, what did not, and where the work can safely resume.

The customer does not want a plausible paragraph. They want the work handled correctly.

That requires more than a prompt. It requires a harness.

What Comes After Prompts and Context

For a long time, the central question in applied AI was:

What should we say to the model?

Prompt structure, examples, instructions, and reasoning guidance all affect output quality. The same model can produce very different results depending on how the request is framed.

Then the question expanded:

What information should the model have before it decides?

That includes retrieving internal documents, loading customer data, preserving conversation history, and condensing long material into usable context. Technologies such as retrieval-augmented generation and memory systems belong here, but the business idea is straightforward: give the model the information it needs to make a better decision.

Agentic systems introduce a third question:

How does that decision become controlled execution in the real world?

The distinction is useful:

• A prompt defines how we ask the model to work.
• Context provides the information the model should consider.
• A harness manages how the agent uses tools, operates within permissions, preserves state, handles failures, and records what happened.

Prompting still matters. Context still matters. Their role has not disappeared.

But once the model can trigger actions, the design focus has to widen from answer quality to execution conditions.

Which tools may the agent call? What data may it access? Which actions need approval? What happens when a tool is unavailable? Can the system explain what it already changed?

Those questions live outside the prompt.

Agent = Language Model + Harness

A practical way to describe the operating structure is:

Agent = language model + harness

This is not a formal scientific equation. It is a design reminder.

The language model interprets the request, reasons about the situation, forms a plan, and decides what should happen next.

The harness turns that decision into bounded execution.

It exposes the tools the model can use. It checks permissions before sensitive actions. It carries session state from one step to the next. It records tool calls and results. It provides recovery paths when execution fails. It also supplies the traces and outcomes needed for evaluation.

In a refund workflow, the model might decide to retrieve an order and inspect the return policy. The harness is responsible for connecting to the order system, authenticating the request, restricting access to the correct customer record, returning a structured result, and requiring approval before an irreversible action.

This distinction matters because model quality and operational quality are not the same thing.

A model may produce an excellent plan and still fail in practice because the tool description is vague, the permission boundary is missing, the session state is lost, or the recovery path is undefined.

When teams evaluate only the model, they can miss the system that determines whether the work is actually repeatable.

That is why harness engineering is not merely an implementation detail for developers. It shapes what the organization can responsibly delegate to an AI Agent.

A Harness Is Designed for Failure, Not Just Success

It is tempting to think of a harness as a way to connect more tools.

Tool connectivity is only the beginning.

Real operations are defined by exceptions. APIs time out. Permissions change. Users make ambiguous requests. Business systems return incomplete data. The model chooses the wrong order of operations. A task that works perfectly in a demo may fail at the third step in production.

The goal is not to make the agent pretend that failure never happens.

The goal is to make failure visible and manageable.

A production-ready execution structure should help the system answer questions such as:

• Where did the task fail?
• Which steps completed successfully?
• Did the agent change any external state?
• Is it safe to retry?
• Should the system stop and ask the user?
• Can the task resume from a checkpoint rather than start over?

Without execution logs, teams cannot locate the failure. Without permission checks, they cannot reliably block risky actions. Without recovery rules, every interruption can force the workflow back to the beginning. Without evaluation, they cannot tell whether a new version improved one task while damaging another.

Harness design is therefore less about giving an agent more freedom and more about defining the conditions under which autonomy is allowed.

A good harness makes agent execution more traceable, repeatable, and recoverable.

It does not guarantee that the agent will never fail. It gives the organization a way to understand and manage failure when it occurs.

Separate Reasoning, Execution, and Memory

One of the most useful design moves is to separate three operating roles:

Reasoning: understand the request, decide what to do, and choose the next tool.

Execution: call external tools and business systems within defined permissions.

Memory: preserve session state, prior actions, and the context needed to continue the work.

These roles are sometimes described as Brain, Hands, and Memory. The labels are less important than the separation.

When all three are treated as one opaque system, failure becomes hard to diagnose.

Did the model choose the wrong action? Did the tool call fail? Was the user missing permission? Did the system lose the result from the previous step?

If teams cannot distinguish those possibilities, they cannot improve the right layer.

Separation makes operations easier to inspect and change. A tool can be replaced without rewriting the reasoning logic. A failed run can resume from recorded state. Execution traces can become evidence for the next evaluation cycle.

This changes the adoption question.

The standard is no longer whether the agent can produce one impressive result. It is whether the system can execute the same class of work repeatedly, explain what happened, and recover when conditions change.

Draw the Harness Before You Deploy the Agent

Before an AI Agent is given real work, teams should be able to answer a concrete set of operating questions:

• Which tools can the agent use?
• Which business systems can it access?
• What information is passed to the language model?
• Which actions can run automatically?
• Which actions require user approval?
• What is the stop condition?
• Where is session state stored?
• What is written to the execution log?
• Where does recovery begin after a failure?
• How will the team know whether an update improved the system?

This is not only a technical checklist. It is a delegation checklist.

If these questions have no clear answer, the organization may still be relying on model output rather than operating an agent as a dependable work system.

Prompt engineering has not become less important. The scope of the design problem has become larger.

After the prompt comes the harness.

Can your team draw the system around the model on one page? Can you show how tools, permissions, state, logs, recovery, and evaluation connect? Can you point to the exact moment when the agent must stop and return control to a person?

If you can, you are beginning to design the agent as an operating structure for real work rather than a compelling demo.

If you cannot, there are still important questions to answer.

How does this AI Agent execute inside the real business, where does it stop, and how does the system learn from failure?

That leads directly to the next article.

Once a harness is in place, teams need to know whether it is getting better. Episode 3 will examine what agent evaluation should measure, where failures occur, and why answer quality alone is no longer enough.

AI Agent Series