This is the point where I stop treating the agent as a conversation. The system has an unresolved action. A retry may create a duplicate appointment; moving on may leave the customer with nothing. Instructions such as “confirm first” and “finish the task” now need definitions the runtime can check.
I write those definitions as a behavioral contract: the actions the agent may take, the proof required before it claims success, and the conditions that make it stop.
Convert values into decisions
Instructions such as “Protect the user,” “Ask before consequential action,” and “Finish the task” are too broad to enforce. Each one must change a specific runtime choice.
For each instruction, ask what decision it must change:
- Which actions count as consequential?
- What evidence constitutes approval?
- Does approval cover one action or a sequence?
- What invalidates it?
- What happens when an action’s result is unknown?
- What evidence distinguishes completion from an attempt?
- When must the system stop?
The review produces a list of situations in which the instruction changes what the system does.
Write the behavioral contract
For every consequential action, define five things:
- Obligation: What must the system do?
- Prohibition: What must it not do?
- Authority: Who or what may permit the action?
- Evidence: What observation confirms the intended effect?
- Termination: When must the system finish, wait, hand off, or stop?
Consider an appointment agent. “Be helpful and confirm with the user” is intent. A behavioral contract is operational:
- The agent may search availability without approval.
- It may propose a time but may not book without authorization tied to that time and participant set.
- A submitted request is not a confirmed appointment.
- Completion requires a provider-issued confirmation.
- If the result is unknown, the agent must reconcile the first attempt before making another.
I do not try to encode every conversational choice. I write one reviewable contract for each class of action that can change the user’s situation or make recovery difficult.
Separate judgment from enforcement
Models can interpret ambiguous requests and propose actions that were never enumerated in advance. Keep that flexibility. Put critical checks in components with narrower responsibilities.
The model may propose sending an email. The system should verify that the recipient, content, and authorization match.
The model may infer that the user approved a purchase. The system should require an authorization record covering the item, amount, and scope.
The model may report that work is complete. The workflow should check the completion criteria against external evidence.
This allocation is practical. The model interprets the request. Narrower components verify the recipient, amount, authorization, completion evidence, and other consequential boundaries. A capability does not clear review if the model alone must enforce the instruction that defines one of those boundaries.
Enforce the contract across the trajectory
Agent behavior unfolds across observations, decisions, tool calls, external effects, retries, and handoffs. A final answer cannot show whether every transition was permitted.
A harmless closing message may follow an impermissible action. A correct action may occur without authorization. A timeout may trigger an unsafe retry. A handoff may be announced without responsibility transferring.
Before each consequential transition, the runtime should be able to answer:
- Are the required facts present and current?
- Is this action permitted in the present state?
- Does the authorization match its parameters?
- Has the action already been attempted?
- Is an earlier outcome unresolved?
- Has a time, cost, or action limit been reached?
- Must control now pass to a person or another system?
For each proposed transition, the runtime records why it allowed, rejected, or deferred the action. That record lets us inspect the whole run instead of inferring it from the closing message.
Make evidence part of correct behavior
The system also needs enough evidence to establish what happened.
The record must answer:
- What happened?
- What remains uncertain?
- What is safe to do next?
A timeout leaves the outcome unknown. A generated success message proves only what the model said. Recording proposals, attempts, and confirmed effects as different states makes retry and repair possible. Every completion claim needs an independently checkable condition.
Turn failures into contract changes
Production behavior will expose conditions the first contract missed. When that happens, I ask what the architecture allowed:
- Was an obligation ambiguous?
- Was a forbidden transition possible?
- Was authorization too broad?
- Did completion lack a verifiable condition?
- Could the workflow proceed while an outcome remained unresolved?
- Was there no safe stop or recovery path?
The answer determines the change: add a scenario, narrow the authority, introduce a state, require evidence, or create a handoff condition. An owner then verifies that the original failure blocks release.
What I check before release
I still use prompts for goals, context, and judgment. Before release, I also want explicit state, scoped authority, runtime checks, proof of external effects, and a repair path. Gaps will remain. The contract makes those gaps observable and gives the team a specific mechanism to change after a failure.