Do not use an AI agent when the next steps are already known, a simpler model call can handle the uncertain input, or your team cannot supervise and recover from its actions. For consequential changes, require verified authority and an appropriate approval process whichever architecture you choose. An agent becomes worth testing when evidence found during the job determines what it should investigate next. Compare that benefit against the cost of operating it and the outcomes you can actually verify.

Separate reading a request from deciding what to do

An email can be hard to interpret while the process for handling it is straightforward. A model might extract the requested change; ordinary code can check permissions and prepare the next step. Giving the model control of the entire process is a separate decision.

Anthropic's Building effective agents, published December 19, 2024, distinguishes predefined workflows from agents that choose their own steps and tools. It recommends starting with a simpler solution. The page now warns that its tooling discussion has aged; the architectural distinction remains useful here.

OpenAI's practical guide, displaying January 15, 2026, also recommends validating the need for an agent before committing. These are vendor design guides, not independent evidence that a particular architecture will save your business money.

Start with the next decision the system must make. If a fixed sequence can answer it, a workflow may be enough even when that sequence contains several model calls.

One billing queue, four different decisions

Fictional example: the following SaaS business, requests and proposed controls are illustrative. None of these designs has been implemented or tested for this article.

Imagine a US B2B software company receiving 500 billing and subscription requests each month, counting portal requests and emails together. That is an assumed workload, not a measured market figure. A proposal offers one agent access to the customer system, billing tools and outbound email.

Before granting that access, split the work by what each request requires. There is no assumed breakdown of the 500 requests among these categories.

1. A portal cancellation follows a fixed path

An authenticated administrator selects "Cancel at period end." The application resolves the administrator's account and subscription from trusted records and checks that their role permits the change.

The next action is known: request the scheduled cancellation and verify the resulting billing state. Conventional code is a reasonable starting point. If a webhook carries the event, verify its signature and check its identifiers against the authorized account. A valid signature alone does not establish the user's authority.

Repeated delivery must not apply the action twice. Retain an operation identifier, check whether the action already completed and read back the subscription's scheduled end state. On a timeout, reconcile the actual state before deciding whether to retry. Missing permissions, an account mismatch or an unresolved result goes to the responsible operator.

An agent would need to demonstrate a benefit beyond choosing the same known action. The code still needs maintenance and failure alerts. Our background-job restart guide explains why durable progress and safe retries matter even without a model.

2. A messy email needs interpretation, then validation

A customer writes: "Please use the new billing contact in the attached note from next month. Keep the service owner as it is."

A single model step could propose the change type, new contact details, effective date and the passage supporting each field. Keep the source passage beside the proposal so a reviewer can check whether the model confused a billing contact with a service owner. Missing information should stay missing.

The output is untrusted. A well-formed response only proves that the fields have the expected shape. It does not prove the request is accurate or authorized.

Resolve the account and requester permissions through trusted authenticated records. An account identifier supplied by the model, a recognizable signature or the sender's email address alone is insufficient. If the email cannot be linked to a verified authorized requester, require confirmation through the authenticated customer channel before proceeding.

In this example, the workflow prepares a billing-contact update for review. Code checks allowed fields and values; an authorized operator checks the proposal against the original request. Ambiguous dates, conflicting records or requests to change a tax identifier stop this path and go to a separate verified process. The model has no billing-write permission.

The uncertain part is interpreting the message. That alone gives no reason for the model to choose tools or control execution.

3. An incident credit needs an accountable decision

Another customer asks for a $2,400 credit after an outage. That amount is part of the fictional request, not an approved entitlement.

A fixed workflow can gather the relevant contract, applicable service-level terms and incident evidence into a review packet. Include each record's version and the unresolved questions. A model may help draft the explanation, but it cannot establish that the customer qualifies for the credit.

The authorized owner decides the amount under the company's policy and approves the exact account, action and outgoing response. Execution then checks that the approval is still applicable, performs the approved action and reads back the resulting record. Record completion separately from the draft response.

If the applicable terms are missing, the incident evidence conflicts or account authority is unclear, stop with incomplete evidence. Do not choose a plausible credit to keep the queue moving.

Human approval can sit inside either a fixed workflow or an agent-assisted investigation. Its usefulness depends on what the reviewer sees, what they can reject and which actions the system prevents before approval.

4. A usage discrepancy may justify a bounded agent

A customer reports that their invoice and usage dashboard disagree. Suppose the useful next check depends on the evidence: a missing time window points toward ingestion records, while duplicate event identifiers point toward deduplication behavior.

An agent could choose which permitted records to inspect next and prepare an explanation with linked evidence. If the same known queries resolve every case, put those queries in a fixed workflow instead.

For this proposed trial, restrict access to read-only records for the authorized account and relevant period. The agent cannot issue credits, change subscriptions or send messages. Treat instructions found inside tickets and logs as data; they cannot grant new permissions.

Before a run, set a maximum number of tool calls, a time limit and a spending limit. Name the person who handles incomplete investigations. Missing records, conflicting evidence, attempts to access another account or an exhausted limit must stop the run and preserve what remains unresolved.

An explanation that sounds convincing is not completion. The owner needs to check whether the cited records support it and whether the investigation answered the customer's actual discrepancy.

Test the boundary before expanding access

Take a permitted, sanitized set of representative requests and agree the expected handling with the workflow owner. Include normal cases and cases that should stop: an unauthorized requester, conflicting account records, an ambiguous instruction and a timeout after a potentially successful write.

Compare the simplest workable approach with the proposed agent on the same cases. Retain the input, trusted account context, retrieved evidence, proposed action and final verified state. Record whether a person had to repair the result. Test repeated runs where model variability matters; one successful demonstration is insufficient.

In the fictional billing pilot, define an unauthorized write as a reason to halt the trial and investigate. A correctly stopped request is useful evidence that a control worked, but it is not a completed billing change. An investigation without enough evidence should remain unresolved rather than be counted as a successful explanation.

Our guide to evaluating an agent across a whole conversation develops the testing question once you have selected a model-assisted approach.

Someone must also own monitoring, recovery and the decision to disable the system. NIST's voluntary AI Risk Management Framework, released January 26, 2023, provides a broader basis for assigning governance responsibilities. The billing controls above are proposed design choices, not a claim of NIST certification.

Compare dollar cost per verified outcome

A low API bill can coexist with a queue that requires constant repair. An expensive call can also be worthwhile if it resolves a task that the simpler approach cannot. Measure both approaches before claiming an advantage.

For one defined workflow and reporting period, fill in:

  • API and tool charges in dollars, including retries and failed runs.
  • Maintenance and recovery hours multiplied by the applicable hourly dollar rate.
  • Review hours multiplied by the applicable hourly dollar rate.
  • Other operating costs allocated to that workflow in dollars.

Add those dollar amounts. Divide the total by the number of verified completed outcomes for the same workflow and period. If there are zero verified outcomes, cost per verified outcome is undefined.

Count each person's time once: supervision already recorded as review or maintenance must not be added again. Keep initial build costs visible separately, or state the allocation period if including them in the comparison.

Define completion before the trial. For a cancellation it might be the authorized, correctly scheduled cancellation confirmed in the billing record. For an investigation it might be an explanation accepted against the evidence by the owner. These are different outputs; report them separately. Unresolved or incorrect attempts stay out of the completion count while their costs remain in the numerator.

Record elapsed time and unresolved volume alongside cost. A system that completes only easy requests may show an attractive ratio while leaving the difficult work to people. Describe potential losses from undetected errors as a separate uncertain risk scenario, with explicit assumptions. They are not measured operating costs or proof of savings.

Choose one workflow to scope

For a proposed agent project, identify the first decision that cannot be handled adequately by a known sequence. Write down the evidence it needs, the permissions it requires and what should make it stop. If the only uncertainty is reading the input, test a bounded model step before delegating the rest.

If you need help making that choice, the practical AI adoption roadmap starts with a workflow, compares AI with conventional options and defines a baseline and stopping conditions. Bring the problem, its owner and the decision you need to support, using sanitized examples. A roadmap and a production build are separate engagements.