The team spent six weeks building an agent. Tool selection, a planning step, a memory layer, retries with reflection. It shipped. Two weeks later someone finally opened the traces in the observability stack, sorted by tool-call sequence, and noticed something awkward. Ninety-two percent of production requests took the same five steps, in the same order: fetch user profile, fetch account state, look up the relevant policy, generate a draft reply, and send it. The other eight percent were mostly errors that retried into the same five steps.

The agent worked, technically. It also paid for a planning call on every request to decide that this request looked exactly like every other request. That's roughly 800 extra input tokens per interaction, an extra second of latency on the critical path, and a Slack channel full of "why did the model use tool X instead of tool Y this one time" threads. Issue 13 was about giving an agent loop the bounds it needs to be safe in production. This issue is one rung further back: whether you needed the loop in the first place.

The short version is that "agent" got treated as the default architecture for anything that touched a model, and it shouldn't be. Most production tasks have a known sequence of steps, and a deterministic workflow that calls the model only at the steps that need judgement is cheaper, faster, and easier to debug. In this issue you'll see the autonomy ladder that names each rung and the question that earns you the next one, plus the trace-mining method that turns an existing agent into the workflow it was probably always going to become.

Why "agent" became the default (and why that's wrong)

If you started shipping LLM features any time from late 2023 onward, the marketing air you breathed was full of agents. LangChain shipped the ReAct-style loop. AutoGPT went viral. Every vendor demo showed a model choosing tools and iterating. The word "agent" started to mean "anything you'd want to do with an LLM", and the loop became the default shape people reached for.

Two things went wrong in production. First, the loop is expensive on every axis. The planning call adds tokens and latency; the tool loop adds more of both; the retries with reflection add more still. On a task that has a known sequence of steps, all of that cost is wasted. Second, the loop is harder to debug. When the model picks the wrong tool, you can't set a breakpoint. You read a transcript, try to intuit what the planner was thinking, and then edit the system prompt and hope. That works for tasks that genuinely need model judgement at every step. It's a bad trade for tasks that don't.

Anthropic's December 2024 post "Building effective agents" (see Further reading) drew the distinction that most teams have since adopted internally: workflows are systems where an LLM is orchestrated through predefined code paths, and agents are systems where the LLM directs its own tool use. The important word is "own". If your code decides which tool runs when, you're doing a workflow. If the model decides, you're doing an agent. The reason the distinction matters is that they cost different things and fail differently, and most tasks want the cheaper shape.

The autonomy ladder

Six rungs, from the most constrained to the most autonomous. Every rung is a valid architecture. The rule is to start at the bottom and only climb when the current rung can't do the job.

Rung 1: Single call. One prompt, one response. If the task is "summarise this ticket" or "extract these fields", a single call is the whole system. The question that would earn the next rung: is the task big enough that a single prompt starts to under-perform on any slice? If not, stop here. Adding more machinery makes the system worse.

Rung 2: Prompt chain. Two or more calls in a fixed sequence, each call's output feeding the next. Extract fields, then classify. Draft the reply, then edit for tone. The question that would earn the next rung: does the sequence branch? If it's always the same steps in the same order, a chain is the right shape.

Rung 3: Router. A model call (or a classifier) picks which of a small set of downstream chains to run. Customer support gets routed to the refund flow, the returns flow, or the technical support flow. Each branch is itself a chain. The question that would earn the next rung: does any single branch need conditional loops of its own, or tool use that depends on intermediate results?

Rung 4: Workflow with LLM steps. A deterministic DAG or state machine written in your code, with LLM calls at specific nodes and tool calls at specific edges. The graph is fixed; the LLM's role is to produce structured outputs that drive transitions. This is the rung most teams should be on and aren't. The question that would earn the next rung: are the transitions in the graph genuinely unknowable in advance, or does the graph just have a lot of edges?

Rung 5: Tool-using agent. A ReAct-style loop where the model picks which tool to call next based on the current state. The loop terminates when the model says it's done or when the bounds from Issue 13 fire. This rung is appropriate when the sequence of tools genuinely depends on what the previous tool returned, and there are too many branches to enumerate by hand. The question that would earn the next rung: is the task genuinely made of sub-tasks with different objectives that need separate contexts?

Rung 6: Multi-agent. Several agents with distinct roles and separate context windows, coordinated by a supervisor agent or an orchestration layer. This is where the cost starts to compound aggressively (every agent has its own token cost, every hand-off is an extra call), so the bar is high. Real-world uses are things like software-engineering agents where a "planner", a "coder", and a "reviewer" have genuinely different jobs.

The ladder from single call to multi-agent, with the question you have to answer yes to before climbing each rung. Read it as a checklist that stops you at the lowest rung that does the job.

The rule of thumb is that you should be able to say out loud what makes the current rung insufficient before you climb. "The chain isn't enough because I have three routes with different toolsets" earns rung 3. "The workflow isn't enough because I don't know which tool to call until the previous tool returns" earns rung 5. If you can't finish the sentence, you don't need the next rung yet.

The trace-mining method

The autonomy ladder is useful when you're starting a new feature. The trace-mining method is what you do to features you already shipped as agents that probably shouldn't have been.

Open your Issue 4 observability stack. Filter to the last few thousand production traces for the agent in question. For each trace, extract the ordered sequence of tool calls the agent made. You now have a distribution of tool-call sequences, and you're going to look at three properties of that distribution.

Path concentration. Sort the sequences by frequency. If the top three sequences cover more than 70 percent of your traffic, most of your agent is doing a small number of things repeatedly. That's the tell.

Path stability. For each of the top sequences, check whether the model consistently produces the same sequence for similar inputs. If the same customer-support ticket type reliably takes the same five-step path, the "which tool next" decision has no information in it. The model is spending planning tokens to arrive at a determined answer.

Judgement steps. For each step in the top sequences, ask whether the model's output at that step is the same across similar inputs or genuinely varies. A step that produces a personalised draft response is a real judgement step. A step that decides which tool to call next when the answer is always the same is not.

When one path dominates and its steps are stable, you compile it into a workflow. Encode the sequence as a deterministic DAG in your code. Replace the planning call with a router at the top (if you have more than one dominant path) or with nothing at all (if you have one). Keep the LLM at the steps that need judgement: the draft response, the summarisation, the classification. The scaffolding, the tool-selection, the sequencing, all of that is now in code.

The result on a real workload usually looks like this: latency drops by 30 to 50 percent because you removed the planning call and cut the extra iterations, token cost drops by 40 to 60 percent for the same reason, and debugging gets dramatically easier because your logs now show a specific node that failed rather than a transcript you have to interpret. Numbers this large only land when there's a dominant path to collapse; the trace-mining method is what tells you whether the collapse is worth doing.

A worked example: support-reply automation

Say the trace shows 92 percent of your customer-support agent traffic runs this sequence: get_user, get_account_state, lookup_policy, draft_reply, send. The remaining 8 percent hits an edge case where lookup_policy returns nothing and the agent has to escalate.

The workflow version is a five-node DAG. Nodes 1 to 3 are deterministic tool calls: no LLM involved, they're just API calls to your own services. Node 4 is an LLM call: draft the reply given the user, account state, and policy. Node 5 is a deterministic send. The edge case is one extra branch off node 3: if lookup_policy returns nothing, route to a "human escalation" node instead of node 4.

The planning call, the ReAct loop, and the "which tool next" reasoning are gone. What's left is exactly one LLM call per request, at the step where judgement actually matters. Ninety-two percent of your traffic is now cheaper and faster; the 8 percent edge case is still handled, just through a specific branch you can see and test. That's the trade the trace-mining method is trying to earn you.

Common mistakes

Starting at "agent" without earning the rung. New team, new feature, someone says "let's build an agent". No one asks whether the task actually branches. Six weeks later you have a system that could have been a five-node DAG. The fix is a short design-review question: what's the sequence of steps for the top three input types, and do they differ? If they don't, you're building a workflow.

Never re-reading traces after shipping. You built the agent, shipped it, and moved on. The traces are sitting in the observability stack showing you the collapse opportunity, but no one reads them. Add a quarterly trace review to the runbook. Even fifteen minutes of sorting by tool-call sequence usually surfaces at least one dominant path worth compiling.

Confusing "the model chose the tools" with "the model needs to choose the tools". These are different claims. In an agent, the model picks the tool. That doesn't mean the tool choice is model-worthy. If a rule engine could pick the tool from the input, a rule engine should, and the model can go back to the judgement step.

Multi-agent as the first solution to complex tasks. A complex task doesn't automatically want multiple agents. It might want a longer workflow with more LLM steps at judgement points. Every agent hand-off is a full context handover, which costs tokens and adds a failure mode where the receiving agent misreads the context. Try one agent (or better, one workflow) before you split.

The takeaway

Six weeks of agent work often collapses into a five-node workflow the moment you look at the traces. The autonomy ladder gives you a way to pick the right rung at design time: single call, prompt chain, router, workflow with LLM steps, tool-using agent, multi-agent, with a specific question that has to be answered yes before you climb each rung. The trace-mining method gives you a way to demote an existing agent to the rung it belongs on: group production runs by tool-call sequence, and when one path dominates and its steps are stable, compile that path into a deterministic workflow that keeps the model only at the judgement steps. Cheaper, faster, and debuggable with stack traces instead of transcripts.

Production checklist

  • Before starting an LLM feature, sketch the top three input types and the sequence of steps each one needs. If the sequences are identical, you're building a workflow, not an agent.

  • Adopt the autonomy ladder as a design-review checkpoint. Name the current rung, and name the question you'd have to answer yes to before climbing to the next.

  • For every shipped agent, sort last month's traces by tool-call sequence and check the top three paths' share of traffic. If it's above 70 percent, schedule a compile-to-workflow spike.

  • For each dominant path, check step-by-step whether the model's output at each step varies with input. Model steps that produce identical output across inputs are candidates for a rule or a deterministic function.

  • Replace the top of a dominant path with a router (if you have more than one dominant path) or with nothing at all (if you have exactly one). Keep the LLM only at the judgement steps.

  • Bound the tool-using agent with the wall-clock and step-cap limits from Issue 13. Workflows don't need those bounds because their step count is known at design time.

  • Re-run the Issue 3 golden set against the workflow version before the cutover, so you have measured evidence that the collapse didn't cost quality.

  • Log which rung of the ladder each surface is currently on, next to the model version, in the Issue 4 observability trace. This makes "what changed" analysis cheaper on the next migration.

  • Schedule a quarterly trace review for every LLM feature. Fifteen minutes of sorting by tool-call sequence is often the difference between paying for planning tokens and not.

Further reading