Agents in production: workflow versus agent loop, cost and tool reach
In a workflow your code fixes the steps and the model fills in each one; an agent lets the model choose the steps. Choose the workflow whenever you can write the steps down, and reach for an agent only when the path genuinely depends on what is found along the way.
What this page answers
Should I use an AI agent or a workflow?
Use a workflow whenever you can write the steps down. Use an agent only when the path depends on what is found along the way, as in debugging code or researching across sources. Read it in “Workflow or agent?”
Workflow or agent?
Most of this track's handbook assistant is a workflow: your code fixes the steps (retrieve, answer, validate) and the model fills in each one. An agent lets the model choose the steps. Choose the workflow whenever you can write the steps down; reach for an agent only when the path genuinely depends on what's found along the way, as in debugging code, researching across sources, or multi-step tasks in an unfamiliar system.
What the loop costs
Track A ran the same 30 small bug-fixing tasks two ways with Qwen2.5-Coder-1.5B-Instruct: as an agent (read the file, edit, run the tests, repeat) and as a single call that sees the code and writes the fix.
Predict first. Which solved more of the 30 tasks with this small model: the agent loop or the single call?
- the agent loop
- the single call
Answer. The single call, 25 against 7. The tasks were small enough to fix in one look, and the small model kept stumbling over the loop's mechanics: malformed tool calls, edits that didn't match the file, and repeating failed steps until it hit the step limit.
| Approach (1.5B) | Solved | Input tokens per task | Output tokens per task | Time per task |
|---|---|---|---|---|
| Agent loop | 7 of 30 | 8,490 | 254 | 78 s |
| Single call | 25 of 30 | 133 | 71 | 19 s |
The agent read about 64 times as many tokens per task (8,490 against 133), took 4.1 times as long, and solved fewer tasks. That's not a verdict on agents: a stronger model uses the loop well, and on tasks you can't solve in one look (an unfamiliar codebase, a bug you have to reproduce) the loop is the only way. It's a reminder to measure, because the loop's costs are certain and its benefits depend on the model and the task.
What the loop can reach
Track A's sandbox blocks file tools from leaving the repository. Lesson A4 then tested whether that's enough: an agent's code runs when it runs the tests, and code can do anything Python can.
Direct: file tool asked to write outside the repo
write_file(path='../canary_written_by_agent.txt')
blocked
Indirect: code that writes outside, run by the tests
open('../canary_written_by_agent.txt', 'w').write('written from inside the test run')
escaped
Indirect, with an audit hook guarding open()
same code, run under a guard
blocked
The file tool's path check stopped the direct attempt, but code the agent wrote, run by the tests, wrote outside the repository anyway; a guard inside the Python process caught that case, though a determined program can get around in-process guards. The lesson for production: put agent-run code in a real sandbox (a container or VM with no network and no secrets), because checks at the tool level only cover the tools.
Continue the lesson
The whole lesson, with its drawings and its questions. The rest of this section and 2 more after it.
