Planning strategies, retry policies and subagent delegation

The basic agent loop is one model, one conversation, one attempt. Production agents wrap it in strategies that spend extra model calls to buy reliability: planning, breaking loops, retrying with a judge, and handing self-contained work to subagents with their own clean context.

In Daveth Pro (AI Engineering)

What this page answers

What is a subagent?

A fresh model, with its own clean context, handed one self-contained piece of work; the main agent takes back only its answer. Read it in the lesson

How do you make an AI agent more reliable?

Wrap the loop in strategies: plan before acting, notice and break loops, retry with a check such as the tests, and hand self-contained work to subagents. Each costs extra model calls. Read it in “Beyond the basic loop”

Beyond the basic loop

The loop from lesson A1 is one model, one conversation, one attempt. Production agents wrap that loop in strategies that cost extra model calls and buy reliability:

  • Planning: have the model write down steps before acting, and often keep that plan as a to-do list it updates.
  • Loop breaking: notice when the agent is repeating itself, and intervene.
  • Retries with a judge: if the result fails a check, such as the tests, throw the attempt away and start fresh.
  • Subagents: hand a self-contained piece of work to a fresh model with its own clean context, and take back only the answer.

Each of these is ordinary harness code. The first three are tested below; subagents are explained at the end, with the reason I didn't measure them on this model.

Predict first. Which single strategy adds the most solved tasks to lesson A1's agent?

  • plan first
  • refuse repeated failing calls
  • up to 3 fresh attempts until the visible tests pass

Answer. Up to 3 fresh attempts: +6 tasks. Plan first: -3. Refuse repeated failing calls: 0. Remember lesson A5: on 30 tasks a change of one or two is within noise, so the time column and the failure counts below matter as much as the totals.

Four strategies, measured

How to read the chart: tasks solved out of 30 (hidden tests) for lesson A1's agent and for each strategy added to it on its own, then for a combined version whose recipe I fixed before seeing any results: lesson A2's line-numbered tools, the loop breaker and retries together. The second number on each bar is the average time per task, because none of these is free.

Drawing: Tasks solved and time per task for each strategy.

Plan first: 4 of 30 (-3), 1.8 min per task. Refuse repeated failing calls: 7 of 30 (0), 70 s per task. Up to 3 fresh attempts: 13 of 30 (+6), 2.8 min per task. Combined with lesson A2's line-numbered tools: 5 of 30 (-2), 6.1 min per task. The loop breaker refused 20 repeated calls across the 30 runs, and the model solved exactly the same tasks as before. Stopping a repeat doesn't give a small model a better idea: replies with no tool call at all rose from 66 to 90 instead. The combined agent is the surprise in the other direction. I chose its recipe before any results were in, and it included lesson A2's line-numbered tools, which on their own scored 2 of 30. Retries can't rescue an agent that rarely makes a valid call: it replied without one 127 times, and retries saved only 3 tasks, at 6.1 min per task. Strategies don't simply add up; each needs measuring in the company it will keep.

VersionSolvedSteps per taskTokens read per taskTime per taskRuns out of steps
Lesson A1 agent7 of 308.78,49078 s20
+ plan first4 of 308.89,6791.8 min22
+ refuse repeated failing calls7 of 308.78,38870 s20
+ up to 3 fresh attempts13 of 307.619,5062.8 min14
Combined: line tools + loop breaker + retries5 of 307.327,9176.1 min14

Retries, and what they cost

The retry strategy is simple: run the agent; if the visible tests don't pass at the end, start a completely new attempt with sampling switched on so it can go a different way, up to three attempts. It's lesson A5's pass@3 put into practice, with the visible tests as the judge of which attempt to keep.

How many attempts each task used: 10 tasks took 1, 4 tasks took 2, 16 tasks took 3. Retries rescued 6 tasks that the first attempt failed. The cost: 2.8 min per task on average against 78 s, 2.2 times as much, and 65% of all that time went on tasks that failed anyway. That's the general shape of retries: cheap on easy tasks, which pass first time, and expensive on hopeless ones, which use every attempt. The judge matters as much as the retries: 5 of the kept attempts passed the visible tests and failed the hidden ones, so a stricter judge would have kept trying on those. In a real project the judge is the project's own test suite, usually far stricter than one visible test.

Does planning help a small model?

The plan-first prompt adds one sentence: “Before your first tool call, write a short numbered plan (at most three steps).” Here's the start of the first reply from one run, exactly as written:

1. List all files in the repository.
2. Read the `stats.py` file.
3. Replace the line that calculates the average with a line that returns 0.0 if the list is empty.
4. Write the updated `stats.py` file.
5. Run the tests.
6. If the tests pass, call `finish` with a summary indicating the change made.

The plan is sensible, though it has 6 steps when the prompt asked for at most three. The score went the wrong way: 4 of 30 against 7 of 30 (-3). The failure counts suggest why: replies without any tool call rose from 66 to 101, and the model wrote 425 tokens per task instead of 254. Asking for prose first pulled this small model out of its tool-calling format. It solved a different set of tasks, too, so part of the change is the noise from lesson A5. On tasks this small the plan is always the same few steps (read, fix, test), which the system prompt already spells out. Planning earns its keep on long tasks with many parts, where the plan becomes a to-do list the agent ticks off and returns to after each step; that's how large agents use it.