How labs train coding agents: pre-training, supervised fine-tuning and RL in sandboxed tasks

Coding agents are trained in three stages. Pre-training teaches what code looks like, supervised fine-tuning on recorded agent traces teaches the tool-call format, and reinforcement learning on real tasks in a sandbox, rewarded when the tests pass, teaches what actually works.

In Daveth Pro (AI Engineering)

What this page answers

How are coding agents trained?

Pre-training on text and code, supervised fine-tuning on examples of the agent loop, then reinforcement learning on real tasks inside a sandbox, where the task's tests decide success. Read it in “Three stages of training”

Why do coding agents need reinforcement learning?

It is the only stage that rewards finishing the job rather than imitating text, which turns a model that can call tools into one that solves tasks with them. Read it in “Three stages of training”

Three stages of training

  1. Pretraining on huge amounts of text and code teaches the model what code looks like and how it tends to continue (Chapters 1 to 10).
  2. Supervised fine-tuning on examples of the agent loop, often recorded traces of tool calls and results, teaches the format: how to call a tool, read its output and continue (Chapter 11; Track B, lesson B2).
  3. Reinforcement learning on real tasks teaches what works. The model attempts a task inside a sandbox, the task's tests decide whether it succeeded, and successful attempts are made more likely (Track B, lesson B4).

The third stage is the one that turns a model that can call tools into one that solves tasks with them, because it's the only stage that rewards finishing the job rather than imitating text.

Tasks, sandboxes and tests

An RL environment for coding looks a lot like lesson A5's evaluation harness, run millions of times. Each task is a repository snapshot, an issue to fix and tests the agent can't see or edit. Each attempt runs in a fresh, isolated sandbox (lesson A4 showed why tool-level checks aren't enough). The reward is computed from the hidden tests, sometimes with extra terms for things like not breaking other tests. Real bug-fix histories from open-source projects are a common source of tasks, because each comes with the fix and the tests that check it.

Where the learning signal comes from

Learning from attempts needs variety in the outcomes. If every attempt at a task fails, nothing distinguishes a good attempt from a bad one; if every attempt succeeds, there's nothing to improve. Lesson A1's results show what that means for a weak starting model.

Predict first. The 1.5B agent from lesson A1 attempted each of 30 tasks once. On how many tasks did it succeed at all?

  • 20 or more
  • 10 to 19
  • fewer than 10

Answer. 7 of 30. On the other 23, a single attempt failed, so RL on those tasks would see only failures.

With one attempt per task, RL could learn only from comparisons across tasks, which mixes up “this attempt was better” with “this task was easier”. Grouped methods like GRPO fix that by sampling several attempts per task and comparing within the group, but they still need at least one success in the group. That's why a strong supervised stage comes first: RL sharpens what a model can already sometimes do; it struggles to create a skill from nothing.