When I started extracting tasks from our codebase, I thought our existing tests would do most of the grading work.
We already had working software. We already had tests. Give an agent the task, run the tests, see whether it passes.
But there's a catch: your tests can pass without actually running.
If the code being evaluated runs inside the test process, and the grader trusts what that process prints, the program can announce “all tests passed” and exit before the tests run. The grader sees success. The task is still unsolved.
This isn't a problem with every test runner. It's a problem with a particular assumption: that the program being tested can't interfere with the evidence used to grade it.
What are we actually grading?
Task extraction means turning real work into a self-contained assignment that an AI agent can attempt, with a way to evaluate the result. That evaluation can become a reward signal during post-training.
Say the assignment is to implement a function that saves a record. In a familiar development workflow, we write a test, call the function, and assert that the record exists. That's useful.
Now imagine the agent can edit code loaded into the same process as the test. That code may run during initialization, before the test body is reached. If a grading wrapper accepts a printed success message or a zero exit code as proof that the checks passed, there's a shortcut around the actual assignment.
The distinction is small, but consequential: “the program reported success” and “the required behavior happened” are different observations.
Keep the tests. Move the decision.
Existing unit tests still matter. Keep them for development and regression checking. But when converting a codebase into a training task, look carefully at who controls the test runner, the outputs it produces, and the final score.
One practical approach is to give the program a small input/output interface, such as a command-line interface, and put the comparisons in a separate, protected harness. A harness is simply the machinery that runs the task and checks the result.
The harness chooses an input. The candidate program returns a result. The harness compares that result with what should have happened and awards credit itself.
For a calculation, that might mean sending in a series of values and comparing the returned answer with an independently computed result. The program provides an answer, not an authoritative “PASS.”
For a storage task, there's another step. A response like {"saved": true} doesn't prove that anything was saved. Use a trusted connection to inspect the database. If persistence is required, check that the record survives the relevant restart or fresh invocation, too.
Choose evidence that establishes the requirement. A correct output can establish a calculation's result. It cannot, by itself, establish where data was stored or which internal function ran.
A separate process is only the beginning
Moving checks into another process helps only if the agent can't change that process's code, expected answers, or score files—or interfere with its execution.
The boundary has to hold in the environment where the task actually runs. A folder called “protected” isn't evidence of protection. Check the real permissions and what the agent can access.
There's also a subtler trap: a trusted wrapper can still link against untrusted code. If that code can intercept execution and print the wrapper's expected response, the wrapper's output doesn't prove the intended operation happened. Changing the output format or adding a random token doesn't automatically fix that.
This is why a CLI is a useful interface, not a universal solution. The right grading design depends on what the task asks the agent to accomplish and what it can control.
Then test the grader itself
Before spending a training run on a task, challenge the grading setup with a few controlled submissions:
- A working implementation. It should earn the credit the task promises.
- An implementation missing an important behavior. It should lose the credit for that requirement.
- A fake-success implementation. Printing a passing message and exiting early should not earn credit.
- A different, valid implementation. It should pass without copying the reference's internal design.
Check what actually executed. If a deliberately broken submission crashes the harness, that isn't the same as the grader correctly recognizing the defect. The run should produce a valid evaluation result, with the missing behavior reflected in the score.
These controls don't prove a grader is impossible to exploit. They do give you better evidence than a reference solution passing once.
The extra work is part of the task
I came to this while trying to turn our company's codebase into something useful for AI training. I expected extraction to mean finding a good engineering problem and packaging the code around it.
It also means engineering the measurement.
The question I now ask is simple: could the program produce this success signal without doing the work?
If the answer is yes, there's more to extract than the task. There's a grader to build.
Further reading: Anthropic's guide to agent evaluations makes a useful distinction between what an agent says it did and the final state it actually produced.