A lifelong stream of ARC puzzles that come back with fresh inputs after long gaps, while new ones keep arriving. It measures whether a learner gets cheaper on work it has seen before, whether it keeps notes like an agent harness or updates its weights.
We ran three agent harnesses through the stream with every learning mechanism they ship switched on, then again with everything they wrote deleted before every instance. A learner that has learned a task should fall below the dotted line, where the demo-cache floor sits once it holds the task's demonstrations. The harnesses start a few points under the bare model, since they can test a guess against the demonstrations, and stay there however often the task returns. The program library, a deliberately naive baseline built around ARC, falls well below.
Replay a run at medium effort:Stateless floorDemo-cache floorProgram libraryPrime Agent, as shippedHermes Agent, as shippedOpenClaw, as shipped
The curves are the paper's Figure 3, on the workstreams stream. The three numbers pair each comparison instance by instance and pool the three orders, with 95 percent intervals by bootstrap over tasks; the returning tasks are those of Appendix F. Every figure of the paper is on Figures.
The learner sees the task's id and a fresh input grid. It can buy a demonstration, one more input with its output, or submit an answer and hear only right or wrong. A demonstration and a wrong answer each cost 1, up to eight of each, and a right answer ends the instance.
25 tasks over 200 instances, four active at a time, each worked in bursts of about three and resting about thirty, with a new task every two. A task came back after a median gap of 5 instances, and at most 141. Then 25 held-out tasks, once each.
An instance is charged its demonstrations and wrong answers out of 16. U is the share of that budget used, averaged over the stream, so lower is better, and P, the share solved within the cap, sits beside it. A learner that has kept a task's rule can answer at once and spend nothing.
Each edition gets its own track and its own number, and no score pools them. The tasks of every track are on Tasks.
| # | Learner | GPT-6 Luna, low | GPT-6 Luna, medium | GPT-6 Luna, high |
|---|---|---|---|---|
| 1 | Program library | 15.7 ±4.2 | 11.7 ±4.2 | 7.8 ±3.4 |
| 2 | Prime Agentas shipped | 42.8 ±4.7 | 23.6 ±2.8 | 20.2 ±2.5 |
| 3 | OpenClawmemory wiped | 39.3 ±4.0 | 25.0 ±3.3 | 23.2 ±2.9 |
| 4 | Prime Agentmemory wiped | 42.1 ±4.5 | 25.1 ±3.5 | 20.5 ±2.3 |
| 5 | Demo-cache floor | 38.1 ±5.2 | 25.3 ±4.7 | 17.4 ±3.8 |
| 6 | OpenClawas shipped | 42.3 ±4.2 | 26.8 ±2.9 | 23.2 ±2.8 |
| 7 | Hermes Agentmemory wiped | 39.0 ±4.4 | 28.0 ±3.8 | 23.6 ±3.4 |
| 8 | Hermes Agentas shipped | 44.2 ±4.9 | 29.7 ±3.8 | 21.6 ±2.6 |
| 9 | Stateless floor | 49.0 ±5.2 | 33.3 ±4.0 | 26.9 ±3.3 |
U in percent of the budget, lower is better, with the standard error over tasks; ranked at medium effort. Each number opens the replay of its run.
Every instance of every run can be replayed, from what the learner bought and submitted to what a harness chose to keep after the verdict. Each pair below ran on the same schedule, so their instances line up one for one.
As shipped, OpenClaw wrote no memory. Asked after every verdict to save anything that could help, it wrote notes under names it chose afresh each time, and before acting almost never looked for them when the task came back.
Prime Agent's lessons live in the session unless the agent marks one global, and as shipped it never did. Asked to store, its global entries opened every later session, and returning tasks got cheaper.
Hermes Agent's background review wrote skills of general advice, many named for steps of the protocol. Told to keep one tested program per task and run it first, it carried the method out with its own tools.
Every answer is a Python program, checked free against the demonstrations. A program that reproduces them all is kept and runs first when its task returns. With its programs wiped before every instance, it writes each one again from scratch.
Or choose any run by track, learner, variant, effort and order on Replay.
The stream and the records of every run are public, and there is no held-out set. Run your learner or harness on the canonical stream, send the run directory as a pull request, and it joins the leaderboard marked reported until we rerun it.
git clone https://github.com/unifyai/continual-arc
cd continual-arc
uv venv && uv pip install -e ".[dev]"
uv run continual-arc data fetch
uv run continual-arc schedule make --config arc1-small \
--mode workstreams --seed 0 --out arc1-ws0.jsonContinual-ARC: Measuring Skill Acquisition over a Working Lifetime. Daniel Lenton, unify, 2026.
@misc{lenton2026continualarc,
title = {Continual-ARC: Measuring Skill Acquisition over a Working Lifetime},
author = {Lenton, Daniel},
year = {2026},
url = {https://unify.ai/continual-arc}
}