Building a brain around a stateless model.

Building software around language models means building on an exponential. Most of the design covers for something the model can't do yet. Every few months a new model does more of that inside its own weights, and the design ages about as fast as the models improve.

Our agent, unify-agent, was meant to work for one person over a long period. We treated the language model inside it as a stateless piece of cortical tissue. Text goes in and text comes out, with nothing kept between calls. Everything else a working agent needs, from what it remembers to how it keeps talking while it works, we built around the model in code and prompts, a bit like assembling the rest of a brain from parts that sit outside the neural network.

We never thought of that harness as the long-term answer, only as the most we could get out of the models of 2025 without training them. Much of what it did for them now happens inside the model, behind the API. Building more structure around the model stopped paying off, and we turned to training the model itself.

Why we built a harness at all

Training a frontier model takes a lab's budget, and in 2025 the strongest models weren't open anyway. From outside a lab, you could only control what went into each call. So we split the agent into parts with one job each, most of them model calls, and used the context window as the boundary between them. Our first technical spec has a part for each job, from the one that picks up the phone to the one that clicks around a browser, and the parts mostly pass text to each other through queues.

We expected those parts to end up trained together eventually, with memory and action entangled in the weights, which is how our own brains do it. Doing that from outside a lab would have taken millions in compute, though. In the meantime, hijacking the context window as a boundary between parts was the best short-term path we could see to an agent that actually did something useful. We saw the harness as a way to carry the agent until the models could do its work themselves.

Designs on the exponential

Each of these designs was built for the models of its moment, around something they couldn't yet do well. We have since tested every one of them twice. The first test ran unify-agent's own code from the time, with the models it used. The second rebuilt each design from scratch, small and without the old bugs, and tested it on today's models against the simplest strong alternatives we could think of. The rebuilds and their results are in our demos repo.

The designs below at the months they went into unify-agent, from April 2025 to the summer of 2026, with the releases of the models they ran on along the bottom.

Talking and acting in separate loops

When OpenAI had just released o3 and o4-mini, a call to a capable model still took several seconds. Because a person keeps talking while an agent works, we ran the conversation in one loop and the work in another. Starting a piece of work gave the conversation a handle to steer it by while it ran, and the work could ask the person a question through the same handle.

The two loops. The person talks only to the conversation loop, which passes requests, corrections and answers down to the work, in blue, and gets progress, results and questions back, in coral.

We ran unify-agent's code from the time with o4-mini in every loop. The two loops answered "how's it going?" in about 7 seconds, where a single loop that couldn't be interrupted took 48. A single loop that could be interrupted answered as quickly and kept working in only 4 of 9 runs. In the rest it either stopped the work to answer or answered once it had finished. When the work needed a fact from the person, though, the uninterruptible single loop got the task right more often, in 15 of 18 runs against 11. In a single loop the agent doing the work is also the one talking to the person, and it can just ask. In the two loops the question and its answer had to pass from one loop to the other, and in four runs that went wrong. The two loops also never told the person how the work turned out, in any of the 8 runs that finished. We read the transcripts and found a bug that wrote each result above everything said in the meantime, and the conversation, which answers the latest message, passed over it. A later change put each result at the end of the conversation instead, but we didn't test that version. We haven't analyzed the rebuild's runs yet.

Storing skills as functions

Working a task out again from notes costs tokens every time, and it can come out differently. We kept what the agent learned about a task as Python functions in a library instead, where they could call one another. A repeated task then just runs its function, and a repair to one shared step reaches every task that uses it.

As notes

Posts about a topic: go to linkedin.com, click the search bar, type the topic and press enter, then click the Posts filter.

People with a name: go to linkedin.com, click the search bar, type the name and press enter, then click the People filter.

Messaging someone: go to linkedin.com, click the search bar, type their name and press enter, click the People filter, then click Message on the first result, type the message and press enter.

As functions

def search_linkedin(query: str):    """Search LinkedIn for `query`, starting from the homepage."""    go_to_linkedin_com()    click_the_search_bar()    type_the_text(query)    press_enter()def send_message(text: str):    """Message the first person in the results with `text`."""    click_message()    type_the_text(text)    press_enter()def find_posts_about(topic: str):    """Show the LinkedIn posts about `topic`."""    search_linkedin(topic)    click_the_filter("Posts")def find_people_named(name: str):    """Show the LinkedIn members called `name`."""    search_linkedin(name)    click_the_filter("People")def message_person(name: str, text: str):    """Send `text` to the LinkedIn member called `name`."""    find_people_named(name)    send_message(text)

Three skills kept as notes, above, and as functions, below. Every note repeats the search, which the functions write once as search_linkedin, and message_person is built from another task, find_people_named.

On the original code, with GPT-4.1, we gave the agent 16 related tasks on a simulated professional network whose search changed halfway through. The library finished every task with under a third of the tokens the notes needed. On today's models the rebuild was never more accurate than the simple alternatives we tried, one of which kept no memory at all, and it was never cheaper than retrieving past solutions. Every design coped when the interface changed partway through, and being able to repair one shared function gained the library nothing. The stronger models solved every task even with no memory, and the library just made them about 60% more expensive.

Requests that carry their conversation

A request that one model writes for another often leaves out what the conversation already settled. Ours were written by o4-mini at the time, and in our tests it left that out of more than a third of them. So the loop passed its conversation down with every call, read-only. The argument that carried it was later hidden from the model altogether, which kept the model from ever writing the conversation out itself.

The same request, handed on two ways. The lookup in aqua gets the conversation, in blue, with the request, and answers with the day of the basketball call. The lookup in coral gets the request alone, finds two conversations with Julia that week, and has to ask which one is meant.

We tested it with o4-mini doing the lookup, as it did then. Given the conversation, it got 95% right where the conversation settled what the request left out, against 75% with the request alone. Where the conversation settled nothing, the lookup that had it mostly answered without asking which exchange was meant, and its accuracy halved. When o4-mini wrote its own requests, the gain was within noise. On today's models the gain shrank as the model doing the lookup got stronger. Where the request left the detail out, it fell from 12 points with Claude Haiku 5.5 to none with Claude Opus 5.5, which asks its caller, who has the conversation. When the model wrote the request itself, even Haiku named the deciding detail 96% of the time, and passing the conversation down did no better than any alternative and cost more.

Memory consolidated in the background

Back then a long context was expensive to fill on every call, and an agent that remembers has to keep deciding what's worth keeping. We summarized its history in the background, away from the conversation, in windows such as the past day and the past week, each built from the one below.

The two ladders of summaries over the record of what the agent did, one by time and one by the number of interactions. Each rung is written only from the rung below, and the number beside an arrow is how many lower summaries make one higher one.

The first version had two bugs that kept most windows from reading the history they summarized. Neither was fixed before we took the summaries out a few months later, and the numbers from that version say little about the idea. On today's cheaper models the rebuild beat every fixed-size memory we tried on questions about a stretch of time, such as which projects were worked on in a given week. On those questions it matched keeping the whole record in a million-token context for a twentieth of the cost or less. Summarizing the raw records at question time did better still, though at 17 times the rebuild's cost per question on Claude Haiku 5.5 and about a hundred times on GPT-6 Luna. Taking every kind of question together, simply extracting facts did as well or better, since a summary drops the old quotes and dates that a search over facts still finds.

Functions and guidance, linked both ways

Some of what an agent learns is a rule rather than a procedure. We kept the procedures as functions and the rules as written guidance in a second library, each rule linked to the functions it governs and kept in one place.

The same knowledge kept as skill folders, on the left, and as two linked libraries, on the right. In the folders the tone rule is copied into two skills, in coral, and drifts. In the libraries it is one guidance entry, in blue, linked to the three functions it governs.

In the original, with GPT-5, changing a linked rule took one edit, against five for copies pasted into skill folders and about two for a rule kept as a skill of its own. Tasks didn't pick up the changed rules any more often, though. The code's own way of updating a rule also failed every time, and our test had to work around it. On today's models, linking helped GPT-6 Luna pick up every rule that applied to a task. Luna otherwise had to judge from the prose which rules applied, and it missed some. Claude Haiku 5.5 came out slightly worse with links than with simpler arrangements, and from GPT-6.1 Sol up linking made almost no difference. A linked rule still took one edit where pasted copies took about three, though a single file of rules needed only one too.

Demonstration without a mode

To teach the agent by showing it, a person shared their screen and talked it through. We made that an ordinary request rather than a mode of its own. The request carried frames of the shared screen, each with what was said over it, and the frames went by reference so they didn't fill every call.

Frames passed by reference. The request carries image ids with what was said over each, in blue, and the loop that receives them looks at a frame only when it needs to. The pixels stay in the image store until then.

In half the runs of our first version the frames never reached the model. When they did, the answers about them came back without the frame they belonged to, and were applied in whatever order they arrived. Those two bugs decided its result. On today's models the rebuild, at 256 to 500 frames, cost 2 to 16% of pasting every frame into the context, and on the cheaper models its accuracy held steady as sessions grew, where pasting every frame lost accuracy. On the cheaper models and on Claude Sonnet 5.5, though, it didn't pass the main test we wrote down before the runs. At 64 and 128 frames simpler options did better, and Claude Opus 5.5 got every request right either way.

A fast brain and a slow brain

A spoken reply has to start within about a second, and a capable model of the time, with every tool, took several seconds just to decide what to do. So a fast voice model held the conversation and told the person it was looking into things. Behind it, a slow model with the whole context did the work and passed the fast one data, leaving all the talking to it.

The two models behind one voice. The fast brain, which the person hears, passes on what was said, and the slow brain answers with guidance, in blue, which carries data and never dialogue.

With GPT-5 mini as the fast model and Claude Opus 4.6 as the slow one, the slow model took about ten seconds to decide on a lookup and eighteen to answer it. The fast model's replies began in 0.7 seconds. In the version we tested, two rules in its prompt told it never to make anything up, and to say it was checking and then stop. With them it invented nothing while it waited. Without them it made up a fact in 56 of 80 replies to lookups, saying things like "The last Delta Freight invoice was $4,320.50", a number it had no way of knowing. The slow model also kept writing whole replies to small talk, which left the agent answering twice.

Execution as a coordinate

Our agent came to act by writing code, with Claude Opus 4.5 writing it, and that code needed somewhere to run. We gave the agent one tool whose arguments choose, on every call, what state survives and which session the code joins.

The state modes against the places code can run. Every cell is reachable from the same tool by changing its arguments, and read-only, in blue, is the mode neither usual executor offers.

In the original, the tool reloaded a slow export a third as often as a fresh sandbox did, but one shared interpreter did just as well and faster. In the rebuild, on today's models, one interpreter shared across several clients let one client's data leak into another's answers, which the tool avoided by choosing the session on each call. One interpreter per client, set up by the harness, avoided both the reloads and the leaks, and it matched the tool on every model for less. When Claude Sonnet 5.5 named the sessions itself, it sometimes gave two clients the same one-letter name, and one client's request ran on the other's data.

The slow brain speaks

We later turned the split around. The slow model wrote every substantive line, spoken word for word, and the fast model only held the floor while it thought.

In scripted calls no turn of small talk got two replies and every answer was heard, at the price of a first word at 1.8 seconds instead of 0.7. On today's models a single fast model with reasoning on and all the records in its prompt was as accurate as either split and cost less. It also finished its answers sooner, though it said nothing for its first few seconds. The same model with reasoning off stated a wrong or invented fact in nearly half its calls.

Stronger models needed less of it

Because the models of 2025 couldn't think and talk at the same time, we split the agent into a fast brain and a slow one. Context was expensive to hold, and we summarized history and passed screens by reference to save it. Models have since become far better at reasoning before they answer and at using a long context, and each of those took over work our harness used to do.

Without the fast brain, a caller asking our agent for an invoice amount on Claude Opus 4.6 would have heard nothing for about ten seconds and waited eighteen for the answer, which on a phone sounds like the line has dropped. Today's models can begin a reply within one to four seconds, though a single model with tools still tends to say nothing while a lookup runs. A canned "one moment" brings its first word to about a second, and with Claude Sonnet 5.5 the answer comes in about five.

Across the rebuilds, most of the gains in accuracy are gone on today's strongest models. The gains that remain are mostly on the cheaper models, or in cost, such as a memory that answers for a twentieth of the price.

The next step is the weights

Our harness kept everything it learned as text, which the model read again on every call. That carried our agent a long way, but the model itself never got any better at the work. We think the model inside an agent should learn from the agent's own work, in its weights, the way a person gets better at their job. The harness is not enough makes that argument, and Continual-ARC is how we measure whether an agent learns from its work at all. We haven't yet worked out how to keep training a model on its own work without it forgetting what it already knew.