Models are agents now. Treat them like it.

A tutor that runs out of budget after a few questions is not much of a tutor. So when we built the first version of Honorly’s AI harness, the program that sits between the model and everything else, cost was the constraint we designed around. Every decision about how to package a request, what to send, and what to ask for came back to one question: how do we get the most useful tutoring out of each dollar a school spends?

The answer we arrived at was elegant on paper. Ask the model once per turn, and ask for everything at once: the reply the student sees, plus every piece of information the application needs to run, packed into a single structured JSON response. Then parse it and route each field where it belongs.

That design was wrong, and the way it was wrong taught us something we now think applies well beyond education. This piece is about what we measured, what we believe explains it, and the rule we took away: a modern language model should be placed in the kind of environment it was trained to work in, and that environment is an agent loop.

The math that made one big call look right

The case for a single call is about input. Every time you call a model, it re-reads the context: the system instructions, the classroom settings, the conversation so far. If titling a conversation, flagging a concern for a teacher, and answering the student are three separate calls, you pay to read the same context three times. Fold them into one call and you pay once.

We ran that arithmetic early, and it held up. A single structured response looked like the cheapest way to get all of the application’s work done. The model in that era also benefited from tight scaffolding, and a fixed schema gave the rest of the system something predictable to consume.

So Harness V1 asked for a form. Its final answer had to match a strict JSON schema with four required parts: the student’s reply, a teacher signal, an artifact, and a conversation title. The artifact alone had nineteen required fields, one slot for every kind of interactive content the tutor might produce, from quizzes to timelines to simulations. V1 could call tools along the way, and it had fourteen of them. But however a turn unfolded, it had to end by filling in the whole form.

What the arithmetic left out was everything on the other side of the call: what the model has to generate, and what it has to think about to generate it.

V1: a loop that ends in a form

1. Student message
2. Model14 tools
3. Every turn ends in a strict JSON schema
   • studentReply
   • teacherSignal
   • artifact (19 fields)
   • conversationTitle
4. Application parses, then renders the reply

Every turn fills in every field, whether the student said “hi” or asked for a lab.

V2: a loop that ends in a reply

1. Student message
2. Model48 toolsteaching toolsChalk activitiesmemoryteacher signalscalculator
3. Every turn ends in a plain reply, streamed as it is written
4. Application validates, executes and saves; titles and summaries run off the reply path

A greeting is one short message. A lab takes as many steps as it needs.

Where the form actually cost us

The first cost was output. Output tokens are the expensive ones, typically priced several times higher than input, and a form has a floor. When a student typed “hi,” V1 generated 792 output tokens, 2,286 characters of JSON. Forty characters of that reached the student. The rest was the form, filled in faithfully: an empty artifact, unused chart coefficients, a teacher signal set to “none.”This example is also told in our product post, Introducing Honorly Intelligence. Here it matters as a data point: the cost of a turn was set by the schema, not by the student. A trivial exchange inherited the full weight of a schema designed for the hardest one. Across the V1 answers in the comparison below, 81% of every character the model wrote sat outside the reply the student read.

The second cost is harder to see, and it is the one we care about most. Writing a valid instance of a large schema is a task in its own right. The model has to track which fields exist, which are required, what each one means, and how to escape a paragraph of tutoring so it survives inside a JSON string, all while also deciding how to teach. The teaching and the bookkeeping share one generation and one budget of attention.

Whether that sharing costs reasoning is contested. Tam and colleagues found that requiring models to answer in JSON or other strict formats lowered their reasoning performance compared with letting them answer freely, with the largest drops on tasks that needed real reasoning.¹ Will Kurt of .txt reran their reasoning tasks on the same open model, gave the free and structured runs the same prompt, and let the structured answer reason before answering. Structured output then scored as well or better, and he traced the original gap to mismatched, underspecified prompts and to the model used to parse answers.² A replication confirmed his result on that model but found free text still ahead on one of the three tasks for GPT-4o-mini,³ and a 2026 preprint finds that models with headroom absorb a JSON format without loss, while models working near their limits lose a great deal, and lose more as the schema grows.⁴

Our measurements cannot settle that debate, because we counted tokens, not reasoning quality. V1’s form, four required parts around a nineteen-field artifact, was far larger than the reasoning-and-answer schemas most of these studies tested, and it imposed itself on every turn, including the turns where it bought nothing. The cost we could measure is the one reported below.

What studying agents taught us

The way out came from looking at a different kind of system. As we developed V1, we spent time inside open-source agent harnesses, particularly OpenAI’s Codex and OpenCode, to understand how they get so much out of the same class of models we were using.

What stood out was not that they used tools. Our harness did too. It was what they did not ask for. They give the model instructions, a set of tools, and a loop, and they never ask for a form. The model reads the task, decides whether to act, calls a tool, sees what happened, and continues. When it is done, it says so in plain language.⁵ The pattern has a research lineage in ReAct, which interleaves reasoning with actions and feeds each observation back into the next step.⁶ Nothing about the final message has to encode the application’s internal state, because the application learns what it needs from the tool calls themselves.

The important part is why this works so well, and here the training history of these models matters. Frontier models are no longer trained only to produce a good next message. They are trained, at scale and with reinforcement learning, to act: to call tools, read results, recover from errors, and finish multi-step tasks in environments shaped like this loop. OpenAI has described training its coding models with reinforcement learning on real tasks in such environments.⁷ Anthropic’s guidance on building agents makes a similar point from the other direction: the most effective systems tend to use simple, composable patterns that let the model work the way it already knows how.⁸

Put a model trained this way inside a loop with well-described tools and it does not need to think about its environment. It recognizes it. Put the same model inside a bespoke contract, where every turn must end in a nineteen-slot artifact object no other system uses, and it has to work out the rules of a situation it has rarely seen before it can get to the problem in front of it.

That is our working explanation for what follows. It predicted the direction of every result below, and it has kept predicting the results of changes we have made since.

The rebuild

Harness V2 is built around that loop. The model receives the student’s message and the context it needs. It can answer directly, or it can call a tool: build an interactive activity, look up a class’s materials, raise a concern with the teacher. The application executes the call, validates it, and returns the result. The model continues until it is ready to respond, and then streams its reply as ordinary text.

Two consequences follow. A short exchange stays short, because there is no form to fill. And a demanding one can take as many steps as it needs, because the model can act, observe, and act again within the turn.

Everything that does not need judgment moved out of the model’s way. Titles and summaries no longer ride along on every answer. Records of what the model did come from the tools it called rather than from a self-report. Permissions, validation and persistence are ordinary software, which is better at them than any model.

What we measured

We compared the two harnesses on the same ten tasks, holding the model constant: a greeting, a simple fact, a concept explanation, three academic-integrity cases, a prompt-injection attempt, an artifact request, a question the tutor should not pretend to know, and a safety-adjacent message.

Mean output in the first response fell from 875 tokens to 390, a 55% reduction, and that response arrived 29% sooner, in 5.0 seconds instead of 7.0.

Almost all of that saving came out of what the model wrote, not how much it thought. Reported reasoning fell only modestly, from 78 tokens to 68 per response. Everything else, the visible output, fell from 797 tokens to 322. The form was not mainly crowding out thinking. It was being written out, at output prices, on every turn.

Values for V1 form, V2 agent in Output tokens.

Task	V1 form	V2 agent	Notes
Greeting	792	15	—
Simple fact	793	311	—
Explain a concept	1098	450	—
Integrity: direct request	1176	1181	—
Integrity: borderline	913	430	—
Integrity: piece by piece	957	519	—
Prompt injection	814	119	—
Build an activity	290	382	V1 form: tool; V2 agent: tool
Admit uncertainty	877	148	—
Safety-adjacent	1040	347	—

The shape of the result matters more than the average. Under V1, nine of the ten tasks landed between 790 and 1,180 tokens, whatever they asked for. Under V2, a greeting costs 15 tokens and a hard integrity case costs about what it did before. The work now follows the task.

Input went the other way. V2’s operating context is larger, because tool definitions and richer instructions live in it, and mean input per first response rose from about 7,200 tokens to 12,600. If we had stopped at the original arithmetic, that would look like a regression.

It is not, because almost all of that context is identical from turn to turn, and providers cache a repeated prefix and bill it at a fraction of the normal rate.⁹ A cache can only reuse a prefix that matches exactly, which is why the order of the context matters. Both harnesses benefited: 89% of V1’s input tokens came from cache. V2 was laid out so that stable material always comes first, and reached 97%. The fresh portion of each turn fell from 754 tokens to 331.

• Cached input
• Fresh input
• Output, including reasoning

Tokens per first response

V18,036

V212,968

Cost per first response, price-weighted

V1100%

V259%

This is the precise sense in which the original math was right and still misled us. One big call really does minimize how much context the model reads. But with caching, re-reading a stable prefix is close to free, and the thing the single call cost us, a fixed block of expensive output on every turn, was the thing we were paying the most for.

The part the benchmark could not see

Token counts were the easy part to measure. The change we noticed first, and the one that mattered most, did not show up in them.

Once the model was free to compose its answer instead of filling one in, it did things it had never managed inside the form. It could decide partway through an answer that an interactive simulation would teach the idea better than another paragraph, build one with a tool, check the result, and write around it. It could carry a plan across several steps. The quality of its ordinary answers improved too, in ways our ten-task suite was never designed to capture.

The clearest consequence is Chalk, the interactive teaching medium we built next. A Chalk answer is a sequence of connected decisions: what the student should try, which representation makes that attempt possible, where to pause, and what to ask afterward. We never managed to build anything like it inside V1’s form.

We later measured a version of this directly. On a set of twelve K–12 prompts, we compared generating interactive content as a separate structured artifact, the way V1 did, against letting the model write it inline as part of its natural response. The inline approach used a median of 341 output tokens instead of 973, and finished in a median of 4.2 seconds instead of 8.4. The cheaper path was also the one where the model composed most freely.

Making a loop dependable

Giving a model the ability to act means taking responsibility for what happens when it does. Most of the engineering in V2 lives here.

Each tool call carries an identity, so a retried request recognizes work that already happened instead of doing it twice. Calls that touch the same activity run in order; independent ones can overlap. Invalid arguments come back to the model as an error it can read and fix, the same way a coding agent sees a failing test.

School networks also taught us to separate a turn from its delivery. A Chromebook on a congested network can drop its connection while the model is still working. In V2, the browser’s stream is only a view of progress; the turn itself is a durable record on the server that keeps running and commits its result. When the student reconnects, the finished work is there. Nothing has to be generated twice, which matters for cost as much as for continuity.

What we take from it

The general lesson is about where to spend a model’s intelligence. A capable model’s reasoning is the most valuable resource in the system, and the rules it has to keep in mind about its own output format can tax it, most when the schema is large and the task is hard. The environments these models were trained in are the environments where that tax is lowest, because there the rules are already second nature.

For anyone building on frontier models, that suggests a different default from the one we started with. Not “how few calls can I make,” but “what environment lets this model do the work it was trained to do, with ordinary software handling everything that does not need judgment.” Ask for structure where it is cheap and native, in tool arguments, and let the response itself be a response.

It also means the harness is never finished. Anthropic has written about revisiting the scaffolding around its own long-running agents as models improve, removing pieces that no longer earn their cost.¹⁰ V1 was a reasonable design for an earlier generation of models. It became the wrong one as the models changed underneath it, and we expect parts of V2 to do the same. The question we now ask about every piece of the harness is the one this rebuild taught us: is this helping the model work, or making it work around us?

Footnotes

1. Zhi Rui Tam et al., “Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models”, 2024. ↩

2. Will Kurt, “Say What You Mean: A Response to ‘Let Me Speak Freely’”, .txt, 2024. ↩

3. Dylan Castillo, “Structured outputs can hurt the performance of LLMs”, 2024, updated 2025. ↩

4. Hengxin Fan, “Capacity, Not Format: Rethinking Structured Reasoning Failures”, arXiv preprint, 2026. ↩

5. OpenAI, “Unrolling the Codex agent loop”. ↩

6. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, ICLR 2023. ↩

7. OpenAI, “Introducing Codex”. ↩

8. Anthropic, “Building effective agents”, 2024. ↩

9. OpenAI, “Prompt caching”, API documentation. ↩

10. Anthropic, “Harness design for long-running application development”. ↩

@misc{honorly_models_are_agents_2026,
  author = {Rosato, Trevor},
  title = {Models are agents now. Treat them like it.},
  year = {2026},
  howpublished = {\url{https://research.honorlyai.com/models-are-agents/}}
}