# Opening the notebook https://research.honorlyai.com/opening-the-notebook/ Trevor Rosato · 2026-10-27 We have spent the past few months running experiments while building an AI tutor for schools, on how it should teach, what it should remember, why it follows its rules, and what it costs to run. Most of that work lived in benchmark scripts and internal notes. Starting today, we publish it. Building an AI tutor turns out to be a long series of experiments. Before we change how the tutor answers, we measure the old answer against the new one. Before we trust a prompt with students, we attack it. Before we buy hardware, we find out how much model fits on it. Since we started building Honorly this spring, the results of that work went straight into the product and nowhere else. They lived in benchmark scripts, result files and long internal notes that a handful of people read. Starting today, we publish them. Honorly Research is where that work now goes. We are starting with seven pieces of research today, and six more will follow over the next three weeks, all of it done over the past three months. Each piece carries two dates: when it was published, and when the work behind it was done. The oldest of it dates to August. Some of it finished last week. ## Why publish The first reason is that the problems are not ours alone. Every team putting a language model in front of children runs into the same failures. A tutor that refuses to write an essay will often write it anyway once the request is dressed up as a study aid. A model that keeps its tutoring rule with its reasoning switched off can drop it almost entirely with its reasoning on. We found these the slow way, with controls and repeated runs, and there is no reason the next team should have to find them again. The second reason is that publishing makes the work better. A result you only have to explain to yourself can survive on a single run and a good feeling. A result written for strangers needs a control, a sample size and an honest account of where it stops. Our main model-behavior study was pre-registered, with hypotheses and stopping rules committed before the data existed. We report the hypotheses that failed next to the ones that held, and we label exploratory work as exploratory. Writing for an outside reader is the discipline that keeps us from fooling ourselves. The third reason is the one the company exists for. We want excellent tutoring to be abundant, and abundance is mostly a question of cost: how many tokens an answer needs, how much hardware a model needs, how cheaply a capable model can be served. Much of what we learn about that applies to anyone building on open models and consumer GPUs. It is more useful in public. ## What we work on The work is organized the way the system is built, from the student’s screen down to the silicon. **Learning experience** asks what an AI answer should look like when the goal is for the student to do the thinking. Today that means [Chalk](https://research.honorlyai.com/handing-students-the-chalk), our medium for answers students work with rather than read, and [a study of the essay requests](https://research.honorlyai.com/the-refusal-that-writes-the-essay-anyway) that slip past a tutor’s integrity rules. **Learner models** asks how a tutor comes to know one student, and a school comes to know thousands, without fooling itself. Two pieces follow shortly: [a tutor that keeps a lab notebook](https://research.honorlyai.com/a-tutor-that-keeps-a-notebook) on each student, and [the agents that read those notebooks](https://research.honorlyai.com/a-team-of-data-scientists-for-every-school) across a whole school. **Agent systems** asks where a model should spend its intelligence. Our first tutor ended every turn by filling out a large structured form. [Letting it work like an agent](https://research.honorlyai.com/models-are-agents) cut its output by more than half. **Model behavior** asks why a model follows its instructions, when it stops, and whether the difference can be seen from the inside. We are publishing three studies of tutor models under pressure, starting with [a model that knows the checkmark it writes is false](https://research.honorlyai.com/the-model-knows-the-checkmark-is-false), and [an essay on looped models](https://research.honorlyai.com/looped-models-are-burning-the-runway) and what they mean for anyone who wants to keep reading a model’s reasoning. **Inference** asks how much capable intelligence fits on hardware cheap enough that no student has to ration it. Our reports on serving large open models from a single gaming card arrive in November, along with the case for building on gaming cards at all. ## What we commit to We will publish regularly, not once. New work will appear here as it is finished, and the backlog that predates this site will keep arriving over the coming weeks. We will report what we measured, under what conditions, and where the result stops, in a short methods note at the end of every piece. When an example was written by our team rather than generated by a model, we will say so. When a result comes from simulation or a single machine, we will say that too. We will publish results that did not go our way. A hypothesis that fails under a fair test is a finding, and often a more useful one than the hypothesis that held. Some things stay private: our production prompts, tuned thresholds and the specific teaching knowledge that makes Honorly work for the schools that use it. Everything else we learn along the way is meant to be read.