ferlogs

~/writing/models-by-role

·Agents·8 min read·Fernando Valdez

Models by role: cheap for the reading, expensive for the decision

One strong model until an eval says otherwise. Then a cheap reader, a middle worker, and a judge that did not write the thing it grades. Raise effort before you raise the model.

Before you build a router, run one strong model and turn the effort down on the tasks that are mostly reading. Splitting the work pays when a chunk is logs, queues, or a search through code. A cheap model does that reading for a fraction of the price, and the expensive one never has to see the raw dump.

The split is not free. Anthropic measured it on their multi-agent research system. A lead with subagents beat a single agent by 90.2% on their research eval, but a single agent already spends about four times the tokens of a chat, and a multi-agent setup about fifteen times. Token volume alone explained about 80% of the quality gap. You can buy a better answer with tokens. You can also buy a more expensive version of the same answer. Evals are how you tell those apart.

Roles, not a lineup

I keep four jobs. I do not let one model hold all of them, and I do not add a fifth because a new model came out on a Thursday.

  • Orchestrator. The expensive model. It holds the question, decides what to ask next, and writes the conclusion a person will read. A diagnosis. The weekly review of lessons. It sees summaries, not dumps.
  • Reader. The cheap model. Search logs, look at a queue, summarise a transcript. The output is a short note with the lines that mattered. If the reader returns the whole file, you paid for a copy.
  • Worker. The middle model. Follow an error through the code. Draft release notes from a diff. It can be wrong in ways a reader cannot, so it does not get the last word.
  • Judge. A strong model, in a clean context, that did not write the thing it is grading. A small judge flips verdicts between runs, and then you edit the wrong file. This is the wrong place to save money.

There is a fifth seat I almost never use: the model I call when the others are stuck, or when I am calibrating the judge. It is too expensive to be the default and too useful to pretend it does not exist. It is a tool for a hard afternoon, not a role in the diagram.

Prices move, so I am dating them. Anthropic's API card on 6 Oct 2026, per million tokens, input then output: Haiku 5.5 at $0.10 / $0.50 up to a 100k prompt, Sonnet 5.5 at $2 / $10, Opus 5.5 at $4 / $20, Fable 5.1 at $10 / $50. Check the card before you budget a quarter on these. The shape is what I expect to survive the next price change: the reader is an order of magnitude cheaper than the orchestrator, and the judge sits with the orchestrator.

Models by role An expensive orchestrator sends independent reads to a cheap model and judgement work to a middle model. Both return short notes. The orchestrator writes the conclusion. A strong judge in a clean context can reject it, with a cap on rounds. orchestrator · expensive reader · cheap logs, queues, transcripts worker · middle code path, release notes short notes, not dumps synthesis · same orchestrator reject, capped judge · strong, clean context did not write the thing it grades
Every handoff carries a goal, an output shape, the tools, and a limit. What comes back is a note.

Anthropic's rule, before you jump a model tier: ask whether it failed to try or failed to know. If it skipped files or tests, raise the effort. If it had the context in front of it and still got it wrong, raise the model. Most of the "we need Opus" afternoons I have watched were the first one.

Patterns that earn the split

A pattern is a reason to spend the extra tokens. If you cannot name which one you are in, you are just calling more models.

  • Chain with gates when the steps are fixed and you can check each one. Release notes are this: diff, classify, draft, check the format. A gate can reject without a person reading a bad draft.
  • Route when requests differ in difficulty. A question with one answer goes to the cheap model. A diagnosis goes to the orchestrator. The router itself should be boring. If the router needs a long prompt, it is a second orchestrator and you should notice.
  • Parallelize independent reads. Logs, queues and metrics at once, then one model reads the three notes. Do not parallelize steps that depend on each other and call it speed.
  • Orchestrator and workers when you do not know the subtasks yet. An incident is this. A mechanical migration usually is not. If you can write the steps down before you start, it is a chain, and a chain is cheaper.
  • Evaluator and optimizer when you can say what good is. The judge can veto. Cap the rounds. Stripe has published a cap of two failed CI rounds before a person takes the task. An agent that retries without a ceiling spends money and gets worse. I use the same cap.

Every handoff to a worker carries the goal, the output shape, the tools and the limits. What comes back is a short summary. If you pass the whole transcript down, you have built a more expensive single agent and called it a team.

Where it runs

Inside the coding agent, a subagent can pin a model, an effort, a tool list and a turn cap. That is enough for interactive work. I do not need a platform to stop a reader from using the expensive model. The frontmatter does it.

Unattended, the same roles go in CI. A read-only tool allow-list, structured output, a schema, and a budget. A later step reads the JSON. It notifies a person or, after that person says yes, it applies. The agent does not apply. The budget is part of the design. A loop without --max-budget-usd or the equivalent is a loop you have not priced.

A managed platform fits work that does not touch your internal systems. Scheduled runs, a shared memory you can version, a grader with a rubric. The moment the job needs your queues or your database, you are back to a sandbox you control and a review. Do not point a hosted agent at production because the demo made the happy path look easy.

What I measure before I add another model

One strong model, with the effort dialed per task, is the baseline. I only keep a cheaper reader, or a separate judge, when an eval says the quality held and the cost moved. OpenAI's practical guide says the same order: prototype with the most capable model, pin the baseline with evals, then step down. Stepping down first, to save money on a task you have not measured, is how you ship a cheap wrong answer and then blame the prompt.

Two published systems are worth copying in miniature. Spotify's Honk work gives the agent few tools and a single verifier, then a judge compares the diff with what was asked and vetoes about a quarter of sessions. Uber's uReview runs several generators and a model whose job is to score comments and drop the ones that are not worth a person's time, across a very large weekly change volume. The pattern is not "more models". It is a role with a check, and a way to throw the result away.

The rule

One strong model until an eval says a cheaper reader, or a separate judge, earns its place. Raise effort before you raise the model. Cap the rounds.

If you cannot say which role a call is serving, it should not be a separate call. The diagram is there to stop me adding boxes.

Where this comes from