Model churn is not your problem

On 9 June, Anthropic shipped Claude Fable 5. Three days later a US export-control order switched it off. It came back on 1 July. Eight days after that, OpenAI answered with GPT-5.6 in three tiers. Four events, one month, no way to predict any of them. If your AI capability was tied to whichever model happened to be ahead, the ground shifted four times without you touching a thing.

Zoom out and it is the same story. OpenAI dominated 2024. Anthropic exploded in 2025 with Claude Code. Now Codex is pulling engineers back. Every flip, the same question lands on someone's desk: do we retool? Retrain? Rewrite everything? If your AI capability lives in tool habits and prompt tricks, each flip costs you.

There are two common reactions. One is to chase the winner every time the leaderboard changes. The other is to freeze and say, reasonably enough, that it might be better to wait until the market settles. Both instincts are understandable. Retooling constantly is exhausting. Waiting feels prudent. But both miss the point: your expertise shouldn't be trapped inside one model.

The part that doesn't churn

The thing that survives model churn is the playbook - a process written down precisely enough that a machine can follow it, stored somewhere outside any single AI tool. Not a clever prompt sitting in one person's chat history. Not a habit that only works because one employee knows how to steer one model. A document the business owns.

Rachel Woods, who runs AI operations firm AMP, keeps hers in plain documents, so the same playbook runs in Claude, ChatGPT, or whatever ships next quarter. Most of AMP's training content hasn't changed in three years, through constant tool churn, because the durable layer is process design, not the tech.

That sounds almost too mundane, which is probably why it is easy to overlook. But it changes the question from "which model are we betting on?" to "what do we know that the model does not?"

Satya Nadella, Microsoft CEO, described a version of the same problem at enterprise scale: can you switch out the "generalist" model without losing the "company veteran"expertise built around it? In other words: can the expertise stay with the company while the model remains interchangeable?

That distinction matters. A frontier model can know a lot about the world and still know nothing about how your firm approves a refund, writes a board update, qualifies a lead, handles a tricky client, or decides whether a proposal is good enough to send.

The engineering world has arrived at the same conclusion from the opposite direction¹:

• Prompts and models churn, but rubrics and test suites preserve the definition of good work across upgrades.
• Teams with evals adopt new models in days; teams without face weeks of re-testing.
• Your eval data is the missing moat.

The link density is deliberate. The same idea keeps arriving from different directions: operators, CTOs, investors, engineers, CEOs. Different vocabularies, same shape of thought. Put bluntly: "The model is rented. The eval is owned."

I shipped a small version of the thesis recently: Ookbook, a markdown file that carries the instructions to update itself - so any agent, running any model, can re-run it next quarter. Same deal: the file is owned, the model that runs it is rented.

Of course, there is an obvious objection: documents decay. Anyone who has worked inside a company knows this. A process doc gets written once, praised, forgotten, and six months later it describes a world that no longer exists.

But that is not a reason to avoid writing the playbook. It is a reason to treat it like an asset. Put it somewhere the business controls. Keep version history. Give it an owner. Update it when the process changes. This is not exotic AI governance - it's just the same boring maintenance you already owe any important company document.

Why now, not later

"We'll document things once the models settle down" gets it backwards, in more ways than one.

First, better models make good playbooks more valuable, not less. If the playbook captures the right process, each model upgrade should execute it better. You do not lose the work. You get a better return from it.

Second, writing the playbook was never really the model's job. This is the part people sometimes want to skip. They hope a stronger model will infer the process for them. Sometimes it can infer the obvious bits. But the valuable parts are usually specialist: the judgment calls, exceptions, house style, thresholds, tradeoffs, the things experienced people check almost without noticing.

A recent paper measured something close to this. Human expert-written playbooks lifted agent pass rates by 16.6 percentage points. AI-generated playbooks performed worse than no playbook at all. The value of a playbook is not that it contains generic instructions. The value is that it captures knowledge a generalist model would get wrong.

The playbook does not have to be huge. The same paper found compact and standard-length skills added 19 to 21.5 points, suggesting that after a modest length, more instructions do not add much. The goal is not to write a manual for everything. It is to write down the five things the model would otherwise miss.

So the useful diagnostic is not "have we documented the whole company?" That is too big and too vague. The better questions are:

• Could a colleague run this process in a different AI tool tomorrow?
• If our most experienced employee retired, what would vanish with them?
• What does good output look like here, and how would we know?
• Where does the model need context, and where does it need constraints?
• Which parts of the process are stable enough to write down, and which parts still need human judgment?

Pick one process. Not the whole business. One process that already matters. Write down the five things a model could not infer. Run it in two different tools. See what breaks. Make adjustments (not too defensive).

The strange consequence is that the faster models improve, the less any individual model matters. Capability shifts away from the frontier lab to the shared drive. The model race is now a spectator sport for businesses - enjoy it from the directors' box. And it starts with a simple document.

---

(1) "Evals" and "test suites" are codified tests of what good output looks like, so you can check any model against the same standard. Basically, playbooks for software developers.