model-ledger: A CLAUDE.md That Routes Work to the Cheapest Reliable Executor

Most agent workflows send everything — including trivial mechanical work — to the most capable, most expensive model. model-ledger-claude is a CLAUDE.md template for Claude Code that treats model capacity as a budgeted resource. It routes work to the cheapest executor that can do it reliably, escalates loudly when that executor is not enough, and keeps a written ledger of every escalation so the routing rules are calibrated by evidence instead of intuition.

The whole system rests on a single test: if this output were wrong and I would not notice, do not delegate it. Clustering commits for release notes passes — the hashes are checkable, and a misfiled commit is obvious at a glance. Writing the PR description fails — it is finished, external-facing text, and a subtle mischaracterisation would ship unnoticed. And "find every TODO" is the trap case: it looks like an LLM task, but it is rg -n "TODO|FIXME", and deterministic facts never go to a model.

The second rule is fail loudly, escalate, and write it down. Lower tiers never retry or silently degrade. local-llm.sh refuses oversized input instead of truncating it, enforces a timeout, exits non-zero with a stderr message on any failure so the agent can never mistake garbage for a result, and logs every invocation — successes included, so failure counts have denominators.

The delegation rules are hypotheses, not doctrine. Every Tier 1 call logs itself to docs/model-ledger-auto.tsv. Soft failures — fluent but wrong, the expensive kind — get a hand-written row in model-ledger.md. At each model epoch, ledger-stats.sh turns the auto-log into per-model success rates, per-instruction success, and failure rate by input size. Three soft failures in one task type are enough to pull that category from the delegation list — and conclusions are held loosely, because an epoch's sample is small and models are non-stationary.

The strongest reason is not the API bill. Everything that enters an agent's context rides along for the rest of the session: a 200KB CI log pasted in at turn 5 is roughly 50,000 tokens every later turn must attend over. Tier 0 and Tier 1 keep raw material out of the context window and let only conclusions in. The repo's own illustrative arithmetic for that log over a 40-turn session, at mid-2026 list prices:

The same asymmetry explains why soft failures dominate the ledger: one wrong-but-fluent digest that sends Claude down a false debugging path can erase the savings of a hundred successful delegations. The template applies the discipline to itself, too — CLAUDE.md is re-sent on every request, so its first line asks whether each rule is worth paying for on every turn.

cp CLAUDE.md /path/to/project/         # then fill in the Project section
mkdir -p scripts docs
cp local-llm.sh ledger-stats.sh scripts/ && chmod +x scripts/*.sh
cp model-ledger.md docs/

git log --oneline -50 | ./scripts/local-llm.sh "Cluster these commits by theme"
./scripts/ledger-stats.sh              # after some real use

The default Tier 1 model, kimi-k2.7-code:cloud, runs on Ollama Cloud through the same ollama run interface and needs an ollama.com sign-in; for a fully local, offline Tier 1, point LOCAL_LLM_MODEL at a local tag such as gemma3:12b or qwen3-coder. An optional PostToolUse hook turns the escalation rule from prose into mechanism. Only the instructions file is Claude Code specific — the scripts, ledger, and criterion port unchanged to AGENTS.md, Cursor rules, or Aider.

We run agents all day across client work, and the waste we kept seeing was not one big expensive call — it was hundreds of small ones: logs pasted whole, TODO hunts sent to a frontier model, digests nobody checked. Routing by intuition did not stick. Routing by a written, falsifiable criterion, with a ledger that tells you when the criterion was wrong, did.