Token Budgets Are the New Payroll: Why Model Routing Is Becoming a Management Decision

AI spend is becoming visible

The first generation of workplace AI was easy to budget. A company bought a fixed number of seats, multiplied them by a monthly price, and treated the result like any other SaaS line.

Agentic tools change that model. They can run longer, make many calls and consume very different amounts of compute for apparently similar requests. The price of experimentation is no longer fully predictable from the number of users.

Uber offers a useful, if extreme, signal. After reaching its annual AI budget earlier than expected, the company introduced a monthly limit of $1,500 per employee and per agentic coding tool. Employees can see their consumption and request an exception. This is not evidence that every company will spend at that level. It is evidence that AI usage is becoming a governed operating cost rather than an invisible feature of a subscription.

Once spend becomes visible, model choice stops being a purely technical preference. It becomes a portfolio decision: where does additional model capability change the outcome enough to justify its cost?

What routing actually does

Model routing assigns different classes of work to different models. A routine extraction task may go to a small model. An ambiguous architectural decision may go to a stronger one. A cascade can start with the cheaper route and escalate when a deterministic check, confidence signal or evaluator rejects the result.

Four patterns cover most practical implementations:

1. Rules: route by task type, risk, data sensitivity or expected length.
2. A learned router: predict which model is likely to satisfy the quality threshold for a specific request.
3. A cascade: start with a lower-cost model and escalate after a failed check.
4. Parallel attempts: compare several outputs when the decision is important enough to justify the extra cost.

Research such as RouteLLM shows that preference data can be used to learn cost-aware routing policies. Anthropic’s guidance on effective agents also describes routing and orchestrator-worker workflows as useful patterns. None of this creates a universal saving rate: the result depends on the workload, the available models and the definition of acceptable quality.

The cheapest token is not always the cheapest outcome

A routing dashboard can show a falling inference bill while the real cost of the system rises. A weaker model may create more retries, longer reviews, customer-facing mistakes or production incidents. Those costs are rarely priced into the model comparison.

This is why the unit of optimization should be a completed, accepted outcome, not a million tokens. For every route, teams should measure at least:

• cost per accepted result,
• first-pass acceptance rate,
• retry and escalation rate,
• human review time,
• defect or rollback rate,
• latency experienced by the user.

If these signals are missing, selecting a cheaper model is a purchasing decision disguised as architecture.

Start with context, not a router

Many systems can cut cost before adding multi-model infrastructure. System prompts grow, the same files are reread, logs are pasted instead of queried, and agents repeat work because state is poorly managed.

Context discipline is often the first intervention:

• keep instructions concise and versioned,
• retrieve only the information required for the current step,
• expose logs and data through tools instead of pasting entire datasets,
• cache stable results,
• detect loops, retries and repeated reads,
• use deterministic code where a model is unnecessary.

A router makes inefficient calls cheaper. Better context design removes some of those calls altogether.

Evaluation is step zero

Routing works only when the organization can answer a deceptively difficult question: good enough for what?

That answer cannot come from a general leaderboard. It comes from a test set representing the organization’s own work: real requests, expected outcomes, known edge cases and consequences of failure. It also requires prompt and outcome logs that make comparisons reproducible.

Models and prices will change. A company may replace one provider with another several times. Its evaluation set can survive those changes and improve after each one. That makes evaluation data more durable than any particular routing implementation.

Better questions for the next AI budget review

1. What is our cost per accepted outcome, by task type?
2. Which tasks demonstrably need the strongest available model?
3. Where is a weak result detected, and what happens if detection fails?
4. How often do cheap routes escalate or retry?
5. Could we compare providers on our own evaluation set within one week?
6. Who owns both the spend metric and the quality metric?

Where Nomtek can help

Nomtek can help teams map AI spend to real workflows, build workload-specific evaluations and introduce routing incrementally. The practical sequence is usually evaluation first, context discipline second, simple rules third and learned routing only when volume justifies the additional system.

‍

Sources

• Bloomberg reporting republished by the Los Angeles Times: Uber’s $1,500 monthly limit
• RouteLLM: Learning to Route LLMs with Preference Data
• Anthropic: Building effective agents

‍