The field guideField notes · 8 min read

How to choose, and when one model beats another

There is no single best AI model — only the best fit for the job in front of you. This is the field guide: start from the work you actually do, then the levers that matter, a heuristic for when to pay up, and an honest map of what's comparable to what.

Skip to the quiz

Start here

By what you're doing

Find the kind of work you actually do and start with a model that clears the bar for it. Most everyday jobs never need the frontier — each pick and the factor that decides it are pulled live from the catalog.

The jobWhat decides itA sensible pick

Everyday questions & writing

Ask, explain, draft an email, rework a paragraph.

Speed & cost — quality is a given

Grok 4.1 Fast$0.28/M · 83/100

Clears the bar. Step up to the frontier only when the writing must land a subtle, high-stakes tone.

Long documents & research

Summarize a report, cross-read many sources, pull out the answer.

Context window & recall

DeepSeek-V4 Pro$0.54/M · 86/100

Clears the bar. Step up to the frontier only when you need dense synthesis across dozens of sources at once.

Content & marketing at scale

Generate product copy, variations, and social posts by the thousand.

Cost per thousand runs

Qwen3-30B-A3B-Instruct-2507$0.09/M · 80/100

Clears the bar. Step up to the frontier only when a flagship campaign needs a distinctive, on-brand voice.

Data & spreadsheets

Clean a table, categorize rows, extract fields into a tidy format.

Structured accuracy

GPT-5.4 mini$1.69/M · 83/100

Clears the bar. Step up to the frontier only when the logic is intricate and a wrong cell is expensive.

Customer support at volume

Answer common questions, route tickets, draft replies around the clock.

Latency & cost per ticket

Qwen3-30B-A3B-Instruct-2507$0.09/M · 81/100

Clears the bar. Step up to the frontier only when a hard escalation needs real reasoning, not a canned reply.

Coding & agents

Write and fix code, or run a tool-using agent through many steps.

Reliability over long tasks

GPT-5.4 mini$1.69/M · 82/100

Clears the bar. Step up to the frontier only when the work is a multi-file refactor or a long autonomous run.

Translation & multilingual

Translate documents and chat fluently across many languages.

Language coverage

Mistral Large 3$0.75/M · 86/100

Clears the bar. Step up to the frontier only when the text is literary, legal, or highly idiomatic.

Creative & brainstorming

Spin up ideas, names, outlines, and first drafts to react to.

Voice & range

DeepSeek-V4 Pro$0.54/M · 83/100

Clears the bar. Step up to the frontier only when you want a genuinely distinctive, publishable voice.

The honest through-line

In every one of these 8 jobs, a non-frontier model clears the bar — averaging about $0.71/M. The frontier flagship, Claude Fable 5 at $20/M, is for the hard, high-stakes, long-horizon minority. Reach for it on purpose, not by default.

Each pick is the best capability-per-dollar model that clears a sensible quality bar for that job, chosen live from the catalog. The score is a 0–100 average across the axes that decide the job; price is blended at a typical 3:1 input-to-output ratio.

The framework

A repeatable way to decide

Six factors, weighed against each other, plus one heuristic that keeps you from overpaying: start cheap, escalate on failure.

Six levers, weighed against each other

There's no single best model — only the best fit for a task's constraints. Every choice trades these six against one another.

Capability

How hard is the task, really?

Most work — answering a question, drafting an email, tidying a spreadsheet, summarizing a report — is solved by mid-tier models. Capability only becomes the binding constraint on genuinely hard problems: book-length analysis, novel reasoning, multi-file code or long-horizon agents where small errors compound.

Lean light: The task is well-defined, has a clear format, or you can verify output cheaply.
Escalate: Quality failures are expensive, the reasoning is open-ended, or errors cascade across many steps.

Cost

What does one call cost — times your volume?

Price is per million tokens, so the gap between tiers explodes at scale. A frontier model can cost 20–50× a budget tier per token. At a million calls a day that is the difference between a rounding error and a line item that gets your project cancelled.

Lean light: High volume, thin margins, or a workload you run continuously.
Escalate: Low volume, high stakes per call, or the human time saved dwarfs the API cost.

Latency

Is a human waiting on this token-by-token?

Reasoning models think before they answer, which is great for correctness and terrible for a chat cursor blinking at a user. Fast tiers and always-on reasoning sit at opposite ends; match the model's temperament to whether the response is interactive or batch.

Lean light: Interactive UX, autocomplete, or real-time agents where responsiveness is the product.
Escalate: Batch jobs, overnight pipelines, or one-shot answers where being right beats being fast.

Context

How much must the model hold in working memory at once?

Context windows have ballooned, but big windows cost more and recall is not free — a model can technically accept a million tokens and still lose the thread in the middle. Size the window to the job, and prefer feeding in only what's relevant over stuffing everything in.

Lean light: Short prompts, a single document, or a setup that surfaces only the relevant passages.
Escalate: Whole books or codebases, long transcripts, or tasks that weigh many documents at once.

Openness & privacy

Can the data leave your walls?

Open-weight models can run on your own hardware, which changes the calculus entirely: no per-token bill, no data egress, full control over versioning and fine-tuning. You trade that for the operational burden of actually running inference well.

Lean light: Regulated data, on-prem requirements, or a need to pin and fine-tune a fixed version.
Escalate: You want the absolute frontier and would rather rent capability than run a GPU fleet.

Modality

What goes in and what comes out?

Text-only models score near zero on vision for a reason — they can't see. If your inputs include images, PDFs, audio, or video, modality is a hard filter, not a preference. Check it first; it eliminates whole columns of the catalog before cost ever enters the picture.

Lean light: Pure text in, pure text out.
Escalate: Documents with layout, screenshots, charts, audio, or any multimodal input.

The core heuristic

Start cheap. Escalate only when a model demonstrably fails — never on a hunch.

1
Filter on hard constraints
Eliminate anything that can't take your modalities, fit your context, or meet a privacy requirement. These are pass/fail — no amount of capability buys back a model that can't see your images or run on-prem.
2
Start at the cheapest plausible tier
Pick the fast or balanced model that could plausibly do the job. Build your prompt and evaluation against it. Most tasks stop here — and you've learned the shape of the problem cheaply.
3
Measure where it fails
Run real inputs and look for specific failure modes: dropped steps, shallow reasoning, format drift, hallucinated facts. Vague dissatisfaction isn't a signal; reproducible failures are.
4
Escalate one tier — only on evidence
Move up to the next tier and re-test the same failures. If they vanish, the upgrade earned its cost. If they don't, the problem is your prompt, your context, or your task design — not the model.
5
Route, don't standardize
In production you rarely need one model for everything. Send the easy 90% to a cheap tier and route only the hard cases up. That blended cost is what good systems actually run on.

Flagship or workhorse

Opus vs Sonnet vs Haiku

The question everyone asks. The answer is capability-per-dollar versus task difficulty — read straight from the live dataset.

frontier

Claude Opus 4.8

Input: $5 /M
Output: $25 /M
Context: 1M
Capability: 94

Capability per dollar9.4

capability 94 ÷ blended $10/M

Reach for the flagship when failure is expensive and the task is genuinely hard — book-length analysis, long-horizon agents, multi-file refactors, frontier reasoning. You're buying reliability on problems where a cheaper model's mistakes cost more than the price gap.

Worth it on the hard 10%.

Opus costs roughly 5× Haiku per token — for +16 capability points. The flagship earns that gap only when the task actually demands it.

balancedDefault pick

Claude Sonnet 4.6

Input: $3 /M
Output: $15 /M
Context: 1M
Capability: 88

Capability per dollar14.7

capability 88 ÷ blended $6/M

The default. Strong enough for the overwhelming majority of real coding, analysis, and writing, at a fraction of flagship cost. If you don't have a specific reason to go up or down, start here.

The right call by default.

fast

Claude Haiku 4.5

Input: $1 /M
Output: $5 /M
Context: 200K
Capability: 78

Capability per dollar39.0

capability 78 ÷ blended $2/M

When a human is waiting or the volume is enormous: chat, everyday questions, autocomplete, classification, extraction. It clears routine work at a price and latency the flagship can't touch — and routes the rare hard case upward.

Unbeatable on volume.

The pattern generalizes across every provider: a flagship, a balanced all-rounder, and a fast tier. The names change; the decision doesn't. Match the tier to the difficulty of the task, not the prestige of the model.

Where the weights live

Closed frontier vs open weights

GLM, Kimi, DeepSeek, Qwen, and Llama made open weights genuinely competitive. Picking a side is about cost, privacy, and control — with eyes open to the tradeoffs.

Closed frontier

Claude, GPT, Gemini, Grok

Peak capability — the highest scores on the hardest benchmarks still live behind an API.
Zero ops: no GPUs to provision, no inference stack to tune, no model weights to host.
Managed safety, tooling, and rapid model updates handled by the lab.

The honest tradeoffs

You pay per token forever, and prices are set by the vendor.
Your data leaves your perimeter on every call.
Models can be deprecated or change behavior under you with little notice.

Open weights

DeepSeek, Qwen, GLM, Kimi, Llama, MiniMax

Run on your own hardware — data never leaves, which clears regulated and on-prem use.
Marginal cost approaches the price of electricity at high, steady volume.
Pin a version forever and fine-tune it to your domain; no surprise deprecations.

The honest tradeoffs

You own the ops: serving, scaling, and GPU economics are now your problem.
At low or bursty volume, a managed API is usually cheaper than idle accelerators.
The very top of the frontier is still, narrowly, a closed-weights game.

What's comparable to what

The capability-class map

Every model in the catalog, grouped into tiers and ranked by core capability. Scan a column to see which open or rival models sit in the same class as Opus, Sonnet, or Haiku.

Frontier

Opus-class flagships. The ceiling on capability — and price.

Balanced

Sonnet-class all-rounders. The default for most real work.

Reasoning

Think-first specialists for math, science, and hard logic.

Fast

Haiku-class workhorses for high-volume, latency-sensitive jobs.

Open weights

Frontier-adjacent models you can self-host and own.

Capability is a 0–100 composite of reasoning, coding, math, knowledge, and agentic scores. Blended price assumes a 3:1 input-to-output ratio. Both are computed live from the dataset.

Myth-buster

Things people get wrong

Short, honest answers to the assumptions that quietly burn budget and ship worse products.

Is the newest, most expensive model always the best choice?

No. It's the most capable on the hardest tasks — but for routine work you're paying a premium you can't measure. The best choice is the cheapest model that reliably clears your quality bar, which is usually a tier or two down from the flagship.

Is open-weight always cheaper than a closed API?

Only at scale. Self-hosting trades a per-token bill for fixed GPU cost. Below a steady, high volume those accelerators sit idle and a managed API wins outright. Open weights pay off on privacy and control long before they pay off on price.

Does a bigger context window mean better answers?

Not by itself. A window is capacity, not comprehension — recall in the middle of a huge prompt is unreliable, and you pay for every token you stuff in. Targeted retrieval beats a maximal context window almost every time.

Are reasoning models just better versions of normal models?

They're different tools. Reasoning models trade latency and token cost for deeper step-by-step thinking, which is a win on math and logic and a waste on a chat reply. Match the temperament to the task, not the marketing.

Should I just standardize on one model for everything?

Tempting, but wasteful. The strongest systems route: a cheap tier handles the easy majority and escalates only the hard cases. One model for everything means overpaying on the easy work or underperforming on the hard.

Do higher benchmark scores guarantee it'll be better for me?

Benchmarks measure averaged, public tasks — not yours. A two-point lead on a leaderboard can vanish on your data, your formats, your edge cases. Treat benchmarks as a shortlist filter, then test the shortlist on your actual workload.

Now put it to work

Take the quiz

Put the framework to work on real scenarios and pressure-test your instincts.

Compare head-to-head

Line up any models side by side on price, context, and capability.

Browse the catalog

Filter the full roster by tier, modality, provider, and openness.