How should you start choosing a model?

The mistake is to start with the leaderboards. A better order is to start with your task: what are you actually asking the model to do, and how would you know it did it well? The model comes last, after you know what you need.

Anthropic’s own guidance frames model choice as balancing a few criteria for your specific case, and it is blunt about the value of doing this first: “Knowing these answers in advance will make narrowing down and deciding which model to use much easier.”^[1]

So write the criteria down before you compare anything. What capabilities does the job require? How fast must the answer come back? What is your budget? Where is the data allowed to go? A task with no privacy constraint and loose latency has very different answers than one handling medical records in real time. The next pages take these one at a time.

Two real tasks make the divergence concrete. A support-ticket triage bot that sorts incoming messages into five queues needs modest capability, must answer in under a second so it doesn’t stall a live chat, runs at high volume so budget adds up fast, and rarely touches anything sensitive — a small, cheap, fast model wins on nearly every axis. A contract-review assistant reading merger agreements needs strong reasoning over long documents, can tolerate a slower answer since a lawyer is reading the output anyway, runs at low volume so budget barely matters, and handles material that cannot leave the firm’s systems — a frontier or self-hosted model wins instead, for close to the opposite reasons. Same four questions, two very different answers, because the task was different going in.

Skipping this step doesn’t remove the decision, it just moves it later and makes it more expensive. Without written criteria, “which model” quietly turns into “which model everyone else is using” or “which model I already have an API key for” — reasonable defaults, but ones you’re choosing by accident rather than because they fit the job. Twenty minutes writing down the four answers above is what turns model selection back into a decision you’re actually making.