What are AI agents good at, and where do they fail?

Agents shine on open-ended tasks with a lot of small steps, where the path is not known up front and there is feedback to learn from at each turn: working through a codebase, researching across many sources, driving a tool with messy inputs.

The catch is arithmetic. Reliability compounds. If each step is right 85% of the time, ten steps in a row succeed only about 0.85¹⁰, which is roughly one in five attempts. Miss a step in the middle of a long task and everything downstream inherits the mistake — often silently, since a wrong-but-plausible action rarely looks different from a right one until much later.

Capability has been climbing, but reliability over long horizons has lagged behind it. METR measures this directly, by finding the task length at which a model succeeds half the time, calibrated against how long the same task takes a skilled human. Their headline result: the length of task frontier models can reliably complete has roughly doubled every seven months, for six years running.^[1] Even so, the gap between short and long tasks stays wide. At the time of their report, models solved nearly every task that took a human under four minutes, but fewer than one in ten of the tasks that took a human over about four hours — and Claude 3.7 Sonnet, a frontier model at the time, sat at roughly a one-hour horizon for 50% reliability.^[1] Picture what those durations mean in practice: a four-minute task looks like fixing an off-by-one error in a function you already understand, while a four-hour one looks like adding a feature that touches several files and needs its own tests — the kind of work where nothing tells you it went wrong until you actually run it. This is the core reason agents are paired with guardrails and human checkpoints rather than turned loose. The workflows and rules pages cover the fixes.