How does batching cut cost?

Most API calls are synchronous: you send a request and wait a few seconds for the answer. Batching flips that. You submit a large pile of requests at once, the provider works through them in the background, and you collect the results later. In return for giving up the instant answer, you pay less.

The discount is steep and consistent across providers. Anthropic’s Message Batches API charges 50% of standard prices on input, output, and special tokens alike.^[1] OpenAI’s Batch API offers the same “50% cost discount compared to synchronous APIs.”^[2] The savings come from the provider being able to schedule the work when it has spare capacity rather than answering on demand.

The cost is latency. Both providers quote a completion window of up to 24 hours, though most batches finish well inside it — Anthropic notes most complete within an hour, and lets you retrieve results once everything is done or after 24 hours, whichever comes first.^[1] OpenAI similarly states batches complete “within 24 hours (and often more quickly).”^[2] Anthropic caps a single batch at 100,000 requests or 256 MB, whichever comes first.^[1]

That makes batching a good fit for work that does not need an immediate reply: running evaluations, classifying or summarizing large datasets, embedding a document collection, or moderating a backlog of content.^[1][2] It is the wrong tool for anything a person is waiting on. For interactive work, the cost lever to reach for instead is prompt caching.

A concrete case: a team processing 50,000 support tickets overnight to pull out category, sentiment, and whether each one needs escalation is exactly the batching use case. Nobody is watching a progress bar at 2am, and at that volume the savings are real money, not rounding error. One detail worth building into any pipeline like this: Anthropic only keeps a batch’s results downloadable for 29 days after creation.^[1] The batch record itself sticks around longer, but the results do not, so a job that runs on a schedule needs something on the other end pulling results down inside that window, not just firing the batch and walking away.