Why do we need structured output from a model?

A language model produces text. That is fine when a person reads the answer, but it is a problem when a program has to. If your code expects a customer’s name, an order total, and a date, a paragraph of friendly prose is hard to work with. You want a small, predictable packet of data instead.

Structured output is the practice of getting that packet. Instead of free text, you ask the model to return data in a fixed shape, almost always JSON, so your code can read each field directly without parsing sentences.

The naive approach is to ask politely in the prompt: “reply with JSON only.” This often works, but not always. The model might wrap the JSON in an explanation, drop a required field, add an extra one, or invent a value where you wanted a fixed choice. Each of those breaks the code downstream. Model providers built dedicated features for exactly this reason. OpenAI describes the goal as ensuring the model “will always generate responses that adhere to your supplied JSON Schema, so you don’t need to worry about the model omitting a required key, or hallucinating an invalid enum value.”^[1]

Take one field as an example: a status your code expects to be exactly active, paused, or cancelled. Asked in plain prose, a model might return “currently active” or “Active” with different capitalization, both reasonable to a person reading it and both a silent bug to code checking if status == "active". The model is not being careless. Free text has no built-in notion of a fixed set of allowed values, so it fills that gap the same way it fills any other one you leave open, with whatever sounds right.

So there are two separate problems. One is making the output valid (well-formed JSON that parses at all). The other is making it match your shape (the exact fields and value types you asked for). The rest of this topic covers how schemas and constrained decoding solve both, and where they stop.