Common failure modes
Why does the model miss things buried in a long prompt?
When a prompt gets long, models tend to use information at the start and end better than information stuck in the middle. Keep prompts short and place the important parts at the edges.
It is tempting to paste everything into the prompt and trust the model to find what matters. With a long input, that backfires. The model can read all of it, but it does not use all of it equally well.
A widely cited study tested where relevant information sits inside a long input. The finding: “performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts,” and this held “even for explicitly long-context models.”[1] The pattern is often described as U-shaped: the edges of a long prompt are used well, the middle is the blind spot.
This has two practical consequences. First, more context is not automatically better. If you paste ten documents to answer a question that needs one, the nine extra ones are noise that can bury the signal. Trim to what the task actually needs.
Second, placement matters. Put the most important material where the model attends best. For large inputs, Anthropic’s guidance is to “put longform data at the top” of the prompt, above your question and instructions, and notes that putting the query at the end “can improve response quality by up to 30%… especially with complex, multi-document inputs.”[2] For long-document tasks it also suggests asking the model to quote the relevant parts first, which “helps Claude focus on the relevant content and ignore the rest of the document.”[2]
So the fix is less about clever wording and more about layout: keep the prompt as short as the task allows, and place the parts that matter at the start and the end.
One practical tell separates this failure from the others in this topic. If the model gets a fact wrong when it is buried in the full-length input but gets the same fact right once you shorten the prompt or move that part to the top, position was the problem, not the model’s knowledge or your wording. That test is worth running before you rewrite an entire prompt chasing a different fix.
There is also a version of this problem too large for trimming and placement to solve: when the material you need is scattered across many long documents rather than sitting in one, you cannot put everything at the start and the end at once. At that point the fix is not a better prompt but a different tool, retrieval that searches your documents first and hands the model only the few passages that are actually relevant, so it is never handed the whole haystack to search through in the first place.
References
- Lost in the Middle: How Language Models Use Long Contexts — Liu et al., Transactions of the ACL
- Prompting best practices — Anthropic