AI stacks
What is an AI stack, and is there one standard setup?
An AI stack is the set of pieces you put together to ship something with a model: the model itself, a serving layer, orchestration, retrieval, tools, and evaluation. There is no one true stack. There are recurring setups, each shaped by the job it does.
People talk about an “AI stack” the way they talk about a web stack, as if there were one fixed list of parts. There is not. What there is instead is a handful of layers that keep showing up, plus a few common ways to combine them. Once you know the layers you can read any setup and tell what it is doing.
The layers, roughly bottom to top: a model that does the reasoning; a serving layer that loads the model and answers requests, whether that is a hosted API or a local runtime; an orchestration layer that strings several model calls and steps together; a retrieval layer that fetches relevant text from your own data so the model can answer from it[1]; a tools layer that lets the model act on the outside world, often through a shared protocol like the Model Context Protocol[2]; and an evaluation and observability layer that tells you whether the thing actually works. Not every project uses all of them.
The reason there is no single stack is that the layers you reach for depend entirely on the job. If you want a model to answer questions about your company handbook, you build a retrieval setup and barely touch tools. If you want a model to file tickets and send messages, you build a tools and orchestration setup and may skip retrieval. If you want to run everything on your own hardware for privacy, the serving layer becomes the part you care about most. Same parts, different center of gravity. The layers are also provider-agnostic: a fully hosted setup buys serving as an API call; a fully self-hosted one runs its own serving layer on local hardware (see the local-inference stack). Same six layers either way. Only who operates each one changes.
The pages that follow walk through five common setups, one per page: the RAG stack for answering from your data, the agent stack for taking actions, the local-inference stack for running on your own machine, the fine-tuning stack for teaching a model new behavior, and the evaluation-and-observability stack for knowing it all holds up in production. Read them as recipes, not as a ladder you must climb in order.
References
- What is Retrieval-Augmented Generation (RAG)? — Pinecone
- Introduction to Model Context Protocol — Anthropic
Common questions
- Do I need every layer to build something useful?
- No. A single prompt to a hosted model is already a working setup. You add a layer only when the job needs it: retrieval when the model must answer from your data, tools when it has to act, observability when it runs in production.
- Is there a best AI stack to learn?
- There is no single best one. Learn the layers and what each does, then pick the smallest combination that answers your problem. The same model shows up in a chatbot, a RAG system, and an agent; what changes is everything around it.