Choosing Your Agentic Stack: A Decision Framework

A stack, not four independent decisions

The four preceding articles in this series each compared one layer in isolation: orchestration frameworks, memory and vector stores, observability tooling, and context/RAG libraries. Read together, those comparisons answer "which tool wins on its own merits" -- they don't answer "which four tools actually fit together for my specific team and constraints," which is a different question with a different, smaller answer set. This article is that second question.

The three variables that actually decide it

Nearly every real decision collapses onto three axes, and most of the apparent complexity in choosing a stack comes from not being explicit about where a given team sits on each one.

Team size and infrastructure capability. A solo developer or a two-person team should weight every layer's decision toward "someone else operates this" -- managed vector database over self-hosted, purpose-built observability platform over building on raw OpenTelemetry, a framework with the gentlest on-ramp over the one with the highest ceiling. A team with dedicated infrastructure engineers can absorb operational surface in exchange for cost, control, or a better long-run fit, and the calculus flips.

Latency and cost constraints. A system with a hard latency budget (a live chat interface) and a system with a generous one (an overnight batch research job) make different framework and tooling choices defensible even for otherwise-identical workloads -- the batch job can afford a more exploratory, less deterministic orchestration style that the live system cannot.

One agent or a fleet. A single well-defined agent and a fleet of many agents sharing infrastructure are different problems. A fleet needs the shared skill/capability registry pattern, consistent observability across every agent (not per-agent ad hoc logging), and an orchestration framework whose composition model (ADK's hierarchical agents, or LangGraph's subgraphs) actually supports multiple agents cleanly -- a framework that's pleasant for one agent can become unmanageable at fleet scale if its composition story is weak.

Why coherence matters more than any single best-in-class pick

The four layers interact, and optimizing each one independently for "best in category" routinely produces a stack that's worse than four honest, coordinated compromises. A purpose-built observability platform's prompt-replay feature is most valuable when the orchestration framework exposes clean prompt/template boundaries for it to hook into -- pairing it with a framework that obscures prompt construction behind heavy abstraction gives up most of that value. A context/RAG library chosen for retrieval sophistication is wasted effort if the orchestration layer can't cleanly hand its output into a multi-step plan. Pick the four layers as a coordinated set, evaluated against the same three variables above, not as four separate "best tool" searches run in isolation.

Three concrete stacks

Solo developer prototyping. Bias every layer toward zero-ops and fast iteration: a lighter orchestration framework or none at all for a single-agent prototype, pgvector if there's already a database in the stack (skip a new vector system entirely at this stage), a purpose-built observability platform's free tier for the prompt-iteration workflow that matters most at this stage, and either a hand-rolled retrieval call or a lightweight LlamaIndex setup if the data is genuinely heterogeneous. The goal is minimizing time-to-signal about whether the product idea works, not building infrastructure for scale that may never arrive.

A team shipping one production support agent. This is where coherence matters most because the system now has real users and a real latency/cost budget. LangGraph if the support flow has a knowable shape worth enforcing (escalation paths, tool-use gates) -- which most support agents do; pgvector for the knowledge base unless query volume has already demonstrated a need to move; a purpose-built observability platform, because prompt iteration against real production failures becomes a weekly activity at this stage and the replay tooling pays for itself; and Haystack or a hand-rolled pipeline for retrieval if the knowledge base has a single, stable shape (most support knowledge bases do).

An enterprise running a multi-agent fleet. Bias every layer toward the option that scales across many agents and many teams, not the one that's fastest for a single team to start with: ADK or LangGraph's hierarchical composition for an orchestration layer that supports genuine multi-agent structure; a self-hosted vector database if query volume across the fleet has actually demonstrated pgvector's ceiling, otherwise still pgvector -- fleet scale doesn't automatically mean vector-store scale; OpenTelemetry extended with fleet-wide dashboards, because a purpose-built platform's per-project pricing and per-project UX often stops fitting once dozens of agents across multiple teams need unified visibility; and LangChain if the fleet's agents genuinely span a heterogeneous set of tools and providers that benefit from its integration breadth.

Walking through the reasoning, not just the picks

The value of the three stacks above is in the reasoning that produced them, which transfers to situations that don't match any of the three exactly.

Take the production support-agent case again. The choice of LangGraph over a lighter framework isn't "LangGraph is better" in the abstract -- it's that a support flow's escalation logic (when to hand off to a human, when a tool result means the agent should stop and ask for clarification) is exactly the kind of enforceable structure LangGraph's explicit graph gives you, and the cost of that explicitness (more upfront design than a looser framework) is worth paying once the flow is handling real customers whose bad experiences are expensive. The pgvector choice isn't "pgvector is always right" -- it's that most support knowledge bases are a few thousand to a few hundred thousand documents, comfortably inside pgvector's comfortable range (see the vector-store article's concrete thresholds), and paying for a dedicated vector database's operational surface before query volume demonstrates the need is optimizing for a problem the team doesn't have yet.

The enterprise fleet case makes the same kind of trade differently for a reason: at fleet scale, the "wait until you need it" logic that applies to a single team's pgvector choice stops applying to observability, because per-project pricing on a purpose-built platform multiplies across dozens of agents in a way it doesn't for a single team, and the point where OpenTelemetry's extra setup cost pays for itself arrives earlier at fleet scale than it does for one team's single agent.

Stack smells: signals you picked wrong

A handful of recurring symptoms are reliable signals that a stack choice should be revisited, independent of which specific tools are involved.

Fighting the orchestration framework's abstractions. If a team is routinely working around a framework's control-flow model -- stuffing custom logic into callback hooks the framework wasn't designed for, or maintaining a fork/patch to bypass a constraint -- the framework's opinionated model no longer matches the actual problem shape, and no amount of workaround code fixes that; it only accumulates a maintenance tax around the mismatch.

Needing a vector-store feature the memory layer doesn't expose. An orchestration framework's built-in memory abstraction is usually a thin, convenient wrapper over a vector store, not a full client. Hitting a wall -- needing metadata filtering, hybrid search, or a specific index type the wrapper doesn't surface -- means the abstraction has become the bottleneck, and the fix is dropping to the vector store's native client directly for that path, not forcing the wrapper to do something it wasn't built for.

Debugging by re-running and guessing. If the team's default response to a production failure is "run it again and see if it happens again" rather than "pull the trace and look," observability was under-invested at the layer-selection stage, regardless of which tool was picked -- this smell points at a gap in adoption/instrumentation more often than a wrong tool choice.

A RAG library doing less work than a plain function would. If the team can't explain what a context/RAG library's abstraction is buying them beyond what a direct embed-and-query call would do in fewer lines, that's the signal described in the RAG-tooling article's "when to skip a framework" section -- the fix there is usually subtraction, not a different framework.

How the stack should evolve, not just start

None of the three example stacks above is meant to be permanent. The realistic pattern is a system that starts closer to the solo-prototype end and migrates layer by layer as specific pressures actually materialize, not a wholesale replatform.

The migration signal for each layer is concrete and measurable, not a vibe: the orchestration layer gets revisited when the team is repeatedly fighting its abstractions (the smell above) or when a single agent is becoming several that need to compose; the memory/vector-store layer gets revisited when query latency or recall at the current scale demonstrably degrades, not preemptively (the vector-search-infrastructure article's concrete thresholds are the trigger, not a calendar date); the observability layer gets revisited when either cost (per-project platform pricing multiplying across a growing team) or coverage (needing fleet-wide visibility a single-project tool doesn't offer) becomes a real constraint rather than a hypothetical one; and the RAG/context layer gets revisited when retrieval sophistication or integration breadth becomes the actual bottleneck, per the trade-offs in that article.

Treating each layer's migration as independent and trigger-based, rather than bundling all four into one big "let's redo the stack" project, keeps each move small, reversible, and justified by a measurement rather than a preference -- which is the same discipline the rest of this series applies to the individual layer choices, just applied over time instead of at a single point.