Why architecture matters here
Agent planning architecture matters because the choice of strategy shapes cost, latency, and success rate. Tree of Thought explores many branches and finds better plans but burns more tokens; ReAct is fast but greedy; Plan-and-Execute is predictable but rigid. Matching strategy to task is the difference between an agent that succeeds and one that spins in circles.
Cost is a real constraint. A ToT-style agent may issue 10-30 LLM calls per user task; a ReAct one may issue 3-5. For high-QPS applications, this matters directly.
Reliability comes from constraints. Step limits, wall-clock deadlines, and reflection-based loop detection prevent an agent from spending forever on a stuck task. Without these, planning becomes an unbounded risk.
The architecture: every piece explained
Walk the diagram top to bottom.
Task Prompt. The user's request: goal, constraints, success criteria. Structured tasks (with clear success conditions) plan more reliably than open-ended ones.
Planning Strategy. The high-level approach. ToT for exploratory problems where you can score partial plans; ReAct for straightforward tool-heavy tasks; Plan-and-Execute for predictable structured tasks.
Plan State. The current state of the plan: steps completed, current step, remaining steps, discovered facts. Serves as persistent working memory across LLM calls.
Tree of Thought. Branch the plan into multiple candidate next-steps; score each with a value model or LLM critique; prune weaker branches; continue on the best. Yao et al. 2023.
ReAct Loop. Interleave Reason (think about next step) and Act (call a tool). Observe result and iterate. Yao et al. 2022. Fast; good for straightforward problems.
Reflection. Periodic self-critique: what happened? Is this progress? Am I stuck? Reflexion pattern (Shinn et al. 2023). Detects loops and course-corrects.
Replanning. When new information invalidates the current plan, revise. Plan-and-Execute agents replan when the plan becomes obsolete.
Sub-agents. Delegate a sub-task to another agent with narrower scope. Useful when the parent's context would otherwise blow up.
Constraint Enforcement. Budget caps (tokens, tools, wall-clock), step limits, and cost limits. The planning layer must respect them structurally.
Stopping Criteria. Goal met, budget exhausted, no progress detected. Explicit criteria prevent runaway.
End-to-end planning flow
Trace a task. User: "Find the cheapest flight from Munich to Bali next week under $800, avoiding overnight layovers."
Planner chooses Plan-and-Execute for this structured task. It generates the plan: (1) search flights for date range, (2) filter by price and layover type, (3) select cheapest, (4) verify availability, (5) present to user.
Executor invokes step 1 tool: flights_search with (Munich, Bali, week). Returns 47 candidates.
Step 2: filter code (structured output from LLM) selects flights under $800 with only daytime layovers. 8 candidates remain.
Step 3: pick the cheapest. Straightforward. Executor updates plan state.
Step 4: flights_availability tool for the top pick. Returns "sold out today; available if booked in 30 min."
Reflection detects mismatch: the plan didn't handle sold-out gracefully. Replans: try second-cheapest.
Retry step 4. Availability confirmed. Step 5: present.
Contrast: if the task were "help me plan a two-week itinerary in Bali based on my interests," a Tree-of-Thought planner would branch into "beach + culture," "adventure + wellness," "family + food" etc, score each, and pick the strongest. More expensive but more thorough.