Why architecture matters here
The architecture matters because it unlocks improvement on tasks where the model can already almost succeed but stumbles on a specific, correctable mistake. Many failures are not capability gaps — the model knows how to write the function — but slips: a misread requirement, an off-by-one, an unhandled edge case, a forgotten constraint. On exactly these failures, a written lesson is enormously leveraged: one sentence of 'you forgot to handle the empty input' can flip a task from failing to passing, at the cost of a single extra attempt rather than a fine-tuning run. Reflexion converts the model's own reasoning ability into a cheap, per-task learning signal.
It matters because it separates the thing being optimized (the actor's next attempt) from the thing providing signal (the evaluator) and the thing turning signal into guidance (the reflector). That separation is what keeps the loop honest. If the actor graded itself, it would rationalize its failures; if reflection were fused with action, the model would produce fluent excuses rather than diagnoses. By making the evaluator an external, trustworthy source of truth — ideally an executable test, not a vibe — and the reflector a dedicated critical pass, the loop gets a real error signal and a real lesson from it.
It matters because episodic memory is the mechanism that makes the improvement cumulative rather than a single retry. Without memory, each attempt starts blank and the agent can loop forever making the same mistake. With a bounded buffer of past reflections carried into each attempt, the actor sees the accumulated lessons from every prior failure at once, so trial three benefits from what went wrong in trials one and two. The memory is what turns a sequence of independent guesses into a trajectory that converges — and, because it is bounded, what keeps the prompt from growing without limit.
Finally, it matters because the loop's structure exposes exactly the knobs you must control to keep it safe and affordable. A self-improving loop that can retry is also a loop that can spin — burning tokens on a task it will never solve, or convincing itself with a weak evaluator that a wrong answer is right. Understanding Reflexion as an explicit loop with a fallible evaluator, a bounded memory, and a trial cap makes those risks visible and controllable, rather than surprises you discover from a bill or a wrong answer shipped with confidence.
The architecture: every piece explained
Top row: the attempt-and-grade path. Each trial starts with the task plus memory — the original prompt together with all reflections accumulated so far. The actor, an LLM, reads that context and produces a trajectory: for a coding task, a solution; for an agentic task, a sequence of tool calls and observations ending in an answer. The environment executes or tests that output — runs the code against tests, executes the tool calls, checks the answer. The evaluator then converts the outcome into a feedback signal: pass/fail, a numeric score, or a critique. The quality of this signal is the ceiling on the whole system, because everything downstream reasons from it.
Middle row: the branch that turns failure into a lesson. First, success? — if the evaluator says the task passed, the loop stops and returns the winning trajectory immediately; there is no reason to keep trying. On failure, a dedicated self-reflection LLM call examines the failed trajectory and the evaluator's signal and reasons about the root cause: not 'it failed' but 'it failed because X, so next time do Y.' That reflection is then written as a concrete lesson — specific and actionable, tied to what actually went wrong — and appended to memory, a bounded episodic buffer that holds the most recent and most useful reflections.
Bottom-left: the feedback edge that closes the loop. On the next trial the actor retries with the lessons in context, so the accumulated critiques directly shape the new attempt. This is the entire learning mechanism: the only thing that changed between attempts is the text of the reflections, yet that text can steer the model away from the exact mistake it just made. Nothing in the model was retrained; the improvement is carried entirely in the prompt.
Bottom-right and ops: the guards that keep the loop from spinning. Stop conditions end the loop on success OR when a maximum trial count is reached, so a task the agent cannot solve terminates instead of retrying forever. The ops strip names the rest of the controls: cap both the trial count and the memory size, guard reflection quality so lessons stay specific and actionable, verify that the evaluator is trustworthy (a weak judge poisons every reflection), and watch cost per solved task so the loop's extra attempts stay worth their token spend.
End-to-end flow
Follow a coding agent solving a function against a hidden unit-test suite, from a confident wrong first attempt to a passing third.
Trial one — the naive attempt: the actor reads the task ('implement a function that merges overlapping intervals') with an empty memory and writes a plausible solution. The environment runs it against the test suite; the evaluator reports that three tests failed, including one with an empty input and one with intervals that touch but do not overlap. The signal is concrete and external — real test failures, not the model's opinion of its own code. Because it failed, the loop proceeds to reflection rather than returning.
Reflection one: the self-reflection call receives the failed code and the specific test failures and reasons: 'The function crashed on empty input because it accessed the first element unconditionally, and it treated intervals that only touch at an endpoint as non-overlapping. Next attempt: handle the empty case first, and merge intervals whose endpoints are equal.' That lesson is written to memory. Note that the reflection is only as good as the evaluator's signal — because the tests pinpointed the exact failing cases, the reflection can be specific rather than a vague 'try harder.'
Trial two — informed retry: the actor now sees the original task plus reflection one. It handles the empty case and the touching-endpoint case, and those tests pass — but a new test, with unsorted input, still fails because the code assumed the intervals arrived sorted. The evaluator reports the one remaining failure, and a second reflection is written: 'The input is not guaranteed sorted; sort by start before merging.' Memory now holds two concrete lessons, both carried into the next attempt.
Trial three — success and stop: the actor reads the task and both reflections, sorts the input, handles empties and touching endpoints, and the full suite passes. The evaluator reports success, the stop condition fires, and the loop returns the winning solution. Three attempts and two paragraphs of self-critique solved a task the model could not do in one shot — with no fine-tuning, no gradient, and a bounded, predictable cost. Had the suite still failed at the trial cap, the loop would have stopped and surfaced the best attempt plus its reflections rather than spinning indefinitely.