Evaluating a Conversational Agent With a Simulated User: Goal Scripts, Stop Rules, Simulator Drift and Blaming the Right Side

Why fixed prompts stop working

A single-turn test sends a prompt and checks the answer. A conversational agent does not fit that mould. It may ask for two pieces of information at once or one at a time, ask a clarifying question you did not anticipate, or confirm before acting, and a fixed list of user messages goes out of step with it after the first turn. The ADK documentation gives exactly this reason for its user simulation feature, available in ADK Python since v1.18.0: when the agent needs two values, it may ask for them separately or together, so the user's side has to be generated in response.

A simulated user is a second program, usually a model, that plays the customer. It knows a goal and some private facts, answers the agent's questions, and decides when the conversation is over. That makes multi-turn evaluation possible, and it introduces a new component that can itself be wrong.

What goes into a goal script

ADK's ConversationScenario holds a starting prompt, a conversation plan and an optional persona; its personas are defined as behaviours with instructions and violation rubrics, and its user simulator configuration caps the conversation with max_allowed_invocations, 20 by default, counting the first prompt. The tau-bench paper takes the same shape and judges each episode by comparing the database state at the end of the conversation with an annotated goal state.

Judge the end state, not the words

Two conversations can be worded completely differently and both be correct, so scoring the transcript against a reference transcript is fragile. Scoring the end state is not: either booking QX7P2 now has the new date, or it does not; either the change inside the 24-hour window was refused, or it was made. Keep a transcript-level check only for things the end state cannot show, such as the agent revealing data it should not, and make that a separate, explicit rubric.

Because both the simulator and the agent are stochastic, one run per task is not enough. The tau-bench paper defines pass^k as the chance that all k independent trials of a task succeed, estimated per task as C(c, k) divided by C(n, k) for c successes in n trials and averaged over tasks. Pass^1 is the ordinary success rate; pass^k falls when an agent is right only some of the time, which is the property a customer actually experiences.

A worked example with a deliberate bug

The companion example has a small date-change agent and a rule-based simulator, so every number is exact and repeatable. The agent has a bug: it checks the 24-hour no-change policy only when it had to ask for the new date, not when the user volunteered the date in the first message. Four goal scripts, four trials each.

The patient persona never triggers the bug, so it reports a perfect agent. The detailed persona fails every trial of the late booking. The mixed persona is the instructive one: the success rate looks excellent at 0.938, but pass^4 is 0.75, because on the late booking the agent is right only when the user happens to wait. The lesson is that the persona is part of the test. A cooperative, patient simulator is the easiest user an agent will ever meet, and a suite built only on it measures that.

Python from the companion example, condensed; standard library only.

from math import comb

def pass_hat_k(successes_per_task, n, k):
    """tau-bench pass^k: mean over tasks of C(c, k) / C(n, k)."""
    return sum(comb(c, k) / comb(n, k) for c in successes_per_task) / len(successes_per_task)

def attribute(task_facts, stated, end_state_ok):
    """Blame the simulator if it stated anything outside its goal script."""
    if end_state_ok:
        return "pass"
    if all(task_facts.get(k) == v for k, v in stated):
        return "agent failure"
    return "simulator failure: discard and re-run"

assert pass_hat_k([4, 3, 4, 4], n=4, k=4) == 0.75

Blaming the right side

A simulator fails in its own ways: it invents a fact that is not in its script, forgets part of the plan, ends early, or answers a question the agent never asked. When it does, the end state is wrong and the agent gets the blame. The tau-bench authors met this directly: of 40 failed retail tasks they examined by hand, 4 were caused by typos or ambiguity in the user instructions rather than by the agent.

Make attribution part of the harness. After every failed episode, check the simulator against its own script and persona rules: did every stated fact come from its private facts, did it follow the plan, did it stop for a valid reason. In the companion example, a simulator that invents a different date 25 percent of the time drops the correct agent to pass^1 0.875 and pass^4 0.75, and both failures are attributed to the simulator. Without that check, the report would show a regression in an agent that did nothing wrong. Discard and re-run simulator failures, and track their rate, because a rising rate means the simulator prompt or model needs attention.

Running it in practice

1. Write goal scripts from real tickets, with the expected end state recorded by someone who knows the domain.
2. Give each script at least two personas, one patient and one that volunteers or corrects, and keep a hostile or confused persona for policy tests.
3. Pin the simulator model and its settings, and keep its instructions under version control like test code.
4. Run several trials per task and report pass^1 and pass^k side by side.
5. Check every failure for simulator faults before counting it against the agent.
6. Store transcripts and end states so a regression can be replayed and inspected.

Failure modes and verification

• The simulator sees too much. ADK defaults include_function_calls to false, so the simulated user does not see tool calls; giving it the agent's internal state lets it answer questions a real user could not.
• No stop rule. Two models can thank each other indefinitely. Cap turns and treat hitting the cap as a failure to investigate.
• Judging with the same model. A judge that shares the agent's model and prompt style can share its blind spots; prefer end-state checks and keep any judge independent.
• Too few trials. Four trials of four tasks, as in the example, illustrate the method but decide nothing; size the suite to the difference you need to detect.

The companion code, simulated_user_eval in Sandeep-BA/agent-decision-lab, has six tests that pin every figure above. It uses rule programs in place of models, so it demonstrates the evaluation logic, not any model's quality.

When a simulator is unnecessary

If the task is single-turn, or the dialogue is a fixed form whose slots are always asked in the same order, scripted conversations are cheaper, deterministic and easier to debug. Reach for a simulated user when the agent decides what to ask and when, because that is exactly what a fixed script cannot follow.

Practice and related reading

Further reading:

• Online evaluation of a production agent
• Scoring agent tool calls offline
• Role prompting, personas and simulated users

Sources and scope

Checked against these sources on 26 October 2026. Figures come from the offline companion example, whose agent and simulator are deterministic rule programs; no model was called.

• Agent Development Kit documentation: User simulation
• Yao et al., tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (arXiv 2406.12045)