JSONL row shape
{"input": "What is the weather in Bangalore?", "expected": "warm", "tags": ["weather", "happy"]}
{"input": "Book me a flight", "expected_tool": "flight_search", "tags": ["tool_call"]}Advertisement
Cover happy path first
10-20 cases the agent should absolutely handle. If it fails these, don't bother with edge cases.
Advertisement
Add edge cases second
Ambiguous inputs. Adversarial phrasing. Empty/null-like queries. Multi-language.