Sources
Expert-labeled cases. Historical prod responses rated good. Synthetic generation for edge cases.
Advertisement
Freshness reviews
Ground truth ages. Every quarter, review 10% for stale expectations. Update or retire.
Advertisement
Multiple valid outputs
Many questions have multiple correct answers. Store as a list; scorer passes if output matches any.