Source: Mastra docs: Evals (scorers) overview
Mastra scorers attached to agents and workflow steps with deterministic trace sampling, stored scores, runEvals, datasets and experiments, and Studio review.
Contents
- What: official documentation page
- Author: Mastra
- Date: read 2026-11-14
- Original: https://mastra.ai/docs/evals/overview
Summary
Scorers are automated evaluators returning (usually 0-1) scores. They run live on sampled production traffic of agents or workflow steps, offline through runEvals(), or against versioned datasets in experiments, with results stored and browsable in Studio.
Key claims
- Package
@mastra/evals; prebuilt scorers such ascreateAnswerRelevancyScorerandcreateToxicityScorer; deterministic quick checks in@mastra/evals/checks(e.g.checks.calledTool()). - Attach with
scorers: { name: { scorer, sampling: { type: 'ratio', rate } } }on an agent or oncreateStep. - Sampling is deterministic per trace ID (falls back to run ID); a
filterpredicate gates eligibility before sampling. - Scores persist to a
mastra_scorerstable only if storage is configured and the scorer is registered on theMastrainstance; otherwise a warning and no persistence. - Datasets hold versioned test cases; experiments run an agent/workflow over a dataset and keep results for comparison.
- Scoring historical traces requires observability to be configured.