Preview
Measure whether a change helped — fixed cases, a simple score, compare before and after so you are not guessing.
Sign in to read the full chapter.
Checking access…
Evaluating Agents
Code panel
Evaluating Agents
Contract
- Keep a fixed list of cases with must-contain rules
- Score weak agent_v1 (baseline) and improved agent_v2
- Print pass rates and compare
evals_demo.py
Highlighting…
Example console
Sample run — exact text can vary
=== Baseline agent_v1 === [FAIL] refund_small ... [FAIL] status_large ... [FAIL] greeting ... Pass rate: 0/3 === Improved agent_v2 === [PASS] refund_small ... [PASS] status_large ... [PASS] greeting ... Pass rate: 3/3 Baseline 0/3 → v2 3/3