Evaluation

Agent evaluation asks whether the system completed a task correctly, safely and efficiently. A final-answer score is not enough because an agent can produce plausible prose after making an unsafe or incorrect intermediate tool call.

Build a task dataset

Create representative cases from real goals. Include normal requests, ambiguous inputs, edge cases, adversarial content and cases where the correct behavior is to refuse or ask for clarification.

Evaluate layers

Measure task success, factual correctness, tool selection, argument correctness, policy compliance, retrieval quality, step count, latency, cost and escalation rate.

Deterministic graders

Use exact assertions for hard constraints: required tool calls, forbidden tools, schema validity, authorization ordering, maximum steps and approval requirements.

Model-based graders

Use model graders for fuzzy qualities such as helpfulness or explanation quality. Calibrate them against human judgments and monitor grader drift.

Regression testing

Every meaningful production incident should become a permanent evaluation case. Run the dataset whenever prompts, models, tools, retrieval or orchestration changes.

Evaluation is a development loop: establish a baseline, change one variable, compare traces and keep improvements that are supported by evidence.

Example: refund agent

A correct run identifies the order, checks eligibility, requests approval and only then calls the refund API. A test that checks only the final sentence can miss an unsafe refund call that happened earlier, so the trace is part of the expected outcome.

Trace assertions

expect(trace.toolCalls.map(x => x.name)).toEqual(["get_order", "check_refund_policy"]);
expect(trace.approvalRequired).toBe(true);
expect(trace.toolCalls.some(x => x.name === "issue_refund")).toBe(false);

Practice

Create 20 evaluation cases. Label expected outcome, allowed tools, forbidden tools, important facts, maximum steps and whether approval is required.

Challenge

Build a regression report comparing two agent versions on task success, unsafe tool calls, average steps, p95 latency and cost per successful task.