Evaluation and operations connect intended outcomes to evidence about a running system. A plausible demonstration is not enough to choose a harness or approve a change. Anthropic's evaluation account distinguishes capability exploration from regression protection and connects automated testing with production feedback. agent-evals
Agent evaluation measures whether an interaction reaches the intended outcome under specified conditions. It must account for environment changes and variable trajectories. tau-bench compares final database state with task goals; this illustrates why a final assistant message is not enough to grade an action-taking system. tau-bench
Observability connects task outcomes to the operations that produced them. OpenTelemetry's trace model supplies spans, parent-child relationships and distributed context propagation. Applying that model to agent runs is a useful engineering synthesis, not proof that all agent telemetry fields have a stable shared schema. otel-traces
Resource budgets bound the time, computation and external activity spent on a task. The objective is useful completion under constraints, not minimizing tokens in isolation. Anthropic's multi-agent research account reports significant token overhead; those ratios describe that system, not universal cost multipliers. multi-agent-research