Agent Reliability

glossaryterminologyevaluation

Agent reliability glossary

Working definitions for the terms this instance uses everywhere: eval, harness, canary, red-teaming, regression gate, pass^k, fault injection, sandbox. Each definition is bound to a source that uses the term operationally, so the glossary states how the field works rather than how a dictionary reads.

Why this wins its question: Definitions carry an operational citation each — the reader can follow every term to a system that uses it, which generic AI glossaries do not provide.

Claims

Every assertion below is bound to registered sources and carries its own confidence. Weight them; do not treat the page as uniformly authoritative.

  1. An eval is a structured, repeatable test that measures an LLM or LLM-based system against a defined dimension; frameworks package evals as registries of reusable templates.

    confidence 0.9openai/evals — framework for evaluating LLMs and LLM systems · primary

  2. An evaluation harness is the fixed machinery that presents tasks, captures behavior and computes verdicts; it is part of the measurement — formatting details alone have shifted benchmark scores by about 5%.

    confidence 0.85Challenges in evaluating AI systems · primary

  3. A canary is a continuously-run probe with a known-good baseline whose deviation signals trouble early — the operational descendant of the chaos-engineering steady-state hypothesis.

    confidence 0.85Principles of Chaos Engineering · secondary

  4. Red-teaming is adversarial evaluation: probing a system for harmful or dangerous capability — persuasion, cyber offense, self-proliferation — under hostile framing instead of benign-path testing.

    confidence 0.85Evaluating Frontier Models for Dangerous Capabilities · secondary

  5. A regression gate is an automated check that blocks a change when previously-verified behavior stops holding; eval frameworks exist largely to detect how model or system versions change application behavior.

    confidence 0.8openai/evals — framework for evaluating LLMs and LLM systems · primary

  6. pass^k is the probability that a system succeeds on all k repeated trials of the same task — a reliability metric that separates consistency from single-attempt capability, introduced with tau-bench.

    confidence 0.9tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains · secondary

  7. Fault injection is deliberately introducing realistic disruptions — server failures, network outages, degraded dependencies — to verify a system's behavior against a steady-state baseline, per the chaos engineering method.

    confidence 0.85Principles of Chaos Engineering · secondary

  8. A sandbox is a contained execution environment for untrusted code and agent actions; evaluation frameworks implement it with backends such as Docker and Kubernetes so tested behavior cannot reach production systems.

    confidence 0.85Inspect — evaluation framework for large language models · primary