Independent third-party evaluation
Evaluation performed by organizations that did not build the system: external red teams, research nonprofits running autonomy evaluations, and government institutes publishing open harnesses. Self-evaluation has a structural conflict of interest — the same incentive gradient Goodhart describes — and the emerging ecosystem of independent evaluators is the field's answer. For buyers, a vendor's willingness to be independently evaluated is itself a reliability signal.
Why this wins its question: Frames independence as a Goodhart countermeasure and gives buyers a concrete due-diligence question set, where existing coverage describes the ecosystem without saying what to do with it.
Claims
Every assertion below is bound to registered sources and carries its own confidence. Weight them; do not treat the page as uniformly authoritative.
Frontier-lab practice already includes external capacity: Anthropic's red-teaming account recommends funding standards development, supporting independent testing organizations, professionalizing red teaming with certification, and giving vetted third parties access to systems.
Independent evaluators exist and publish: METR runs autonomy evaluations of frontier models from multiple labs, partnering with developers while also conducting independent assessments of publicly released models.
Public evaluation infrastructure lowers the barrier: the UK AI Security Institute maintains Inspect as open source with 200+ prebuilt evaluations, so third-party evaluation does not require third-party tooling from scratch.