Build the test that decides whether verification belongs in the loop.
The programme does not need a larger collection of monitor scores. It needs measurements that change a concrete choice: use this checker, choose another evidence mode, or abstain.
- Build the reference-composition ladder firstFirst experiment. Measure the same verifier across ordered levels of answer access, from a supplied answer through partial references to full recomputation.
Deliverable: A curve showing which reference forms buy reliable checking and where extra inference cannot compensate for missing access. - Publish a model-pair scorecardEvaluation. Estimate verifier competence and generator-verifier error overlap on one paired task set, with false alarms and catches reported separately.
Deliverable: A decision record for choosing a checker by observed fit rather than parameter count or brand tier. - Turn monitor inversion into a release testSafety control. Keep a small bank of grounded sentinel tasks spanning the known difficulty boundary and compute signed discrimination on every monitor release.
Deliverable: A release gate that fails when a monitor becomes evidence against correct work, even if its other health metrics improve. - Route by evidence modeSystem design. Choose among sampling, model verification, executable tests, retrieval, human review, and abstention according to available reference access and consequence.
Deliverable: A local router that names why a check was selected and refuses when no available mode clears its threshold. - Make every result reproducible without provider accessResearch infrastructure. Freeze prompts, raw responses where publication is permitted, graders, seeds, manifests, and a standard-library reproduction script for every public result.
Deliverable: Small evidence bundles a reviewer can rerun without an API key, a private harness, or the author present. - Give self-reports an explicit claim ceilingInterface contract. Separate observed behavior, system-generated explanation, inferred mechanism, and unresolved cause in logs and user-facing reports.
Deliverable: A result format that keeps a useful explanation from being mistaken for independent evidence about the system that wrote it.
The ordering rule
Measure access before optimizing inference. Measure model-pair behavior before choosing a verifier. Establish a grounded failure signal before automating a release gate. Build the router only after those thresholds exist.