Most of verification is the gap between checking an answer and reproducing one.

Give a verifier the answer and a predicate to apply, and it succeeds almost always. Make it derive the answer before it can judge, and it collapses. Everything below sits on that one measured slope, and none of it depends on the verifier being clever.

Research map

The site separates measurements, unresolved questions, and proposed engineering work. The status on each entry is part of the content, not a confidence badge.

The access law

Cost is irrelevant at both ends: spending more at the top changes nothing because it is already solved, and spending more at the bottom changes nothing because the problem is access rather than effort. The architectural claim this makes falsifiable is uncomfortable: oversight cannot be bootstrapped from models alone unless a fallible reference suffices. A composition ladder testing exactly that is running now.

Three results that sit on it

Extra inference compute helps only in the middle

Sampling a model many times and voting buys accuracy where single-sample competence is partial, and close to nothing at either end. An inverted U, not a rising line.

claude-opus-4-8 · effect 0.156, CI [0.093, 0.216] · MEOI 0.05 ·n = 89 graded · 768 tasks · 6144 requests · seed 20260728 · manifest sha256 pinned · backend anthropic-replay

Two caveats travel with this result and are not footnotes. It is underpowered: MDE is 0.063 against an MEOI of 0.05, so the two shoulders are undetected, not shown to be flat. And the backend is a hash-pinned replay rather than a live call. Raising the graded n from 89 to 143, a factor of 1.61, would bring MDE under MEOI and turn both shoulders from inconclusive into informative nulls. That would turn one confirmed point into a confirmed shape, and it is the cheapest experiment here.

Which model, not how big

A 9B model out-verified a 12B by 0.77 on the same paired tasks, while a 31B of the 9B's own family beat that same 12B by 0.92. Family and training lineage dominate parameter count, which means the monitor you should pick may run on a desk.

qwen3.5:9b 0.804 vs gemma4:12b 0.034 · n = 500 · pre-registered · paired seed · zero overlap

A monitor does not degrade. It inverts.

Holding the predicate identical and moving only arithmetic width, verification went from 0.536 to −0.648. Below zero means the monitor rejects correct answers more often than corrupted ones: it is now evidence against the thing it is checking.

At the moment it crossed, three of five other metrics improved. A monitor-health dashboard would have shown the system getting better. Every metric that does not require ground truth is blind to this, which is what makes it a deployment problem. A twelve-design ladder is queued to locate the crossing and test whether peer disagreement detects it without an answer key.

What this demonstration does not show

The findings above are measured. This one is not. It is textbook arithmetic, shown so a reader can check it. Set how many verifiers you run, how often each is right alone, and how much of each verdict is the part they all share.

A single-factor latent model: each verifier errs when its own draw falls below a threshold, and part of that draw is shared with every other verifier. At zero shared component it reduces exactly to the textbook answer, which is the reason for using it rather than an invented curve. The familiar arithmetic is the special case, and the shortfall is what happens when you leave it. Verified against the closed form to within 0.06%.

What follows from it

Count evidence, not votes. Three correlated verifiers are closer to one than to three, so confidence belongs against measured decorrelation rather than against a headcount. If you want a second opinion, it has to differ in lineage, in modality, or in how it fails; a second prompt to the same family is the same opinion asked twice.

The strongest check is a different kind of thing altogether: an executing test, a physical measurement, a person who has not seen the answer. That is why every tool in this ecosystem publishes something checkable instead of asking to be believed, and it is why a system's account of its own reasoning has a ceiling. Self-report is a further output of the same system, carrying the same shared component as everything else it produces.

Four questions this domain has to answer on its own

A result about the limits of correlated judgment should not depend on a single person's name to stay findable. So this domain is being built into the programme's own index, and the test it has to pass is that four questions can be settled here without leaving: what the work has found, what it has failed to resolve, which experiments and tools are worth building next, and where any one claim can be checked.

Three of those have pages today. The fourth does not. Methods, preregistrations, the grading boundary and the corrections all still live on the dated personal research record, linked from here rather than copied, and until they can move without leaving two versions behind, this site summarises that record and does not replace it.

Three rules govern how the rest arrives. A figure appears here only if it renders from the canonical findings data, so this domain cannot become a second hand-maintained set of numbers that quietly drifts from the first. Anything proposed is labelled proposed, and an unresolved question is never written as a forecast. A correction stays beside the claim it changed, which makes the record longer over time instead of tidier.

The programme is small and it is honest about that. One line of enquiry with results, one second line in progress, no institute of forty people implied by a logo.

The applied side

This is not a detached research interest. Deliberation here is short-range because long trust chains fail for the reason above, and every ranking tool in the constellation shows its arithmetic so a reader can disagree with the method rather than the answer. The finding shaped the products.