Equal evaluation scores do not certify equivalent AI behaviour.
The project studies evaluation equivalence, selected identity and downstream behaviour as separate objects of analysis—and develops an exact control for how much selection change an updated ranking objective truly requires.
When does the same score conceal a different system state?
Aggregate evaluation can preserve a performance summary while leaving the identity of selected evidence, actions or recipients unresolved. RIDI measures that unresolved state directly and tests when it changes downstream behaviour.
17.27% benchmark-defined correctness change across 800 queries under exact equality of the complete assessed grade sequence and five standard retrieval metrics.
10.70% equal-dataset correctness change versus a 1.37% repeat floor; 9.33-point repeat-adjusted excess (95% CI 7.13–11.57).
P5 remained negative at the strict inferential gate (0/760). P5b, locked separately before generation, qualified 103/760 cells and produced 4.36 points semantic-disagreement excess above repeat variability (95% CI 3.27–5.58).
565 of the top 1,000 priorities changed in the v2→v3 update, far above adjacent same-version controls of 0 and 7.
The exact frontier separates observed turnover from the minimum turnover required to stay within a declared updated-score utility budget.
Negative and conditional results remain part of the evidence.
- P5 is not rescued or relabelled by P5b.
- P5b is conditional on the fixed 20-system roster and common candidate pool.
- Correctness changes are bidirectional and are not a net-harm claim.
- EPSS delayed outcomes are sparse and retrospective.
- CMS is retained as a rule-version boundary case, not a hidden-reallocation headline.
- No formal CODECHECK certificate has been issued.
Public protocol, software and evidence.
The repository includes the open-source package, Quick Start, public evidence synopsis and registration links. The journal submission files are not published in the repository.