Interactive evidence replay · preregistered RAG

Same retrieval evaluation. Different selected evidence. Different answer.

A single registered SciFact case illustrates the core identification problem behind the current manuscript, Equal evaluation scores do not certify equivalent AI behaviour.

This replay is an illustrative case, not the full 800-query endpoint and not the later GPT-5.6/P5b extensions.

Replay the registered example

Same positive passage. Same rank. Same relevance pattern. Same five retrieval metrics.

Claim 275
Combining phosphatidylinositide 3-kinase and MEK 1/2 inhibitors is effective at treating KRAS mutant tumors.

RETRIEVAL EVALUATION: IDENTICAL

Context A · reference

1
49610382
856657413
49203764
30785505
12857136
286516437
22726148
112006859
2298020510
24190159

Context B · random zero-grade replacement

1
49610382
97674443
95054024
219310055
333879536
293215307
153225188
274285099
4165041710
9021186

precision@100.1000

recall@100.3333

nDCG@100.4693

MRR@101.0000

MAP@100.3333

Context A verdictSUPPORTS

Context B verdictREFUTES

Press Run demo to reveal the decisions.

Interpretation boundary. The original result is bidirectional at the aggregate level and is not a net-harm claim. The current project also reports a registered GPT-5.6 Sol transfer, a negative P5 inferential-equivalence gate and the separately locked P5b practitioner-equivalence extension. Read the full public evidence synopsis →