- Published on
On real market data there is no answer key, so you can never tell whether an agent discovered real structure or overfit noise. superquant-bench inverts the trade-off: a fully synthetic 100-asset price panel with 22 known alpha patterns injected into it, giving the grader the exact true conditional mean of every return. That buys three things reality cannot — a prediction score with a literal zero noise floor (oracle 100, iid noise 0), a directly measured false-discovery rate via a pattern-free twin universe, and statistical look-ahead enforcement. A red-team of seven exploits validates the metrics, then four frontier models run under one identical Claude Code scaffold. The spread is 10×, model identity explains 75% of prediction variance — and the strongest model is the least disciplined, spraying false discoveries on the universe that contains nothing.