Synthetic-data

Jev, the first model from TypeSafe AI, does not generate text. You give it a state and a set of typed questions, and it returns a probability distribution over the answers you defined, in a single parallel pass that costs you nothing for output. Your code then branches on the result, so Jev works as a smart if-statement rather than a chatbot. TypeSafe trains it with what it calls RLCD, Reinforcement Learning for Calibrated Decisions: a new training target, in the way RLHF was, whose goal is probabilities that mean what they say rather than text a rater likes or an answer a verifier accepts. This post explains what Jev is and how to use it, then goes through RLCD in pictures. It covers what calibration means, why RLHF's reward destroys it and why RLVR's reward cannot produce it, and why a proper scoring rule is the only objective that fits the stated goal. It also shows why that makes the 'reinforcement' in the name mostly a story about where the data comes from, and why the one number where Jev still leads its open clones is confidence ordering, which temperature scaling cannot fix. Nothing about RLCD has been published, so every claim is labelled by who made it.

On real market data there is no answer key, so you can never tell whether an agent discovered real structure or overfit noise. superquant-bench inverts the trade-off: a fully synthetic 100-asset price panel with 22 known alpha patterns injected into it, giving the grader the exact true conditional mean of every return. That buys three things reality cannot — a prediction score with a literal zero noise floor (oracle 100, iid noise 0), a directly measured false-discovery rate via a pattern-free twin universe, and statistical look-ahead enforcement. A red-team of seven exploits validates the metrics, then four frontier models run under one identical Claude Code scaffold. The spread is 10×, model identity explains 75% of prediction variance — and the strongest model is the least disciplined, spraying false discoveries on the universe that contains nothing.

LangChain's harness-engineering recipe treats evals as the training data for agents — but what do you do when you don't have evals, can't touch production customer data, and need to probe very specific corners of agent behaviour? You build a harness whose output is the dataset itself. This post walks through a synthetic-data-generation harness that runs Claude Code in a loop over Phoenix traces and generates edge-case companies designed to break the agent under test, grounded in what Meta-Harness and the LangChain canon actually say (and don't say) about how such loops should be built.