Harel Lidar

Article summary · Oct 2026

Where Training Data Comes From - Synthesis, Verification and Distillation Across the Training Stack

Mostly someone else’s words, from Jiaxin Zhang: Where Training Data Comes From. I keep it as reading, not as my own writing. Read the original ↗

TL;DR

Data is the new moat, this can be inferred from the fact that architecture details are shared quite commonly and training data is nothing more than “a diverse mixture of public and licensed sources”. The question that one needs to always be able to answer is “At each stage of training, what counts as a single training example, and who guarantees that it is correct?” All other steps, filtering, synthesis, mixtures, rubrics, distillation and decontamination is downstream from the answer to the above question.

The proper Schema

Most of the confusion about data stems from the fact that we are using a bad word for it. By giving a single name to six different things we are bound to be confused, and this is easily negated if we instead reframe these as different choices along 4 separate axes -

  1. What makes a single data “unit”
  2. What is the data source
  3. What verifies/certifies a data sample
  4. What is the characteristic failure
    1. Mental note-to-self: notice that post training is split into 3, and that continual training already has a separate, dedicated step just for this.
    2. The unit of data stops being text. In pre-training you buy tokens. In RLVR you buy problems with checkers — the answer is not the data, the checker is. In agentic RL you buy environments, which is to say you buy software. This is why data work migrates from data engineering to software engineering as you go up, and why the headcount profile of a post-training team looks nothing like a pre-training data team.
    3. The economics invert. A pre-training corpus is 10⁹–10¹⁰ documents with essentially zero verification cost per document. An RLVR run is 10³–10⁵ prompts, each of which needs a checker that someone wrote and validated. Agentic RL is 10²–10⁴ tasks, each of which may need a Docker image. Per-example cost rises by five or six orders of magnitude while the example count falls by five or six — which is why “just get more data” is good advice at the bottom of the stack and nearly always wrong at the top.
      1. Ankify these numbers Who gives the supervision signal - Zhang makes this point explicit, and says that the answer to this question is crucial, more than the “real” vs “artificial” debate. It can be a human, a teacher model or the “world”. Each of these have characteristics regarding cost, performance ceiling and failure patterns. The current best practice is largely - teachers for the cold start, the world for the climb, humans for taste and for the checkers you can’t automate.

How to Synthesize

Ways of synthesizing new data can be grouped into eight recurring patterns, each with it’s own failure pattern and key points of relevance.

Verification Ladder

The most important things to internalize from this ladder are:

Pretraining - Filtering is the Name of the Game

Mid Training - what it is and how we optimize for it

Operationally it is everything that happens after you decide pre-training is finished and before you start collecting preference or reward data: up-weighting a domain, extending context, annealing on a high-quality mixture, installing an output format, injecting a proprietary corpus, and preparing the model so that RL has something to amplify. The unit of data here is a curated pack, and the certifier is the weakest of any stage: a proxy eval plus a replay ratio. Nobody checks individual documents. What you are actually deciding is a distribution and a schedule, and the recurring lesson across the literature is that the same facts in a different format are worth roughly an order of magnitude more.

Knowledge Injection - A model can perfectly memorize a document and still be unable to answer any question about it. This is because in a single-format corpus the attribute is encoded diffusely across the document’s tokens rather than linearly in the hidden state adjacent to the entity’s name. Fine tuning cannot fix it becuase where knowledge is stored is a pre-trianing-time property of the corpus. The way to remedy it is through augmentation, specifically diversity Long Context is an unlocked rather than an installed capability

SFT Stage - a Readiness Gate

A sample here is a triplet of a (prompt, response, optional - reasoning trace). The certifier is a design choice, can be a verifier, judge or a human.

RL - The Data is Prompts, Not Answers

Agentic Data is Software

At the top of the stack, a training example is a piece of software. A triplet of a resettable environment, a task specification and a “verifier” of some sort that runs against the environment state and determines if the task was accomplished.

Distillation are a data-acquisition strategy

Distillation isn’t about losses, it’s about acquiring supervision. When deciding what suits you, you need to ask yourself:

  1. What data can you physically extract from the teacher
  2. How far apart you and the teacher are

Contamination - eval is a part of the Pipeline

The Data Platform

Given that data is the new product, a serious lab builds a platform for data rather than maintaining datasets. Here are the components this entails:

  1. A provenance and lineage service. Every training example needs to carry where it came from, which generator and prompt produced it, which certifier passed it, and which runs consumed it.
  2. A decontamination service, not a decontamination script. Prompt-side n-gram matching, run against every eval you report, on every corpus you add, including synthetic corpora and RL prompt sets. It belongs in the platform rather than in each project because the published overlap rates are alarming enough that you will not remember to run it manually.
  3. A scorer registry. Quality classifiers are the highest-leverage knob in pre-training and the most dangerous one in mid-training, because they are length-biased and will silently delete your long-context and agent-trajectory data. Register each scorer with the source distribution it was calibrated on, its length-response curve, and the threshold in use — and always report random selection with source preservation as a baseline, since it beats most published filters.
  4. A verifier registry with adversarial suites attached. Every checker is a piece of software with a version, a measured precision and recall on your data, and a suite of adversarial patterns it must resist. The non-negotiable lesson from the ladder section is that static accuracy does not predict RL outcome, so the artifact you version is the verifier plus its attack suite, and the suite has to be regenerated as policies get stronger.
  5. An oracle channel. At every checkpoint, sample ~1,000 training prompts, regenerate, and score them with a strong external model. Plot training reward and oracle reward on the same axes. Divergence between those two curves is the most reliable reward-hacking alarm I know of, and unlike a held-out eval it costs one generation pass per checkpoint.
  6. An environment registry. Container images, seeds, task specifications, rubrics, and a two-sided validation record showing the checker fails on the unfixed instance and passes on the fixed one. This is the part that is unambiguously software engineering, and the part where headcount goes at the top of the stack.
  7. Private, rotating, and freshly collected evals. Whatever you publish, keep a held-out set that never leaves the building, rotate a fraction of it on a schedule, and maintain a non-regression suite that runs on every checkpoint. The reason is in the contamination section, and the incentive analysis behind it is simple: contamination is not only an accident of web crawling, it is what an organization drifts toward when its objective function collapses to a score.