Recognize a pairing failure in the model output
Look for recurring answers that satisfy an older constraint, refer to the wrong project or reproduce a plausible but unrelated file. Inspect the corresponding training records before deciding the model ignores instructions. A target can be professionally produced and still be wrong for the human input attached to it.
Our practical approach is to reconstruct a record independently: give a reviewer the instruction and its references, then ask which available output belongs to that task. Compare their selection with the dataset link. Record disagreements as link defects, missing evidence or ambiguous task boundaries. Do not use the model’s similarity score as the sole judge of correctness. Similar language and geometry can help locate candidates, but they do not establish that a file resulted from a particular request.
Separate file identity from task identity
A filename such as final_model tells you little about the task it answers. Give every source file an immutable identifier and preserve its original path, format and source package. A content hash can identify identical bytes; it does not by itself establish the meaning, author or approved status of a document.
Represent the task separately. Useful preparation fields include project_id, initial_instruction_ref, revision_ref, artifact_id, artifact_version and link_status. Add the basis for each link: an explicit document reference, a source system relationship or a reviewed inference. Keep that basis distinguishable from the link itself. If two exports contain the same design in different formats, record the relationship without counting them as independent task outcomes. Define how identifiers survive translation, extraction and conversion.
Check the revision sequence without assuming dates tell the story
Compare declared revision identifiers, available message order and references inside the documents. File modification time can change during copying or extraction, so treat it as a clue rather than a reliable task timestamp. Preserve original source dates and identify which events those dates describe.
Look for impossible or unresolved chains: a review cites a version absent from the package, a target predates a relevant change, or two files share a revision label but differ materially. Some projects have parallel workstreams rather than one linear sequence. Represent that structure explicitly. Where correspondence was not retained, document edits may support a narrower revision task, but not a reconstructed conversation. Mark unavailable stages so downstream preparation does not silently invent them to satisfy a preferred schema.
Inspect the changed requirement against the linked output
Illustrative example: an initial room brief requests a desk on the south wall. A later instruction moves it to the window while retaining a clear entry path. The prepared record attaches the later instruction to the first drawing because the files share a room identifier. Both files are genuine; the pair is still incorrect.
Check the relevant drawing versions, the revision reference and any retained change message. A reviewer should confirm which output expresses the changed request and whether the earlier constraint remains satisfied. Record an unknown outcome if the revised drawing is missing. Do not substitute a synthetic drawing and label it as the original project result. Synthetic examples may test the linking pipeline, but the real source relationship must remain distinguishable.
Validate joins before exporting training records
Run structural checks on the proposed preparation: referenced identifiers exist, projects match, duplicate links are intentional and review records point to specific result versions. Flag joins made through partial filenames or nearest dates for inspection. Report how many links are explicit, reviewed, inferred or unresolved without treating those categories as equivalent.
Use domain review for relationships that structural checks cannot establish. Ask whether the output addresses the instruction, whether the relevant change is visible and whether the stated review actually assessed that change. A result artifact and a validation file serve different purposes. Preserve the lineage from prepared record to source files so a corrected link can be rebuilt deterministically. Sample difficult joins, including parallel revisions and translated briefs, before expanding the export.
Ask what the supplier is delivering
- Is the purchase source material, reviewed linked records or both?
- Which links are supported directly by source references?
- How are uncertain links, missing outputs and parallel revisions represented?
- Can an exported record be traced to the exact source versions?
Common question: can we repair the links automatically? Automated candidate matching can reduce search work, but acceptance depends on the task and evidence. Agree on what must be checked mechanically and what needs domain judgment. Keep unresolved examples out of targets that imply a verified relationship, or give them a separately defined task. Measure model behavior after the repair on independent projects; a cleaner manifest is necessary evidence of a data change, not a guaranteed improvement in the trained model.