YourSOTAGet a sample
Home/Insights/DATA PREPARATION

DATA PREPARATION

Your redacted dataset lost the context that made it useful.

Removing a sensitive string is straightforward. Preserving the reasoning task around it can be harder. A dataset can look cleaner while leaving the model unable to tell which actor, object or revision a requirement concerns.

THE PRACTICAL TAKEAWAY

Review the allowed use, preserve necessary relationships where appropriate, and test the transformed task before acceptance.

A task-focused preparation process with separate handling review.
A task-focused preparation process with separate handling review.

Identify the relationships the task needs

Begin with a task map. What must the model distinguish to answer correctly? A change request may refer to a specific room, discipline, revision or decision owner. A source may contain identifying details alongside those relationships. Treat their role in the task as an explicit design question.

For a requirements assistant, personal names may be unnecessary while the distinction between requester and reviewer matters. For a project coordination task, a repeated object identifier may be necessary to connect records. Write these dependencies down before transformation. Otherwise, automated removal can quietly turn a solvable problem into an ambiguous one without generating a technical error.

Follow the permission scope through preparation

Determine which material and intended uses are within the agreed scope before selecting transformations. Redaction is a preparation method; it does not itself supply permission for training, distribution or commercial use. Ask the responsible reviewers to examine the proposed source fields and delivery arrangement.

Keep a record of what is included, excluded and transformed. Avoid moving additional source details into prompts or external processing systems without considering their permitted handling. The workflow should make review decisions visible to the preparation team. These are practical process recommendations, not legal conclusions about a dataset or a guarantee that a particular transformation makes it suitable for use.

Test a transformation with an illustrative task

Imagine two fictional instructions: “Change Room A lighting to the evening setting” and “Keep Room B lighting unchanged.” If every room reference becomes the same generic placeholder, the relationship disappears. An answer can no longer reliably distinguish the change target from the preserved target.

Where approved for the task, consistent replacement labels can preserve that distinction: Space 01 and Space 02. This example demonstrates task continuity only. It does not establish anonymity or rule out identification through other details. Test whether the prepared instructions and associated artifacts still connect correctly, and review whether the remaining context is appropriate for the proposed delivery.

Examine the whole package, including metadata

A document’s visible text is only part of its contents. Filenames, model properties, embedded attachments, comments and revision metadata may carry details that a text-only pass does not inspect. Build the review inventory around the actual formats you plan to deliver.

Research on training data extraction demonstrates that language models can reproduce individual training examples. Our preparation recommendation is therefore to review retained content rather than assume that training obscures it. This does not prescribe an extraction attack or a legal standard. The practical objective is a documented handling decision for the package and its intended workflow.

Check task usefulness after each transformation

Run a small before-and-after review within the permitted environment. Does the transformed task still contain enough information for its target answer? Can a reviewer trace referenced entities? Do constraints retain their ordering and units? Does removing a detail change the reason the answer is correct?

  • Check consistency across linked files.
  • Check whether replacement labels collide.
  • Check whether deletions create unsupported answers.
  • Check residual detail in the final delivery formats.

If the target now depends on removed context, revise the task or exclude the record. Retaining a confident answer beside an incomplete instruction teaches a behavior you may not want.

Label transformed examples honestly

Identify preparation methods and known limitations in the delivery record. Distinguish source redaction from newly written abstractions. If the team rewrites a case to preserve its general decision pattern, label that as a prepared example rather than an untouched historical instruction.

Likewise, distinguish draft responses from reviewed targets. A useful transformed source is not automatically a validated training example. Review whether the proposed response remains supported after transformation. Keep lineage and review records under the agreed handling controls, and deliver only the fields in scope. The buyer should understand both what the sample contains and which additional work would be needed for production use.

Can we remove every name and proceed?

Name removal is a narrow operation. Its effect depends on what else remains, how records link together and what the buyer intends to do. It can leave unnecessary identifying context, or it can remove a distinction needed for correct reasoning. Neither result follows from a count of deleted names.

Choose one task and prepare a small representative sample with the responsible review team. Measure task continuity and inspect remaining details in the delivered formats. Expand only after the process and acceptance criteria are understood. That produces a more useful acquisition conversation: available fields, necessary relationships, permitted use, preparation requirements and the limits of the resulting examples.

Useful for your team?Share on LinkedIn ↗

FROM THE GUIDE TO YOUR NEXT STEP

Which relationships must survive preparation?

Describe the task context your model needs. YourSOTA can discuss available source fields, scoped handling requirements and separately agreed preparation for a representative sample.

Request a sample & data brief