1. Start with the task, not the terabytes
Write a short description of the behavior your product needs. For an engineering assistant, that might be extracting requirements from a brief, updating a layout after a change request, or checking whether a drawing meets stated constraints. These require different examples, labels and evaluation methods.
Corpus size helps describe supply. It does not tell you how many useful training records can be created. A native BIM model, a PDF and a linked instruction–result pair are different units. Ask the supplier to report file volume and usable task coverage separately.
2. Ask for a sample you can actually inspect
A sample should resemble the proposed purchase across relevant disciplines, source languages, file formats and project types. Ask how it was selected. A polished demonstration is useful, but it should not quietly stand in for the whole corpus.
- Open the files in the software your pipeline will use.
- Check whether instructions are linked to the correct outputs.
- Identify missing references, duplicate exports and incomplete records.
- Separate result files from validation files: an output shows what was made; a review record shows what was checked.
For multi-step tasks, request the original instruction, later changes and associated file versions where those records exist. Missing history should be visible in the package brief.
3. Get the data brief in writing
The research paper Datasheets for Datasets proposes documenting a dataset’s purpose, composition, collection and intended uses. That is a useful starting point for a supplier conversation.
Our practical recommendation is to turn the brief into a purchase-specific record: source formats, measured scope, language coverage, linking methods, known gaps and checks completed. Keep the source corpus distinct from any extracted, translated or annotated records you commission.
4. Match the license to your intended use
Give your procurement and legal teams the proposed source information and intended model uses early. Ask what authorization supports the supplier’s ability to offer the license, which restrictions apply, and whether third-party content needs separate treatment.
Put the scope in the agreement: training, evaluation or commercial model use; permitted recipients; duration; territory; redistribution; derived datasets; and treatment of trained models after the license ends. Access to files and permission for every intended use should be reviewed as separate questions.
5. Compare the total cost of usable data
Look beyond the corpus price. Your team may also need extraction, application-specific conversion, translation, redaction, linking and expert annotation. Ask which work is included, which is optional and how its output will be accepted.
Compare proposals using the same task definition. A cheaper source corpus can become expensive if your team spends weeks reconstructing context. A prepared dataset can be useful, but only if its preparation and review match your actual pipeline.
6. Make the first purchase a measurable pilot
Agree on the sample, preparation deliverables, acceptance checks and evaluation plan before a larger commitment. Keep evaluation projects separate from training projects, including related files and revisions. Compare against your current model with the same tools and prompts.
Measure the result your users care about: task completion, constraint adherence, time spent correcting outputs or another defined outcome. Also record engineering effort and serving cost. Expand only when the pilot establishes fit for your product.
For the next decision, read our RAG vs. fine-tuning guide and our guide to multi-step instruction datasets.