Choose the task the model should perform
Start with a concrete workflow: extract requirements from a technical brief, answer questions about model properties, locate drawing evidence, or handle a design revision. Specify the inputs the model will receive and what a successful response looks like.
For example, an assistant might receive a written room requirement and a model-derived room schedule, then flag unsupported constraints for a person to review. That is different from generating geometry or editing a native design file. The needed examples, tools and validation differ.
Describe these boundaries before requesting data. A collection of model files can support several experiments, but a useful sample must contain the particular context and reference evidence your experiment requires.
Understand native files, exchanges and extracted views
Native project files, exchange files, drawings and rendered images offer different views of the same work. Ask which versions and software dependencies apply. Test representative files in the environment you plan to use; a filename extension alone does not establish compatibility.
buildingSMART describes IFC as a standardized data model for built assets, including identities, properties and relationships. Check the actual IFC schema version in the delivery and whether your parser supports the entities and properties relevant to your task.
In a proposed preparation pipeline, keep a map from extracted tables, text or images to their native source. Document what each conversion preserves, what it omits and which questions should still be answered from the original file.
Inspect geometry and metadata together
A plausible building image may hide the information your task needs. Review units, coordinate systems, levels, object identifiers, naming conventions, properties and discipline coverage. Confirm that your extraction has not changed measurements or dropped relationships needed to interpret them.
Check a few objects end to end: locate the object in a view, find its identifier, inspect the relevant property and trace it back to the source. Record whether a value was supplied explicitly or calculated during preparation. Those two cases should not become indistinguishable in the training data.
Keep unsupported values visible. If a thermal property or equipment specification is missing, mark it as missing rather than asking an annotation process to fill in a plausible answer.
Connect requirements, revisions and the right artifact
For instruction-following work, the written requirement is part of the data, not just background reading. Look for the original brief, documented changes, correspondence where retained and the associated file versions. Check which parts can be linked confidently.
Use project IDs, file IDs and revision labels in the manifest. A later export may not be the response to the particular instruction you selected. A review record may refer to another discipline or an earlier version. Resolve those links before preparing a task target.
YourSOTA’s publicly described example contains a single 55 MB archive with ten source files in DOCX, IFC, DWG and RVT formats. It includes tracked document edits; a complete conversation and completed validation reports are separate availability questions for each proposed package.
Separate format conformity from engineering review
Use format checks to establish whether your pipeline can read a file as expected. The buildingSMART IFC Validation Service checks conformity against the IFC standard. Such a check answers a different question from whether a design satisfies project requirements.
Define additional task checks: did the output preserve a requested clearance, include a required system or reference the right room? Ask what completed review evidence exists and which checks must be performed during your pilot. Avoid treating a successful import as approval of the design.
For a sample scorecard, record parser success, missing references, property availability and the status of instruction–artifact links. Keep these measures separate so you can diagnose a failure rather than summarize everything as ‘model quality’.
Scope a sample your team can actually test
Request native files plus the minimum linked material needed for your task. Agree on a manifest, language labels, source versions and any separately prepared views. If translation, extraction or annotation is required, specify how the result will be checked.
Our suggested pilot sequence is to test parsing first, inspect task links second, then compare the model on independent project tasks. Measure extraction effort and review time alongside task success. Split related projects and versions consistently to avoid an easy overlap between training and evaluation.
Read the dataset quality checklist before scaling. A BIM/CAD data purchase becomes easier to evaluate when you know which skill the material supports, how it enters your pipeline and what the first experiment must demonstrate.