Choose a decision the sample can support
Write the business question before extracting data. “Can the evaluator join defect labels to images?” is testable; “is this useful for AI?” is too broad. Name the evaluator and the evidence needed. Agree a decision date and budget for preparation work without implying that a successful test commits either party to a purchase.
Separate a schema demo from a representative sample
A few clean records can show file structure, but cannot establish archive-wide quality. Document the population, selection method, date range, and excluded categories. If coverage matters, list the groups that must appear, such as languages, time periods, or defect classes. If a rare group is oversampled, state that the sample proportions differ from the full collection.
Prevent the same underlying item crossing test boundaries
For repeated views, clips, revisions, or variants, identify the parent unit. Group by physical part, original recording, document family, or product family when planning separate subsets. Otherwise the evaluator may see near-identical material in more than one subset. Have the evaluator review the split design for its task; there is no universal split ratio that fits every collection.
Write acceptance evidence, not a universal quality score
Define what will be measured, the denominator, and how failures are counted. Examples include unmatched image-label joins, unreadable pages, and missing units in a numeric field. Ask the evaluator to set task-specific thresholds before seeing results. Report unknowns and excluded records alongside any pass count. A small sample cannot prove full-population performance.
Resolve handling before transferring anything
Document proposed access, storage location, review period, recipients, and treatment of copies and outputs after evaluation. Keep permission review separate from technical delivery. If you cannot confirm that a sample may be shared, use a clearly marked fictional schema example for discussion while the review remains open. Do not send raw samples through this website.
Close the loop with a written decision
Record the release tested, checks run, failures, and remaining uncertainty. Choose a next action: stop, request targeted evidence, revise the package, or discuss commercial terms separately. Keep test records with the scope proposal so that later delivery can be compared with what the evaluator actually assessed.
| Question | Evidence to capture | Decision owner |
|---|---|---|
| Can the labels be joined? | Matched, missing, and duplicate image IDs | Evaluator data engineer |
| Does coverage match the use? | Counts by required class and capture setup | Evaluator task lead |
| What failed? | Failure log tied to sample release | Both teams at the review date |
Evaluation plan worksheet
Edit this outline, then copy or download it. It stays in this page and is not submitted or saved by this tool. Keep confidential records and personal information out. Save your work before leaving or reloading.
You can select and copy the text manually. This is a preparation worksheet, not a legal agreement.