Synthetic Data: The Questions to Ask
A one-page fitness-for-purpose guide for research buyers, covering where synthetic data is viable, where it isn't, and the ten critical questions to ask any supplier.
A one-page fitness-for-purpose guide for research buyers.
Where synthetic data is a viable option today
- ·
Boosting small samples for subgroup analysis where robust real seed data exists (n >= 200 per segment)
- ·
Privacy protection (sharing sensitive datasets without exposing real personal data)
- ·
Early-stage exploration (hypothesis generation, concept screening, discussion guide piloting before real fieldwork)
- ·
Pre-testing survey instruments (identifying comprehension failures cheaply before launch)
- ·
Hard-to-reach populations at discovery stage where recruitment is prohibitively expensive or ethically complex
Where it is not viable right now
- ·
Tracking studies, brand equity, or U&A benchmarks (synthetic trend data has no anchor in real-world change)
- ·
Decision-grade research as sole evidence (major launches, pricing, or investment decisions need real respondents)
- ·
Emotional or behavioural insight (synthetic participants report what people write about, not what they feel or do)
- ·
Rare events and tail behaviour (models systematically miss low-frequency but commercially critical patterns)
- ·
Regulated categories without legal review (health, financial vulnerability, and children require enhanced scrutiny)
Ten critical questions to ask any supplier
- ·
What real data underpins your synthetic outputs, and can you evidence its provenance and consent basis?
- ·
How do you validate synthetic outputs against real respondents? Ask for the specific tests and pass thresholds.
- ·
What calibration benchmark do you run, and what percentage of synthetic outputs are checked against real humans?
- ·
How do you assess and correct for bias, particularly on protected characteristics and non-Western populations?
- ·
Which model and version generated this data, and can you reproduce the same outputs on request?
- ·
What re-identification risk assessment have you done? "It's synthetic so it's anonymous" is not an answer.
- ·
Will synthetic elements be explicitly labelled in every deliverable, including verbatims and charts?
- ·
What are you telling us this data cannot be used for? A credible supplier volunteers the limitations.
- ·
How do you comply with MRS/ESOMAR requirements and EU AI Act transparency obligations for AI-generated content?
- ·
If we audited your methodology tomorrow, what documentation would you show us?
The pattern to watch for: suppliers who lead with speed and cost but cannot answer questions 2, 3, and 5 quickly and specifically. Validation, calibration, and reproducibility are where synthetic data programmes succeed or fail.