What have published studies of simulated respondents validated?

Papers and vendors report that simulated respondents match people. The reports use different human references, question types and metrics, so the results cannot be set side by side. Many tests replay public surveys a model may have seen. Few state how far two samples of real people differ. A map shows which claims rest on questions the model could not have seen, for which question types, against which human reference.

Asking a model for the split beat simulating respondents and counting on 1,222 of 1,680 distributions (Gong et al., 2026). Interview-grounded agents reached 83% of participants' own two-week retest agreement (Park et al., 2024). Full personas scored 0.748 accuracy against 0.746 for demographics alone across 164 outcomes (Peng et al., 2025). Model variance ran 1.9 to 3.9 times below human among professional philosophers (Shi and Haupt, 2026). Training on answer distributions cut Wasserstein distance 32 to 46%, with gains on unseen surveys and groups (Suh et al., 2025). Pew found mean errors of 12.4 points when synthetic samples were set against its panel (Pew Research Center, 2026). Vendor figures are self-reported.

The unit is a published study or public vendor page with model answers and human answers to the same questions. The search strings, databases and inclusion rule are registered first. Each study is coded for respondent group, question type, method, human reference, replay or held-out tag and reported metrics. A second coder codes a fifth of the studies, and agreement is reported per field. The map counts and compares and does not pool.

A table that shows, for each question type and respondent group, the number of studies, the human references used and the share tagged held-out. The answer holds only if two coders agree on the replay or held-out tag for most studies.

A narrative review that cites the strongest results and repeats the authors' own summary claims. The map must show what that review cannot: how many studies tested questions a model could not have seen, and for which question types and respondent groups.

The map describes what was published. Studies differ in metric and reference, so figures are not pooled. The search is bounded by its strings and by sources that load. Preprints may change. Vendor pages report figures without audit. A gap in the map shows an absence of reports, not an absence of evidence.