To assess the replicability of social priming findings we reviewed the extant close replication attempts in the field. In total, we found 65 close replications, that replicated 46 unique findings. Ninety-four percent of the replications had effect sizes smaller than the effect they replicated, only 18% of the replications reported a significant p-value in the original direction, and the 95% confidence interval of the replication effects included the original effects only 26% of the time. The strongest predictor of replication success was whether or not the replication team included at least one of the authors of the original paper. Twelve of the 16 replications with at least one original author produced a significant effect in the original direction and the meta-analytic average of these studies suggest a significant priming effect (d = 0.33, 95% CI[0.26; 0.65]). In stark contrast, none of the 49 replications by independent research teams produced a significant effect in the original direction and the meta-analytic average was virtually zero (d = 0.001, 95% CI[-0.03; 0.03]). We argue that these results have shifted the burden of proof back onto advocates of social priming. Successful replications from independent research teams will likely be required to convince sceptics that social priming exists at all.
Latest posts by Ryan Watkins (see all)
- AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap - September 22, 2023
- The Robotic Herd: Using Human-Bot Interactions to Explore Irrational Herding - September 22, 2023
- Human-AI Interactions and Societal Pitfalls - September 19, 2023