Impossible Hypotheses and Effect Size Limits

Abstract

Psychological science moves towards specification of effect sizes in formulating hypotheses, performing power-analyses, and when consideration of the relevance of findings. This development has sparked an appreciation for the wider context in which such effect sizes are found, as the importance assigned to specific sizes may vary from situation to situation. We add to this development a crucial but in psychology hitherto underappreciated contingency: there are mathematical limits to the magnitudes that population effect sizes can take within the common multivariate context in which psychology is situated, and these limits can be far more restrictive than typically assumed. The implication is that some hypothesized or pre-registered effect sizes may be impossible. At the same time, these restrictions offer a way of statistically triangulating the plausible range of unknown effect sizes. We explain the reason for the existence of these limits, illustrate how to identify them, and offer recommendations for improving hypothesized effect sizes exploiting the broader multivariate context in which they are situated.