Addressing the theory crisis in psychology
- PDF / 873,329 Bytes
- 23 Pages / 595.276 x 790.866 pts Page_size
- 36 Downloads / 208 Views
THEORETICAL REVIEW
Addressing the theory crisis in psychology Klaus Oberauer 1 & Stephan Lewandowsky 2,3
# The Psychonomic Society, Inc. 2019
Abstract A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans. Keywords Replication . Scientific inference . Hypothesis testing . Computational modeling . Preregistration
Psychology has a problem. Over the past decade it has become clear that many findings, among them some deemed well established, are not replicable (Marsman et al., 2017; Open Science Collaboration, 2015). Numerous recommendations have been made for how to address this “replication crisis” (Asendorpf et al., 2013; Munafò et al., 2017). Virtually all these recommendations pertain to our methods of data collection, analysis, and publication. Here, we argue that, in addition to poor methods, the replication crisis is also due to the prevalence of theories that have only a weak logical relation to the hypotheses through which they are evaluated empirically. We suggest that the replication crisis is best resolved by focusing attention on the role of theorizing, and we do not believe that current recommendations that focus entirely on data * Klaus Oberauer [email protected] 1
Department of Psychology–Cognitive Psychology, University of Zurich, Binzmühlestrasse 14/22, 8050 Zürich, Switzerland
2
University of Bristol, Bristol, UK
3
University of Western Australia, Crawley, Australia
generation are sufficient to overcome the crisis. To help clarify our argument, we summarize the intended meaning of some key terms in Table 1. Scientific reasoning relies on inferences on two levels (see Fig. 1). On the first, the empirical level, we link hypotheses (e.g., X, Y, Z) to data (e.g., “x,” “y,” “z”
Data Loading...
