Why optional stopping can be a problem for Bayesians
- PDF / 1,767,655 Bytes
- 18 Pages / 595.224 x 790.955 pts Page_size
- 56 Downloads / 185 Views
THEORETICAL REVIEW
Why optional stopping can be a problem for Bayesians 1,2 ¨ Rianne de Heide1,2 · Peter D. Grunwald
© The Author(s) 2020
Abstract Recently, optional stopping has been a subject of debate in the Bayesian psychology community. Rouder (Psychonomic Bulletin & Review 21(2), 301–308, 2014) argues that optional stopping is no problem for Bayesians, and even recommends the use of optional stopping in practice, as do (Wagenmakers, Wetzels, Borsboom, van der Maas & Kievit, Perspectives on Psychological Science 7, 627–633, 2012). This article addresses the question of whether optional stopping is problematic for Bayesian methods, and specifies under which circumstances and in which sense it is and is not. By slightly varying and extending Rouder’s (Psychonomic Bulletin & Review 21(2), 301–308, 2014) experiments, we illustrate that, as soon as the parameters of interest are equipped with default or pragmatic priors—which means, in most practical applications of Bayes factor hypothesis testing—resilience to optional stopping can break down. We distinguish between three types of default priors, each having their own specific issues with optional stopping, ranging from no-problem-at-all (type 0 priors) to quite severe (type II priors). Keywords Bayesian statistics · Hypothesis testing · Model selection · Statistical inference
Introduction P value-based null-hypothesis significance testing (NHST) is widely used in the life and behavioral sciences, even though the use of p values has been severely criticized for at least the last 50 years. During the last decade, within the field of psychology, several authors have advocated the Bayes factor as the most principled alternative to resolve the problems with p values. Subsequently, these authors have made an admirable effort to provide practitioners with default Bayes factors for common hypothesis tests. Key references include, among many others, Rouder, Speckman, Sun, Morey, and Iverson (2009), Jamil, Ly, Morey, Love, Marsman, and Wagenmakers (2016), Rouder, Morey, Speckman, and Province (2012). We agree with the objections against the use of p value-based NHST and the view that this paradigm is inappropriate (or at least far from optimal) for scientific research, and we agree that the Bayes factor has many
Peter D. Gr¨unwald
[email protected] 1
Leiden University, Leiden, Amsterdam, The Netherlands
2
The Netherlands Centre for Mathematics & Computer Science (CWI), Amsterdam, The Netherlands
advantages. However, as also noted by Gigerenzer and Marewski (2014), it is not the panacea for hypothesis testing that a lot of articles make it appear. The Bayes factor has its limitations (cf. also Tendeiro & Kiers, 2019), and it seems that the subtleties of when those limitations apply sometimes get lost in the overwhelming effort to provide a solution to the pervasive problems of p values. In this article, we elucidate the intricacies of handling optional stopping with Bayes factors, primarily in response to Rouder (2014). Optional stopping refers to ‘looking at the resul
Data Loading...
