Selection bias and pseudo discoveries on the constancy of stock return anomalies
- PDF / 1,050,999 Bytes
- 20 Pages / 439.37 x 666.142 pts Page_size
- 47 Downloads / 163 Views
Selection bias and pseudo discoveries on the constancy of stock return anomalies Russell P. Robins1 · Geoffrey Peter Smith2
© Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract There are now a large and rapidly growing number of studies that test the constancy of stock return anomalies. In this study, we produce new and convincing evidence that the standard constancy test is heavily influenced by selection bias. Backed by a carefully designed Monte Carlo simulation, we show that selection bias predisposes the standard constancy test to reject the null by a factor of five to 12 times more than normally expected. Failure to recognize this bias can result in publication of the type of pseudo discoveries that Harvey (J Finance 72(4):1399–1440, 2017) warns about in his Presidential Address to the American Finance Association. We then describe the Quandt/Andrews test, a correct and unbiased test for anomalies and changes in anomalies, and apply it to test the constancy of 15 well-known stock return anomalies. Keywords Selection bias · Stock return anomaly · Constancy test JEL Classification G10 · G14 · G19
1 Introduction In his 2017 Presidential Address to the American Finance Association, Harvey (2017) focuses attention on the problem of statistical bias in financial research. Put simply, statistical bias is a catch-all term encompassing the troublesome problems of “multiple testing,” “selection bias,” “p-hacking,” and “HARKing: hypothesizing after the results are known.” Harvey (2017) argues statistical bias is a serious problem that can lead to an embarrassing number of published studies where type I errors masquerade as meaningful results.1 1 See also Harvey et al. (2016) and Harvey’s Keynote Address at the 2019 Annual Meeting of the Financial Management Association entitled “Fake Research” for more on statistical bias in financial research.
* Geoffrey Peter Smith [email protected] Russell P. Robins [email protected] 1
AB Freeman School of Business, Tulane University, New Orleans, LA 70118, USA
2
WP Carey School of Business, Arizona State University, Tempe, AZ 85287, USA
13
Vol.:(0123456789)
R. P. Robins, G. P. Smith
There are now a large and rapidly growing number of studies that test the constancy of stock return anomalies.2 In this study, we produce new and convincing evidence that the standard constancy test for stock return anomalies is heavily influenced by selection bias. We do this in two ways. First, we begin by formulating a detailed mathematical proof of selection bias through the allegory of testing for a hypothetical publication effect. This proof follows from the fundamental problem of causal inference described in Holland (1986). The basic idea is that the returns on a published stock return anomaly are not otherwise equal to the returns that would have been observed had the anomaly not been published. This absence of a proper counterfactual introduces selection bias. Second, we quantify the effect of selection bias by Monte Carlo simulation. Our results show
Data Loading...
