Nonparametric Predictive Inference for Test Reproducibility by Sampling Future Data Orderings
- PDF / 1,716,191 Bytes
- 22 Pages / 439.37 x 666.142 pts Page_size
- 98 Downloads / 199 Views
Nonparametric Predictive Inference for Test Reproducibility by Sampling Future Data Orderings Frank P. A. Coolen1 · Filipe J. Marques2
© Grace Scientific Publishing 2020
Abstract This paper considers nonparametric predictive inference (NPI) for reproducibility of likelihood ratio tests with the test criterion in terms of the sample mean. Given a sample of size n used for the actual test, the NPI approach provides lower and upper probabilities for the event that a repeat of the test, also with n observations, will lead to the same overall test conclusion, that is rejecting a null-hypothesis or not. This is achieved by considering all orderings of n future observations among the n data observations, which based on an exchangeability assumption are equally likely. However, exact lower and upper probabilities can only be derived for relatively small values of n due to computational limitations. Therefore, the main aim of this paper is to explore sampling of the orderings of the future data among the observed data in order to approximate the lower and upper reproducibility probabilities. The approach is applied for the Exponential and Normal distributions, and the performance of the ordering sampling for approximation of the NPI lower and upper reproducibility probabilities is investigated. An application with real data of the methodology developed is provided. Keywords Exponential family · Likelihood ratio test · Lower and upper probabilities · Nonparametric predictive inference · Normal distribution · Reproducibility probability · Sampling orderings of future observations Mathematics Subject Classification 62A99 · 62G99 · 62P30
* Filipe J. Marques [email protected] Frank P. A. Coolen [email protected] 1
Durham University, Durham, UK
2
Universidade Nova de Lisboa, Lisbon, Portugal
13
Vol.:(0123456789)
62
Page 2 of 22
Journal of Statistical Theory and Practice
(2020) 14:62
1 Introduction The reproducibility probability (RP) of a statistical test measures how likely it is that if a statistical test were repeated under the same circumstances, it would lead to the same conclusion, that is the rejection or non-rejection of the null hypothesis. This is an important property which was first addressed by Goodman [11] and then by Shao and Chow [17], De Martini [10], De Capitani and De Martini [9] and Shao and Chow [17] who dealt with this issue as being an estimation of the power of a test problem. In Coolen and Bin Himd [7] a new perspective was presented using the nonparametric predictive inference (NPI) framework of frequentist statistical methods [2, 5, 6]. This NPI approach for the reproducibility probability of a test (NPI-RP) considers the test result for a predicted future sample of the same size as the original sample; this approach will be detailed in Sect. 2. The NPI approach for reproducibility of likelihood ratio tests was introduced in [15] and used in [16] to study the reproducibility of hypotheses testing between two Beta distributions. In [15] the authors considered only simple hypoth
Data Loading...
