How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framewor
- PDF / 534,111 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 100 Downloads / 149 Views
CORRESPONDENCE
Open Access
How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framework Brennan C. Kahan1*, Gordon Forbes2 and Suzie Cro3
Abstract Results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the method that provides the most favourable result (commonly referred to as ‘p-hacking’). Pre-specification of the planned analysis approach is essential to help reduce such bias, as it ensures analytical methods are chosen in advance of seeing the trial data. For this reason, guidelines such as SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and ICH-E9 (International Conference for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use) require the statistical methods for a trial’s primary outcome be pre-specified in the trial protocol. However, prespecification is only effective if done in a way that does not allow p-hacking. For example, investigators may prespecify a certain statistical method such as multiple imputation, but give little detail on how it will be implemented. Because there are many different ways to perform multiple imputation, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in different ways before deciding on a final approach. In this article, we describe a five-point framework (the Pre-SPEC framework) for designing a pre-specified analysis approach that does not allow p-hacking. This framework was designed based on the principles in the SPIRIT and ICH-E9 guidelines and is intended to be used in conjunction with these guidelines to help investigators design the statistical analysis strategy for the trial’s primary outcome in the trial protocol. Keywords: Randomised trial, Pre-specification, Transparency, Bias, p-hacking
Background Results from clinical trials depend upon the statistical methods used for analysis [1–5]. Different methods of analysis applied to the same trial can lead to different conclusions around effectiveness and safety [1–14]. Therefore, results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the approach that provides the most favourable result. This is commonly referred to as ‘p-hacking’ and can lead to bias in treatment effect estimates, confidence * Correspondence: [email protected] 1 MRC Clinical Trials Unit at UCL, 90 High Holborn, London WC1V 6LJ, UK Full list of author information is available at the end of the article
intervals, and p values [1–5, 7–10, 12, 15]. Pre-specification of the planned analysis approach is therefore essential to help reduce such bias, as it ensures that analytical methods are chosen in advance of seeing the trial data [1–5, 7, 9, 10, 12]. The SPIRIT (Standard Protocol Items: Recommendations for In
Data Loading...
