A Novel USMLE Step 1 Projection Model Using a Single Comprehensive Basic Science Self-Assessment Taken During a Brief In
- PDF / 326,950 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 52 Downloads / 147 Views
ORIGINAL RESEARCH
A Novel USMLE Step 1 Projection Model Using a Single Comprehensive Basic Science Self-Assessment Taken During a Brief Intense Study Period Stephen D. Bigach 1,2
&
Robert D. Winkelman 2 & Jonathan C. Savakus 2,3 & Klara K. Papp 2
Accepted: 1 October 2020 # International Association of Medical Science Educators 2020
Abstract Background Comprehensive Basic Science Self-Assessments (CBSSAs) offered by the National Board of Medical Examiners (NBME) are used by students to gauge preparedness for the United States Medical Licensing (USMLE) Step 1. Because residency programs value Step 1 scores, students expend many resources attempting to score highly on this exam. We sought to generate a predicted Step 1 score from a single CBSSA taken several days out from a planned exam date to inform student testing and study plans. Methods 2016 and 2017 Step 1 test takers at one US medical school were surveyed. The average daily score improvement from CBSSA to Step 1 during the 2016 study period was calculated and used to generate a predicted Step 1 score as well as mean absolute prediction errors (MAPEs). The predictive model was validated on 2017 data. Results In total, 43 of 61 respondents totaling 141 CBSSAs in 2016 and 37 of 43 respondents totaling 122 CBSSAs in 2017 were included. The final prediction model was [Predicted Step 1 = 292 - (292 - CBSSA score) * 0.987527 ^ (number of days out)]. In 2016, the average difference between predicted and actual scores was -0.81 (10.2) and the MAPE was 7.8. In 2017, 88 (72.1%) and 118 (96.7%) of true Step 1 scores fell within one and two standard deviations of a student’s predicted score. There was a MAPE of 7.7. Practice form used (p = 0.19, 0.07) and how far out from actual Step 1 it was taken (p = 0.82, 0.38) were not significant in either year of study. Conclusion This projection model is reasonable for students to use to gauge their readiness for Step 1 while it remains a scored exam and provides a framework for future predictive model generation as the landscape of standardized testing changes in medical education. Keywords Step 1 . Testing . Projection modeling . USMLE
Introduction For several years, the National Board of Medical Examiners (NBME) has offered Comprehensive Basic Science Self-
* Stephen D. Bigach [email protected] 1
Department of Orthopaedic Surgery, McGaw Medical Center of Northwestern University, IL Chicago, USA
2
School of Medicine, Case Western Reserve University, Cleveland, OH, USA
3
Department of Orthopaedic Surgery, Vanderbilt University Medical Center, Nashville, TN, USA
Assessments (CBSSAs) to assist students preparing for the United States Medical Licensing Examination (USMLE) Step 1 examination. The self-assessments have been shown to predict students’ readiness to take the exam [1] when taken in test-like conditions [2]. However, much to the dismay of some [3, 4], in recent years, students’ Step 1 scores have been used increasingly more frequently to rank applicants for interviews and the match process [5–7]. T
Data Loading...
