Psychiatry Clinical Skills Evaluation: Interrater Reliability of the American Board of Psychiatry and Neurology Required
- PDF / 201,087 Bytes
- 5 Pages / 595.276 x 790.866 pts Page_size
- 87 Downloads / 207 Views
EMPIRICAL REPORT
Psychiatry Clinical Skills Evaluation: Interrater Reliability of the American Board of Psychiatry and Neurology Required Assessment Michael D. Jibson 1
&
Nahid Keshavarzi 1
Received: 24 April 2020 / Accepted: 5 October 2020 # Academic Psychiatry 2020
Abstract Objective The American Board of Psychiatry and Neurology (ABPN) requires applicants to successfully complete at least 3 Clinical Skills Evaluations (CSE) as part of the credentialing requirements to sit for the psychiatry certification examination. The authors examined a database of CSE assessments conducted at a single site to determine the interrater and test-retest reliability of the evaluation. Methods The authors examined 159 CSE assessments of 51 practicing psychiatrists who completed residency prior to the implementation of CSEs, but did not previously sit for the ABPN examination and later required CSEs to do so. Of these, 36 were simultaneously observed by 2 evaluators, at least 1 of whom had extensive experience with the CSE and 8 of whom were novice evaluators. At least 3 CSEs were done on the same day by each candidate. Results The intraclass correlations for CSEs observed by 2 evaluators for the 3 clinical skills assessed (patient–physician relationship, conduct of the interview, and case presentation), were in the low-moderate “good” range for interrater reliability. Test-retest reliability for all candidates was in the moderate–high “good” range. Significant learning took place between the 1st and 3rd CSEs, with a mean improvement of 0.42–0.49 points on an 8-point scale (p < 0.01). Conclusions These data demonstrate good interrater reliability for the CSE even when conducted by novice faculty evaluators and good test-retest reliability despite significant learning that occurred over the series of assessments. The study supports both the educational and assessment value of the CSE. Keywords Certification . Education . Evaluation . Psychiatry . Interrater reliability . Test-retest reliability
In 2011, the American Board of Psychiatry and Neurology (ABPN) became the last member of the American Board of Medical Specialties to discontinue the live-patient evaluation as a component of its psychiatry certification examination. In its place, the ABPN now requires applicants to successfully complete at least 3 Clinical Skills Evaluations (CSE; sometimes referred to as “clinical skills verification” or “CSV”) of physician–patient relationship, psychiatric interview including mental status examination, and case presentation as part of the credentialing requirements to sit for the psychiatry certification examination [1]. Shortly thereafter, the Psychiatry Review Committee of the Accreditation Council of Graduate Medical
* Michael D. Jibson [email protected] 1
University of Michigan, Ann Arbor, MI, USA
Education (ACGME) required that residency programs conduct annual clinical skills assessments of residents that include all components of the CSE [2]. The CSE is now an integral component of all ACGME-accredited psychiatry progr
Data Loading...
