John Gimpel¹
Richard Labaere^2,1, Jeanne Sandella¹ and John Boulet¹
1 NBOME
2 ATSU

After the pandemic-associated discontinuation of the Level-2-PE clinical skills component of the COMLEX-USA national licensure exam in 2020-2021, a Special Commission recommended research on a school-based national standardized assessment of core osteopathic clinical skills, including communication skills, medical interviewing, and physical examination. The NBOME committed resources to developing a core competency capstone for osteopathic medical students (C3DO). 

The C3DO pilot evaluated the feasibility of administering a centrally developed multi-station OSCE at osteopathic colleges. Senior medical students from 4 schools completed the 8-station OSCE exam prototype. The prototype incorporated nationally standardized case content and extensive standardized patient and physician examiner training. The analysis cohort included 811 senior medical students. Descriptive statistics were used to compare performances both within and across schools. The reliability of the component and total scores was estimated using generalizability theory. Evidence to support the construct validity of the scores was gathered by quantifying the relationships between component scores (Pearson r) and summarizing student survey responses regarding the adequacy and comprehensiveness of the assessment. 

Preliminary results revealed some variability in performance across medical schools and attributable to the choice of evaluator. The reliability of the patient-physician communication summary score was ρ2=0.80. The ability to gather data (history taking, physical examination) was moderately correlated with communication ability (r=0.38). The summary of student survey responses indicated that most students felt that the assessment was fair and measured the relevant competencies. 

The results from the initial pilot studies suggest that administering a school-based clinical skills assessment is feasible. While some evidence was gathered to support the psychometric adequacy of the scores, further studies are needed to buttress the validity argument. The NBOME, as a part of strategy to build a standardized, school-based, clinical skills assessment, will continue to collaborate with colleges of osteopathic medicine to adapt and refine the C3DO prototype. 

References (maximum three) 

National Board of Osteopathic Medical Examiners. COMLEX-USA · Level 2-PE — NBOME. Retrieved November 30, 2022, from https://www.nbome.org/assessments/comlex-usa/comlex- usa-level-2-pe/ 

National Board of Osteopathic Medical Examiners. The Special Commission on Osteopathic Medical Licensure Assessment Final Report — NBOME. Retrieved January 12, 2023, from https://www.nbome.org/special-commission-on-osteopathic-medical-licensure- assessment/final-report/