Presentation Description
Stefan Schauber1
Shewatatek Gedamu Wonde2,1
1 University of Oslo
2 Jimma University
Shewatatek Gedamu Wonde2,1
1 University of Oslo
2 Jimma University
Background
To improve health service coverage, Ethiopia has implemented a “flooding strategy” to health professions education and, across several years, established multiple medical schools across the country (1). In this context, a national medical licensing exam plays an important role to assure minimal standards of competency of medical graduates (2). In a joint project between Jimma University, Ethiopia, and the University of Oslo, Norway, we conducted a mixed- methods study to provide evidence for the quality of the Ethiopian licensing examination in medicine.
To improve health service coverage, Ethiopia has implemented a “flooding strategy” to health professions education and, across several years, established multiple medical schools across the country (1). In this context, a national medical licensing exam plays an important role to assure minimal standards of competency of medical graduates (2). In a joint project between Jimma University, Ethiopia, and the University of Oslo, Norway, we conducted a mixed- methods study to provide evidence for the quality of the Ethiopian licensing examination in medicine.
Summary of work
We collected both qualitative (expert interviews) and quantitative data (student responses to exam items and expert judgments from standard setting). We will present background on the Ethiopian context of medical training and psychometric analyses of exam data. Preliminary results from the qualitative part of the study will be presented, too.
Results
Data on 600 items and N=2,213 candidates were collected. We analyzed results from three administrations of the medical licensing exam using both Classical Test Theory and Item Response Theory (IRT). Results indicate that the exams are sufficiently reliable, with coefficient alpha varying between 0.83 to 0.85. Comparable results were obtained from IRT analyses. Items generally fit well with the assumptions of IRT. More in-depth analyses indicated potential issues with distractors on selected multiple-choice items.
Discussion
Our results indicate that the exams were reaching recommended standards of reliability. However, a closer look at the particular exam content is warranted. Finally, we will discuss future possibilities to address research questions using this dataset.
Conclusions
MLEs are especially important in the given context using a “flooding strategy” to medical education. Future research should elaborate on differences between medical schools or gender- related differences in exam results.
Take-home messages
We provide crucial evidence for the defensibleness of a high-stakes MLE in Ethiopia.
We provide crucial evidence for the defensibleness of a high-stakes MLE in Ethiopia.
References (maximum three)
(1) Derbew M, Animut N, Talib ZM, Mehtsun S, Hamburger EK. Ethiopian Medical Schools’ Rapid Scale-up to Support the Government’s Goal of Universal Coverage. Academic Medicine. 2014 Aug;89(8):S40.
(2) Price T, Lynn N, Coombes L, Roberts M, Gale T, de Bere SR, et al. The International Landscape of Medical Licensing Examinations: A Typology Derived From a Systematic Review. Int J Health Policy Manag. 2018 Apr 28;7(9):782–90.
(3) Tavakol M, Dennick R. Post-examination interpretation of objective test data: Monitoring and improving the quality of high-stakes examinations: AMEE Guide No. 66. Medical Teacher. 2012 Mar 1;34(3):e161–75.