Presentation Description
Siu Hong Michael Wan1
Hui Yin Wan2
1 The University of Notre Dame. Australia
2 Bond University
Hui Yin Wan2
1 The University of Notre Dame. Australia
2 Bond University
Background
Both modified Angoff and Cohen methods are used in the standard setting of the pass mark of Multiple-Choice Questions (MCQ) in written examinations. Angoff methods are labour intensive with many academics involved and the Cohen method is relatively simple to apply. We compared these standard setting methods in the final year high-stakes examinations of the graduate-entry MD program.
Both modified Angoff and Cohen methods are used in the standard setting of the pass mark of Multiple-Choice Questions (MCQ) in written examinations. Angoff methods are labour intensive with many academics involved and the Cohen method is relatively simple to apply. We compared these standard setting methods in the final year high-stakes examinations of the graduate-entry MD program.
Summary of Work
From 2018 to 2022, cohorts of final year medical students, comprising of 115-125 students, sat the 100-item MCQ paper as part of their year-end summative examinations. A multi- disciplinary panel of clinicians set the pass marks for the papers using the modified Angoff method. The modified Cohen mark was calculated according to the cohort’s performance scores.
Results
Pass marks derived from the modified Angoff method for the 2018-2022 cohorts were 53.3%, 51.2%, 52.2%, 54.8% & 53.0% respectively. The modified Cohen mark was calculated at 65% of the 90th percentile of each student cohort. The Cohen pass marks for the cohorts were 52.0%, 53.3%, 51.9%, 50.2% & 51.3% respectively. There were no significant differences in the number of students who failed the MCQ exam using both standard setting methods.
Discussion
Modified Cohen method gave very similar pass marks as the modified Angoff method in the high-stakes summative MCQ examinations. Limitation of the study includes data from a single medical school.
Modified Cohen method gave very similar pass marks as the modified Angoff method in the high-stakes summative MCQ examinations. Limitation of the study includes data from a single medical school.
Conclusions
This is an encouraging finding. As modified Cohen method requires only simple calculations post examination, this might be a more efficient method in standard setting of high-stake summative examinations.
Take-home Message
The new Modified Cohen method could be used in high-stakes summative examination to set the pass mark of written MCQ papers, minimising human resources and time as well as maximising efficiency. More historical data can be used to verify this method compared to any current standard setting methods.
References (maximum three)
Taylor CA. Development of a modified Cohen method of standard setting. Med Teach. 2011;33(12):e678-82. doi: 10.3109/0142159X.2011.611192. PMID: 22225450.