Quang Ngo¹
Keyna Bracken², Helen Neighbour¹, Mike Lee-Poy¹, Rebecca Long¹, Jeffrey McCarthy¹, Jeremy Sandor¹ and Matthew Sibbald¹
1 DeGroote School of Medicine, McMaster University
2 Michael G. DeGroote School of Medicine, McMaster University, AMEE member

Multiple choice exams (MCEs) in medical education are an efficient means to assess knowledge and clinical reasoning while maintaining standardization. 

There is debate in the literature as to whether MCEs should penalize wrong answers. Proponents argue this rewards accuracy while maintaining test validity. Those against state that it unfairly penalizes risk takers who are making educated, rather than uninformed guesses. Evidence is emerging that this may contribute to gender inequity in MCEs. 

The undergraduate medical education program at McMaster University uses a longitudinal progress test called the Personal Progress Inventory (PPI) to monitor knowledge acquisition, taken 8 times over the course of the program. Traditionally there has been a 0.25 mark penalty for incorrect answers. 

The penalty for guessing was removed in January 2023 due to concerns the penalty unfairly targeted risk averse groups and increased test anxiety without improving validity. This natural experiment allowed us to compare the effects of the penalty on exam outcomes. 

Means of the class scores before and after the removal of the guessing penalty were compared using ANOVA. Data was available for 2 sittings of the PPI (Feb and May/23) for the 1st and 2nd year cohorts. Compared to matched historical cohorts, the means were higher after elimination of the penalty for both sittings of the PPI and for both cohorts (p<0.05). No interaction was found between penalty and year. 

The reasons for the change in scores is likely multifactorial. Anxiety is known to hinder performance and reducing anxiety could contribute to improvement. With respect to test validity, we continue to see scores increase over time, regardless of penalty, suggesting the expertise gradient is preserved. Cut scores for this exam are norm-referenced and the number of below-threshold students has not changed. Future work should identify risk averse groups (i.e gender or specialty choice). 

References (maximum three) 

Lucy R. Betts, Tracey J. Elder, James Hartley & Mark Trueman (2009) Does correction for guessing reduce students’ performance on multiple‐choice examinations? Yes? No? Sometimes?, Assessment & Evaluation in Higher Education, 34:1, 1- 15, DOI: 10.1080/02602930701773091 

Coffman KB, Klinowski D. The impact of penalties for wrong answers on the gender gap in test scores. Proceedings of the National Academy of Sciences. 2020;117(16). pmid:32253310