Chi-Wei Lin^1,2,3
Kuo-Chin Huang^4,5,3, Pei-Chun Kuo¹, I-Ting Liu^1,2,3 and Ming-Nan Lin^6,3
1 E-Da Hospital
2 I-Shou University
3 Taiwan Association of Family Medicine
4 National Taiwan University Hospital
5 National Taiwan University
6 Dalin Tzu Chi Hospital

Since 1986, the Taiwan Association of Family Medicine has overseen the training, evaluations, and specialist examinations of family medicine in Taiwan. The Family Medicine specialist exam includes a preliminary 125 multiple choice questions (MCQs), followed by an oral test for those who pass. Despite rigorous MCQ design, issues like high pass rates and question flaws persisted. The study aims to elucidate the impact of a 2018-initiated intervention to enhance question quality. 

Summary of work: 

A five-year program was introduced wherein testing experts trained MCQ composers. Following each exam, questions underwent a difficulty and discrimination assessment. Feedback was provided to composers, flagging overly simplistic, challenging, or low- discrimination questions for subsequent revisions. 

Results: 

Approximately 130 candidates sit for the board examination annually. Post-intervention data showed that the average discrimination index increased from 0.15 (2017) to 0.23 (2022) (p=0.003). The proportion of questions with low discrimination (D<0.2) decreased from 72% to 46%. Question difficulty remained stable, ranging between 0.70 and 0.75. 

Discussion: 

MCQs, pivotal in global medical exams, present both benefits (flexibility, control over difficulty) and challenges (bias towards memory recall, difficulties in creating high- discrimination questions). This study underscores the potential of continued education and feedback mechanisms to augment the quality of MCQs. 

Conclusions: 

The Taiwan Association of Family Medicine's intervention emphasizes the critical role of ongoing training and feedback in boosting MCQ test quality, suggesting a promising model for other medical education boards globally. 

Take-home messages: 

Addressing challenges in MCQ design, especially those favoring memory recall, requires a dedicated and systematic approach. 

Continued education and feedback mechanisms can significantly enhance the discrimination index and overall quality of MCQs in medical exams. 

The Taiwan Association of Family Medicine initiative's success serves as a model for medical education boards globally, emphasizing the value of expert-led training and consistent feedback. 

References (maximum three) 

PA Coughlin, CR Featherstone. How to Write a High Quality Multiple Choice Question (MCQ): A Guide for Clinicians. Review Eur J Vasc Endovasc Surg. 2017 Nov;54(5):654-658. 

Ghada Khafagy, Marwa Ahmed, Nagwa Saad. Stepping up of MCQs' quality through a multi- stage reviewing process. Educ Prim Care. 2016 Jul;27(4):299-303. 

Piyush Gupta, Pinky Meena, Amir Maroof Khan, Rajeev Kumar Malhotra, Tejinder Singh. Effect of Faculty Training on Quality of Multiple-Choice Questions. Int J Appl Basic Med Res. 2020 Jul-Sep;10(3):210-214.