Presentation Description
Korakrit Imwattana1
1 Faculty of Medicine Siriraj Hospital, Mahidol University
1 Faculty of Medicine Siriraj Hospital, Mahidol University
Background:
The multiple-choice examination is an essential tool for the assessment of basic medical knowledge. Some test items may include a short clinical scenario (a stem). If used appropriately, these stems can provide clinical context and may help students prepare for their clinical-based assessment in the future, without much impact on the question quality. However, generating stems can be challenging, especially for educators without clinical background or experience.
The multiple-choice examination is an essential tool for the assessment of basic medical knowledge. Some test items may include a short clinical scenario (a stem). If used appropriately, these stems can provide clinical context and may help students prepare for their clinical-based assessment in the future, without much impact on the question quality. However, generating stems can be challenging, especially for educators without clinical background or experience.
Summary of work:
An Excel-based exam stem generator was developed in-house. This prototype focused on basic bacterial infections, following the Thai Medical Council’s criteria, for use in the Principles of Microbiology class. The program was designed to be user-friendly, requiring only 5 inputs (bacterium of interest; patient’s age, gender, extra conditions [risk factors, comorbidities etc.]; and whether to include laboratory information). It also provided suggestions for non-clinical users. The program then generated a clinical scenario with sufficient keywords for the most likely causative organism. This study evaluates the efficacy of this program.
An Excel-based exam stem generator was developed in-house. This prototype focused on basic bacterial infections, following the Thai Medical Council’s criteria, for use in the Principles of Microbiology class. The program was designed to be user-friendly, requiring only 5 inputs (bacterium of interest; patient’s age, gender, extra conditions [risk factors, comorbidities etc.]; and whether to include laboratory information). It also provided suggestions for non-clinical users. The program then generated a clinical scenario with sufficient keywords for the most likely causative organism. This study evaluates the efficacy of this program.
Results and discussion:
Compared to an experienced educator in an ideal condition (38.97 seconds/stem) this in-house stem generator significantly reduced the time to create stems (20.49 seconds/stem, p < 0.0001). Among the 50 randomly generated stems, only 12% (95% CI: 4 – 24%) required minor editing, excluding scientific name formatting, and none required major editing.
Compared to an experienced educator in an ideal condition (38.97 seconds/stem) this in-house stem generator significantly reduced the time to create stems (20.49 seconds/stem, p < 0.0001). Among the 50 randomly generated stems, only 12% (95% CI: 4 – 24%) required minor editing, excluding scientific name formatting, and none required major editing.
Conclusion:
This exam stem generator is a simple and useful tool to reduce the educator's workload in preparing exam items. Advanced users can also easily expand on this prototype to include more diseases and specific conditions to fit the need of different courses.
This exam stem generator is a simple and useful tool to reduce the educator's workload in preparing exam items. Advanced users can also easily expand on this prototype to include more diseases and specific conditions to fit the need of different courses.
Take-home message:
An Excel-based program can be designed to help reduce the educator's workload as well as standardise the workflow.
An Excel-based program can be designed to help reduce the educator's workload as well as standardise the workflow.
References (maximum three)
1 Medical Competency Assessment Criteria for National License 2012: The Medical Council of Thailand; 2012 [Retrieved from: http://www.tmc.or.th/download/medical2555.pdf].
2. Hays RB, Coventry P, Wilcock D, Hartley K. Short and long multiple-choice question stems in a primary care oriented undergraduate medical curriculum. Educ Prim Care. 2009 May;20(3):173-7.