Jillian Yeo¹
1 Centre for Medical Education

The broad, overarching question for the action research study was: How can assessment practices in NUS Medicine medical undergraduates’ OSCEs be optimized to ensure fairness? The use of action research methodology in this study stems from its fundamental intent to solve real-life problems (Lewin, 1946). In cycle 1, faculty were recruited on a voluntary basis to participate in the semi-structured interview to gather their perceptions on the fairness of the OSCEs. The 15 minute long survey was disseminated to all medical undergraduates who fit the inclusion criteria of having taken an OSCE previously and were in Phase 2 to Phase 5 of NUS Medicine. Initial and descriptive coding was adopted to code the transcripts and open- ended survey responses manually. 

The goal of the Cycle 2 action step is to work collaboratively with OSCE examiners and the Faculty Assessment Committee to develop the examiner feedback framework. A pilot study will be conducted in the Phase 4 medical undergraduate OSCEs. An interview will be conducted with 5 examiners involved in the assessment of Phase 4 medical undergraduates OSCEs to gather their feedback on the usefulness, ease of understanding, gaps in the feedback report. After revision of the report, another 5 examiners will be interviewed to evaluate its effectiveness. 

Ten faculty were interviewed and 51 medical undergraduates responded to the survey. Sixty- three percent of medical undergraduates felt that the OSCE was not a fair form of assessment. A general consensus among faculty was the importance of striving to attain fairness in all assessments, however, complete fairness is unattainable. Within the institution, themes that contributed to unfairness concentrated around stakeholders’ behavior: human variability, deviant behaviours and cultural awareness. Themes that contributed to fairness in the OSCEs were the systemic infrastructure imposed by the assessment committee: structured systems and fairness in numbers. 

References (maximum three) 

Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2, 4, 34- 46. https://doi.org/10.1111/j.1540-4560.1946.tb02295.x 

Malau-Aduli, B. S., Jones, K., Saad, S., & Richmond, C. (2022). Has the OSCE Met Its Final Demise? Rebalancing Clinical Assessment Approaches in the Peri-Pandemic World. Front Med (Lausanne), 9, 825502. https://doi.org/10.3389/fmed.2022.825502 

Valentine, N., Shanahan, E. M., Durning, S. J., & Schuwirth, L. (2021, Sep). Making it fair: Learners' and assessors' perspectives of the attributes of fair judgement. Med Educ, 55(9), 1056-1066. https://doi.org/10.1111/medu.14574