Jillian Yeo¹
1 Centre for Medical Education

The broad, overarching question for the action research study was: How can assessment practices in NUS Medicine medical undergraduates’ OSCEs be optimized to ensure fairness? The use of action research methodology in this study stems from its fundamental intent to solve real-life problems (Lewin, 1946). In cycle 1, faculty were recruited on a voluntary basis to participate in the semi-structured interview to gather their perceptions on the fairness of the OSCEs. The 15 minute long survey was disseminated to all medical undergraduates who fit the inclusion criteria of having taken an OSCE previously and were in Phase 2 to Phase 5 of NUS Medicine. Initial and descriptive coding was adopted to code the transcripts and open- ended survey responses manually. 

The goal of the Cycle 2 action step is to work collaboratively with OSCE examiners and the Faculty Assessment Committee to develop the examiner feedback framework. A pilot study will be conducted in the Phase 4 medical undergraduate OSCEs. An interview will be conducted with 5 examiners involved in the assessment of Phase 4 medical undergraduates OSCEs to gather their feedback on the usefulness, ease of understanding, gaps in the feedback report. After revision of the report, another 5 examiners will be interviewed to evaluate its effectiveness. 

Ten faculty were interviewed and 51 medical undergraduates responded to the survey. Sixty- three percent of medical undergraduates felt that the OSCE was not a fair form of assessment. A general consensus among faculty was the importance of striving to attain fairness in all assessments, however, complete fairness is unattainable. Within the institution, themes that contributed to unfairness concentrated around stakeholders’ behavior: human variability, deviant behaviours and cultural awareness. Themes that contributed to fairness in the OSCEs were the systemic infrastructure imposed by the assessment committee: structured systems and fairness in numbers. 

References (maximum three) 

Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2, 4, 34- 46. https://doi.org/10.1111/j.1540-4560.1946.tb02295.x 

Malau-Aduli, B. S., Jones, K., Saad, S., & Richmond, C. (2022). Has the OSCE Met Its Final Demise? Rebalancing Clinical Assessment Approaches in the Peri-Pandemic World. Front Med (Lausanne), 9, 825502. https://doi.org/10.3389/fmed.2022.825502 

Valentine, N., Shanahan, E. M., Durning, S. J., & Schuwirth, L. (2021, Sep). Making it fair: Learners' and assessors' perspectives of the attributes of fair judgement. Med Educ, 55(9), 1056-1066. https://doi.org/10.1111/medu.14574 

Elena Pascoe^1,2
Conor Gilligan³, Katie Wynne⁴ and Brian Jolly^5,3
1 School of Medicine & Public Health, College of Health, Medicine and Wellbeing, University of Newcastle.
2 risr/
3 University of Newcastle
4 The University of Newcastle / University of New England
5 School of Medicine & Public Health, College of Health, Medicine and Wellbeing

1. Research Question(s)
This research will explore the influence of the supervisor and student role in providing/receiving feedback for Workplace-based Assessments (WBAs) assessed in Years 4/5 of the BSCI/MD programmes by investigating: 

2. Analysis / Methodology
This project will utilise three strands of inquiry and multiple methodologies in separate phases: 

3. Findings so far
This is work in progress with results unavailable. 

How can qualitative methodologies be used to add value/evidence in health professions assessment? 
Could these research methodologies inform improvements to assessment implementation? 

5. Will your supervisor be in attendance? 
Yes 

References (maximum three) 

Fernando, N., Cleland, J., McKenzie, H., & Cassar, K. (2008). Identifying the factors that determine feedback given to undergraduate medical students following formative mini‐CEX assessments. Medical Education, 42(1), 89-95. 

Molloy, E., Boud, D., & Henderson, M. (2020). Developing a learning-centred framework for feedback literacy. Assessment & Evaluation in Higher Education, 45(4), 527-540. 

Stokoe, E. (2013). Applying findings and creating impact from conversation analytic studies of gender and communication. Economic and Industrial Democracy, 34(3), 537-552.