Nara Jones^1,2
Linda Grose¹, Francesco Amico¹ and Conor Gilligan^3,1
1 University of Newcastle
2 University of Tasmania
3 EACH

The surgical viva voce exam (Viva) at our institution is a high stakes assessment delivered to over 200 students per year through multiple multi-question exam stations. Ensuring its integrity proves challenging for heterogenous reasons, including reliance on time-poor clinicians as examiners. 

Concerns raised in 2022 over assessor variability, fail rates and student feedback mandated a structural revision. Questions, marking guidance, and exam delivery, were modified aiming for improved exam validity and reliability. 

We re-wrote exam questions to improve their ability to explore the students’ clinical knowledge. We constructed example answers and assessor guidance mapped to the marking schema. These were provided to assessors for calibration, along with a short online briefing. 

Each student was assessed by eight different assessors across the stations to reduce bias. Each station was standard set separately using the Borderline Regression Method to limit the impact of variable station difficulty. 

A hybrid, online and face-to-face, delivery was chosen to enhance assessor recruitment and academic integrity. A survey is scheduled in November 2023, to explore assessor experience. 

Data obtained from half of the 2023 cohort demonstrates low assessor standard deviations, strong station statistics, and perceived improvements in the quality and specificity of assessor feedback. Similar results are expected for the full cohort in November 2023. 

Unintended consequences of such approaches include rigidity which limits the capacity of assessors to apply expert judgement and can force student results into pre-established categories. Such limitation can be counterbalanced by selecting ad-hoc sub-specialty examiners. 

Conclusions
Surgery Viva revision proved beneficial to overall exam quality. 

A proactive approach is paramount for continuous revision of assessment methods contributing significantly towards student progression decision in healthcare education. 

References (maximum three) 

1. Imran M, Doshi C, Kharadi D. Structured and unstructured viva voce assessment: A double- blind, randomized, comparative evaluation of medical students. Int J Health Sci (Qassim). 2019 Mar-Apr;13(2):3-9. PMID: 30983939; PMCID: PMC6436443.