Presentation Description
Bradley Williams1
Renee Harvey1, Lizzi Shires1, Nara Jones1, Anthea Dallas1 and Rohan Church1
1 University of Tasmania
Renee Harvey1, Lizzi Shires1, Nara Jones1, Anthea Dallas1 and Rohan Church1
1 University of Tasmania
Background
Medical education providers must ensure they deliver fair and defensible quality-assured assessment processes.[1] The Objective Structured Clinical Examination (OSCE) is a complex assessment, requiring a highly coordinated group of clinicians, simulated patients (SPs) and administrative staff, geographically dispersed across testing centres.
Summary of work
Placing quality assurance (QA) at the centre, our OSCE has been refined through the innovative and creative use of technology to deliver a fair assessment, acceptable to all participants.
Briefing:
Examiners are provided station material online with calibrated video examples which they mark and receive immediate feedback from experienced markers. SPs receive their material in the same format. An online briefing session is offered prior to the assessment for examiners and SPs to meet and clarify their roles.
Examiners are provided station material online with calibrated video examples which they mark and receive immediate feedback from experienced markers. SPs receive their material in the same format. An online briefing session is offered prior to the assessment for examiners and SPs to meet and clarify their roles.
Live QA:
QA examiners monitor station performance remotely via livestream through a laptop strategically placed in each station and record their observations using an electronic QA marksheet.
QA examiners monitor station performance remotely via livestream through a laptop strategically placed in each station and record their observations using an electronic QA marksheet.
Results:
Marking is collected electronically using online examination software and results moderated using a custom designed spreadsheet template allowing timely release of results.
Marking is collected electronically using online examination software and results moderated using a custom designed spreadsheet template allowing timely release of results.
Evaluation: All examiners and SPs are invited to participate in an online survey, composed of quantitative (Likert scale) and qualitative (free text), questions to provide feedback on their experience of the QA process.
Results
We have collected several years of survey data and will present an analysis of the 2023 results, anticipating approximately 30 SPs, and 30 examiners to respond.
Descriptive statistics and thematic analysis of free text responses will be reported, related to perceived efficacy of examiner and SP calibration processes.
Discussion & Conclusions
Innovative use of online delivery allows QA processes to be conducted remotely in a way that is acceptable to participants and ensures comparable OSCE delivery across sites.
Take-home messages
Sharing this approach to remote QA may inspire innovation at other educational institutions.
References (maximum three)
1. Malau-Aduli B, Hays R, Van Der Vleuten C. Understanding Assessment in Medical Education Through Quality Assurance. 1 ed: McGraw Hill; 2021 Aug 27.