Presentation Description
Carmel Tepper1
1 Bond University
1 Bond University
Background:
In 2022, Bond transitioned high-stakes written clinical examinations to Progress Tests; longitudinal knowledge examinations of ‘exit level’ content, required to be known by interns. As use of online resources is now considered common practice - and sometimes best practice - in the clinical workplace, the Progress Tests were conducted Open Book. January 2023 brought the rise of Generative Artificial Intelligence (Gen-AI) technology with potential to disrupt assessments, particularly knowledge based tests with open internet access.
Summary of work
We opted to embrace this technology and provide students with clear permission to access Gen- AI during Open Book written exams. We then advised students on effective use of Gen-AI for clinical exam questions and warned them of current limitations of the tool. I will share with you steps taken to secure the integrity of our Open Book examinations with Gen-AI.
Results
We used a variety of methods during construction of our exam questions to maximise requirements for students’ to activate their clinical reasoning prior to use of any online tools. The aim was to ensure that our written exams remained fit for purpose in determining student clinical knowledge competency for progression. Exam performance data of two Open Book Progress Tests will be shared that indicates we need not fear Gen-AI but can work with our medical students to teach them how to use these tools effectively to best prepare them for authentic clinical work practices.
Conclusions
Early data suggests that use of Gen-AI in Open Book Progress Tests provides minimal advantage to medical students in conducting Open Book Progress Tests. Gen-AI continues to evolve and grow in the online space. We can best prepare medical students for their authentic future professional activity by educating them on effective use of Gen-AI to support their core medical knowledge and clinical reasoning.
References (maximum three)
1. Loh E. BMJ Leader Published Online First: Accessed 2nd August 2023. doi:10.1136/leader- 2023-000797