Debra Sibbald¹
Andrea Sweezey¹
1 Leslie Dan Faculty of Pharmacy, University of Toronto 

Online admissions examinations, inherently vulnerable to academic dishonesty, are at heightened risk due to rapid emergence of generative artificial intelligence (AI) tools such as Chat GPT, since November 2022. We explored the impact of strategies to reduce or address AI threats prior to and after the 2023 spring admission cycle at the Faculty of Pharmacy, UT. 

Key threats to online exam vulnerabilities were identified through appraisal of published evidence. We instituted deterrent stratagems: increasing question and answer length, shortening timing, disabling copy functions and personalizing questions for individual contexts. Academic misconduct penalties were emphasized. In a field test, selected participants were asked to circumvent possible detection and use AI. Assessors were trained in awareness and recognition using examples of AI responses and practiced rating authentic vs AI samples. A red flag system was employed to alert for suspicious examples. Admission test results were analyzed for detection and red flags comments. 

On analysis, it was not possible for assessors to detect AI responses. Tactics and approaches aimed at altering question elements were considered ineffective. 

Published threat themes include detection, academic and professional credibility, implications for knowledge work and application, ethics and digital equity. Potentially effective strategies include actionable misconduct penalties; revision of assessment items to increase personalized elements and discourage prompt writing; and focused assessor training with AI samples. 

The defensibility of high stakes admissions tests is increasingly threatened by rapidly evolving technologies which overcome deterrent approaches. Challenges to authenticity include validity, reliability, equivalence, impact on learning, feasibility, acceptability and sustainability. 

AI-generated responses challenge the fairness, transparency, and objectivity of virtual high- stakes online tests impacting applicants, assessors, assessments, and institutions. Vigilant scrutiny for threats and realistic timely responsive strategies are imperative. Revisit possibilities of one-on-one live in person or online interviews, resources permitting. 

References (maximum three) 

Dwivedi YK, Kshetri N, Hughes L, Slade EL, Jeyaraj A, Kar AK, et al. “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management. 2023;71. 

Giannos P, Delardas O. Performance of ChatGPT on UK Standardized Admission Tests: Insights From the BMAT, TMUA, LNAT, and TSA Examinations. JMIR Medical Education. 2023;9(1):e47737 

Abd-Elaal E-S, Gamage SH, Mills JE. Assisting academics to identify computer generated writing. European Journal of Engineering Education. 2022;47(5):725-45.