Mahmoud Suleiman¹
Karen Fung¹, Tara Leslie² and Li-Ann Kuan
1 Pharmacy Examining Board of Canada (PEBC)
2 University of Alberta

Candidates can be tempted to utilize technology to give them an unfair advantage in their test performance. Technology, however, offers innovative strategies to counter these threats, improving exam security and enhancing assessment processes. 

Through the lens of high stakes assessments, the audience will be taken through the experience of the Pharmacy Examining Board of Canada (PEBC) in piloting and implementing Automatic Item Generation (AIG) and Linear-on-the-Fly Testing (LOFT) for its Pharmacist Qualifying Examination Part I (MCQ), a component of the entry-to-practice competency assessment for pharmacy in Canada. These two distinct, yet related strategic initiatives, intertwine to underpin PEBC's focus to enhance the quality of its assessments, improve efficiency, and safeguard validity by ensuring the security of the assessment processes. 

AIG involves the development of models and the use of computer algorithms to generate a bank of thousands of multiple choice questions. LOFT is an exam construction method using artificial intelligence (AI) to automate assembly and delivery of exam forms affording each exam taker a parallel, yet unique form. Those interested in MCQ development, exam delivery, and item bank security will find the session valuable. The topic is highly applicable for those involved with large scale testing and setting strategic priorities for assessment organizations. The audience will learn about PEBC’s recent pilot of AIG and imminent launch of LOFT, sparking the exploration of applicability to their own organization. 

This interactive presentation will provide the audience with an introduction to, and demonstration of AIG and LOFT methodologies with a focus on three components: how the underlying technologies work and their benefits and limitations, the psychometric considerations, and the change management journey from inception to implementation. The challenges presented and solutions utilized are not unique to PEBC and the participants will be encouraged to share their thoughts, insights and experiences. Attendees will be able to examine if these innovations could enhance their own assessment processes and also explore strategies for implementation. 

Further Audience Engagement and Size:
The presenters will consider the feasibility of a hands-on activity to demonstrate how AIG works which may limit the desired attendance to sixty participants, otherwise a larger audience is welcomed. 

References (maximum three) 

Leslie, T., Gierl, M. (2023). Using Automatic Item Generation to Create Multiple-Choice Questions for Pharmacy Assessment. American Journal of Pharmaceutical Education. 100081, ISSN 0002-9459. https://doi.org/10.1016/j.ajpe.2023.100081.