Yogesh S¹
Kavitha M²
1 ASSISTANT PROFESSOR, INSTITUTE OF INTERNAL MEDICINE, MADRAS MEDICAL COLLEGE, CHENNAI, TAMILNADU, INDIA
2 VICE PRINCIPAL, MADRAS MEDICAL COLLEGE, CHENNAI, TAMIL NADU, INDIA

1. Background:
Excellence in medical education relies on proficient skills assessment and development. The transition from a traditional curriculum of two decades (1997) to the reformed competency-based medical education (CBME) in 2019 brought significant upheaval [1]. Evaluating practical skills of medical students using conventional methods has proven cumbersome and complex, especially with large cohorts. The demand for consistent assessment, prompt feedback and comprehensive evaluations mirroring real-life situations has driven innovation. This research tackles these issues by incorporating SimCapture technology to enhance the assessment. 

2. Summary of Work: 
A study was conducted at Madras Medical College, Chennai, India, involving Trained Faculties of different departments, by integrating SimCapture technology for assessing 250 pre-final year MBBS students, using Peyton's 4 Step Approach, focusing on key domains: student knowledge, technical proficiency, critical thinking, communication skills, and safe practice. The assessment process included online pre-tests, structured checklists, peer-to-peer learning with digital checklists on SimCapture, self-recorded videos, online post-tests, and virtual OSCE evaluation. 

3. Results: 
The integration of SimCapture streamlined data collection and provided a comprehensive analysis of student progress. Individual competencies and cohort-wide performance were revealed through a 360-degree analysis. A digital repository was created, enhancing accessibility and supporting longitudinal analysis. The interactive learning environment improved engagement and empowered students in their skill development journey, promoting retention, application, and overall competence. 

4,5. Discussion/Conclusions: 
The emotional component ‘feel’ plays a pivotal role in changing the attitude and behavior, making transition process smooth. The adoption of SimCapture redefined assessment practices and contributed to cultivating proficient, confident, and competent medical professionals. The study's findings emphasize the potential of technology in shaping medical education and enhancing healthcare provision. 

6. Take-home Messages:
This study establishes a new standard for future endeavors in medical education by encouraging educators, institutions, and stakeholders to leverage technology's potential for improved assessment and skill development. 

1. Medical Council of India.Competency Based Undergraduate Curriculum.2018. Medical Council of India. Available from: https://www.mciindia.org/CMS/wp- content/uploads/2020/01/UG-Curriculum.pdf [Google Scholar] 

KWANG CHIEN YEE¹
1 University of Tasmania

With the widespread use of artificial intelligence, especially generative AI using large language model, such as ChatGPT, many assessment tasks are under spotlight. The use of essay and/or open book examinations become an issue as current plagiarism detection software cannot reliably detect the use of AI in assignment completion. 

The best assessment that will reliably as well as provide validity to practical skill assessment in medical education is via workplace, competency and/or skill assessments. These assessments take time, especially if this is being organised in a large group. Assessment by workplace supervisors, however face the issue of assessment standard, and the difficulties in monitoring student progress. 

Blockchain is a distributed ledger, which is being used in many industry. The use of blockchain in medical education assessment has rarely been considered. With the concept of workplace/competency/skill assessment, as well as the recent shift towards industry-acredited micro-credentialling process for higher education, blockchain technology could be a perfect fit to support continual-individualised assessment, while maintaining monitoring, controlled and standards 

This presentation will present the theory of blockchain in medical education and a model of how blockchain-based medical education assessment will work. One of the benefits of blockchain, is the use of smart-contract, which could provide feedback and/or direct further study. Furthermore, blockchain supported assessment will allow individualised, student- centred learning and assessment process. This presentation will discuss our framework and our model in using blockchain in medical education assessment how it will benefit the current teaching and learning environment. 

References (maximum three) 

Wong MC, Yee KC, Nøhr C.
 Socio-Technical Considerations for the Use of Blockchain Technology in Healthcare. Stud Health Technol Inform. 2018;247:636-640. 

Mahmoud Suleiman¹
Karen Fung¹, Tara Leslie² and Li-Ann Kuan
1 Pharmacy Examining Board of Canada (PEBC)
2 University of Alberta

Candidates can be tempted to utilize technology to give them an unfair advantage in their test performance. Technology, however, offers innovative strategies to counter these threats, improving exam security and enhancing assessment processes. 

Through the lens of high stakes assessments, the audience will be taken through the experience of the Pharmacy Examining Board of Canada (PEBC) in piloting and implementing Automatic Item Generation (AIG) and Linear-on-the-Fly Testing (LOFT) for its Pharmacist Qualifying Examination Part I (MCQ), a component of the entry-to-practice competency assessment for pharmacy in Canada. These two distinct, yet related strategic initiatives, intertwine to underpin PEBC's focus to enhance the quality of its assessments, improve efficiency, and safeguard validity by ensuring the security of the assessment processes. 

AIG involves the development of models and the use of computer algorithms to generate a bank of thousands of multiple choice questions. LOFT is an exam construction method using artificial intelligence (AI) to automate assembly and delivery of exam forms affording each exam taker a parallel, yet unique form. Those interested in MCQ development, exam delivery, and item bank security will find the session valuable. The topic is highly applicable for those involved with large scale testing and setting strategic priorities for assessment organizations. The audience will learn about PEBC’s recent pilot of AIG and imminent launch of LOFT, sparking the exploration of applicability to their own organization. 

This interactive presentation will provide the audience with an introduction to, and demonstration of AIG and LOFT methodologies with a focus on three components: how the underlying technologies work and their benefits and limitations, the psychometric considerations, and the change management journey from inception to implementation. The challenges presented and solutions utilized are not unique to PEBC and the participants will be encouraged to share their thoughts, insights and experiences. Attendees will be able to examine if these innovations could enhance their own assessment processes and also explore strategies for implementation. 

Further Audience Engagement and Size:
The presenters will consider the feasibility of a hands-on activity to demonstrate how AIG works which may limit the desired attendance to sixty participants, otherwise a larger audience is welcomed. 

References (maximum three) 

Leslie, T., Gierl, M. (2023). Using Automatic Item Generation to Create Multiple-Choice Questions for Pharmacy Assessment. American Journal of Pharmaceutical Education. 100081, ISSN 0002-9459. https://doi.org/10.1016/j.ajpe.2023.100081. 

Mahalakshmi Venkatesan Natrajan¹
Rajkumar Elanjeran²
1 AVMC, VMRF DU, India
2 AVMC, VMRF DU 

Creating real-time performance capture and assessment system for large number of medical students to master procedural skills in low-resource setting poses unique challenges and requires technology-driven, targeted approach. We describe our experience using Simcapture using mobile phones to train and assess 64 medical interns in procedural skills during the ‘Boot camp’ conducted at the beginning of internship at a medical college in India. 

Methods:
We identified 14 Essential Procedural Skills for training medical interns using manikins and task trainers and developed checklists. Real-Time Performance Capture of learners through live Video Recording using handheld or mounted Smartphones was done. 

Multiple reviews of the recordings for formative learning was done using guided checklists, by self, peers and facilitators. Trials and deliberate practice was encouraged. Based on real- time analysis, detailed immediate feedback on errors, deviations from proper technique, or safety concerns, in each attempt was offered ensuring tailored, adaptive learning. Assessment and certification was completed. 

To ensure successful implementation, 

a. Capacity Building and Train-the-Trainers session to facilitate the technology's transfer prior to the actual program was done 

b. Support on technical/ instructional issues 

c. Pilot Testing was carried to identify usability challenges, refine checklists and adaptability to local conditions. 

o Partnerships with the technology provider ensured resource-sharing and planning.

Results:

The data was analysed for students' achievement. Students requiring additional training and error-prone steps/ stations were identified for remediation. Concordance of marking was found between self and peer assessment with some variations with facilitator marking. The feedback confirmed the usefulness and viability of the process and ease of use of technology. 

Summary:
Success of our technology-driven approach hinges on its adaptability to unique challenges of low-resource setting, including device availability, connectivity limitations, and local educational contexts. Regular feedback from educators and learners is crucial for continuous improvement. 

References (maximum three) 

1. Sarah Yardley, Pim W. Teunissen & Tim Dornan (2012) Experiential learning: AMEE Guide No. 63, Medical Teacher, 34:2, e102-e115, DOI: 10.3109/0142159X.2012.650741 

2. Motola I, Devine LA, Chung HS, Sullivan JE, Issenberg SB. Simulation in healthcare education: a best evidence practical guide. AMEE Guide No. 82. Med Teach. 2013 Oct;35(10):e1511-30. doi: 10.3109/0142159X.2013.818632. Epub 2013 Aug 13. PMID: 23941678. 

3. Guze PA. Using Technology to Meet the Challenges of Medical Education. Trans Am Clin Climatol Assoc. 2015;126:260-70. PMID: 26330687; PMCID: PMC4530721.