piyaporn sirijanchune¹
Panomkorn Lakham²
1 medical education center, Chiangrai Prachanukroh hospital 

Clinical skills remain fundamental to medicine and form a core component of the professional identity of medical students. Artificial intelligence (AI) is the simulation of human intelligence rapidly advancing technology to revolutionize medical education. AI provides personalized assistance with the assessment. This study explored the value of AI programs to assess the clinical skills of medical students and compared them to traditional assessment methods. 

A total of 37 fifth-year medical students enrolled in the study. A cross-sectional study of medical students from June 2022-2023. During the clinical skills assessment, the medical students interact with the examination station for diagnosis and treatment. There are 6 stations of the formative evaluation of clinical internal medicine categorized into 3 domains; 1. history taking and physical examination, 2. diagnosis and treatment, 3. advice and patient education. The AI program uses processing to score the medical student’s diagnosis in real-time with immediate insight into their performance. AI to score student performance on in-person components of clinical skills assessment compared to the traditional assessment methods. 

The AI group had a higher score than the traditional group, the mean score was 85.27+4.28 and 81.25+5.12 out of 100 points, respectively. There was no significant difference in mean score with P-value of 0.07. The Interrater reliability between AI and traditional assessment was 46.36%, Kappa 0.443. The stress in the AI group was 43% compared to the traditional group with 80%. The AI assessment agreed with traditional assessment tools in 80% of cases, with 92% sensitivity and 90% positive predictive value. 

AI can be used to assess clinical skills with good quality and accuracy as traditional methods, demonstrated favorable performance with consistent results and immediate feedback to the students. The AI had the advantage of decreasing the error of the assessment which improved the reliability and validity of the assessment. 

References (maximum three) 

1. Daniel, Michelle & Rencic, Joseph & Durning, Steven & Holmboe, Eric & Santen, Sally & Lang, Valerie & Ratcliffe, Temple & Gordon, David & Heist, Brian & Lubarsky, Stuart & Estrada, Carlos & Ballard, Tiffany & Artino, Anthony & Da Silva, Ana & Cleary, Tim & Stojan, Jennifer & Gruppen, Larry. (2019). 

2. Ahmed, Ishtiaq & Ishtiaq, Sundas. (2020). ASSESSMENT METHODS IN MEDICAL EDUCATION: A REVIEW 1 2 ISHTIAQ AHMED, SUNDAS ISHTIAQ. 6. 95-102. Clinical Reasoning Assessment Methods: A Scoping Review and Practical Guidance. Academic Medicine. 1. 10.1097/ACM.0000000000002618. 

3. González-Calatayud V, Prendes-Espinosa P, Roig-Vila R. Artificial Intelligence for Student Assessment: A Systematic Review. Applied Sciences. 2021; 11(12):5467. https://doi.org/10.3390/app11125467