Yvette Hlophe¹
Masikisiki Baphumelele², Werner Cordier³, Astrid Turner⁴, Vukosi Marivate⁵ and Sumaiya Adam⁶
1 University of Pretoria
2 Computer Science Department, Faculty of Engineering,University of Pretoria
3 Pharmacology Department, School of Medicine, Faculty of Health Sciences,University of Pretoria
4 School of Public Health, Faculty of Health Sciences,University of Pretoria
5 Computer Science Department, Faculty of Engineering, University of Pretoria
6 Department of Obstetrics and Gynaecology,School of Medicine, Faculty of Health Sciences,University of Pretoria

In designing the new medical curriculum, student attributes were selected that will facilitate a Day 1 doctor to excel. The Day 1 doctor emanating from the University of Pretoria should reflect and think critically. Literature has shown the benefits of using portfolios as a tool to promote longitudinal, continuous learning. An online portfolio will promote easy access to students to allow them to reflect at their convenience. 

Large Language Models (LLMs), such as ChatGPT-3 (Generative Pre-trained Transformer3), have been developed to understand language through the analysis of extensive text data. LLMs have demonstrated impressive performance across various text-related tasks. 

The aim of the study is to implement an online reflective portfolio to teach critical thinking and promote LLMs as a tool to interpret reflective data. 

Third year medical students were the study population. A rubric to assess critical thinking and reflection was provided to students and facilitators prior to reflections. A range of LLM’s based on their parameter capacities were selected. Our selection included Alpaca/Llama, Chat and GPT-3 . 

Preclinical exposure: Some students feared they would not be able to communicate adequately with patients. Post clinical exposure: students mentioned the importance of theoretical work taught in year 1-2 of the medical degree and the importance of languages skills, team work when solving case studies, the need to re-visit skills taught in the skills lab. Facilitators provided feedback to students. Positive correlations between Facilitator-score and both ChatGPT-3-score and Bard- in addition to being cost-effective and are valuable for reflective assessments score. 

The implementation of a functional, longitudinal online reflective portfolio will emphasize a student centered medical program. LLM’s have shown to be reliable and unbiased when 

compared to human raters. The implementation of the reflective portfolio in the medical curriculum will be substantiated by evidence based research. 

References (maximum three) 

1. AZER, S. A. 2008. Use of portfolios by medical students: significance of critical thinking. The Kaohsiung journal of medical sciences, 24, 361-366. 

2. White, A.D., Hocky, G.M., Gandhi, H.A., Ansari, M., Cox, S., Wellawatte, G.P., Sasmal, S., Yang, Z., Liu, K., Singh, Y. and Ccoa, W.J.P., 2023. Assessment of chemistry knowledge in large language models that generate code. Digital Discovery, 2(2), pp.368-376.