Guat Keng Yvonne Goh¹
Han Nee Gan¹, Elaine Ching Ching Tan¹, Shin Ying Thng¹ and Peck Har Ang¹
1 SingHealth, Changi General Hospital

Working in a fast-paced and busy emergency department presents numerous challenges for healthcare professionals. Clinical event debriefing (CED) has emerged as a valuable tool to improve patient safety and enhance healthcare professional competencies. CED is a process of reviewing and reflecting on clinical events, allowing healthcare teams to analyze their performance, identify errors, and foster a culture of continuous learning through reflective practices. 

Our team, experienced in simulation-based education and inter-professional training, sought to implement CED in our emergency department. We conducted a faculty development session introducing the STOP-5 model, tailored for our department's needs. To encourage CED adoption, we displayed posters with QR codes for data collection, focusing on major trauma and pediatric resuscitation cases. 

We hypothesize that CED serves as an effective tool for providing valuable feedback to healthcare professionals on their performance. By fostering a culture of continuous learning and improvement, debriefing sessions offer a powerful platform for competency development and continuous professional growth. Participants' experiences and perceptions of the debriefing process will be carefully studied to ascertain the extent to which they find it helpful in identifying areas for improvement and increasing their confidence in managing similar clinical events in the future. 

We also aim to collect comprehensive data on the impact of CED on work processes, patient safety, and the identification and mitigation of latent safety threats. By understanding these aspects, we can demonstrate the potential of CED in enhancing overall healthcare quality. 

CED holds promise as an effective formative assessment tool for healthcare professionals. Its implementation in emergency departments and other healthcare settings could lead to improved patient safety and overall care quality. Further exploration of CED's applicability across diverse healthcare environments and professional populations is warranted to unlock its full potential in optimizing healthcare outcomes. 

References (maximum three) 

1. Kessler DO, Cheng A, Mullan PC. Debriefing in the emergency department after clinical events: a practical guide. Ann Emerg Med. 2015 Jun;65(6):690-8. doi: 10.1016/j.annemergmed.2014.10.019. Epub 2014 Nov 15. PMID: 25455910 

2. Walker CA, McGregor L, Taylor C, Robinson S. STOP5: a hot debrief model for resuscitation cases in the emergency department. Clin Exp Emerg Med. 2020 Dec;7(4):259- 266. doi: 10.15441/ceem.19.086. Epub 2020 Dec 31. PMID: 33440103; PMCID: PMC7808839. 

3. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008 Nov;15(11):1010-6. doi: 10.1111/j.1553-2712.2008.00248.x. Epub 2008 Oct 20. PMID: 18945231. 

Thanatorn Puthiganon 

Background:
New medical knowledge has been expanding exponentially in recent years. Self reflection is a useful technique to promote continuous professional development. We apply self reflection in Social Pediatrics to improve knowledge and attitude in pediatric residents. Aims to measure residents’ attitudes concerning self reflection in continuous professional development and compare their knowledge before and after the curriculum. 

Summary of work:
A descriptive cross-sectional study that includes 18 second-year pediatric residents at Buddhachinaraj Hospital, Phitsanulok, Thailand. The course aimed at applying self reflection techniques (What did you observe, learn, and plan). At the end of Social Pediatrics curriculum, we used questionnaires to compare knowledge before and after the curriculum. We used median in descriptive analysis and inferential analysis was done with Wilcoxon signed rank test. 

Results:
The study included 18 residents. All of them thought that this curriculum could improve knowledge and holistic approach. 72% of respondents believed that the curriculum boosted their interest in Social Pediatrics. 61% of respondents strongly agree that self reflection could help improve their continuous professional development. Furthermore, 66% of respondents strongly agree that they have more confidence to use self reflection in their practice. In comparing knowledge, before the curriculum, the median of their knowledge was (Median(Q1,Q3)) = 3(1.2,5) and the outcome after the curriculum completion was (Median(Q1,Q3)) = 9(8.2,10). Their knowledge before and after the curriculum was statistically significant (p<0.001). 

Discussion and Conclusions:
Self reflection in Social Pediatrics could improve the knowledge of pediatric residents and after finishing the curriculum. In regard to the attitude towards self reflection in continuous professional development, pediatric residents have better attitude and more confidence to apply self reflection in their practice. 

Take-home Messages:
Everyday practice in self reflection could help build up capability, knowledge and create more confidence in order to improve the continuous professional development. 

References (maximum three) 

Yvette Hlophe¹
Masikisiki Baphumelele², Werner Cordier³, Astrid Turner⁴, Vukosi Marivate⁵ and Sumaiya Adam⁶
1 University of Pretoria
2 Computer Science Department, Faculty of Engineering,University of Pretoria
3 Pharmacology Department, School of Medicine, Faculty of Health Sciences,University of Pretoria
4 School of Public Health, Faculty of Health Sciences,University of Pretoria
5 Computer Science Department, Faculty of Engineering, University of Pretoria
6 Department of Obstetrics and Gynaecology,School of Medicine, Faculty of Health Sciences,University of Pretoria

In designing the new medical curriculum, student attributes were selected that will facilitate a Day 1 doctor to excel. The Day 1 doctor emanating from the University of Pretoria should reflect and think critically. Literature has shown the benefits of using portfolios as a tool to promote longitudinal, continuous learning. An online portfolio will promote easy access to students to allow them to reflect at their convenience. 

Large Language Models (LLMs), such as ChatGPT-3 (Generative Pre-trained Transformer3), have been developed to understand language through the analysis of extensive text data. LLMs have demonstrated impressive performance across various text-related tasks. 

The aim of the study is to implement an online reflective portfolio to teach critical thinking and promote LLMs as a tool to interpret reflective data. 

Third year medical students were the study population. A rubric to assess critical thinking and reflection was provided to students and facilitators prior to reflections. A range of LLM’s based on their parameter capacities were selected. Our selection included Alpaca/Llama, Chat and GPT-3 . 

Preclinical exposure: Some students feared they would not be able to communicate adequately with patients. Post clinical exposure: students mentioned the importance of theoretical work taught in year 1-2 of the medical degree and the importance of languages skills, team work when solving case studies, the need to re-visit skills taught in the skills lab. Facilitators provided feedback to students. Positive correlations between Facilitator-score and both ChatGPT-3-score and Bard- in addition to being cost-effective and are valuable for reflective assessments score. 

The implementation of a functional, longitudinal online reflective portfolio will emphasize a student centered medical program. LLM’s have shown to be reliable and unbiased when 

compared to human raters. The implementation of the reflective portfolio in the medical curriculum will be substantiated by evidence based research. 

References (maximum three) 

1. AZER, S. A. 2008. Use of portfolios by medical students: significance of critical thinking. The Kaohsiung journal of medical sciences, 24, 361-366. 

2. White, A.D., Hocky, G.M., Gandhi, H.A., Ansari, M., Cox, S., Wellawatte, G.P., Sasmal, S., Yang, Z., Liu, K., Singh, Y. and Ccoa, W.J.P., 2023. Assessment of chemistry knowledge in large language models that generate code. Digital Discovery, 2(2), pp.368-376. 

Jinelle Ramlackhansingh¹
1 Memorial University 

Background:
The students do reflection assignments as part of their formal curriculum assessment. Students are required to reflect on their learning experiences in the community critically. 

Summary:
This work is part of a more extensive critical ethnography examining the professional identity development of pre-clinical medical students. There were 31 volunteers in the first year, with attrition to 25 in the second year. Focus groups were held with students every six weeks. Participant observation of classes and governance meetings was done. Faculty and administrative staff interviews were completed. The theories of Bourdieu and Foucault were used in the analysis. 

Results:
The students spoke about unhelpful feedback received. The students did not find the feedback helpful. The students thought they would receive "correct" answers to their reflections. The students complained their feedback did not tell them what they should have written. 

Discussion:
Students rely on receiving feedback about reflection assignments to facilitate learning and correct mistakes. The students expected their reflections would be assessed as “right or wrong” answers to determine whether they passed the assignment. Reflection is a personal subjective experience with no wrong or correct answer. The dominant culture of medicine tends to reward busyness and productivity and not take time to pause and reflect. It would be seen as heterodoxic to use the time for reflecting on action and giving feedback, thus making change difficult. 

Conclusions:
The lack of actionable feedback was a lost opportunity for students to reflect and learn from their experiences. It has been suggested that reflection should not become a competency to be assessed; instead, it should be taught and understood as a way of seeing and being. 

Take-home: Guidance for students on the importance of self-reflection in learning is needed. Robust faculty development in providing effective feedback is required. 

References (maximum three) 

Lovy, A., Paskhover, B., & Trachtman, H. (2010). Teaching bioethics: The tale of a “soft” science in a hard world. Teaching and Learning in Medicine, 22(4), 319–322. https://doi.org/10.1080/10401334.2010.513196 

Ng, S. L., Kinsella, E. A., Friesen, F., & Hodges, B. (2015). Reclaiming a theoretical orientation to reflection in medical education research: A critical narrative review. Medical Education, 49(5), 461–475. https://doi.org/10.1111/medu.12680 

Westberg, J., & Jason, H. (2001). Fostering reflection and providing feedback helping others learn from experience. Springer Pub. 

Kath Hoare¹
1 AUT

Background
The tertiary education sector grapples with the complexity of student success. It is understood clinical health science students’ course engagement together with their learning motivation influences assessment achievement, completion, and programme progression. Exploring conditions for different students’ educational experiences can contribute to the integration of optimal approaches for engagement, assessment development, and curriculum design to promote student success. 

Summary of work
Through the lens of Ryan and Deci (2000, 2017) self-determination theory, this research aims to explore the educational experience for undergraduate health science students. A mixed design will be used with a primary focus on descriptive quantitative analysis for participants invited to complete a questionnaire identifying learner types and motivation, together with reviewing their course engagement, assessment achievement, and success. A secondary focus using descriptive qualitative response analysis will be applied to rated and optional open-ended questions. This combination can offer a more in-depth understanding of students’ educational experiences. 

Results
The preliminary research findings will be presented in late 2023, including differentiation between learner types, their motivation (intrinsic/extrinsic), engagement or non-engagement, assessment achievement, and success. 

Discussion
While there is a commonly held belief that students value the experience of being motivated, their learner type differences can present challenges to maintaining course engagement and meeting the assessment requirements. 

Conclusions
Understanding different learner types and motivations assists in the development of more effective strategies for curriculum design. By explicating the educational experience from the student's perspective courses can be aligned in a way that considers different learner types and motivations alongside the quality and type of assessments, educational experience, and success. 

Take-home messages/implications for further research

Key findings can be used to inform recommendations for the identification of different learners, support for engagement, and enhancement of assessments. The findings may be relevant for other educational contexts and learner groups. 

References (maximum three) 

Ryan, R. M. & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. 

Ryan, R. M. & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford Publishing. 

Tipaporn Thongmak¹
Pairoj Boonlucksiri¹, Araya Khaimook¹ and Wasin Kampeera¹
1 Hatyai Medical Education Center 

Background:
Objective Structured Clinical Examination (OSCE) is commonly used for assessing clinical competencies. Students are stressful, and their level of anxiety is high. Mock OSCE is a method for the students to improve their performance and relieve their anxiety. This study aims to evaluate whether Mock OSCE reduces the anxiety and increases the confidence of students for summative OSCE. 

Summary of work:
Mock OSCE was provided at the mid-block of the Rural medicine rotation of the 5th-year medical student. There were 15 stations including history taking, physical examination, and procedural skills. A cross-section survey was conducted in 38 students after they finished the summative OSCE. A 5-point-likert e-questionnaire comprised 4 parts: a) the help of Mock OSCE on summative OSCE preparation, b) the confidence in summative OSCE, c) the anxiety relieved by Mock OSCE, and d) the feedback on Mock OSCE. 

Results:
The overall satisfaction mean of Mock OSCE was 3.41+/- 0.71. The help mean of Mock OSCE on the summative OSCE preparation was 3.51+/- 0.5. The student received more knowledge and skills with the feedback mean of 3.62+/-0.70. However, the students felt more confident and relieved anxiety of summative OSCE with mean of 3.23+/- 0.71 and 3.05+/- 0.71, respectively. Through narrative feedback, some students suggested that they needed more time for giving feedbacks in each station, and they learnt the missing points and how to improve them. 

Discussion and conclusion:
Even though, Mock OSCE is a formative assessment for preparing the students before the summative OSCE. Some students still had anxiety and diffidence. The adequate time for giving feedbacks in each station and other formative OSCE such as Mini- CEX and DOPS would be helpful for the summative examination. 

Take home messages:
Mock OSCE is a method for students to practice, that be familiar and have confidence before summative examination. 

References (maximum three) 

Arafat Mulla
Prajwal Khairnar¹, Simon Fleming², Philip Webb¹ and Rowland Bright-Thomas³
1 Northern Care Alliance
2 Royal North Shore Hospital
3 Manchester University NHS Foundation Trust

Deanery-led teaching plays a crucial role in the development of Higher Specialty Trainees in the UK. During the COVID-19 pandemic, the traditionally face-to-face educational program transitioned to an online format. Feedback forms were also adapted for this new virtual context. This three-year study examined the feedback provided by Trainees during monthly regional deanery teachings across multiple medical departments. 

Utilising "rating" and "impact" numeric values, the study categorised the feedback based on the gender of both trainees and speakers. These rating scales range from 1 to 5, where 1 indicates poor performance and 5 represents excellent delivery. An mean difference of 0.22 to 0.35 points in this context is statistically and educationally significant. 

Statistical analysis revealed gender disparities. Female speakers received on lower ratings from male trainees by 0.22 points, compared to their female counterparts. Conversely, male speakers received higher ratings from male trainees, with a mean difference of 0.24 points. In terms of impact—a measure reflecting the perceived utility of the session—female speakers scored mean 0.4 points lower from male trainees, whereas male speakers scored an 0.35 points higher. 

While these findings may be influenced by confounding factors, they highlight an inherent gender bias in the feedback mechanisms examined. Tools, such as Likert scales, are known to have their limitations. While we updated teaching sessions to meet challenges, an opportunity was missed to update our feedback mechanisms. 

In summary, our study highlights gender-based disparities in educational feedback and raises questions about the adaptability and fairness of traditional feedback tools in an increasingly virtual academic environment. The data emphasise the urgent need for a comprehensive review of existing feedback mechanisms, ensuring they are suited for our rapidly evolving educational methods. 

Mandy Liu¹
Tomasz Cecot², Jian Yang1, Chia Di Lee², Guilherme Fonseca² and Michael Manio²
1 School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong
2 The University of Hong Kong

[Background]
Formative assessments with high engagement can improve students' learning experience in dissection classes [1]. With increasing demand for anatomy education and growing class sizes, it is necessary to adopt innovative teaching methods to ensure an effective learning experience for students. One approach is using near-peer teachers (NPTs), who have a similar level of training as the students they instruct. This form of teaching has been shown to have positive effects on both students and NPTs [2].

[Summary of Work]
 In this study, we explored the implementation of NPTs to conduct formative assessments in abdomen dissection classes using checklists. A quasi-experimental study was carried out. Teachers and NPTs were randomly assigned to either the experimental group (with checklists) or the control group (without checklists). Data on student performance, NPTs’ competence in formative assessment, and overall satisfaction were collected.


[Preliminary results]
Both students and NPTs expressed positive feedback on the checklist in facilitating effective formative assessment. NPTs using checklists reported higher levels of confidence and satisfaction in their teaching compared to the control group. Post-dissection quiz of four groups will be further analysed.

[Conclusion and discussion]
This study highlights potential benefits of using checklists in near-peer teaching for large cohort dissection classes. NPTs can effectively guide students through formative assessments using the checklist, leading to effective learning experience and overall satisfaction. However, one challenge was the time required for NPTs to familiarize themselves with the checklist and incorporate it effectively into teaching.


[Implications for Further Research]
It would be beneficial to investigate the potential technology-enhanced tools for supporting the implementation of checklists as formative assessment tools in medical education. Future research could also explore the use of checklists in other areas of medical education, as well as investigate long-term impacts of using checklist- based tools on student learning and NPTs’ development. 

References (maximum three) 

1. Evans, D.J.R., Zeun, P. and Stanier, R.A. (2014). Motivating student learning using a formative assessment journey. J. Anat., 224: 296-303. 

2. McDaniel, K. G., Brown, T., Radford, C. C., McDermott, C. H., van Houten, T., Katz, M. E. and Hildebrandt, S. (2021). Anatomy as a model environment for acquiring professional 

competencies in medicine: Experiences at Harvard Medical School. Anatomical sciences education, 14(2), 241-251.