Hossein Karimi Moonaghi¹
Neda Negahban¹, Horak Poorzand¹ and Mohammad Khajedaluee¹
1 Mashhad University of Medical Sciences 

Background:
By examining the comprehensive system for evaluating the academic progress of general medical students, often the objectives of the cognitive domain and the form of cumulative evaluation were used, and the tests were not used much for feedback to the students. The aim of the study is to develop a model that fits the levels of Miller's evaluation pyramid in formative-cumulative forms. 

Methods: 
The search was started in Iranian and international databases, journals, curriculum of prestigious universities in the world. To find out about the latest events in the field of assessment, AMEE international virtual conferences in August 2021 and the summary booklet of medical education articles of national conference of medical education were reviewed. Data analysis was done by Beredy's adaptive model. Finally, a model was developed according to Miller's evaluation pyramid. Its validation was done in the focus group meeting in two ways, in person at Mashhad Medical School and virtual. 

Results:
The approach of assessment is towards formative assessment format and improvement of traditional methods along with modern methods, which was clearly observed in the study of the curriculum of Harvard-Stanford University in America and Oxford University in England. Integrating the results with Miller's evaluation levels, and the formative and cumulative evaluation format, led to the formulation of a model with the most favorable opinions of experts. In addition, the majority of opinions and suggestions of experts were related to the change in the way of executive policies of universities and providing a context for the emergence of new idea. 

Conclusion:
A model including measurement methods according to the levels of Miller's evaluation pyramid was developed in formative-cumulative. It is suggested that the model be reviewed by the relevant experts and notified by taking into account the implementation conditions for the correct evaluation process. 

References (maximum three) 

Eugene Teoh 

Background:
The term reality shock has fittingly been used to express the novice nurses’ (NN) reaction upon recognising the apparent mismatch between their clinical experience, values and ideas imbibed when they first transit from a student to a NN. 

Methods:
This study aimed at examining the effectiveness of the new hospital nursing orientation programme for NNs in increasing job satisfaction and reducing work-related stress in their transition to practice journey. A validated questionnaire was used to survey the NNs. 

Results:
A total of 151 nurses were recruited into the study, comprises 86 for the control and 65 nurses for the intervention group. 

The new nursing orientation programme has showed to significantly increase the autonomy of the nurses from 3rd month to 6th month [mean (SD) 3rd month vs 6th month: 3.66 (0.52) vs 3.86 (0.47); p = 0.019]. 

In terms of reducing stress, it was observed that lower stress level were reported at 3rd month, for all aspects including conflict with physicians, uncertainty, inadequate preparation, conflict with other nurses, lack of support, workload, and also death and dying. 

Significant decrease in stress was observed from week 1 to 3rd month in terms of inadequate preparation [mean diff (95%CI): 0.238 (0.044, 0.432); p = 0.017], workload [mean diff (95%CI): 0.285 (0.079, 0.490); p = 0.007], and also death and dying [mean diff (95%CI): 0.187 (0.069, 0.305); p = 0.002]. 

Discussion & Conclusion:
The old hospital nursing orientation programme does not seem to be effective in promoting job satisfaction and reduce stress for nursing staff, by indicating no significant difference in all aspects except conflict with physicians. The new hospital nursing orientation programme has reported increased autonomy and decreased stress level of NNs. 

References (maximum three) 

Aalia Zaib¹
1 Gandhara University Peshawar, Pakistan. 

This study aimed to provide insights into the attitudes of medical students towards daily reflection writing and to explore the potential benefits of this practice for personal and professional development. The findings of this study could be used to inform medical education policy and practice, emphasizing the need to integrate daily reflection writing into medical curriculums. 

A total of 320 medical students (1st, 2nd, and 3rd year MBBS students) from Kabir Medical College Peshawar were included in the study from January 2023 till March 2023. Ethical approval was obtained from the Ethical Committee of the Gandhara University. Two pre-tested questionnaires using a Likert scale were modified to suit the study's objectives. The questionnaire was self-administered, and content validity was ensured before conducting the research. Descriptive and inferential analysis were applied using SPSS version 26.0. Demographic information such as age, gender, and academic major were collected from participants. 

The study found that most medical students had a positive perception towards daily reflection writing. Almost half of the medical students (47.8%) found that writing in their logbook improved their information retention, while 46.9% noticed positive changes in their learning approach. Additionally, nearly half of the participants (49.1%) reported being satisfied with the experience of daily reflection writing. These results highlight the potential benefits of integrating daily reflection writing in logbooks for medical students. 

The findings suggest that medical students have a positive attitude towards daily reflection writing and recognize its potential benefits for personal and professional development. The study highlights the need to integrate daily reflection writing into medical education curricula to foster critical thinking and self-awareness among medical students. 

Keywords
reflective practice, reflective writing, reflective practice in medical education. 

References (maximum three) 

1. Sandars J. The use of reflection in medical education: AMEE Guide No. 44. Med Teach [Internet]. 2009 [cited 2023 Mar 13];31(8):685–95. Available from: https://pubmed.ncbi.nlm.nih.gov/19811204/ 

2. Bernard AW, Gorgas D, Greenberger S, Jacques A, Khandelwal S. The use of reflection in emergency medicine education. Academic Emergency Medicine. 2012 Aug;19(8):978–82. 

3. Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract [Internet]. 2009 Sep [cited 2023 Mar 13];14(4):595–621. Available from: https://pubmed.ncbi.nlm.nih.gov/18034364/ 

Toby Jackman 

Traditional undergraduate medical education prepares students inadequately for their Foundation Year One role on clinical ward rounds (1). Final-year student confidence has been undermined by the COVID-19 pandemic and reduced face-to-face patient encounters (2). Ward round simulation may help improve student confidence in non-technical skills (3). 

We conducted paired 7-point Likert-like questionnaires of final-year medical students before and after a simulated ward round. Student responses were analysed using paired t-Test to determine whether the simulation had evoked a change in confidence in non-technical skills. 

There was statistically significant improvement in confidence of students in almost all non- technical domains. Students likely develop confidence in information gathering early in undergraduate education; followership is a relatively new field of study and medical students receive less training in, and opportunity to practice, these skills. Simulation remains a key tool in engendering confidence in nontechnical skills in medical students. 

References (maximum three) 

1. Nikendei C, Kraus B, Schrauth M, Briem S, Jünger J. Ward rounds: how prepared are future doctors?. Medical teacher. 2008 Jan 1;30(1):88-91. 

2. Choi B, Jegatheeswaran L, Minocha A, Alhilani M, Nakhoul M, Mutengesa E. The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey. BMC medical education. 2020 Dec;20(1):1-1. 

3. Harvey R, Mellanby E, Dearden E, Medjoub K, Edgar S. Developing non–technical ward‐round skills. The clinical teacher. 2015 Oct;12(5):336-40. 

Methas Arunnart 

Abstract 

References (maximum three) 

Atsushi Tanikawa¹
Ryo Sagisaka², Koshi Nakagawa³ and Shoji Yokobori⁴
1 Department of Emergency and Critical Care Medicine, Tohoku University Hospital
2 Department of Integrated Science and Engineering for Sustainable Societies, Faculty of Science and Engineering, Chuo University
3 Department of Emergency Medical System, Graduate School, Kokushikan University
4 Department of Emergency and Critical Care Medicine, Graduate School of Medicine, Nippon Medical School

Background:
Virtual reality (VR) has potential for effective learning in medical education. There are increasing reports of simulation and procedures education using VR. Compared to conventional two-dimensional (2D) videos, however, the effects of VR in medical education are not clear. This study aims to compare VR and 2D videos on distance learning of live-action Focused Assessment with Sonography for Trauma (FAST) in initial trauma care and to examine psychological learning effects such as self-efficacy and comprehension. 

Methods:
We conducted a randomized controlled trial using distance learning. Eligible participants for inclusion were fourth- to sixth-year medical students and first- and second-year residents in five medical schools and university hospitals. We conducted stratified randomization by institution and participants were assigned to 2D and VR groups. Participants attended approximately 30 minutes of remote lectures on initial trauma care and watched live- action FAST practices in the emergency room in 2D or VR. Primary outcomes were self- efficacy, intrinsic value and emotional engagement to assess learning effectiveness. Multiple regression analysis was used to evaluate the association between VR use and outcomes. 

Results:
Sixty-four participants were eligible for analysis (2D, n = 33; VR, n = 31). There were no significant differences in participant characteristics; however, the median pre-test score for measuring medical knowledge differed by two points (2D, 20.0; VR, 18.0). In multiple regression analysis to evaluate the association between VR and outcomes, all outcomes showed no significant association (B, -0.62, 0.44, 0.98; 95% CI, -5.62 to 4.38, -2.72 to 3.59, -2.12, 4.08; p-value, 0.80, 0.78, 0.53, self-efficacy, endogenous value and emotional engagement, respectively). 

Conclusion:
We evaluated VR use and psychological learning effects in distance learning of FAST in initial trauma care. In this study, using VR was not significantly associated with learning effectiveness. 

MOHAMMED ALGABGAB¹
Michal Tombs²
1 A PhD student in Medical Education, School of Medicine, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS
2 Reader in Medical Education, School of Medicine, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff 

Clinical feedback is a critical component of formative assessment in EMS education.However,research suggests that students and faculty face a variety of challenges and highlighted various training needs of clinical supervisors.The aim of this study was to ascertain the way in which these training needs can best be addressed.The study aimed to identify and propose a strategy to support EMS institutions in Saudi Arabia in developing a supportive framework for clinical supervisors that promotes effective delivery of clinical feedback.This study employed a merged-method approach using a modified Delphi methodology across three rounds.The participants included an expert panel of deans,directors,and clinical supervisors across EMS education in Saudi Arabia,with a prerequisite of experience in EMS clinical supervision or faculty support roles for clinical feedback.In the first round,interviews were conducted that informed the development of a Delphi questionnaire.The second and third rounds used an online survey,that consisted of 73items divided into three categories:clinical feedback training needs and support,ongoing support and quality assurance,and clinical feedback guidelines and tools.Twelve EMS experts participated in the first round,39 in the second,and38 completed the third round.A consensus threshold of 75% was established, with items reaching this agreement deemed accepted or rejected. 

Consensus was reached on 66 outof 88 items across the three categories:1)Clinical Feedback Training(44 out of 59 items);2)Clinical Feedback Quality Assurance(11 out of 13 items);and3)Guidelines and Tools for Improving Clinical Feedback(11 out of 16 items). 

Participants agreed on how prior clinical feedback challenges can be best addressed through training,support,and guidelines,emphasising the importance of formal educational training and workshops,mentorship,peer observation,and ongoing support for clinical supervisors.Interestingly,clinical supervisors expressed a preference for a mixture of face to face and online methods of training.Simulated scenarios are essential for realistic,risk-free clinical feedback training.The need for collaboration among all Saudi EMS institutions for quality assurance was highlighted,emphasising shared responsibility in ensuring clinical feedback quality. 

References (maximum three) 

No references 

Sonia Ijaz Haider¹
Qamar Riaz² and Muhammad Furqan Bari¹
1 Dow University of Health Sciences
2 Aga Khan University

Presently in Pakistan there is no formal and structured mentoring system for postgraduate medical trainees. Therefore, the study aimed to assess, develop, implement and evaluate the impact of a one-year peer mentoring program for postgraduate medical residents. 

This was a mixed methods study design. It was conducted in three phases. In the first phase, a survey was administered to junior residents in Medicine, Paediatrics, Pathology and Radiology to assess their academic needs at a private medical university. Final year residents, who served peer mentors were provided a one-day mentoring workshop, including details of the identified needs assessment. The second phase was followed by one-year intervention in which mentors- mentees met as per mutual feasibility, either weekly or fortnightly. In the third phase, a structured survey and focus group discussions were conducted to inquire the effectiveness of mentoring. 

A total of 33 out of 52 (63%) year 1 residents completed needs assessment survey. Four essential areas identified for intervention were research (63%), memory (59%), reading (55%), time management (53%). Findings of the one-year intervention indicated that residents reported exceling in areas of research, memory and reading, however time management needed improvement. Focus group discussions emphasized mentoring as mandatory as it supported in their personal/professional development, however time management remained a challenge for both the mentors and mentee. Several recommendations were shared regarding the format of the mentoring programme. 

Peer mentoring programme is needed to facilitate academic progress, social acclimatization, psychological well-being and personal growth of junior residents. 

Senior residents should be trained as mentors of juniors, and a semi-structured formal and informal mentoring can be implemented addressing a wide range of assessment needs across all residency programmes. 

Implication for further practice
To implement the mentorship across all the residency programme at the University. 

References (maximum three) 

Venktaramana V, Ong YT, Yeo JW, Pisupati A, Krishna LKR. Understanding mentoring relationships between mentees, peer and senior mentors. BMC Med Educ. 2023:31;23(1):76 

Ong Y, Quek C, Pisupati A, Loh E, Venktaramana V, Chiam M, et al. Mentoring future mentors in undergraduate medical education. PLoS One. 2022;17(9): e0273358 

Krishna LKR, Tan LHE, Ong YT, Tay KT, Hee JM, Chiam M, et al. Enhancing mentoring in palliative care: an evidence based mentoring framework. J Med Educ Curric Dev. 2020;7:2382120520957649 

Chaya Prasad¹
Fanglong Dong¹
1 Western University of Health Sciences 

Medical students acquire knowledge, skills, competency through curriculum delivered by experienced practicing clinical faculty. Research has shown that a stress-free environment with free exchange of ideas between tutor and tutees results in a higher level of engagement, increased clinical knowledge, competency, communication skills. This can be achieved by peer tutoring (PT) if the tutors have been well trained by the faculty. PT can be effective in sharing of curriculum related clinical knowledge and skills. We wished to evaluate the efficacy of PT in sharing of non-curriculum soft topics, such as Lifestyle Medicine (LM), with our pre-clinical students. 

LM longitudinal track in our 2 campuses enrolls 35 of students each year. In this pilot, Year 2 LM students were tutors, and their non LM peers (year 2), and incoming year 1 students were tutees. The tutors were trained by LM director, wherein they developed a relevant clinical case presentation. This was followed by an in-depth discussion of the topic with clinical reference. Tutees were asked to complete a voluntary survey, which assessed case presentations, skills of the tutors, level of engagement of tutees and overall experience. 

208/367 year 1 students and 141/377 year 2 students completed voluntary survey. Host campus tutees had highest percentage of enjoying chatting with peers(97.5%), with a statistically significant differences between 2 campuses (p=0.0289), likely due to differences in LM curriculum. 95% at both campuses had positive responses to several questions pertaining to PT experience. Host campus year 2 students had highest percentage on topic of usefulness of LM (88.1%, p=0.0325), satisfaction with presenters (98.2%,p=0.0359), “train the trainer” experience (88.1%, p=0.0116),interactive nature of large group session (88.1%, p=0.0002),enjoyable overall experience (90.8%, p=0.028), new overall experience (85.3%, p=0.0455), and usefulness of provided questions (89%, p=0.0068). Our study demonstrates that PT can be an effective teaching and learning modality. 

References (maximum three) 

1. Lessans S, Pasarica M, Kay D. Do Medical Students Appreciate Lifestyle Medicine and Are They Confident in Their Skills? PRiMER. 2022 Jun 16;6:15. doi: 10.22454/PRiMER.2022.968910. PMID: 35801195; PMCID: PMC9256293. 

2. Shenoy A, Petersen KH. Peer Tutoring in Preclinical Medical Education: A Review of the Literature. Med Sci Educ. 2019 Dec 23;30(1):537-544. doi: 10.1007/s40670-019-00895-y. PMID: 34457698; PMCID: PMC8368558. 

3. Morgan KM, Northey EE, Khalil MK. The effect of near-peer tutoring on medical students' performance in anatomical and physiological sciences. Clin Anat. 2017 Oct;30(7):922-928. doi: 10.1002/ca.22954. Epub 2017 Aug 11. PMID: 28726243. 

FIRDOUS JAHAN¹
Nausheen Saeed, Mazin Saleh Al-Rudaini, P. J. Joseph Francis and Siddiqui Muhammad 1 WONCA World organization of Family Doctors 

Medical student teaching, learning, and evaluation are going through a paradigm shift due to the ongoing evolution of medical education. Medical students' professionalism is important for them to develop into capable and trustworthy doctors for the community. 

This is a questionnaire-based cross-sectional study. Participants were selected on the basis of a convenient nonprobability sampling technique. An online questionnaire was used for Self- Assessment of professionalism in medical students. The questionnaire was pilot tested .A favorable ethical opinion was obtained from the Research Ethics Board 

A total of 199 students participated. 58% were clinical students. (91%) of the participants believe that they are aware of their responsibilities as a student and practice them. Students be certain that they are empathetic, trustworthy, and accountable for their actions (95%), know how to maintain professional relationships (92%), and are aware of ethics, patients' privacy confidentiality (95%). Nearly half of the students consider themselves role models for others (46%) and take the initiative to answer questions during sessions (49%) 

A significant difference was observed (p > 0.5) between clinical and non-clinical students' responses related to knowledge of responsibilities, awareness of ethics, patients' privacy, confidentiality, and team player skills . 

According to the literature, professionalism is a skill that calls for formal instruction. It is vitally important to evaluate performance using a variety of criteria and approaches. There are numerous ways to develop professionalism in medical students. This study reflects the need of teaching learning and assessment of professionalism in pre-clinical students. 

Conclusions 

Honoring the values of truthfulness, and accountability is mandatory in professionalism training. 

All medical professionals must show medical professionalism build trust with patients and the public, so, learning and assessment of professionalism should start early years of medical college. 

References (maximum three) 

Palmer, E.G., Reddy, R.K. & Laughey, W. Teaching Professionalism to Medical Students Using Dissection-Based Anatomy Education: a Practical Guide. Med.Sci.Educ. 31, 203–213 (2021). https://doi.org/10.1007/s40670-020-01137-2 

Hsieh JG, Kuo LC, Wang YW. Learning medical professionalism – the application of appreciative inquiry and social media. Med Educ Online. 2019;24(1). https://doi.org/10.1080/10872981.2019.1586507 

Firdous Jahan , Nisar Ahmad, Ikram A Burney. Teaching, learning and assessment of professionalism in medical education: A citation-based systematic review of literature. International Journal of Life Science Research Archive, 2023, 04(01), 006–016 

Jongweon Shin¹
1 Department of Plastic and Reconstructive Surgery, Eunpyeong St. Mary’s Hospital, College of Medicine, The Catholic University of Korea 

Background:
Widespread use of smartphones and wireless internet have made YouTube an easily accessible educational modality. Many residents use YouTube to acquire knowledge regarding microsurgical techniques; however, its quality and effect has not been verified. 

Summary of Work:
The authors included 22 residents working in the Department of Plastic and Reconstructive Surgery at our institute. Using block randomization, seven were allocated to a textbook group (TG), eight to a free-searching group (FSG), and seven to a designated- video group (DVG). After reviewing textbooks, YouTube videos, or designated videos, respectively, each group performed microsurgical anastomosis using artificial vessels. The total procedure time, Objective Structured Assessment of Technical Skills (OSATS), operative errors, and degree of leakage were assessed by blinded evaluators. Self-confidence rates were also compared. 

Results:
The YouTube groups (FSG and DVG) performed better than the TG. Although procedure time was significantly longer in the DVG (p=.006), the performance of DVG was better than that of TG in all assessments (OSATS: p=.012; operative errors: p=.002; leakage: p=.010). FSG showed more operative errors (p=.004) and leakage (p=.007) compared to DVG but had higher OSATS (p=.008) and fewer operative errors (p=.002) than TG. The post- intervention confidence rates were significantly higher in FSG and DVG compared to TG (p=.002 and p=.001, respectively). 

Discussion:
Microsurgery videos on YouTube had positive effects on microsurgery practice judging by the fact that both FSG and DVG showed better results than TG, although there are concerns regarding the reliability of YouTube videos. Also, considering that DVG showed better results than FSG, the educational effects may be enhanced if a quality control system is introduced for YouTube videos. 

Conclusions:
YouTube may help to improve the microsurgical skills of residents. 

Implications for further research or practice:
Methodology for improving the quality of educational contents of media. 

References (maximum three) 

David Mathew 
Nien Yue Koh 

Background 

Basic communication skills were taught in the earlier years of study in the Lee Kong Chian School of Medicine; communications in the clinical context is typically taught from the third year of study onward. 

Students desire to be in the "hot seat" and be assessed on their performance. However, due to a progressively larger student cohort and challenges in recruiting faculty to perform real time feedback on-site due to their individual clinical commitments, not all students would get the chance. 

We introduced the concept of role play videos, whereby students were tasked to role play as either the medical student or the patient and film themselves. The videos were then submitted to the clinical faculty, who would review and provide feedback on the students’ performance. 

50% fewer faculty were required for remote assessment of the students’ performance compared to real-time assessment on-site. Students gave a feedback score of 4.35/5 for this learning activity and deemed feedback provided by the faculty to be helpful and informative. 

The use of role play videos brought about an additional dimension to clinical communications learning. The primary aim of requiring fewer faculty for the module was achieved. In addition, more students could benefit from evaluation by the clinical faculty. Given that faculty could watch the videos at their own time, they were less constrained by time, were able to review the students' performance closely at a relatively relaxed pace and subsequently provided more detailed feedback. 

Role play videos with remote assessment by the clinical faculty appears to be one of the feasible strategies to manage the challenges in the teaching and assessment of clinical communications, allowing for reduction in faculty resource requirement without loss of quality of learning experience and with added benefit of more detailed feedback for students. 

Wen-Hsuan Hou^1,2
Jeng-Cheng Wu³, Faith Liao^4,5 and Chien-Yeh Lu⁶
1 Department of Physical Medicine and Rehabilitation, Taipei Medical University Hospital 2 College of Medicine, National Cheng Kung University
3 Taipei Medical University Hospital
4 Department of Education, Taipei Medical University Hospital
5 Graduate Institute of Humanities in Medicine, Taipei Medical University
6 College of Medicine, National Cheng Kung University, Tainan, Taiwan 

Background:
Shared Decision Making (SDM) is a critical clinical competency for physicians to provide patient-centered care, but the integration of SDM in clinical practice has been slow. This study aimed to assess the factors influencing SDM skills among undergraduate medical students (UGYs) on the basis the framework of Social Cognitive Theory (SCT). 

Summary of Work:
We conducted a pretest-posttest study in a regional teaching hospital. The curriculum design was based on Kolb’s experience learning cycle, contained blended teaching of self-learning materials including online video and printed information, hands-on simulation with standard patients (SP), and debriefing with formative Objective Structured Clinical Examination (OSCE). SCT related factors of SDM skills (behavior), self-efficacy (personal factor), facilitators and barriers (environmental factor) regarding to SDM was collected. 

Results:
A total of 88 UGY have completed this blended SDM training program. The behavior in SDM skills observed by the clinical teachers (OPTION5) were significantly improved (t=20.411, p<.05). UGY’s personal factor of SDM self-efficacy significantly correlated to the SDM skills behavior. For environmental factor, barrier but not facilitator of SDM were significantly associated with the SDM skills. In the multiple regression, UGYs of the 6th grade had better SDM skills than the 5th (β=1.37, p<.05); and they had more perceived more barriers had worse SDM skills (β=-.56, p<.05). The most common barriers were lack of time to practice (86.4%), lack of practical experience or examples (68.2%), and lack of knowledge in practice (68.2%). 

Discussions:
To the best of our knowledge, this is the first study well examined and explained the effectiveness SDM teaching among UGYs based on the factors of SCT. 

Conclusion:
Both personal and environmental factors were associated with the behavior of SDM. 

Take-home messages/implications for further research or practice:
SCT is a practical theoretical framework in exploring the effectives of SDM blended teaching among UGYs. 

References (maximum three) 

Chiao-Hui Lin¹
Tzu-Han Weng²
1 Department of Nursing/Chi Mei Medical Center, Liouying, Taiwan
2 Yuzen Retirement Center

Utilizing traditional Taiwanese night market stalls, the following games will be incorporated into the curriculum on emergency medical techniques and intravenous catheter insertion skills: 

1. Retro Pinball Machine: The retro pinball machine will be used to simulate the process of assessing and responding to emergency situations. Nurses will navigate through the game, making decisions and performing appropriate interventions based on the scenarios presented. 

2. Fishing Game: The fishing game will be adapted to simulate the skill of intravenous catheter insertion. Nurses will use a fishing rod-like device to "catch" veins on a model arm, practicing the technique of locating and inserting an intravenous catheter. 

3. Whack-a-Mole Game: The whack-a-mole game will be modified to enhance nurses' proficiency in venipuncture. Instead of moles, simulated veins will pop up, and nurses will use a tool to "poke" the veins in the correct manner, practicing the proper technique of venipuncture. 

By incorporating these games into the curriculum in a lively and interactive manner, nurses can learn and enhance their skills in a joyful and stress-free environment. This approach aims to promote better retention of knowledge, internalization of skills, and improvement of any existing deficiencies. 

After the implementation of the Taiwanese traditional night market game-based approach, the scores for nurses in emergency medical techniques and intravenous catheter insertion skills improved from 54.9 to 91.4. Moreover, the course satisfaction level, averaged at 4.8. 

Utilizing traditional games to enhance in-service education or related courses for nurses proves to be effective in creating a sense of fun, interactivity, and receiving valuable feedback from learners. This approach ultimately leads to improved learning outcomes and cognitive development. 

It is recommended that hospitals consider incorporating game-based approaches into their educational programs in order to enhance nurses' understanding of course content and promote the achievement of learning objectives. 

References (maximum three) 

Chang, C. Y., Chung, M. H., & Yang, J. C. (2022). Facilitating nursing students' skill training in distance education via online game-based learning with the watch-summarize-question approach during the COVID-19 pandemic: A quasi-experimental study. Nurse education today, 109, 105256. https://doi.org/10.1016/j.nedt.2021.105256 

Crevacore, C., Coventry, L., Duffield, C., & Jacob, E. (2022). Factors impacting delegation decision making by registered nurses to assistants in nursing in the acute care setting: A mixed method study. International Journal of Nursing Studies, 136, 104366. https://doi.org/10.1016/j.ijnurstu.2022.104366 

Yu Chen Su¹ 
1 NTUNHS 

Abstract 

In the early stages of the COVID‐19 pandemic, Out of necessity, nursing education programs significantly increased their reliance on virtual simulation during the early stages of the COVID‐19 pandemic. as clinical site restrictions were implemented, education programs leading to student Nurse degrees increasingly relied on virtual simulation-based experiences to provide clinical training to their students. this change and the various modalities employed by student Nurses across Taiwan. 

Virtual simulation education is a teaching method that utilizes virtual reality or computer simulation technology. In nursing education, virtual simulation education provides students with an experience closely resembling real clinical situations, aiming to cultivate their clinical judgment, skills. 

This teaching has become particularly crucial during the COVID-19 pandemic, as students have been unable to participate in traditional in-person clinical placements. Virtual simulation education has an effective alternative, ensuring nursing students still receive a high-quality learning experience and enhancing their preparedness for future clinical practicum. 

In Taiwan, the impact of the COVID-19 pandemic is still a concern, and many parents worry about their children getting infected during hospital internships. Hospitals are also concerned about the potential spread of the virus to the community. As a solution, a three-week clinical internship program has been implemented, with the first two weeks focusing on clinical nursing technique education in the school's experimental classrooms. The final week is spent in hospitals for practical training, thereby reducing the chances of exposure to potential sources of infection. This arrangement provides reassurance to both students' parents and the hospitals. 

Satisfaction assessments were conducted to evaluate the results of this approach, and it received positive feedback from students, parents, and hospitals. The implementation of this learning approach demonstrated that prior simulated practice enhances memory and skills, ultimately leading to better learning outcomes for the students. 

References (maximum three) 

Aebersold M. Simulation-based learning: No longer a novelty in undergraduate education. OJIN: The. Online Journal of Issues in Nursing. 2018;23(2) doi: 10.3912/OJIN.Vol23No02PPT39. 

Benner P. Finding online clinical replacement solutions during the COVID-19 pandemic. Educating Nurses. 2020, March 19 

Khwaja K, Deban M, Iqbal S, et al. The McGill simulation complexity Score (MSCS): a novel complexity scoring system for simulations in trauma. Can J Surg 2023;66:E206-11. 

Adedeji Oyefeso¹
1 University College London

Background
Internal Medicine Trainee doctors are required to complete the MRCP(UK) diploma in order to progress onto higher specialty training. The final part of the diploma is a practical exam – Practical Assessment of Clinical Examination Skills (PACES). PACES is notoriously difficult to pass, with a relatively low pass rate, and expensive to take. Therefore, preparation for PACES is vital. Despite the multitude of learning methods available to prepare for PACES, there is little consensus on which methods are effective. 

Summary of work
This study aimed to understand trainee perceptions of the educational effectiveness of different learning methods used to prepare for the PACES examination and what characteristics made them effective, with the ultimate aim of improving trainees’ chances of success on their first attempt. Thirty-five participants who had passed PACES within the last five years volunteered to complete a questionnaire survey. Data from the questionnaires were analysed using a mixed methods approach. 

Results/Discussion
Trainees reported receiving benefits from almost all learning methods. Trainees highly valued learning methods such as revision courses, mock exams, senior-led bedside teaching, peer learning, online resources, and mentoring. Features of educationally effective learning methods included their ability to replicate the exam, provision of good clinical exposure and ample feedback, familiarisation of the exam procedure, help with knowledge acquisition and consolidation, accessibility/convenience, and their ability to evoke an emotional response. Trainees also received benefits from solo practice and textbooks; however, these were not rated as highly as other methods. 

Conclusion
Trainees should use a combination of learning methods to prepare for the PACES examination. 

Take home messages

Jada Chia Di Lee¹
Tomasz Stanislaw Cecot¹, Guilherme dos Reis Borges Coelho da Fonseca¹, Mandy Liu¹, Michael Magtoto Manio¹, Timothy Ka Chun Un² and Jian Yang¹
1 School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong
2 Division of Cardiology, Department of Medicine, Queen Mary Hospital

Background
Anatomical variation refers to body features that differ from classic anatomy textbook descriptions. Though they may not affect normal function, they can impact clinical practice. Traditionally, anatomical variation is only taught during the clinical phase of medical school, but recent data suggests early exposure can enhance students' knowledge throughout the curriculum1. 

Summary of work
We introduced the building of an AVD repository for medical students of preclinical years to observe and identify anatomical variations during dissection practicals. Suspected variations are photographed and uploaded for verification by teachers and senior students. The platform also includes a quiz section with selected images and questions contributed by the students to test the knowledge of their peers on anatomical variations. 

Results 
The AVD repository has two ways to evaluate preclinical students' knowledge of anatomical variations: 1) checking the validity of the identified structures uploaded on the platform; and 2) quiz performance on the AVD repository. Overall, the AVD repository provides a comprehensive way to evaluate preclinical students' knowledge of anatomical variations, and teachers can assess the student's ability to identify and understand anatomical variations practically and interactively. 

Discussion and Conclusions
Co-creating learning materials and assessments are effective in higher education, and the AVD repository can promote students' motivation for learning anatomical variations by increasing engagement and ownership2. Furthermore, anatomy learning requires the spatial ability to understand the spatial orientation and relationships between body parts. The creation of the AVD repository provides a valuable platform for students to study and be assessed on different anatomical variation structures which can help to train students' cue interpretation and knowledge during preclinical training3. 

Take-home message
Using an AVD repository to generate different types of questions can reinforce students' knowledge of common anatomical variations and increase their awareness of possible variations in future clinical practice. 

References (maximum three) 

MANISH TAYWADE¹
Binod Kumar Behera², Debkumar Pal³ and Bimal Kumar Sahoo⁴
1 All India Institute of Medical Sciences Bhubaneswar
2 AIl India Institute of Medical Sciences Bhubaneswar
3 Junior Resident AIl India Institute of Medical Sciences Bhubaneswar 4 Shri Jagannath Medical College and Hospital, Puri

Background:
The pandemic provides us with alternate teaching-learning methods for adaptation. The use of information technology is a great opportunity in such a critical situation. The sudden shift from classroom teaching and assessment to online offers multiple challenges and with no experience of the online teaching-learning environment. The researchers conducted an online assessment of the participants with the objective to evaluate the crossword puzzle and feedback on the online session 

Material and Methods:
We use google meet for one online lecture with various innovative methods of teaching like Crossword, Pictorial presentation, and Multiple choice questions to assess the understanding of a topic taught. Feedback was also taken for further improvement in the online lecture. 

Results:
Sixty percent of participants scored Ninety-four percent for crossword. Twenty percent got 100 percent for crossword. The feedback suggests 50 percent were in favor of a change in the online teaching-learning and assessment methods that require more effort. 50 percent were comfortable with online learning and teaching due to remote location and flexibility. 

Conclusion:
Online platform is an alternative teaching-learning of the classroom in a crisis like a pandemic unless used judiciously. 

References (maximum three) 

1.Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. The Lancet infectious diseases. 2020 Jul 1;20(7):777-8. 

2.Rose S. Medical student education in the time of COVID-19. Jama. 2020 Jun 2;323(21):2131-2. 

3.Khaleel M. Effectiveness of Crossword Puzzle as an Adjunct Tool for Active Learning and Critical Thinking in Microbiology. Journal of Clinical & Diagnostic Research. 2022 Jan 2;16. 

Brekhna Jamil¹
Susie Schofield² and Mandy Moffat²
1 Khyber Medical University
2 University of Dundee

Background:
A sense of belongingness (SoB) is one of the most important requirements for students in all types of learning situations, including online distance learning. Many of the existing SoB scales are available for programs offered on campus. However, these cannot be used to evaluate programs offered through online distance learning. This study aims to develop a validated tool for measuring the SoB in postgraduate online distance learners.


Methods:
The study was done in three phases. In the first phase, a qualitative study was conducted to explore the concept of belongingness in postgraduate online distance learning. In the second phase, a consensus was established on the sense of belongingness questionnaire items using the Modified Delphi Technique using two rounds, and in the third phase a psychometric validation was performed. 

Results:
Six themes emerged: Convenience and Flexibility, e-immediacy, Technology and Accessibility, Virtual Community Building, Communication, and Comfort and Support. Initially, 67 items were created under these themes. After the Delphi process with 51 and 43 experts respectively, 43 items reached consensus. Content validity by 33 experts reduced items to 37. Following cognitive interviews with 14 students, the list expanded to 40. Reliability and construct validity were confirmed from 894 online postgraduate students, achieving a Cronbach’s Alpha of 0.95. 

Conclusion:
A promising new SoB scale for postgraduate online learners was developed with promising initial validity and reliability. Additional research is needed to enhance its generalizability across diverse institutions and online program types. 

Keywords:
sense of belonging; postgraduate; online learning; distance learning; belongingness 

References (maximum three) 

1. Zhao, L., & Kuh, D. (2017). Learning online: Evaluating the student experience. Journal of Distance Education, 31(2), 45-58. 

2. Brown, M., & Lee, M. J. (2015). The influence of online community on perceived belonging and commitment to course. Online Learning Journal, 19(4), 9-29. 

3. Peters, K., & Kumar, V. (2019). Student engagement in online learning: What works and why. Distance Education Quarterly, 66(3), 21-40. 

João Pedro Monteiro^1,2
José Miguel Pêgo^1,2 and Carlos Fernando Collares³
1 Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal.
2 ICVS/3B's, PT Government Associate Laboratory, Braga, Portugal.
3 University of Minho

Background:
Multiple choice questions (MCQs) are a widely used method of assessment, due to their ability to objectively evaluate knowledge and skill. However, MCQs are subjected to flaws, whose detection is time and resources-consuming. We aimed to develop an open-access software that automatically identifies Item-Writing Flaws (IWFs) in MCQs. (1, 2) 

Summary of work:
A database of 1150 items from high-stake Portuguese exams was analyzed. Criteria established by Rush et al. (2016) was used for identification of IWF. Accuracy calculations and statistical analysis to calculate the degree of agreement, the systematic effect and the validity were made comparing both analyses. 

Results:
The developed tool successfully identified 78,0% of items with at least one flaw. Individually, the sensitivity and specificity were higher than 61,29% and 91,0% respectively with an accuracy higher than 92%. Five tests showed a substantial or an almost perfect agreement and only two tests showed a statistically significant result. 

Discussion:
The tool demonstrated good efficacy for most IWFs categories, particularly for negative stems, true/false options, and absolute terms. The overall higher specificity of the tests shows that the tool is more reliable in the identification of items without IWF than otherwise. 

Conclusions:
The tool showed promising results, with high accuracy, sensitivity and specificity, although it had certain limitations in specific types of IWFs. The software has a high reliability in identifying items with at least one IWF but a lower reliability in identifying the number of each item’s IWF. Further research and development are necessary, including the use of Artificial Intelligence tools to enhance IWFs identification. 

Take home messages:
We have shown that it is possible to automatically identify IWFs and reduce the burden of MCQ reviewing work using software solutions that could also contribute to reducing the cost and the time needed to construct excellent MCQ. 

References (maximum three) 

Voraluck Phatarakijnirund
Tim Phetthong, Detchvijit Suwanpakdee, Boonchai Boonyawat, Wirongrong Arunyanart and Nopaorn Phavichitr 

Background:
The rating scale is a widely used for assessment medical student performance in classroom activity. However, this method might be inaccuracy especially in setting of large group teaching when the teacher usually rates all of the student with the same score. In this study, we developed the structured rating scale using for assessment of fifth-year medical student performance in Case-based learning (CBL) class during Pediatric course. 

Summary of work:
The new two-step rating scale was developed. In the first part, teachers have to rate the overall group performance in the 5 areas; including team work, class engagement, critical thinking, problem-solving ability and communication skills. In the second part, teachers have to rate the student individually to the level of 1 to 5 according to their performance in class compare to the overall group performance. The full score is 5 and the first and second part gave the 70% and 30% of total score, respectively. The mean + SD score of the new rating scale in each of medical students were compared to score of old rating scale (5 areas of assessment in each individual student) using in the previous year. 

Results:
Score of 198 students from 48 CBL classes were assessed. The mean + SD score of the student using the new 2-step rating scale were significantly different compare to the old rating scale and well correlated with the MCQ examination score. The range of score more than 0.5 (10% of total score) was observe in 58.2% of teachers using new rating scale compare to 0% in old rating scale. 

Conclusion:
The overall and individual rating scale is an effective assessment method and suitable to use to evaluate medical student performance during CBL class. 

Take home: Good quality assessment method provides important data to determine the student performance 

References (maximum three) 

O'Brien CE, Franks AM, Stowe CD. Multiple rubric-based assessments of student case presentations. Am J Pharm Educ. 2008;72(3):58 

Paul Didaskalu¹
Craig Hassed² and Chris Barton²
1 Monash University
2 Monash University

Background
Psychological distress among medical students and junior doctors is a significant concern. Compassion training has shown promise in improving psychological well-being among healthcare workers, including medical students. However, the feasibility and student perceptions of compassion training in medical curriculum are understudied. 

Summary of work
This study assessed the feasibility of an online compassion training course as part of medical student curriculum. We used a mixed-methods approach to assess survey results from final- year medical students participating in the Compassion Training for Healthcare Workers course. We conducted descriptive and inductive content analyses on quantitative data from Likert scale responses and qualitative data from open-ended responses. 

Results 

Most of the 306 participants responded positively to the training, appreciating the content, variety of teaching mediums, discussion forums, and practical strategies learnt. Students emphasised the importance of compassion in healthcare and reported a positive shift in attitude towards compassion. Although some students suggested a face-to-face component, the online asynchronous format was well-received for its flexibility and self-paced learning. 

Discussion
This study demonstrates the feasibility and positive reception of incorporating an online compassion training course in medical curriculum, supporting the growing interest in compassion training for healthcare professionals. Acknowledging the limitations, including lack of demographics and objective measures, it provides proof of concept, and a valuable understanding of what learners want from a compassion training curriculum. 

Conclusions
This study demonstrates the feasibility and positive reception of incorporating an online compassion training course into final year medical student curriculum. 

Take-home messages
Medical students’ and junior doctors’ psychological wellbeing is a growing problem and compassion training shows promise in addressing it. This study serves as a proof of concept for a scalable medical education intervention, laying the groundwork for future research to assess the long-term impact and efficacy of compassion training in medical education. 

References (maximum three) 

1. Beyond Blue. National Mental Health Survey of Doctors and Medical Students 2019 [Available from: https://medicine.uq.edu.au/files/42088/Beyondblue%20Doctors%20Mental%20health.pdf]. 

2. Singer T, Klimecki OM. Empathy and compassion. Current Biology. 2014;24(18):R875-R8. 

3. Weingartner LA, Sawning S, Shaw MA, Klein JB. Compassion cultivation training promotes medical student wellness and enhanced clinical care. BMC medical education. 2019;19(1):139. 

Richard Arnett¹
Muríosa Prendergast¹
1 RCSI University of Medicine & Health Sciences 

A key component of the new RCSI undergraduate medical curriculum is the use of Progress Testing. RCSI Progress Testing involves 4 diets per year each consisting of 160 SBAs with 3 options plus a ‘Don’t Know’ option. Penalty scoring is applied: +1 (correct), -0.5 (incorrect), and 0 (Don’t Know). 

2. Summary of work 

Students in the new curriculum are in Year 2 but eventually, Progress Testing will be happening simultaneously throughout the programme. The methodology for setting standards needs to balance the need for robustness and transparency, the use of penalty scoring, and the variety and workload of staff involved. 

3. Results 

We are using a modified Bookmark methodology. Response data are analysed under a Rasch model to produce an Ordered Item Booklet (OIB) in which questions are presented in order of increasing difficulty. Each question is accompanied by additional information for each cohort taking the test (including mean score, discrimination, estimated total score, and pass rates). Judges identify the point in the OIB where the probability of a Minimally Competent Candidate (MCC) drops below 50% (for each cohort). The group cut score is the mean ability level of these estimates. To account for penalty scoring, an ‘allowance’ is subtracted from the cut score. This allowance is based on a percentage of the total test items and decreases in each year of the programme. 

4. Discussion 

Incorporating negative marking into the standard-setting process has been challenging & staff have required additional support. 

5. Conclusions 

Progress Testing is currently confined to the first two years but the initial experience suggests this methodology should scale as additional cohorts come online. Applying a decreasing penalty allowance to the cut score allows some level of educated guessing. 

6. Take-home messages/implications for further research or practice
Planning & ongoing communication are crucial. 

References (maximum three) 

Baldwin P. A Problem with the Bookmark Procedure’s Correction for Guessing. Educ Meas: Issues Pr. 2021;40(2):7–15 

Karantonis A, Sireci SG. The Bookmark Standard-Setting Method: A Literature Review. Educational Measurement: Issues and Practice. 25(1):4–12 

Lucy Wilding¹
Lishan Yang¹ and Jessica Ang
1 Nanyang Technological University, Lee Kong Chian School of Medicine 

Background
Tools like TurnItIn have allowed educators to check student assignments for plagiarism, but the advent of generative artificial intelligence (AI) such as ChatGPT has brought forth an even greater challenge for educators: the use of these resources in writing student assignments. 

Summary of work
ChatGPT-use has been identified in students’ work, leading to concerns about their integrity and professionalism. As faculty, it is our responsibility to address this. A 2-hour focus-group discussion of 20 educators was undertaken as a staff development session to explore the challenges faced with ChatGPT. 

Results
Issues identified include current-version ChatGPT’s propensity to “hallucinate”, citing entirely made-up references and studies. Solutions proposed included educating students regarding the responsible utilisation of ChatGPT. Faculty also shared that ChatGPT was even able to generate reflective writing assignments. Students must be reminded that reflection is about the process rather than producing a perfect piece of writing. Although TurnItIn has created an AI- generated content detector that is 98% accurate, the best way to hold students accountable remains to question them directly on their work with some degree of rigour. 

Discussion
Aside from the ethical issues surrounding this, students using these platforms must recognise that ChatGPT is a tool to be used with caution – the information given may not be accurate, nor the sources peer-reviewed. Further, given the known “hallucination” issue, all references need to be corroborated. 

Conclusions
Whilst students should ideally complete their own assignments without the use of AI, it is becoming a part of everyday life. Educators need to be aware and engage with students about how best to use it.


Take-home
As educators we must be cognisant and need to embrace AI as the future. 

We need to adapt our own practices to ensure continued delivery of high-quality up-to date education 

References (maximum three) 

No references 

William Lam¹
Priya Khanna¹
1 The University of Sydney

The recent advancements within the field of Artificial Intelligence (AI), particularly large language models such as AI powered chatbots (e.g., Chat GPT and Google Bard) seems to have created unprecedented disruptions in higher education. The use of AI solutions in automation of tasks has penetrated student learning and assessment of learning. Global concerns around student use of chatbots in assignment submission is created concerns and scepticism among faculty around academic integrity. Whilst much has been written about the challenges and affordances of AI in higher education, not much is known about the ways in which students in programs such as Medicine are capitalising on using AI for assessment of and for learning. This presentation provides research being undertaken by group of medical students around developing and validating a scale to assess the usage and attitudes of medical students towards AI technologies in education and practice. Technology Assisted Model and Kane’s Validity framework are being referred to as theoretical frameworks for scale development and validation. The initial survey will be administered to a sample of medical students affiliated to various universities in Australia, Sheffield University, UK and New Orleans, USA. Participants will also be invited to take part in semi-structured interviews. 

The validated scale and qualitative findings will contribute to the field of medical education by providing a reliable and valid tool to assess medical students' readiness for AI-assisted assessment. The findings will inform both students, faculty and administrators in curriculum development, instructional design, and policymaking regarding better and informed integration of AI technologies in medical education programs. 

References (maximum three) 

No References 

Jeannie Watkins 

Professional behaviour is something that we expect students who are training to be healthcare professionals to have knowledge of and demonstrate in preparation for professional practice. Researchinthisfieldshowsastrongcorrelationbetweenthosewhodemonstratepoor professional behaviour as students and disciplinary action once qualified. Within the university system there is a process for managing unprofessional behaviour but this is normally at the institutional level when there are clear professionalism issues and does not always make provision for students with low level professionalism issues which could be managed and remediated early. Whilst students are taught about professional expectations, there is a lack of formal assessment of low level issues in spite of the importance placed on it. This session proposes a fair, supportive and transparent process to formally recognise, record, and manage students' with low level professionalism concerns at programme level and building the evidence to escalate where necessary. 

References (maximum three) 

Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board Maxine A Papadakis 1, Carol S Hodgson, Arianne Teherani, Neal D Kohatsu 

Achieving good medical practice: interim guidance for PA and AA students (GMC, 2022) 

Ruri Ashida¹
Makoto Kurata², Satoshi Takeda³ and Motoko Kita⁴
1 University of the Sacred Heart, Tokyo
2 Tokyo Medical University
3 The Jikei University School of Medicine, Tokyo
4 THe Jikei University School of Nursing, Tokyo

Background: 
The need to care for patients with diverse sociocultural backgrounds is increasingly addressed in medical education curricula. What students learn must be practiced by clinical faculty, but faculty in Japan have had little such training. Faculty development (FD) is necessary. 

Summary of work: 
A questionnaire eliciting actual cases of non-Japanese patients was sent to 201 institutions including 2021 Olympics-designated hospitals and health care institutions in the top 20 prefectures where Chinese, Vietnamese, Korean, Filipino, and Brazilian residents resided. The results were qualitatively analyzed and used to develop FD cases. Faculty development was then conducted, consisting of 1) case studies, in which faculty’s present knowledge and awareness of sociocultural issues were assessed, and 2) simulated patient practice with a follow-up assessment. 

Results:
Responses were received from 37 institutions (94 cases). Language problems, religious concerns, differences in the concept of time (e.g. patients coming late for appointments), and different ideas about medication (e.g. sharing medication with others) were noted, among others, and these issues occasionally caused frustration, even anger, on both the healthcare professionals and the patients. 

Discussion:
Some healthcare professionals had no knowledge of broad cultural generalizations: what might be required of Muslim patients, patients from cultures with a relaxed attitude towards time, or those from cultures where medicine is regarded as something that can be shared. Some assumptions or biases of the healthcare professionals themselves were also noted: e.g., coming late for an appointment also occurs with Japanese patients. 

Conclusion:
Scenarios developed from real foreign patient cases can help faculty reflect on their own biases, acknowledge cultural generalizations as an essential starting point of person- centered care, and enhance cultural humility in the clinical setting. 

Take-home messages
Cases developed for FD can also be adjusted for use with students. 

References (maximum three) 

1. Deliz, J.R., Fears, F.F., Jones, K.E. et al. Cultural Competency Interventions During Medical School: a Scoping Review and Narrative Synthesis. J GEN INTERN MED 35, 568–577 (2020). https://doi.org/10.1007/s11606-019-05417-5 

2. Ministry of Education, Culture, Sports, Science and Technology. Model Core Curriculum for Medical Education in Japan: 2022 Revision. 

https://www.mext.go.jp/content/20230315-mxt_igaku-000026049_00003.pdf Accessed August 10, 2023. 

Mahwish Arooj¹
1 AMEE

Background:
Feedback plays a pivotal role in the field of education, serving as a powerful tool to enhance learning outcomes and student performance. While extensive research exists on the general importance of feedback, a noticeable gap exists in the literature regarding its potential as a dedicated assessment tool, particularly within the context of Pakistan's education system. 

In Pakistan, as in many other educational contexts, assessment is primarily used as a measure of student performance rather than a strategic instrument for enhancing learning. Therefore, this study seeks to investigate the overlooked area of utilising feedback as an assessment tool, aiming to bridge the gap between its theoretical significance and practical application. 

This cross-sectional study was designed to analyze changes in Mid Stake Examination scores over time. A comparison was made between students who attended feedback sessions and those who did not. To assess the relationship between attendance at feedback sessions and exam scores, a Chi-square test was applied. 

The results indicated that for the 1st, 2nd, and 3rd years, the calculated p-values were all less than 0.5, suggesting a statistically significant association between attending feedback sessions after low stake assessments (LSA) and exam scores. In contrast, for the 4th year, the calculated p-value exceeded 0.5, indicating that attending feedback sessions after LSA did not have a significant impact on exam scores for this group. 

The outcomes of this study highlight the value of using feedback as a strategic assessment tool, especially during the initial or basic years. It highlights the importance of timely and helpful feedback sessions that accommodate students changing needs throughout the duration of their academic careers. 

Conclusion:
This study highlights the significance of specific feedback and advocates for the exploration of different feedback approaches, as the development of a feedback-centred culture can create opportunities for ongoing student improvement. 

References (maximum three) 

Educators and institutions should consider strategically integrating timely and constructive feedback sessions as a core component of their assessment strategies, especially in earlier academic years. 

Future research could delve into identifying specific factors contributing to the varying impact of feedback on different academic year levels. Conducting longitudinal studies with extended follow-up periods can reveal the long-term effects of integrating feedback as an assessment tool 

Bakke, B. M., Sheu, L., & Hauer, K. E. (2020). Fostering a feedback mindset: a qualitative exploration of medical students’ feedback experiences with longitudinal coaches. Academic Medicine, 95(7), 1057-1065 

McGinness, H. T., Caldwell, P. H., Gunasekera, H., & Scott, K. M. (2020). An educational intervention to increase student engagement in feedback. Medical Teacher, 42(11), 1289-1297. 

Preston, R., Gratani, M., Owens, K., Roche, P., Zimanyi, M., & Malau-Aduli, B. (2020). Exploring the impact of assessment on medical students’ learning. Assessment & Evaluation in Higher Education, 45(1), 109-124. 

Jose Marcos Felix¹
1 Universidad Anahuac 

Different types of questions that tell us how much the student knows, interprets. False and true questions, multiple choice, to rank, among others. Adding some images of patients or their laboratory or radiological results to clinical vignettes is a big step forward, but more can be done. (Preston 2019, Tabish 2008) 

The diversification of the way of presenting the reagents (items) of theoretical subjects with clinical content allows the student not only to be more interested in his own exam but also to recognize that there are ways to get closer to the clinical context that he will face in years to come. 

In theoretical examination of clinical content in a subject for 2nd year medical students, we used 30 to 40-second video-based format reagents that contextualizes a clinical or hospital situation accompanied by laboratory or clinical image. These types of reagents were mixed with usual reagents (with only text or with text plus images). After the test, a questionnaire based on a Likert scale was applied. 

Of the 84 students surveyed, 91.7% of them had not seen a video format in reagents of their previously. 86.9% of them liked it. 76.2% of them consider this mix much more understandable or more understandable. 90.4% of the students consider that the video question format places them in an environment closer to a clinic situation and 84.5% of the students consider that it greatly or somewhat facilitates the understanding of what is being questioned. And in general mixing question formats with text, text with image and video improves the evaluation experience in up to 88.1% of the participants. 

The mixture of reagents in theoretical questions trying to contextualize the clinical aspects tries to make an "opzonization" of the assessment and makes the student's evaluation "more digestible". Its acceptance is high among students. 

References (maximum three) 

Preston R, et al (2019): Exploring the Impact of Assessment on Medical Students’ Learning, Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2019.1614145. 

Tabish SA. (2008) Assessment methods in medical education. Int J Health Sci (Qassim). Jul;2(2):3-7. PMID: 21475483; PMCID: PMC3068728. 

Julia Wimmers-Klick¹
1 University of Northern British Columbia 

The integration of artificial intelligence (AI) into medical education necessitates a reevaluation of our approach to Portfolio. In the context of a longitudinal program in a medical school in British Columbia, Canada, Portfolio enhances professional development, focusing on soft-skill topics such as communication skills, professionalism, diversity, and equity. Traditionally, students submit written reflective pieces after in-person sessions, which are assessed and graded by individual coaches. However, the emergence of ChatGPT has rendered the verification of students' written submissions for authenticity challenging. This situation prompts us to reconsider our approach to portfolio assessment in the age of AI. 

To investigate this issue, we conducted focus group interviews involving site leads, coaches, students, and administrators. Data analysis was performed using a SWOT framework. The results of these interviews have sparked a discourse regarding the goals and objectives of the portfolio program. The primary outcome indicated a shift towards nurturing students' intrinsic motivation to reflect on relevant topics, drawing from their experiences both within and outside their portfolio group. Moving away from rigid submission requirements, alternative assessment methods such as art pieces or audio recordings have been proposed. Moreover, a subgroup of interviewees suggested restructuring the portfolio framework and incorporating follow-up in- person sessions where students can deliver short presentations on their personal reflections. However, concerns regarding confidentiality and personal pressure in implementing this approach were acknowledged. 

In conclusion, all participants reached a consensus that Portfolio is essential and plays a vital role in medical education. Given its nature, portfolio assessments require specific attention. The key lies in cultivating students' intrinsic motivation to reflect and learn within the realm of soft skills. 

Looking ahead, there is a need to explore diverse approaches that effectively foster intrinsic motivation. Identifying the most successful strategies in this regard will shape future of Portfolio assessment in medical education. 

References (maximum three) 

Hauer, K. E., Iverson, N., Quach, A., Yuan, P., Kaner, S., & Boscardin, C. (2018). Fostering Medical Students’ lifelong learning skills with a dashboard, coaching and learning planning. Perspectives on Medical Education, 7(5), 311–317. https://doi.org/10.1007/s40037- 018-0449-2 

Van der Gulden, R., Timmerman, A., Muris, J. W., Thoonen, B. P., Heeneman, S., & Scherpbier-de Haan, N. D. (2022). How does portfolio use affect self-regulated learning in 

clinical workplace learning: What works, for whom, and in what contexts? Perspectives on Medical Education, 11(5), 1–11. https://doi.org/10.1007/s40037-022-00727-7 

Priya Khanna 

Limitations of positivist nature of traditional assessment system in capturing the nuances of clinical competence has led to the uprising popularity of programmatic assessment as an alternate approach that embraces subjectivity of human judgement in holistic decision making of complex knowledge, skills and behaviours. Although educationally sound, programmatic assessment is highly complex educational approach-both i its design and implementation. In this large-scale implementation study of programmatic assessment, we unpacked the implementation factors, as perceived by students and faculty, using critical realist perspectives. 

Data was derived from in-depth focus groups with medical students and interviews with the faculty. Based on critical realist methodology, Bhaskar’s concept of ontological stratification as a theoretical framework allowed unpacking the domains of reality of student and faculty experiences of programmatic assessment namely, the empirical (data gathered from observations and experiences), the actual (events or non-events that students and faculty report within the assessment program), and the real (underlying causal structures and mechanisms). Archer’s concept of morphogenesis/morphostasis further enabled us to retroduce conditions that may impact the sustenance and growth of programmatic assessment as a complex change. 

We found a significant mismatch and miscommunication between the intended program theories and observable/ reported outcomes. We theorise underlying mechanisms as a state of anomie (sense of alienation and dysfunctionality); agnosis (superficial implementation without attention to structural agility and cultural adaptability) and agenesis (naïve, incomplete, and eclectic system of assessment without attention to underlying education design). 

Achieving balance between agency, structure, and culture can promote sustained changes in assessment practices. Promoting collective reflexivity and agency among faculty and students can be achieved via optimising assessment structures by explicit integration of theory with practice and changing learning culture to promote faculty and students’ acceptance and trust related to the new norms, beliefs and behaviours in assessing for, as and of learning. 

References (maximum three) 

Archer, M. 1995. Realist social theory: The morphogenetic approach. Cambridge university press. 

Bhaskar, Roy. 1975. A realist theory of science: Routledge 

Roberts, C., Khanna, P. et al, 2022. Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis. Medical Education, 56(9), pp.901-914. 

Noa Gazit
Gilad Ben-Gal and Ron Eliashar 

Background.
The current selection for surgical training is based on ineffective methods. The use of virtual reality and game-based simulation assessments for selection is promising since they offer many advantages over traditional assessment methods: they provide real-life-like experiences, collect rich data and eliminate the bias associated with human assessment. 

Summary of Work.
Two gamified virtual reality assessment tests were developed – one for the assessment of technical skills and one for the assessment of cognitive abilities and personality characteristics. The technical skills test is performed on a laparoscopic simulator and includes 10 different gamified tasks. The non-technical skills test is performed on a computer and includes three gamified tasks. Based on these tasks 13 competencies are evaluated. A sample of 150 candidates and 30 expert surgeons took the test to evaluate the feasibility and validity of these tests for selection for surgical training. Participants gave their feedback regarding the tests and item analysis was conducted for each test separately. 

Summary of Results.
Candidates and expert surgeons considered the tests to be relevant and suitable for selection. Item analysis showed positive psychometric properties of the tests such as reliability (0.83 for the technical skills tests and 0.70 for the non-technical skills test) and discrimination of test items. Items that were too difficult were omitted from the tests and changes in test structure and instructions were implemented according to the participants’ feedback. 

Discussion and Conclusion. 
Initial evidence regarding the validity of these innovative tests for selection for surgical training was obtained. Further evidence (e.g., relations with criterion) is needed to establish the validity of these tasks for selection. 

Take Home Messages.
Virtual reality and game-based simulation are promising tools that have the potential to improve surgical residents' selection. 

1. Wanzel KR, Ward M, Reznick RK. Teaching the surgical craft: From selection to certification. Curr Probl Surg. 2002;39(6):583-659. doi:10.1067/MOG.2002.123481 

2. Louridas M, Szasz P, Fecso AB, et al. Practice does not always make perfect: Need for selection curricula in modern surgical training. Surg Endosc. 2017;31(9):3718-3727. doi:10.1007/s00464-017-5572-3 

3. Bann S, Darzi A. Selection of individuals for training in surgery. Am J Surg. 2005;190(1):98-102. doi:10.1016/j.amjsurg.2005.04.002 

ZARRIN SIDDIQUI 

Background
Rasch mathematics modelling is used in educational assessment. In medical education, it is commonly used in knowledge-based assessment and to some extent in the clinical assessment. Objective Structured Clinical Examination (OSCE) is considered to be the gold standard in the assessment of clinical competence. The examination comprises of a series of authentic clinical tasks called stations. Most of these tasks are observed by an examiner and involves standardised patients. 

There are few studies which have applied Rasch modelling to explore the uni-dimensionality of the exam as such, effect of the training. No study has looked at the performance of students in individual stations. 

Aim of the study
To use Rasch analysis to estimate the ability of students on each of the OSCE stations. 

Methods
Scores from two stations used in OSCE were used for analysis. There are three domains assessed at this station. 

It was observed that the first two domains technique and description of signs were easier compared to the differential diagnosis. This information is important for academics to set the standard for each academic year and will be discussed in the presentation.


Results
We have used the Rasch analysis for the scores received on each checklist as part of the Quality Assurance process and feedback to the academics. 

suyoung yoo¹
1 Sungkyunkwan University 

The integration of technology into traditional education is revolutionizing learning across various fields, including nursing. Augmented Reality (AR) and Virtual Reality (VR) are recognized for their potential in enhancing remote clinical skills training, yet research on AR in nursing education is limited. This study aimed to create an AR-based educational program for future nurses, assessing user satisfaction, usability, and acceptance. 

The program focused on educating ICU nurses in device usage, using Kern's six-step approach for teaching innovation. Nursing education experts were interviewed to identify issues and improve the AR program. After development, the AR programs were tested with users, followed by evaluations involving developers, operators, and program users. 

The results were positive, with participants stating that AR improved their clinical performance. AR was particularly well-received by those interested in self-directed and hands- on learning. Nurses showed a willingness to embrace new technology, emphasizing AR's potential in nursing education for self-directed learning and practical training. 

Incorporating AR strategically into educational programs allowed trainees to gain practical experience in a controlled environment, fostering active participation and learner-centered self- directed learning. This technological advancement increased the overall efficiency of medical education programs. 

Rania Zaini 

All clinical training programmes are designed to fulfil specific defied core clinical competencies. National regulation bodies defined core competencies to be attained by clinical trainees at different level of training; this includes undergraduate students and residents. Studies recommend implementing variety approach in the assessment of clinical competencies; this includes perspective of self-assessment and external evaluation. Reflection helps trainees in developing skills and reviews the effectiveness of training. 

Aim :
This study aims to measure trainees’ self-perceptions of the level of core clinical competencies before and after the mock –OSCE that designed as part of the preparatory course for final Internal Medicine (IM) Board exams.In addition, the study investigated any possible significant differences between students’ self-assessment and their actual observed performance. 

Method:
An online four-point Likert scale survey was designed to measure trainees’ self-perception of ten core clinical competencies, ranged from (clear fail, borderline, pass, clear pass). Also eight stations mock OSCE was designed. Each station consists of many clinical competencies, which was tagged in the electronic evaluation form. 

Results.
About 45 final year Internal Medicine residents participated in the Internal Medicine (IM) Board Preparatory Course. All candidate were invited to participate in the study, about 30 (66.6%) IM residents agreed to participate in the study and completed the post and pre exam self-assessment survey. The majority of respondents were female 17 (57%), and passes the IM Board Mock OSCE 22 (73%). Respondents’ self-assessment of their clinical competencies showed confidence to pass the exam of all clinical competences. Also respondents’ self- assessment did not change before and after the mock OSCE. Yet there is a significant difference between the self-assessment and the observed external evaluation in the following clinical competencies (prioritising and interpret investigation, clinical Diagnosis, patient management). Most respondents fail the previous clinical competencies.

Respondents’ self-perceptions did not changed before and after the Mock OSCE. Female respondents showed lower perceptions of their clinical competencies comparing to male peers. Yet this is not statistically significant. Trainees’ reflection on clinical competencies was not clearly accurate with the observed preceptor external evaluation. Reflective training must developed trainees reflection skills and self-assessment.



References (maximum three)