Peter Boedeker
Nadia Ismail and Nancy Moreno 

Background
Medical and health professions programs invest heavily in curriculum-renewal. Programmatic assessment often aims at endpoint measures rather than the implementation process and individuals involved. Knowing individuals' concerns can inform continuous quality improvement (CQI) and ultimately impact return on investment. The Concerns-Based Adoption Model, a framework for understanding adoption of innovative educational practices (1),positsthatfacultytransitionthroughsevenStagesofConcern(SoC). Eachincreasingstage indicates a greater level of comfort and skill with the innovation (2). The SoC Questionnaire helps leaders identify, address, and track faculty concerns over time. At Baylor College of Medicine, we are using the SoC Questionnaire to inform CQI efforts as we implement a new curriculum. 

Summary of work
We administered the 35-item SoC Questionnaire to 143 individuals implementing the curriculum. Open-ended items captured concerns and solutions. 

Results
Of those surveyed, 42 (29.4%) responded; 18 (42.9% of respondents) were at the lowest SoC (stage 0). The next highest concern was equally stage 1 and stage 5. Most faculty are at a low stage of concern and may be passively engaging with the new curriculum. Suggestions for improvement included regular meetings between implementers, opportunities for advance preparation, and improved logistics. 

Discussion and Conclusions
Reponses to the SoC Questionnaire provided valuable insights. The low SoC of many faculty respondents was anticipated given the recent introduction of the new curriculum. Faculty suggestions for addressing concerns will inform CQI efforts with the ultimate goal of increasing faculty's comfort and skill with implementing the new curriculum, evidenced by an increase in their SoC. 

Take-home/further research
Focusing on the concerns of those tasked with implementing an innovation can inform CQI efforts and increase program fidelity. The SoC Questionnaire is one tool for collecting this information. Future work includes re-distribution of the survey to evaluate change in concern over time. 

References (maximum three) 

(1) Hord S, Rutherford W, Huling L, Hall G. (2014). Taking Charge of Change. Austin, TX USA: SEDL. 

(2) George, A. A., Hall, G. E., & Stiegelbauer, S. M. (2006). Measuring implementation in schools: The stages of concern questionnaire. Southwest Educational Development Laboratory. 

Megan Anakin¹
Josie Macfarlane^1,2 and Alexandra McHaffie^1,2
1 University of Otago
2 Te Whatu Ora

Background
Medical imaging is core to clinical practice; however, junior doctors report they do not feel adequately prepared by their medical imaging curriculum for practice.1 This feeling might be related to how students experience an integrated medical imaging curriculum. In a previous study involving medical students and clinical educators, we found that when medical imaging learning is integrated into other areas of the medical curriculum, it appears to be experienced an two sets of contradictory states, which impacts assessment.2 We wondered if how we could investigate these findings further. 

Summary of work
Our analysis was underpinned by an interpretive perspective where we viewed learning, teaching, knowledge application, and assessment as socially situated activities. Previous research findings2 and new interview data generated with junior doctors were analysed with a reflexive thematic approach guided by principles of assessment.3 

Results
If medical imaging is experienced as ‘everywhere and nowhere, and ‘repetitive and patchy’ then assessment may be less visible, unpredictable, and provide incomplete feedback than ideally possible. Additionally, students were rarely asked to perform demonstrations of applied understandings, therefore the frequency of assessing how student complete medical imaging related tasks could be increased to emphasise assessment for learning. 

Discussion
These findings can be used to inform curriculum evaluation and assessment development initiatives. Educators can heighten the visibility, predictability, and feedback about medical imaging knowledge and its application to students. An example of an online learning resource will be presented to illustrate how this approach can be applied by clinical educators to other contexts. 

Conclusions
The principles of assessment can help us to investigate and alter an integrated medical imaging curriculum. 

Implications for further research
A next step will be to evaluate the impact of the assessment embedded in online learning resources on student learning and junior doctors’ feelings about preparedness for practice. 

References (maximum three) 

ANUPONG KANTIWONG 

Background:
Rational drug use (RDU) is the process of appropriate prescribing, dispensing, and patient use of medications[1]. In pharmacology course, third-year medical students are encouraged to create media tool for promote health awareness about RDU concept. The purpose of this study was to determine the reliability of multiple-perspective assessment across different source of raters (instructors, representative students and audiences) by using G-theory. 

Summary of work:
This study enrolled 100 students, and divided into 10 groups to create a short-communication video about common irrational drug use such as ATBs smart use, drug over the counter, non-essential national drug lists, drug administration and cannabinoid abuse. The G-study with a fully crossed one-facet design (GroupXRater) quantifies the amount of variance and G-coefficient. The D-study provides information about the optimal number of raters in different sources needed to obtain reliable measurements. 

Results:
G-study shows the G-coefficients of assessing by single instructor, representative student or audience were 0.75,0.54 and 0.66, respectively and D-study suggested that dependability level can be achieved by using two instructors, five students or three audiences for a one-time assessment (Absolute coefficients = 0.75, 0.78 and 0.85, respectively). 

Discussion and conclusions:
This research demonstrated that all source of raters has high reliability score and suggested that greater improvements in dependability might be made by increasing the number of instructors rather than by increase representative student or audience numbers. Although instructor had the highest D-coefficient and increasing the number is efficient in increasing reliability score, but evaluation from multiple sources will contribute to the diverse perspectives in terms of utilization and feedback. 

Take-home Messages:
G-theory allows the estimation of the magnitude of multiple sources of variance and helps decision makers to determine the number of raters in multiple sources needed to obtain reliable measurements and should be used for limit number of occasions. 

References (maximum three) 

[1] World Health Organization. (2012). The pursuit of responsible use of medicines: sharing and learning from country experiences. World Health Organization. 

Arunee Tipwong¹
Parinya Chamnan²
1 Surat Thani Medical Education Center, Surat Thani Hospital, Thailand
2 Sunphasitthiprasong Medical Education Center, Sunphasitthiprasong Hospital, Thailand

Background:
Clinician teachers need multifacet teaching capabilities to help learners achieve good learning outcomes. A previous study found positive association between clinical teaching self-efficacy and professional fulfilment in clinician teachers. This study aimed to further explore how different domains of teacher competency are associated with professional fulfilment. 

Summary of works:
We conducted secondary data analysis from a previous survey study among Thai clinician teachers who work in Medical Education Centers. Maastricht Clinical Teacher Questionnaire was used to assess teaching competency. Domains of clinical teaching competency: articulation, exploration, coaching and role-modelling, were calculated in the mean score. The Professional Fulfilment Index Scale was used to measure professional fulfilment and determined at a cut-off point mean score of 3. Descriptive and analytic statistics were used. 

Summary of result: 
Among 227 respondents, 67.8% were female, and the mean teaching experience was 10.6 years (SD=6.4). The clinical teachers reported 20% of professional fulfillment. Logistic regression showed competency in exploration (OR=3.75, 95%CI 1.75-8.02) and articulation (OR=3.64, 95%CI 1.72-7.71) had higher predictive power of professional fulfillment over coaching (OR=2.63, 95%CI 1.25-5.58) and role-modelling (OR=2.32, 95%CI 1.17-4.60). 

Discussion and conclusion:
Four domains of clinical teaching competency: exploration, articulation, coaching, and role- modelling can promote professional fulfillment in clinician teachers. Basic teaching skills, exploration and articulation, showed higher predictive power on professional fulfillment than 

coaching and role-modeling. Although role-modelling and coaching are important for teaching medicine, basic pedagogy like articulation and exploration are fundamental in teaching skill development. It ensures clinician teachers of self-confidence in knowledge transferring, which consequently promotes positive psychological well-being. Therefore, a faculty development program for clinician teachers that provide an assessment of teaching competency and useful feedback to enhance teaching skills may help increase their professional fulfillment. 

Take home message:
Basic pedagogy is vital and assessing teaching skills can help promote clinician teacher’s professional fulfillment. 

References (maximum three) 

Tipwong A. Exploring Internal and External Contributors to Mental Health 

among Thai Physicians in Clinical Teaching Contexts. eScholarship@McGill: McGill University; 2022. 

Bajwa NM, De Grasset J, Audétat M-C, et al. Training junior faculty to become clinical teachers: The value of personalized coaching. Medical Teacher. 2020;42(6):663-672. 

Trockel M, Bohman B, Lesure E, et al. A Brief Instrument to Assess Both Burnout and Professional Fulfillment in Physicians: Reliability and Validity, Including Correlation with Self-Reported Medical Errors, in a Sample of Resident and Practicing Physicians. Academic Psychiatry. 2018;42(1):11-24. 

Cheng Yi Huang^1,2
YU-MING WANG³, Chia-Hui Hung⁴, Yi-Ching Li⁵ and Yung-Chuan Ho⁶
1 Department of Nursing/Chung Shan Medical University
2 Department of Nursing /Chung Shan Medical University Hospital
3 Department of Psychology/Chung Shan Medical University
4 Department of Occupational Therapy/Chung Shan Medical University
5 School of Medicine /Chung Shan Medical University Hospital
6 Department of Medical Applied Chemistry/ Chung Shan Medical University

USR is increasingly recognized as a critical aspect of higher education. Developing students' competence so they may be productive, responsible citizens and a tremendous resource for communities is a crucial component of universities' social responsibility. 

From September 2020 to January 2021, a self-reported, cross-sectional study was conducted. We utilized convenience cluster sampling from a medical university. Two hundred students were admitted for the first phase, and EFA was performed to confirm four factors. For the second phase, 405 students were accepted, CFA was performed, and the adjusted model was three factors in achieving the best fit. 

We proceeded to the CFA by using 17 items and a three-factor structure. The fit of model 2 was optimal without modification, representing a good fit between the hypothetic model and the collected data. The internal consistency reliability of the HPLS-USR Scale was calculated at 0.91; the total variance explained is 51.02%. 

Our final model of literacy-based scale for measuring reflections is also consistent with evaluating students’ professional competence in personal growth, the responsibility of citizenship, and social interaction. The CFA validation of the scale model is slightly different from the results of the previous EFA. Social interaction (Factor 3) correlates highly with intellectual growth (Factor 4). 

There was a good fit for the structural equation model, suggesting that personal growth, the responsibility of citizenship, and social interaction were latent variables of reflections on a USR curriculum. 

The HPLS-USR has undergone the extraction process of EFA and the rigorous cross-validation of CFA, and the HPLS-USR has successfully passed the rigorous verification, proving that it can be applied to the evaluation of USR courses in medical universities. 

References (maximum three) 

Awad, A., & Mohammed, A. A. A. (2022). Social Responsibilities at Universities How to Enforce the Social Responsibility of Faculty Members Applied Study at Abu Dhabi University–UAE. Journal of Positive School Psychology, 9640–9660-9640–9660. 

Hung, C. H., Huang, C. Y., Wang, Y. M., Li, Y. C., & Ho, Y. C. (2022). The Literacy-Based Scale for Measuring Reflections on a University Social Responsibility Curriculum: Development and Validation. International Journal of Environmental Research and Public Health, 19(8), Article 4545. https://doi.org/10.3390/ijerph19084545 

Chris Harlow¹
Stephanie Williams², Arun Sivananthan³, Neel Kapoor, Louise Schofield⁴ and Clifford Lisk⁵
1 King's College Hospital
2 Royal Free Hospital
3 Imperial College London
4 Royal Free London
5 Barnet Hospital

The London Medical Trainee Network was set up to provide virtual teaching via Zoom for Internal Medicine Trainees in London, UK following the cessation of in-person teaching because of the COVID-19 pandemic. We outline the standard operating procedure (SOP), process of communication and lessons learned over the three years of the program. 

A SOP was created by Education fellows and Supervising Trainer; one afternoon teaching per month consisting of three 40-minute talks on a medical specialty, followed by 10 minutes of Q&A. Talks are guideline-based with three to five multiple-choice questions (answered via an interactive polling platform). To reduce technical problems, we run a test call in the preceding week. 

We communicate primarily through WhatsApp groups with only administrators allowed to post, with no external advertising, and maximum once-daily posts during normal working hours. All our slides and posters are formatted to a distinctive, uniform design. Anonymised feedback is required for automated certification. Panel discussions have been introduced. 

Twenty-eight sessions have occurred in 2022/23. The mean attendance this year was 333 trainees. 98% feel the sessions ran smoothly, 99% would recommend to others, 92% felt able to ask questions, and 86% feel our teaching is at least as effective as face-to-face. Benefits include no travel (90%) and ability to watch later (86%), while loss of social interaction (60%) was felt to be a negative. 100% gave positive qualitative feedback on slide design. 

Our experience shows that a well organised virtual teaching program has a significant role in the delivery of consistently high-quality teaching that is at least as effective as face-to-face teaching, with the advantage of being available as recordings and no travel. We need to do work on integrating virtual teaching and face to face teaching to get the best of both worlds. 

References (maximum three) 

N/A 

Jillian Clarke¹
Yobelli Jimenez¹
1 Sydney School of Health Sciences, The University of Sydney 

This work explores an ePortfolio capstone project confined to a single end-of-degree subject, which asks students to retrospectively reflect on skill development and professional identity, to create an employment-focussed ePortfolio. We reflect on 5 years of iterative development of this assessment. 

We aimed to embrace a prior Portfolio assessment experience that had focussed on learning development while ultimately serving the need for summative assessment1, albeit in a single unit rather than throughout, for pragmatic reasons. The task has three components: Professional Capabilities, University Graduate Qualities and Learning to Learn skills. The context is an entry-level postgraduate medical imaging degree with 80 students. The learning framework involves 3x2hr interactive lectures plus 2x2hr tutorials per student group. The assessment involves 2 small-group reports based on tutorial work, which culminates in the creation of a personal ePortfolio of depth and complexity as discussed in tutorials. 

Student feedback, staff reflections, expert educational designer feedback and the resulting changes over each iteration has resulted in an effective, sustainable task for students and faculty. Students report the task assists them articulate their achievements in interviews for their first position. 

While the evidence for programmatic assessment strategies, such as portfolio work which spans a students’ degree are compelling, traditional portfolios can create tensions2, can be cumbersome to compile and curate, time-consuming to mark and difficult to integrate into an existing degree3. The retrospective ePortfolio can provide a sensible and effective solution. 

Conclusions 

We have iteratively designed a sustainable end-of-degree assessment in which students articulate their unique strengths and achievements, to create an employment focussed ePortfolio. Clear instructions and scaffolded tutorial activities which help students brainstorm their learnings over their degree to date, and class discussion of standards, are essential to bring degree-long learnings and evidential artifacts together in a professional ePortfolio. 

References (maximum three) 

Dorai Raj D Appadorai 

The prevalence of the ageing population is phenomenally escalating and thus, the sprouting of nursing homes globally is set to grow. Transferring an unwell patient from the nursing home to a tertiary centre for medical care is almost a knee jerk practice, owing largely to less competent or confident healthcare professionals in these settings. This compels the necessity of ensuring that healthcare professionals are periodically assessed and competent in managing common acute medical scenarios within the nursing home setting. Primarily, this ensures that nursing home residents receive prompt and accurate care by providers who are robust in their approach, within the safe, familiar and controlled space of the nursing home. In situ simulation has been used to assess and remodel performance in real time, while providing an interactive and reflective learning experience. Simulation based education is widely known to unravel hidden cognitive errors at workplace, reframe mental models and enhance interprofessional collaboration during a crisis. Nursing homes often face manpower crunch, due to the heavy caregiving or medical burden posed by residents. The focus at nursing homes is largely to comply to routine care and ensuring that each resident survives to live another day. Conitunous education is commonly neglected. Most clinicians or nurses are not able to leave the confines of the home to attend simulation training at dedicated centres or tertiary settings. It is however, imperative that their clinical competence and decision making skills are assessed and refined periodically. Periodic assessments and reflection informs and drives future practice. Therefore, in situ simulation based training brings the experience to nursing homes. Trainee performance is assessed and collated through qualitative and quantitative data. Outcomes from in situ training can be shared and extrapolated across other elder care institutions for improved patient safety outcomes and potentially reduced unnecessary hospitalization. 

References (maximum three) 

1) Using in situ simulation to improve in-hospital cardiopulmonary resuscitation: Authors - G. Lighthall, T. Poon, T. K. Harrison 

2) Integrating Human Patient Simulation into Associate Degree Nursing Curricula: Faculty Experiences, Barriers, and Facilitators: Authors - K. Adamson