Sonya Moore¹
Alex Tsirgialos¹, Sia Kazantzis¹ and Selina Parry¹
1 University of Melbourne 

Post-professional clinical practice includes navigating uncertainty, complex clinical reasoning and decision making in different clinical situations. Knowledge acquisition does not guarantee translation into practice, calling for assessment designs which demonstrate integration of advanced clinical skills learning into real-world clinical practice. 

We designed a new wholly online semester-long subject, Advanced Clinical Practice, for practicing physiotherapists and podiatrists. Constructive alignment principles were used to design online learning activities matched to industry-oriented learning outcomes. Clinicians continued to practice remotely in their chosen practice context, where they videoed their own real-world clinical encounters. Clinicians then engaged in self and peer review of these videos, tutor-faculitated online discussions and a reflective professional skills portfolio. Ethics approval (University of Melbourne, ref:2023-23744-42190-1) was obtained to evaluate cohort assessment grade performance and student experience. 

All subject-completing students (n=10) demonstrated an advanced scenario-integrated skill-set repertoire in real-world clinical encounter video assessments. All subject evaluation student survey participants (n=4) agreed online tutorial discussions improved clinical video assessment preparation. Qualitatively, students valued tutor feedback and instructional guidance towards meeting assessment requirements. 

Clinical encounter video assessments required development of new digital technology guidance, including ethical conduct and rapid-response troubleshooting solutions. They provided an authentic demonstration of real-world clinical performance, in remote and context- specific practice settings, expanding accessibility and experiential opportunities. Future cohort evaluation with larger numbers will extend upon early insights, which advocate tutor engagement, instructional communication and transparent outcome expectations. Clinical video assessments will be informed by future digital capability and stakeholder voices. 

Conclusions

Constructively aligned clinical encounter video assessments enabled post-professional clinicians to demonstrate advanced, industry-recognized real-world clinical skills. 

Take-home messages
Constructively aligned clinical video assessments: 

Kellie Charles¹
Paul Chin², Bridin Murnion³, Treasure McGuire⁴, Sarah Hilmer^5,6, Jennifer Martin⁷, David Reith², David Joyce⁸, Catherine Lucas⁹, Nicholas Holford¹⁰, Richard Day¹¹, Jennifer Schneider⁷, Matthew Doogue¹², Catherine Han¹⁰, Sarah Herd¹³, Claire Harrison¹⁴ and Deborah O'Mara¹⁵

1 ANZAHPE
2 University of Otago
3 Royal North Shore Hospital, Northern Sydney Local Health District + University of Sydney 4 Bond University
5 Northern Clinical School, University of Sydney
6 Royal North Shore Hospital
7 University of Newcastle
8 Univerisity of Western Australia
9 Royal Brisbane & Women's Hospital.
10 University of Auckland
11 University of NSW + St Vincent's Hospital, Darlinghurst
12 Department of Medicine, University of Otago, Christchurch, New Zealand Department of Clinical Pharmacology, Canterbury District Health Board, Canterbury, New Zealand
13 University of Tasmania
14 Monash University
15 University of Sydney Medical School/ AMEE/ ANZAHPE

Background:
Undergraduate medical students globally do not feel confident in their prescribing skills at the point of transition from medical school to clinical practice. Currently assessment of prescribing skills across medical schools in Australia and NZ remains inconsistently assessed. 

Summary of work:
The UK Prescribing Safety Assessment was modified for use in Australia and New Zealand (ANZ) as the Prescribing Skills Assessment (PSA). We investigated the implementation, student performance and acceptability of the ANZ PSA for final-year medical students (2). This study used a mixed-method approach involving student data (n = 6440) for 2017–2019. Data were also aggregated by medical school and included student evaluation survey results. Quantitative data were analysed using descriptive and multivariate analyses. The pass rate was established by a modified Angoff method. Thematic analyses of open-ended survey comments were conducted. 

Results:
The average pass rate was slightly higher in 2017 (89%) which used a different examination to 2018 (85%) and 2019 (86%). Little difference was identified between schools for the PSA overall performance or domain sub-scores. Most students provided positive feedback about the PSA regarding the interface and clarity of questions, but an average of 35% reported insufficient time for completion. Further, 70% on average felt unprepared by their school curricula for the PSA, which is in part explained by the low prescribing experience; 69% reported completing ≤10 prescriptions during training. 

Conclusions:
The ANZ PSA was associated with high pass rates and acceptability, although student preparedness was highlighted as a concern for further investigation. 

Takeaway points.

References (maximum three) 

Aline D. Scherff₁
Stefan Kääb2₂, Martin R. Fischer₁ and Markus Berndt₁
1 LMU University Hospital, LMU Munich, Institute of Medical Education, Munich, Germany
2 LMU University Hospital, LMU Munich, Department of Medicine I, Munich, Germany

Background
Electrocardiography (ECG) interpretation is a crucial part of medical education and an important professional activity that medical students must master prior to graduation. However, students often struggle with ECG interpretation, with a recent meta-analysis showing a mean diagnostic accuracy of only 40%. Improving ECG interpretation skills early on has the potential to positively impact patient care. Currently, ECG interpretation is usually taught using visual schemas, but research has shown that as expertise increases, physicians rely less on schemas and arrive at more accurate diagnoses more quickly. 

Summary of work
A study by Scherff et al. (submitted) used eye-tracking and verbal explanations from an expert cardiologist to develop a learning intervention for medical students (EYE-ECG1). This first study showed that students who received the intervention tended to have better ECG interpretation skills and gave more favorable feedback compared to training as usual. The present study EYE-ECG2 built upon the previous study's recommendations to modify the video-audio synchronization of the intervention video and randomize the learning cases. 

In the EYE-ECG2 study, 94 medical students underwent a 3.5-hour online learning intervention, which included a pre-test, four cases presenting complex ECG scenarios, and a post-test (cf. Berndt et al., 2020; Schwehr, 2018). The intervention group viewed a 12-minute expert video before the learning cases. This video allowed the students to see the expert's gaze on the ECGs during interpretation (eye tracking) and to hear the expert's verbal explanations. 

Summary of results
Employing a multiple regression model, the study found moderate and significant improvement in ECG interpretation skills and replicated the results of the previous study, showing greater gains when using the expert video. However, modifying the video-audio synchronization did not improve student performance, and all patient scenarios meaningfully predicted learning outcome, regardless of the order of presentation. 

Discussion and conclusion
In conclusion, presenting students with expert ECG interpretation using eye-tracking and verbal explanations has the potential to improve ECG interpretation skills. 

Take Home Messages
All presented patient scenarios are important for learning outcomes. The conference presentation will include further analyses suggesting that expert videos could hold greater value for advanced learners compared to early learners.



References (maximum three)

Berndt M, Thomas F, Bauer D, Hartl A, Hege I, Kaab S, et al. The influence of prompts on final year medical students’ learning process and achievement in ECG interpretation. GMS Journal for Medical Education. 2020;37(1):Doc 11. 

Jarodzka H, Scheiter K, Gerjets P, Van Gog T. In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction. 2010;20(2):146-54. 

Schwehr KA. Klassifizierung und Analyse von Fehlern bei der EKG-Beschreibung, Befundung und Interpretation: LMU München; 2018. 

Gerrard Phillips¹
Chris McManus² and Liliana Chis³
1 Executive Medical Director, The Federation of Royal Colleges of Physicians of the United Kingdom
2 Emeritus Professor, Research Department, University College London
3 MRCP(UK) 

Background:
PACES assesses the clinical knowledge, behaviours and skills of trainee doctors who intend to enter higher specialist training in the physician specialties. The PACES carousel has five clinical stations, each having one or two encounters, eight encounters overall. Seven physician skills (A-G) are assessed: A: Physical Examination, B: Identifying Physical Signs, C: Clinical Communication, D: Differential Diagnosis, E: Clinical Judgement, F: Managing Patients’ Concerns, G: Maintaining Patient Welfare. Two examiners at each station mark candidates independently. Overall, candidate performance in each skill will have been independently assessed between 8 and 16 times. Candidates need to achieve a minimum passing score in each skill plus achieve an overall total score of 130 (maximum: 176) to pass. 

Summary of work:
Using a retrospective longitudinal cohort sample of 9,018 first attempt UK trainees sitting PACES in 2017-2022, we investigated the impact of PMQ, gender and ethnicity on the pass rate in each of the seven skills. We also applied a multivariate analysis to explore how passing MRCP(UK) Part 1 and Part 2 written examinations at first attempt, PMQ, gender and ethnicity predict passing PACES overall. 

Results:
The highest pass rate was in Skill G and the lowest in Skill B. Compared with their counterparts, females, white candidates and UK graduates achieved higher pass rates in all clinical skills and in Part 1 and Part 2 written examinations when sitting for the first time. Passing Part 1 and Part 2 written examinations at first attempt is also a predictor of passing PACES. 

Discussion: 
Differential attainment in clinical skills performance is a complex multifaceted issue which needs exploring further. 

Conclusion:
Being female, white, UK graduate and passing written examinations at first attempt are good predictors for PACES success. 

Take-home messages:
Differential attainment in clinical exams is a significant issue that needs further investigation. 

References (maximum three) 

1. Woolf K. “Differential attainment in medical education and training.” BMJ; 2020, 368:m339. https://www.bmj.com/content/368/bmj.m339 

1. S. Rupal and S. Ahluvalia. “The challenges of understanding differential attainment in postgraduate medical education.” British Journal of General Practice 2019; 69 (686): 426-427. DOI:https://doi.org/10.3399/bjgp19X705161. https://bjgp.org/content/69/686/426