Times are shown in your local time zone GMT
Ad-blocker Detected - Your browser has an ad-blocker enabled, please disable it to ensure your attendance is not impacted, such as CPD tracking (if relevant). For technical help, contact Support.
Assessment of clinical skills and attributes
E Poster
ePoster
11:30 am
26 February 2024
Exhibition Hall (Poster 1)
Session Program
11:30 am
Seng Khee Gan1
Rashmi Watson1
1 Medical School, University of Western Australia
Rashmi Watson1
1 Medical School, University of Western Australia
Background
- Supporting final year students underperforming with clinical skills (CS) performance is challenging (refs 1,2,3). Whether targeted workshops, complementing existing pastoral support and academic monitoring, to review skills performance assists confidence and self-agency with improving CS is unknown.
Summary
- Students underperforming in CS with the MD3 OSCE 2022, or in workplace assessments in final year MD4 2023, were invited to 20 CS development (CSD) group sessions (face-to-face or on-line options available).
- CSD workshops were facilitated by a faculty clinician addressing common CS deficiencies with paired-peer practice opportunities, peer feedback and group debrief. Attendance monitoring and student evaluations were implemented to monitor student engagement and reflection.
Results
- 35 students were invited, with a further 10 added by mid-2023; 14 sessions delivered thus far in 2023. Of invited students: 40% attended ≥50% sessions. 30% attended <25% sessions. Mean evaluation response rate with sessions was 37%.
- The average of session scores to date using a 5-point Likert scale is tabulated (1- strongly disagree; 5-strongly agree):
Adequate opportunity for practice - 4.14
Received useful feedback - 4.27
Increased confidence in approaching specific clinical skills addressed in session - 4.46
Knew what to work on further to improve my clinical skills after this session - 4.48
Had adequate opportunity to ask questions - 4.60
Able to learn more about technical aspects in approaching OSCEs - 4.47
Overall rating of session - 4.65
Discussion
- Students reported positive learning experiences with CSD sessions and reported increased confidence with CS; with responses suggestive of self-agency with personal preparations. Some students showed little engagement, for unclear reasons. Correlation with data from pre and post evaluations and summative performance will be evaluated at year-end.
Conclusion
- Support sessions targeting CS practice in struggling final year students can assist confidence and reported self-agency in attending students.
Implications
- Earlier identification and support are in place and benefits remain to be explored.
References (maximum three)
- Chou CL, Kalet A, Costa MJ, Cleland J, Winston K. Guidelines: The dos, don’ts and don’t knows of remediation in medical education. Perspect Med Educ 2019; 8:322- 338.
- Cleland J, Leggett H, Sandars J, Costa MJ, Patel R, Moffat M. The remediation challenge: theoretical and methodological insights from a systematic review. Med Educ 2013; 47:242-251.
- Kalet A, Guerrasio J, Chou CL. Twelve tips for developing and maintaining a remediation program in medical education. Med Teach 2016; 38:787-792
11:35 am
Jessica Bellamy1
1 University of Wollongong
1 University of Wollongong
Background:
Team-based learning (TBL) is a structured form of small-group learning that emphasises student preparation out of class and application of knowledge in class. TBLs help identify inconsistencies between students’ understanding and their practicum experiences; stimulating development of new frameworks (Parmelee et.al., 2012; Zimmerman et.al., 2011).
Team-based learning (TBL) is a structured form of small-group learning that emphasises student preparation out of class and application of knowledge in class. TBLs help identify inconsistencies between students’ understanding and their practicum experiences; stimulating development of new frameworks (Parmelee et.al., 2012; Zimmerman et.al., 2011).
Summary of work:
Most of my teaching has been directed towards 3rd and 4th year Exercise Physiology students; a period where clinical reasoning is developed. Clinical decision making is a key for Exercise Physiologists, and therefore is often reflected in assessment items. A previous strategy utilised to assess clinical reasoning was through participation in Simulated Case Study Learning Tutorials (SCSLs). Through reflection, I noted that students had found SCSLs over challenging. I recognised that the SCSL format favoured high-performing, extroverted students, thus, I shifted towards the TBL format.
Most of my teaching has been directed towards 3rd and 4th year Exercise Physiology students; a period where clinical reasoning is developed. Clinical decision making is a key for Exercise Physiologists, and therefore is often reflected in assessment items. A previous strategy utilised to assess clinical reasoning was through participation in Simulated Case Study Learning Tutorials (SCSLs). Through reflection, I noted that students had found SCSLs over challenging. I recognised that the SCSL format favoured high-performing, extroverted students, thus, I shifted towards the TBL format.
Results:
Informal student feedback was positive.
Informal student feedback was positive.
- ‘The TBL was great. I liked listening to others’ ideas and thoughts on similar topics. Working as a team to complete the quiz – listening to their reasoning – I find that the best way to learn’
- ‘The TBLs are really great. Less pressure, and learnt from colleagues when discussing ideas. TBLs are better than SCSLs’
Discussion:
I continue to reflect and modify both the case studies involved, and the marking criteria associated with each TBL. I strive to ensure assessment task marks accurately reflect the performance and effort of students, including the weighting of individual vs team marks, facilitator feedback and reflection.
Conclusions:
TBLs provide a safe and effective platform for 3rd & 4th year Exercise Physiology students to develop clinical reasoning.
Take home messages:
- TBL case studies should consider embedded or applied WIL for greatest translation
- The balance between individual and team scores must be considered
The use of interactive platforms may be useful for larger class size
References (maximum three)
- Parmelee, D., Michaelsen, L.K., Cook, S., & Hudes, P.D. (2012). Team-based learning: a practical guide: AMEE guide no.65. Medical Teacher, 34(5), e275-287.
- Zimmerman, S.F., Lester Short, G.F., Hendrix, E.M., & Timson, B.F. (2011). Impact of interdisciplinary learning on critical thinking using case study method in allied health care graduate students. Journal of Allied Health, 40(1), 15-18.
11:40 am
Weeming Lau1
Kyi Kyi Tha1, Sean Atkinson2 and Priyia Pusparajah1
1 Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia
2 School of Rural Health, Gippsland, Monash University Australia
Kyi Kyi Tha1, Sean Atkinson2 and Priyia Pusparajah1
1 Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia
2 School of Rural Health, Gippsland, Monash University Australia
Background
Assessment drives learning. It is important that assessment aligns with the curriculum to ensure meaningful outcomes for the learners. There are many methods to assess clinical skills [CS], each with its pros and cons [1]. The Objective Structured Clinical Examinations [OSCE] is used by many institutions and can be conducted in different ways [2]. Over the time, assessment in CS has moved to one that is enriched with immediate feedback from the assessor within a safe learning environment.
Assessment drives learning. It is important that assessment aligns with the curriculum to ensure meaningful outcomes for the learners. There are many methods to assess clinical skills [CS], each with its pros and cons [1]. The Objective Structured Clinical Examinations [OSCE] is used by many institutions and can be conducted in different ways [2]. Over the time, assessment in CS has moved to one that is enriched with immediate feedback from the assessor within a safe learning environment.
We will discuss a different approach in CS assessment using the Assessment for Progression Examination (APEx) approach. We designed APEx with a focus on effective learning, and constructive feedback. The student are provided with adequate time to complete the task and up to four re-sits if they are deemed nearly at standard or not yet at standard.
The student’s skills performance is not graded using absolute scores. Instead, APEx is marked using three different standards [not yet at standard, nearly at standard and at standard] to determine the outcome of the task.
Why is the topic important for practice?
Most assessments are designed to align with the curriculum. However, we also need to design an assessment approach in CS that is stress free ‘for’ and ‘as learning’ which will benefit students. Sharing this innovation will help participants brainstorm other alternatives in assessment.
Symposium format, including participant engagement methods Introduction of speakers = 5 mins
Engagement with participants - Overview of assessment in clinical skills using interactive tools e.g. polling, Kahoot = 10 mins
Introduction of APEx and the process - 15 mins
Engagement with participants - Assessment approach used at their workplace - the pros and cons. = 20 mins
APex - challenges faced at different sites = 30 mins Engagement with participants - Take home message = 10 mins
Take-home messages / symposium outcomes / implications for further research and / or practice
Creating a new CS assessment approach takes significant time. It needs to be trialed out and be modified based on feedback received during the test.
References (maximum three)
- Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009). Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. Jama, 302(12), 1316-1326.
- Harden, R. M., Lilley, Patricia, & Patricio, Madalena. (2016). The definitive guide to the OSCE : the Objective Structured Clinical Examination as a performance assessment.
11:45 am
Jessica Bellamy1
1 University of Wollongong
1 University of Wollongong
Background:
In 2019, national Exercise & Sports Science Australia (ESSA) accreditation updates required the inclusion (and assessment) of all Exercise Physiology students’ competence to construct and deliver Functional Capacity Evaluations (FCE) for injured workers within the NSW workers compensation scheme.
In 2019, national Exercise & Sports Science Australia (ESSA) accreditation updates required the inclusion (and assessment) of all Exercise Physiology students’ competence to construct and deliver Functional Capacity Evaluations (FCE) for injured workers within the NSW workers compensation scheme.
Summary of work:
I drew on my 10 years of industry experience, combined with my teaching philosophy that emphasises student development of ‘real-life clinical skills’ to develop an applied work-integrated learning (WIL) assessment task. Utilising constructive alignment principles (Biggs & Tang 2015), the scaffolded learning supporting this task incorporated multiple learning modalities including online asynchronous interactive theoretical content, WIL case-studies and practical, before completion and submission of an individual FCE report.
I drew on my 10 years of industry experience, combined with my teaching philosophy that emphasises student development of ‘real-life clinical skills’ to develop an applied work-integrated learning (WIL) assessment task. Utilising constructive alignment principles (Biggs & Tang 2015), the scaffolded learning supporting this task incorporated multiple learning modalities including online asynchronous interactive theoretical content, WIL case-studies and practical, before completion and submission of an individual FCE report.
Results:
Five iterations of this assessment task have allowed >300 Exercise Physiology students to obtain essential ESSA accreditation and work within the NSW workers compensation scheme. The approach has been peer-reviewed by academics, industry partners, students and ESSA, each recognising the authentic learning opportunities and relevance to clinical skills. Utilising the Perceived Utility of Learning Activities Scale (PULAS), this assessment task and associated learning significantly increased (p<.0001) student understanding of FCEs.
Five iterations of this assessment task have allowed >300 Exercise Physiology students to obtain essential ESSA accreditation and work within the NSW workers compensation scheme. The approach has been peer-reviewed by academics, industry partners, students and ESSA, each recognising the authentic learning opportunities and relevance to clinical skills. Utilising the Perceived Utility of Learning Activities Scale (PULAS), this assessment task and associated learning significantly increased (p<.0001) student understanding of FCEs.
Discussion:
Small changes have been made based on student feedback, grade distribution and self-reflection, resulting in a greater focus on integrating transition (Kift 2015) and translation (Leggat et.al., 2021) pedagogy. Modifications include: incorporation of a WIL video and 3hr practical before submission of the FCE report.
Small changes have been made based on student feedback, grade distribution and self-reflection, resulting in a greater focus on integrating transition (Kift 2015) and translation (Leggat et.al., 2021) pedagogy. Modifications include: incorporation of a WIL video and 3hr practical before submission of the FCE report.
Conclusions:
This assessment task and associated learning has resulted student competency to complete FCEs, providing >30 graduating students with employment within the workplace rehabilitation sector.
This assessment task and associated learning has resulted student competency to complete FCEs, providing >30 graduating students with employment within the workplace rehabilitation sector.
Take home messages:
- While professional standards drive learning outcomes, I encourage educators to consider how they can implement authentic and meaningful tasks.
- Scaffolding learning through inclusion of WIL may assist students transition from theory to practice.
- Reflect and receive feedback from multiple sources in order to ‘improve’ learning experiences.
References (maximum three)
- Biggs J., & Tang C. (2015). Constructive alignment: An outcomes-based approach to teaching anatomy. In: Chan, L., Pawlina W. (Eds). Teaching Anatomy. Springer.
- Kift S. (2015). A decade of transition pedagogy: a quantum leap in conceptualising the first-year experience. Nursing & Health Sciences, 4(4), 201-208.
- Leggat F., Wadey R., Day M.C., Winter S., & Saunders P. (2021). Bridging the know- do gap using integrated knowledge translation and qualitative inquiry: a narrative review. Qualitative Research in Sport, Exercise and Health, 1, 1-4.