Keith Wilson¹
 1 Dalhousie University 

Assessment systems in health professions education are complex and multifaceted. They sit within larger structures that too are often complex. When making changes in these systems, many educators quickly discover that there are several challenges that need to be addressed. For example, educators involved in assessment of student performance may want to integrate a new tool to meet a specific assessment need. Implementation will need to address many interrelated parts and as a result it is oft times challenging to bring these educational innovations to fruition. Systems thinking is an approach to address complex problems and is in use in many professions. It allows us to think more broadly when addressing a problem, considering multiple facets. To this end, a main goal in using systems thinking to address assessment problems is to understand the behaviour of involved processes, ultimately leading to creative ways to manage these complex problems. 

Given the complexities noted above, gaining skill in using systems thinking to address assessment problems allows educators to have greater chance of success. This workshop intends to aid participants in understanding, and ultimately applying, systems thinking to assessment problems in their own context. 

After introductions and icebreakers, participants will be led through a brief introductory session on system thinking, related examples, and how it applies to health professions assessment. Participants will be introduced to practical tools to address and analyze complex problems found in assessment. In small groups, participants will apply their knowledge of systems thinking using a case study. This application of systems thinking principles will then be debriefed in the larger group. Further brainstorming to apply these to participants’ own contexts will be explored before concluding the workshop. 

Educators involved in planning and implementing assessment change(s). 

Intermediate 

- Define systems thinking and explore its importance in health professions education and assessment 

- Recognize the interdependency of components within a complex system
- Apply systems thinking principles to a sample health professions assessment problem 

40 

References (maximum three) 

Khanna P, Roberts C, Lane AS. Designing health professional education curricula using systems thinking perspectives. BMC Medical Education. 2021; 21:20. 

Hassan IS et al. A Systems Thinking approach for the creation of effective competency-based medical education programs. Pan African Medical Journal. 2022;41(203). 

Tim Vincent¹
Bradley Williams², Nadia Mahmood¹, Chee Yeen Fung³, Sam Brandsema², David Howlett¹, Lizzi Shires², Amir H. Sam³ and Malcolm Reed¹
1 Brighton and Sussex Medical School
2 University of Tasmania
3 Imperial College School of Medicine 

Most medical education assessment resources are developed for institutional or national audiences with limited international exchange despite a large overlap in content. Sharing educational resources between institutions internationally is an excellent opportunity to pool knowledge and expertise globally whilst also maximising the use of highly quality-assured educational materials. 

Capsule is a case-based formative assessment resource developed by the Brighton and Sussex Medical School (BSMS) in 2015. The mobile app and web platform delivers clinical scenario- based learning across the whole UK undergraduate medicine curriculum. It is widely used across the UK to support medical students in applying medical knowledge to clinical practice[1,2]. The cases are exclusively developed and quality-assured by senior medical educators in the UK. 

In 2022, BSMS entered a collaboration with the University of Tasmania (UTAS) to adapt existing Capsule cases for Australian medical students. The joint venture has revealed the possibilities and challenges of successfully adapting formative assessment content to meet the needs of local curricular and health contexts. 

Following the experience of adapting assessment cases for Australia, Capsule has continued to work internationally to develop educational resources for Rwanda, Singapore, Ireland and Ukraine. 

High-quality formative assessment tools for undergraduate trainees are available and sharing these across international borders can cost-effectively maximise the global expertise and educational value across the medical education community. However, it does take expertise and time to analyse where content differs between countries and to adapt it appropriately and efficiently. Sharing our experience of adapting an established high-quality, formative 

assessment resource ('Capsule') between the UK and Tasmania will provide real-life insight into the opportunities and challenges of these processes, and further explore how these methods can be implemented in local contexts across the world. 

Workshop format, including participant engagement methods 

Participants will gain practical insight into the realities of adapting formative assessment cases to their local contexts and curricula. The workshop will provide them with the opportunity to explore and practise the process of adapting assessment items for their own nation in small groups. This will be followed by an opportunity to feedback, reflect, and learn from the range of experiences in the wider group and to apply these to formative assessment practice in their own contexts. 

Who should participate? 

Medical educators who are interested in: 

Level of workshop (beginner/intermediate/advanced) 

Take-home messages / workshop outcomes / implications for further research or practice 

References (maximum three)

1. Sadler J, Wright J, Vincent T, Kurka T, Howlett D. What is the impact of Apps in medical education? A study of CAPSULE, a case-based learning App. BMJ 

Simul Technol Enhanc Learn. 2020 Sep 24;7(5):293-296. doi: 10.1136/bmjstel-2020- 

000593. PMID: 35515743; PMCID: PMC8936662.
 2. Karunaratne D, Karunaratne N, Wilmot J, Vincent T, Wright J, Mahmood N, Tang A, 

Sam AH, Reed M, Howlett D. An Online Teaching Resource to Support UK Medical Student Education During the COVID-19 Pandemic: A Descriptive Account. Adv Med Educ Pract. 2021 Nov 13;12:1317-1327. doi: 10.2147/AMEP.S337544. PMID: 34803422; PMCID: PMC8599888.