Skip to main content
Ottawa 2024
Times are shown in your local time zone GMT

Perceptions of assessment, teams and equality in assessment

Oral Presentation

Oral Presentation

10:00 am

28 February 2024

M210

Session Program

Caroline Wright1
Dragan Ilic1, Marcelo Maghidman1, Liesl Heinrich1, Kirsten Schliephake1 and Marilyn Baird1
1 Monash University 



Background:
An online case-based assessment program, integrating Science And Practice (iSAP) has been developed to evaluate critical thinking and use of science to inform professional judgements.(1) 

The study aimed to explore student experiences of iSAP assessment. Final year students, enrolled in a range of health professions courses were invited to participate in an anonymous online survey. Questions with Likert scale responses investigated student perceptions on case authenticity, critical thinking and reflection. An open-ended question also allowed students to comment freely on their experiences. Descriptive statistics and thematic analysis were used to analyse the first round of results. The survey will remain open over the next three months and second recruitment drive will attempt to increase participation. 


Results:
24 students have participated to date in the survey. 18/24 students agreed that iSAP allowed them to integrate theory and practice, 16/24 agreed that the assessment helped them develop reflective skills with 19/24 agreeing iSAP assisted in development of critical thinking skills. Preliminary analysis of qualitative comments revealed positive and negative aspects including; the usefulness of the expert response; challenges of adhering to prescribed assessment word limits; and perceived inconsistencies in expert responses causing confusion about expectations. 


Discussion:
Evidence of iSAP’s usefulness in promoting critical thinking, reflection and closing the theory-practice gap has been demonstrated with albeit small participant numbers to date. The results of this survey can be used in conjunction with the university student evaluation processes in the continued enhancement of case-based assessments. 


Conclusion:
The iSAP assessment approach is effective to evaluate student knowledge, reflection and analysis and it also allows students to bridge the theory-practice gap, preparing them for work-integrated learning and professional practice. 



References (maximum three) 

1 Williams I, Schliephake K, Heinrich L, Baird M. Integrating science and practice (iSAP): An interactive case-based clinical decision-making radiography training program. MedEdPublish. 2017 Apr 4;6(65):65. 

Nicole Kain1
Nigel Ashworth1,2, Nancy Hernandez-Ceron1, Ed Jess1, Iryna Hurava1 and Parisa Hamayeli- Mehrabani1
1 College of Physicians & Surgeons of Alberta
2 University of Alberta




1. Background 
Growing numbers of physicians are working in a group practice/clinic setting. Yet, little is known about how this impacts physician performance and patient care. The College of Physicians & Surgeons of Alberta (CPSA) is the medical regulatory authority (MRA) for the province of Alberta, Canada. CPSA is mandated to assess physician performance and continuing competence, and has historically done this at the individual level. The Group Practice Review (GPR), assesses performance at the group/clinic level. The goal of this study is to use the GPR to identify potential risk and protective factors of family medicine/general practitioner (FM/GP) physician group performance. 


2. Summary of work 
Seventy FM/GP clinics participated in the GPR program from 2017-2019. Group performance was determined via the groups’ compliance with the CPSA’s Standards of Practice (SOP). A logistic regression was performed to identify predictors of high performing group practices; those more likely to comply with the SOPs. 


3. Results 
Three factors were identified to predict FM/GP group SOP compliance: prescribing practices, workload and chart scores. Groups with potentially harmful prescribing practices; and/or higher than average volume of patients were less likely to comply with CPSA’s SOPs. Groups with more physicians and better chart scores were more likely to comply with the SOPs. 


4. Discussion 
This study is one of the first to consider predictors of physician group performance. While this research has limitations, it increases understanding of the impact of group practice on physician performance. 


5. Conclusions 
Prescribing, workload and charting all appear to have an impact on FM/GP group performance. With increasing trends towards physician group practice, consideration of the impact of group practice is important for MRAs to both support physicians and protect the public. 


6. Take-home messages / implications for further research or practice
MRAs can identify potentially poorly performing FM/GP clinics. 



References (maximum three) 

No references 

Mumtaz Patel1
Jane Mamelok1
1 NHS England



Background 
Inequality in medical education has existed for many years. The evidence of the attainment gap is at all levels. Between 5 and 30 percentage point difference in education performance exists between UK White trainees, UK black and minority ethnic trainees and International Medical Graduates. Similar gaps are seen for supportive environments, annual review of competence progression (ARCP) outcomes and postgraduate exams. 

Our research at NHS England has focused on earlier identification and interventions to better support trainees. Our cross-collaborative differential attainment(DA) research is with key stakeholders including, the General Medical Council, Royal Colleges, Universities to enable research to be applied in practice and integrated into policy. We have developed a programmatic cross-specialty approach with a stepwise targeted set of interventions with novel approaches to improve training experience, educational outcomes and improve recruitment and retention. 

We have linked trainee interventions with educator development programmes to improve personalised support for trainees. A Learning Needs Analysis tool has been developed providing an effective individualised approach, identifying bespoke trainee needs, which informs personalised interventions to improve outcomes. This work is potentially transferable to other healthcare and training programmes. 


Why is the topic important for research and / or practice? 
Tacking inequality and addressing DA is hugely important from a moral, ethical, personal impact, regulatory and practical perspective. We all lose out if doctors do not receive the support or opportunities they need to achieve their potential. We need to ensure that training environments, assessments and curricula are fair and that any barriers to progression are valid, fair, and justifiable to protect patients. Attainment gaps represent wasted potential and there is a need to apply research into practice and consider interventions that have been shown to have made a difference. A better trained, more confident workforce will have a benefit to all, in particular our patients. 


Workshop format, including participant engagement methods 
Highly interactive workshop, sharing ideas, themes, considering these in the context of a variety of healthcare settings and training pathways. 

There will be facilitated discussion both in a main group and smaller group setting, with ideas considered to tackle inequality in medical education, reduce the attainment gap and enhance training. 


Who should participate? 
All educators, trainees, stakeholders interested to learn about tackling inequality in medical education. 


Level of workshop (beginner / intermediate / advanced)
Intermediate 


Take-home messages / workshop outcomes / implications for further research or practice 
The priority is to build the evidence on what works and share lessons learned. There is no one intervention can eliminate inequality in medical education and tackle DA effectively. There is a need for system wide interventions with earlier identification of learning needs and tailored interventions for trainees and trainers. 


Intended outcomes: 
- Improved understanding of inequality in medical education, differential attainment and ways to address within own learning environment. 
- Share current research and how this can be applied in practice
- Dissemination of good practice across disciplines and institutions - Improve experiences and outcomes for learners. 


Maximum number of participants
50 



References (maximum three) 

1. https://www.gmc-uk.org/-/media/documents/2016-04-28-fairpathwaysfinalreport_pdf- 66939685_pdf-67089434_pdf-70345187.pdf 

2. Woolfe K, Rich A, Viney R, Needleman S, Griffin A. Perceived causes of differential attainment in UK postgraduate medical training: a national qualitative study. BMJ Open 2016;6:e013429 

3. Jeremy Brown, Liam Jenkins, John Sandars, Julie Bridson, Mumtaz Patel (2023) Evaluation of the impact of the Royal College of Psychiatrists Clinical Assessment of Skill and Applied Knowledge masterclass on reducing the attainment gap General Medical Council