Brahm Marjadi¹
Neville Chiavaroli², Olanrewaju Sorinola³, Veronica Milos Nymberg⁴, Caroline Joyce¹, Carl Parsons¹ and Anna Ryan⁵
1 School of Medicine, Western Sydney University
2 Australian Council for Educational Research
3 Warwick Medical School, University of Warwick
4 Department of Clinical Sciences, Lund University
5 Melbourne Medical School, The University of Melbourne

Background:
The importance of teaching diversity in medical curricula has been well- established (1). However, the role of assessment for teaching and learning about diversity is still very much under-researched. Assessment questions may form part of the hidden curriculum (2) which either strengthens or undermines diversity teaching. 

Summary:
We audited pre-clinical and clinical multiple choice and short answer examination questions from the same year (2018) at three medical school at three western countries. The audit focused on patient portrayals in the examination question stems. We used an expansion of Critical Race Theory (3) in identifying a wide range of diversity characteristics: age, ethnicity, gender, sexual identity, marital status, residency status, geographic residence, socio- economic status, religion/spirituality, and ability. Both explicit and implicit diversity characteristics were included in the audit. 

Results:
We audited 1,537 patient portrayals from 3,566 examination questions. Apart from age (89.4%) and binary genders (93.9%), other diversity characteristics were rarely portrayed (ethnicity 7.2%, relationship status 1.9%, sexual identity 1.1%, socio-economic status 0.5%, geographic residence 0.1%, disability 0.1%), or not at all (non-binary genders; residency status; religion/spirituality). 

Discussion:
The three schools are embedded in highly diverse communities and teach about patients’ diversity, yet the vast majority of patients presented in written examination question stems were non-descript beyond their age and binary genders. While presenting excessive and unnecessary patient characteristics in examination questions should be avoided, the absence of many diversity aspects may reduce examination authenticity and undermine the teaching of diversity in the curriculum. 

Conclusion:
There is room for improvement in diversity presentation in written examinations at the three audited schools. 

Implications:
Medical schools should consider a routine audit and potential broadening of the diversity features of patients in examination questions to support teaching about diversity. 

References (maximum three) 

Colleen Robb¹
Kelly Dore^1,2
1 Acuity Insights
2 McMaster University

While a more diverse workforce is required to reduce health disparities, many communities are still underrepresented across allied health(1). To achieve workforce representation, a more diverse student body must first be admitted into educational programs. This is often additionally difficult for minority applicants who must complete knowledge-based exams (KBE) which are often biased in favour of majority groups. However, additional tools like situational judgment tests (SJT) can facilitate a more holistic process and potentially mitigate the demographic impact of KBEs. Casper, for example, is an SJT which tends to produce smaller demographic group differences (DGD) than KBEs(2). Of particular interest is Casper’s inclusion of video responses, which could further reduce DGD. 

This study explores DGD and compares Casper in its previous format (typed responses) to the new format (typed and video responses). Casper formats were also compared to publicly available 2022-2023 data for traditional admissions metrics (MCAT(3)). Applicants (n=18,685) to health science programs completed Casper and effect sizes (Cohen’s d) were used to evaluate DGD. 

With the new Casper format, DGD were either reduced or maintained relative to the old format. With regard to race, White applicants were compared to (i) Asian (0.05 to 0.01), (ii) Black/African/Caribbean/African American (0.77 to 0.54), and (iii) Hispanic/Latinx/Spanish applicants (0.40 to 0.42). Results persisted when comparing native and non-native English speakers (0.77 to 0.75), rural and non-rural applicants (0.15 to 0.16), low- and high-income applicants (0.31 maintained), and applicants who did and did not identify as having a disability (0.02 maintained). Gender DGD increased slightly with the new format (0.14 to 0.21). Both Casper formats evidenced smaller DGD than MCAT 

These findings suggest the new Casper format can further reduce DGD and when used alongside other academic measures early in the selection process, may help to dilute larger DGD often observed in KBEs. 

References (maximum three) 

(1) Wilbur, K., Snyder, C., Essary, A. C., Reddy, S., Will, K. K., & Saxon, M. (2020). Developing workforce diversity in the health professions: A social justice perspective. Health Professions Education, 6(2), 222-229. https://doi.org/10.1016/j.hpe.2020.01.002 

(2) Acuity Insights. (2023). Casper Technical Manual. https://acuityinsights.com/casper- technical-manual/ 

(3) AAMC. (2023). 2022 FACTS: Applicants and Matriculants Data. https://www.aamc.org/media/6066/download?attachment 

Clare Owen¹
1 MSC 

The Medical Schools Council (MSC) is working with UK medical schools to introduce a national applied knowledge test (AKT) that will form part of the Medical Licencing Assessment regulated by the General Medical Council (GMC). A voluntary Policy Framework has been put in place that covers issues such as the adjustments available to students with disability. 

Summary of work
The introduction of the AKT provided an opportunity to put in place a policy for the provision of accommodations for students with a disability that followed best practice and was compliant with UK equalities legislation. 

Survey data on the percentage of students receiving adjustments will be shared. 

The main considerations in putting together the policy on adjustments were; 

Conclusions
The policy asks medical schools to make decisions on adjustments for their students to avoid any unnecessary bureaucracy for students and ensure continuity of provision. MSC will accept any adjustment signed off by a school that it can reasonably accommodate. 

Take-home messages / implications for further research or practice 
To create an inclusive environment for students with disabilities care must be taken to ensure processes for receiving adjustments are straightforward and cater to them as individuals. 

References (maximum three) 

GMC Guidance - Welcomed and Valued https://www.gmc-uk.org/- /media/documents/welcomed-and-valued-2021-english_pdf-86053468.pdf accessed August 2023 

Mumtaz Patel¹
Jeremy Brown², Liam Jenkins², John Sandars² and Julie Bridson²
1 NHS England
2 Edge Hill University

Background
Research has identified significant barriers that International Medical Graduates (IMGs) and UK ethnic minority trainees face in their postgraduate medical training, including a lack of familiarity with UK assessments, and a lack of good quality feedback. The impact of these barriers can be seen in attainment gap observed in postgraduate examination pass rates. 

Summary of work
This evaluation focused on eight Masterclasses designed by the Royal College of Psychiatry, and commissioned by the General Medical Council and Health Education England (now NHS England). These were designed to provide targeted support for trainee groups identified as being at higher risk of failing to progress in training, and focused on teaching trainees key principles relating to the CASC examination. Both qualitative and quantitative research was carried out to evaluate the impact of this targeted examination preparation support. 

Results
Early quantitative findings from the September 2021 and January 2022 CASC sitting indicate higher pass rates and a narrowed attainment gap for masterclass participants, particularly for trainees from IMG (11.3% higher for masterclass attendees) or ethnic minority (12.9% higher for masterclass attendees) backgrounds. 

Three key qualitative themes were identified: 1) raised awareness of key factors relating to the examination; 2) development of skills related to examination preparation and performance; 3) application of these skills. 

Discussion
Participants reported a raised awareness of the CASC examination requirements, and developed some key generalisable examination skills in what was perceived to be a safe and trusted learning environment. 

Conclusions
These findings add to the existing evidence (Hawkridge and Molyneux, 2019) that early, targeted support improves outcomes for groups at greater risk of examination fails. 

Take home messages
Interventions can be developed to provide early, targeted support to trainees. There needs to be balance between familiarisation of the examination itself and the development of more generalisable skills. 

References (maximum three) 

References 

Hawkridge A, Molyneux D. (2019) A description and evaluation of an educational programme for North West England GP trainees who have multiple fails in the Clinical Skills Assessment (CSA). Education for Primary Care. 30(3):167-172. 

Panisinee Lawasut^1,2
Thanakorn Jirasevijinda^3,4, Sira Vachatimanont^1,2, Panot Sainamthip^5,6 and Nijasri Charnnarong Suwanwela^1,2
1 Chulalongkorn University
2 King Chulalongkorn Memorial Hospital, Thai Red Cross Society
3 Weill Cornell Medical College
4 NewYork-Presbyterian Hospital/Weill Cornell Medical Center
5 Division of Academic Affairs, Faculty of Medicine, Chulalongkorn University
6 Department of Pharmacology, Faculty of Medicine, Chulalongkorn University

The Chulalongkorn University International Medical Program (CU-MEDi) in Thailand is dedicated to nurturing a diverse cohort of globally oriented and versatile physicians. A departure from convention, CU-MEDi admits graduate students without requiring a science major in their undergraduate studies. Only the Medical College Admission Test (MACT) is required. This project compares the performance of students from science and non-science backgrounds in our program on standardized exams. 

Summary of work 
We assess standardized exam performance among science and non-science students in the academic year 2021-2022 preclinical phase. Evaluation includes summative scores over the first three semesters and Thai National Licensing Examination Step 1 (NLE1) scores mid- fourth semester. A foundational two-month course precedes the curriculum, introducing students to fundamental medical science concepts. The program emphasizes active learning, early clinical exposure, and continuous mentorship from student advisers. 

Results 
In the first class of 35 students, 10 (29%) comprised the non-scientific group. Although MCAT scores for the non-science group were slightly lower, this disparity lacked statistical significance. Across semesters, mean summative scores of the science group consistently outperformed the non-science group, yet the gap progressively diminished (mean difference = 6.81, 6.72, 5.10 points for the first, second, and third semester). Ninety-one percent of the students successfully passed the NLE1, with the cohort achieving a mean score surpassing the national average. 

Discussion 
This study suggests that students from non-science background can perform as well as their peers with science backgrounds under a curriculum that consists of a foundational course followed by an integrative approach to learning . The findings underscore the potential within students from varied academic origins and emphasize the significance of learning environments. 

Implications for further research 
Future work involves studying individual learning paths, non-science students' success factors, and comparing the two groups’ performance in the clinical rotations. 

Gabrielle Finn¹
1 ASME

Differential attainment (DA) is the unexplained variation in attainment between groups when split by a number of protected characteristics, including age, gender and race. Like many institutions, the University of Manchester has awarding gaps for Black, Asian, and Disabled students. Institutions are under pressure to reduce differential attainment, but to date there are no agreed causes of these variations. 

We conducted a realist evaluation to explore the contexts, mechanisms, and outcomes (CMOs) associated with DA within the Faculty of Biology, Medicine, and Health at the University of Manchester. Over150 realist interviews have been conducted with healthcare students, and University stakeholders. 

Results:
We will highlight our programme theory as to the causes of, and solutions to, differential attainment - with a particular lens on assessment. Our data highlight the staggering impact of the lack of inclusive assessment across the student journey. Findings demonstrate how bias can also perpetuate discriminatory patterns that contribute to health inequalities. 

Discussion: 
We often look at end point assessment as the issue when considering DA, yet the reality from our data is that it is also a consequence of the daily discriminatory behaviours students face. These issues manifest in the culture, curriculum, and assessment resulting in a detrimental impact on student wellbeing, learning, and subsequent attainment. Educators exacerbate this problem due to their lack awareness that assessments are neither value neutral, nor culture free. 

Conclusion:
It is important to appreciate that with diverse cohorts, there are often cultural taboos preventing students from seeking help, especially concerning assessment. Assessors are often reluctant to make accommodations for learners, allowing the fear of students cheating or the notion of less robust assessments take over. 

Take home/implications:
Differential attainment is a complex issue but moves towards inclusive and socially just assessments go some way to levelling the playing field for students. 

References (maximum three) 

- 

Miriam Armstrong¹
David Black¹, Liliana Chis², Hristina Evans¹, David Hope³, Abigail Schneider¹ and Mike Jones¹
1 Federation of Royal Colleges of the Physicians of the United Kingdom
2 MRCP(UK)
3 University of Edinburgh

Background:
We examined the contribution of sex, ethnicity, age and PMQ to variation in UK physician recruitment scores and MRCP Part 1 results, extending the evidence-base for interactions between demographic characteristics and performance. We evaluated self- assessment as a component of selection and associations with interview and exam performance. Using continuous score outcomes measures attainment gaps among passing candidates and lets us explore nonlinear effects. 

Summary of work:
Using a retrospective longitudinal cohort sample of N=11,791 physicians recruited into UK Core Medical Training in 2012-2018, we applied multilevel modelling to test the associations of self-assessment and interview scores and MRCP Part 1 exam scores, and then measure the effects of sex, ethnicity, age and PMQ on those associations. 

Results:
Self-assessment and interview scores were non-linearly associated. Medium-large associations were observed for low-scorers, who scored poorly on self-assessment and interview, but there was effectively no association for those who provided a high self- assessment. The association between recruitment scores and MRCP Part 1 performance was linear, with a large effect. The multilevel model showed that sex, ethnicity, age and PMQ impacted the strength of the associations between these variables, indicating demographic factors influence scores in postgraduate assessment. Age correlated moderately and non- linearly with recruitment and exam results, with younger physicians (<30) scoring slightly higher. 

Discussion:
All four examined demographic characteristics impact performance in postgraduate assessment. The support required by different physicians in training should be tailored according to the requirements of their specific demographic characteristics. 

Conclusions:
Sex, ethnicity, age and PMQ intersect in strong and complex ways with recruitment scores and examinations performance. Interactions between demographic variables need further examination and understanding before their implications can be fully understood. 

Take-home messages:
These interactions in postgraduate training data must be further investigated if we are to minimise attainment gaps. 

References (maximum three) 

Soc Med 95, no. 10 (Oct 2002): 511-3. https://doi.org/10.1177/014107680209501013. https://www.ncbi.nlm.nih.gov/pubme d/12356978