Ye Ye¹
1 National Board lf Medical Examiners

Abstract 

In recent years, many programs are evaluating how they can better integrate considerations for diversity, equity, and inclusion (DEI) within their program practices. This begins with a commitment to respecting and representing the perspectives of all candidates and those individuals and organizations they serve in professional practice. Candidates are most likely to feel this inclusivity based on whether the content of the assessment reflects their world view from either their personal background or that of those who they serve. A diverse, equitable, and inclusive physician workforce can lead to better health outcomes and patient experience for all patients, with particular benefit for traditionally underserved populations (Marrast, Zallman, Woolhandler, Bor, & McCormick, 2014). 

When we develop assessments in medical education, what can we do to avoid having bias and stereotypes in our test content? In this presentation, we would like to offer some practical strategies on a number of dimensions: subject matter experts (SME) composition and training, item writing guidelines, patient characteristics and its use in test content, statistical techniques to detect bias, and important considerations when using technology for content creation, especially in the generative AI era. 

It is important to note that developing equitable assessments is not limited to a set of checklists. Rather, we propose a holistic approach where DEI practices are embedded throughout the development process, from item development, to test assembly, to scoring and report. The authors would like to offer our practical guidelines in creating assessment through the equity lens. 

References (maximum three) 

Marrast LM, Zallman L, Woolhandler S, Bor DH, McCormick D. Minority physicians’ role in the care of underserved patients: Diversifying the physician workforce may be key in addressing health disparities. JAMA Intern Med. 2014;174(2):289-291 

Hill KA, Samuels EA, Gross CP, Desai MM, Sitkin Zelin N, Latimore D, Huot SJ, Cramer LD, Wong AH, Boatright D. Assessment of the Prevalence of Medical Student Mistreatment by Sex, Race/Ethnicity, and Sexual Orientation. JAMA Intern Med. 2020 May 1;180(5):653-665. doi: 10.1001/jamainternmed.2020.0030. PMID: 32091540; PMCID: PMC7042809. 

Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ. Differences in words used to describe racial and gender groups in Medical Student Performance Evaluations. 

PLoS One. 2017 Aug 9;12(8):e0181659. doi: 10.1371/journal.pone.0181659. PMID: 28792940; PMCID: PMC5549898.