Colleen Robb¹
Kelly Dore^1,2
1 Acuity Insights
2 McMaster University

While a more diverse workforce is required to reduce health disparities, many communities are still underrepresented across allied health(1). To achieve workforce representation, a more diverse student body must first be admitted into educational programs. This is often additionally difficult for minority applicants who must complete knowledge-based exams (KBE) which are often biased in favour of majority groups. However, additional tools like situational judgment tests (SJT) can facilitate a more holistic process and potentially mitigate the demographic impact of KBEs. Casper, for example, is an SJT which tends to produce smaller demographic group differences (DGD) than KBEs(2). Of particular interest is Casper’s inclusion of video responses, which could further reduce DGD. 

This study explores DGD and compares Casper in its previous format (typed responses) to the new format (typed and video responses). Casper formats were also compared to publicly available 2022-2023 data for traditional admissions metrics (MCAT(3)). Applicants (n=18,685) to health science programs completed Casper and effect sizes (Cohen’s d) were used to evaluate DGD. 

With the new Casper format, DGD were either reduced or maintained relative to the old format. With regard to race, White applicants were compared to (i) Asian (0.05 to 0.01), (ii) Black/African/Caribbean/African American (0.77 to 0.54), and (iii) Hispanic/Latinx/Spanish applicants (0.40 to 0.42). Results persisted when comparing native and non-native English speakers (0.77 to 0.75), rural and non-rural applicants (0.15 to 0.16), low- and high-income applicants (0.31 maintained), and applicants who did and did not identify as having a disability (0.02 maintained). Gender DGD increased slightly with the new format (0.14 to 0.21). Both Casper formats evidenced smaller DGD than MCAT 

These findings suggest the new Casper format can further reduce DGD and when used alongside other academic measures early in the selection process, may help to dilute larger DGD often observed in KBEs. 

References (maximum three) 

(1) Wilbur, K., Snyder, C., Essary, A. C., Reddy, S., Will, K. K., & Saxon, M. (2020). Developing workforce diversity in the health professions: A social justice perspective. Health Professions Education, 6(2), 222-229. https://doi.org/10.1016/j.hpe.2020.01.002 

(2) Acuity Insights. (2023). Casper Technical Manual. https://acuityinsights.com/casper- technical-manual/ 

(3) AAMC. (2023). 2022 FACTS: Applicants and Matriculants Data. https://www.aamc.org/media/6066/download?attachment