Ngoc-Thanh-Van Nguyen¹
Sy Van Hoang¹, Duc Trong Quach¹, Khanh Duc Nguyen¹, Thi-My-Hanh Nguyen¹, Lam Ho Nguyen¹, Kha Minh Nguyen¹, Thi-Bich-Thuy Van¹, Duy Cong Tran¹, Tran-Tuyet-Trinh Nguyen¹, Tuan Thanh Tran¹ and Hoa Ngoc Chau¹
 1 University of Medicine and Pharmacy at Ho Chi Minh city

Critical reasoning is a crucial competence of medical graduates. Current OSCE grading in Vietnam’s medical schools are heavily checklist-based, emphasizing more on the number of performed items, rather than the comprehensive and logical approach to arrive at the most probable diagnoses. Furthermore, a conventional fixed cutscore (4 out of 10) was applied regardless of cohorts and testing materials. We conducted the first Vietnamese study to improve standard setting in summative OSCE cardiovascular station, focusing on critical reasoning. 

We had two types of summative OSCE. One focused on history-taking. The other required students to came up with reasonable diagnoses and order appropriate investigations using information provided. We revised the OSCE questions in both types. For the former, at the end of history-taking, learners must conclude most probable diagnoses with reasons. For the latter, more complex scenarios were given to evaluate high-order thinking. To execute the change, we trained subject matter experts, formed Clinical Competence Committee, revised all OSCE questions, checklists and global ratings. Borderline regression group method (BRGM) was used to establish the new passing score. Content Validity Index was 100% for each item and the entire checklists. 

296 third-year students were evaluated in academic year 2022-2023. In both semesters, BRGM-derived passing scores were 5 out of 10, higher than fixed cutscore. Consequently, passing rates were lower with BRGM (82.8% vs 56.5% in semester 1 and 100% vs 89.2% in semester 2) compared to conventional method. Difference between cutscores were attributed to disparity of item weights between checklists and borderline group definition. Items relating to critical reasoning were given inadequate weights in checklists. Therefore, even if learners missed these critical items, they would still able to score more than 4 and passed the exam. 

High-stake examination should be regularly standardized to accurately assess critical reasoning, distinguishing between competent and incompetent candidates. 

References (maximum three) 

1. Javaeed A. Assessment of Higher Ordered Thinking in Medical Education: Multiple Choice Questions and Modified Essay Questions [version 1]. MedEdPublish 2018, 7:128 (https://doi.org/10.15694/mep.2018.0000128.1) 

2. Lai, JH., Cheng, KH., Wu, YJ. et al. Assessing clinical reasoning ability in fourth-year medical students via an integrative group history-taking with an individual reasoning activity. BMC Med Educ 22, 573 (2022). https://doi.org/10.1186/s12909-022-03649-4