Skip to main content
Ottawa 2024
Times are shown in your local time zone GMT

Assessing Clinical Reasoning and decision making

Workshop

Workshop

1:30 pm

26 February 2024

M212

Session Program

Su Somay1
Christopher Feddock1, Thai Ong2 and Debra Klamen3
1 NBME
2 National Board of Medical Examiners
3 Southern Illinois University School of Medicine




1. Background 
The primary objective of NBME's Assessment Alliance is to foster collaborative communities comprising medical educators, learners, and NBME staff, united in addressing the most pressing assessment challenges in medical education. This initiative aims to establish different collaborations, each dedicated to exploring various themes or scopes of work, delving into areas such as the assessment of constructs or exploration of novel assessment methods or formats. In 2022, a significant milestone was achieved as NBME partnered with faculty members from ten diverse U.S. medical schools to establish the first Creative Community. 

The focus of the first Creative Community was shaped based on the feedback and insights gathered from the medical education community. It was determined that enhancing the assessment of clinical reasoning (CR) within the context of the Objective Structured Clinical Examination (OSCE) format would be the starting point. The OSCE for CR Creative Community was launched with a mission to develop innovative solutions that both promote and assess CR skills through school-based OSCEs. This marks a crucial step forward in advancing assessment and aligning it with the evolving needs of learners and educators. 


2. Why is the topic important for research/practice? 
CR skills lie at the core of health professions education, making the assessment of CR an essential aspect of medical education. Nevertheless, assessing CR remains challenging due to its multifaceted and context-specific nature (e.g., Daniel et al. 2019, Young et al., 2018). To tackle these complexities effectively, the Creative Community leveraged Evidence-Centered Design (ECD; Mislevy, Steinberg, & Almond, 1999), a systematic and rigorous framework for assessment development that ensures alignment between assessments and the intended inferences about learners. 

3. Workshop format

Presentation: 
Overview of a framework for Clinical Reasoning, including the differentiation of process vs. outcome measures. 

Breakout groups Part I:
Participants will identify evidence of learner skill for specified clinical reasoning components.

Part I Report Out

Presentation:
Overview of effective frameworks for formative feedback to learners.
 
Breakout Groups Part II:
Group members will identify profiles of learner skills that provide effective feedback for learner growth.

Part II Report Out and Q&A
 
4. Who should participate?
Participants should include medical educators and educational researchers who are interested in formative assessment and/or clinical reasoning. This workshop is focused on the OSCE context, but the principles are applicable to multiple assessment formats. No specific prerequisites are required.


5. Level of workshop (beginner / intermediate / advanced)
Beginner to intermediate level


6. Take-home messages / workshop outcomes / implications for further research or practice
The Creative Community's work promises to advance medical education and practice. By adopting ECD framework for assessment development, significant progress has been made in valid and meaningful assessment of these complex skills.




References (maximum three)
Daniel, M., Rencic, J., Durning, S. J., Holmboe, E., Santen, S. A., Lang, V., Ratcliffe, T., Gordon, D., Heist, B., Lubarsky, S., Estrada, C. A., Ballard, T., Artino, A. R., Jr, Sergio Da Silva, A., Cleary, T., Stojan, J., & Gruppen, L. D. (2019). Clinical Reasoning Assessment Methods: A Scoping Review and Practical Guidance. Academic medicine : journal of the Association of American Medical Colleges, 94(6), 902–912. https://doi.org/10.1097/ACM.0000000000002618 
Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., & Johnson, L. (1999). A cognitive task analysis with implications for designing simulation-based performance assessments. Computers in Human Behavior, 15, 335-374. https://doi.org/10.1016/S0747-5632(99)00027-8 
Young, M., Thomas, A., Lubarsky, S., Ballard, T., Gordon, D., Gruppen, L. D., Holmboe, E., Ratcliffe, T., Rencic, J., Schuwirth, L., & Durning, S. J. (2018). Drawing Boundaries: The Difficulty in Defining Clinical Reasoning. Academic medicine : journal of the Association of American Medical Colleges, 93(7), 990–995. https://doi.org/10.1097/ACM.0000000000002142