Su Somay¹
Christopher Feddock¹ and Thai Ong²
1 NBME
2 National Board of Medical Examiners 

1. Background
Evidence-Centered Design (ECD; Mislevy, Steinberg, & Almond, 1999) is a structured approach to developing assessment content, which includes four developmental steps: specifying the purpose of assessment and the intended inferences about the learners; specifying the evidence that is necessary for evaluating the learner in the identified areas; designing the most appropriate tasks to yield that evidence; and specifying the data that should be collected for each of those tasks. 

ECD is an important framework for assessment development because it directly aligns the evidence collected in an assessment and the intended inferences about the learners. Assessments developed through ECD are conceptualized as evidentiary arguments, building the key components of test score validity. Further, using the structured steps listed above provides a unified approach to assessment development and implementation. Thus, ECD provides a comprehensive approach for even complex domains that lack clear assessment criteria and objective assessment measures. 

In this workshop, participants will learn about the ECD framework through an application to assessment of clinical reasoning, a complex, multidimensional domain. The workshop will involve hands-on activities and group discussions to provide participants with the opportunity to apply the concepts learned to their own educational contexts. 

2. Why is the topic important for research and / or practice?
Using ECD to develop assessments for clinical reasoning ensures that the assessment scores are valid, reliable, and aligned with the desired learning outcomes. ECD starts by clearly defining the learning objectives and competencies that students are expected to achieve, resulting in assessments that are directly related to the key skills and knowledge. ECD also involves a detailed task analysis, which helps identify the cognitive processes and steps involved in clinical reasoning. This analysis allows educators to break down the complex process of clinical reasoning into manageable components and design assessment approaches that measure these components effectively. 

3. Workshop format
Part I: Presentation
-Overview of ECD and its use in designing and developing assessments -Overview of a framework for Clinical Reasoning
 
Part II: Breakout Groups 
-Participants will choose a subcompetency for Clinical Reasoning and structure an assessment through the first two steps in ECD. Presenters will rotate between groups. Presenters will rotate between groups. 

Part III: Report Out and Q&A 

4. Who should participate?
Participants should include medical educators and educational researchers interested in practicing a structured process to develop assessment content. This workshop uses the domain of clinical reasoning for illustration purposes, but the process can be adapted to any content domain. 

5. Level of workshop (beginner / intermediate / advanced)
Introductory to intermediate level 

6. Take-home messages / workshop outcomes / implications for further research or practice Participants will:
 i. Understand fundamentals of ECD and its role in developing assessments.
 ii. Practice ECD through a clinical reasoning example. 

iii. Exchange ideas on how to incorporate ECD into their own medical education programs. 

7. Maximum number of participants 30 

References (maximum three) 

Daniel, M., Rencic, J., Durning, S. J., Holmboe, E., Santen, S. A., Lang, V., Ratcliffe, T., Gordon, D., Heist, B., Lubarsky, S., Estrada, C. A., Ballard, T., Artino, A. R., Jr, Sergio Da Silva, A., Cleary, T., Stojan, J., & Gruppen, L. D. (2019). Clinical Reasoning Assessment Methods: A Scoping Review and Practical Guidance. Academic Medicine: Journal of the Association of American Medical Colleges, 94(6), 902–912. https://doi.org/10.1097/ACM.0000000000002618 

Mislevy, R.J. and Riconscente, M.M. (2006). Evidence-Centered Assessment Design. In T.M. Haladyna & S. M. Downing (Eds.). Handbook of Test Development (1st ed.). Routledge. https://doi.org/10.4324/9780203874776 

Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., & Johnson, L. (1999). A cognitive task analysis with implications for designing simulation-based performance assessment☆. Computers in Human Behavior, 15, 335-374. https://doi.org/10.1016/S0747-5632(99)00027-8